Display device and voice wakeup method for display device
By identifying the signal-to-noise ratio and system volume of the user's voice, and adjusting the amplitude of the wake-up sound of the display device, the problem of unwell sound volume of the device in different environments is solved, and the user experience is improved.
Patent Information
- Application Number
- PCT/CN2024/081175
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-06-29
- Filing Date
- 2024-03-12
- Publication Date
- 2024-11-28
AI Technical Summary
The system volume is too loud to detect when the display device is in a noisy environment, causing the user to mistakenly believe that the device has not been awakened. However, when the system volume is high in a quiet environment, the alarm sound is too abrupt, affecting the user experience.
By identifying the signal-to-noise ratio of the user's voice and the current system volume, adjusting the audio amplitude of the wake-up prompt to adapt to the noisy environment and the system volume, ensuring that the volume of the wake-up prompt tone is moderate and easy for users to detect.
Improves the user's wake-up experience under different environmental conditions, ensuring that the wake-up prompt sound is easy to detect while not interfering with the user.
Smart Images

Figure CN2024081175_28112024_PF_FP_ABST
Abstract
Description
Display device and voice wake-up method for display device
[0001] CROSS-REFERENCE TO RELATED APPLICATIONS
[0002] This application claims priority to Chinese patent applications filed on March 22, 2023, with application number 202310286393.6; filed on June 29, 2023, with application number 202310791929.X; filed on June 29, 2023, with application number 202310793945.2; filed on June 29, 2023, with application number 202310791970.7; and filed on June 29, 2023, with application number 202310791851.1, the entire contents of which are incorporated herein by reference. Technical Field
[0003] The present application relates to the technical field of display devices, and in particular to a display device and a voice wake-up method for the display device. Background Art
[0004] A display device refers to a terminal device capable of outputting a specific display image. These devices can include smart TVs, mobile terminals, smart advertising screens, projectors, and other terminal devices. Taking smart TVs as an example, they are based on Internet application technology, feature an open operating system and chip, and possess an open application platform. They enable two-way human-computer interaction and integrate multiple functions, including audio, video, entertainment, and data, to meet the diverse and personalized needs of users.
[0005] The display device is equipped with a far-field voice control function, and the user can wake up the display device through voice. Voice wake-up, also known as keyword recognition / detection, is an important branch of voice recognition technology. It means that the display device recognizes specific keywords (for example, wake-up words or trigger words) from the voice input by the user. After recognizing the voice of the user inputting the wake-up word, it is awakened and plays the wake-up prompt tone. The display device plays the wake-up prompt tone according to the system volume. If the system volume is high, the volume of the wake-up prompt tone is also high. If the system volume is low, the volume of the wake-up prompt tone is also low.
[0006] The effect of the wake-up tone played by the display device on the user's hearing will be affected by the noise level of the environment. For example, if the display device is in a noisy environment and the system volume of the display device is low, the volume of the wake-up tone played by the display device will also be low. In this case, the user will not easily hear the wake-up tone, causing the user to mistakenly believe that the display device has not been awakened. For another example, if the system volume of the display device is high and the environment is quiet, the volume of the wake-up tone played by the display device will also be high. In this case, the high-volume wake-up tone will be more abrupt in the environment and easily disturb the user.
[0007] Summary of the Invention
[0008] According to some embodiments of the present application, a display device includes: a sound collector, configured to: collect user voice; a memory, configured to: store computer instructions and data associated with the display device; at least one processor, connected to the sound collector and the memory, and configured to execute the computer instructions to enable the display device to execute: when recognizing that the user voice contains a wake-up word, obtaining the signal-to-noise ratio and the current system volume of the user voice, the signal-to-noise ratio being used to characterize the noisiness of the environment in which the display device is located; adjusting the amplitude of the audio of the wake-up prompt sound according to the signal-to-noise ratio and the current system volume to obtain the audio to be played of the wake-up prompt sound; and controlling the speaker to play the audio to be played of the wake-up prompt sound at the current system volume.
[0009] According to some embodiments of the present application, a voice wake-up method for a display device includes: collecting user voice; when recognizing that the user voice contains a wake-up word, obtaining the signal-to-noise ratio and the current system volume of the user voice, the signal-to-noise ratio being used to characterize the noisiness of the environment in which the display device is located; adjusting the amplitude of the audio of the wake-up prompt sound according to the signal-to-noise ratio and the current system volume to obtain the audio to be played of the wake-up prompt sound; and playing the audio to be played of the wake-up prompt sound at the current system volume. BRIEF DESCRIPTION OF THE DRAWINGS
[0010] FIG1A shows a usage scenario of a display device according to some embodiments of the present application;
[0011] FIG1B is a schematic diagram of an interactive scenario of a display device according to some embodiments of the present application;
[0012] FIG2 is a configuration block diagram of a control device according to some embodiments of the present application;
[0013] FIG3 is a configuration diagram of a display device according to some embodiments of the present application;
[0014] FIG4A is a schematic diagram of software configuration of a control device according to some embodiments of the present application;
[0015] FIG4B is a schematic diagram of a network architecture for voice interaction according to some embodiments of the present application;
[0016] FIG5 is a schematic diagram of a lighting effect control system structure composed of a terminal device and an indicator light according to some embodiments of the present application;
[0017] FIG6 is a schematic diagram of a dual-chip architecture according to some embodiments of the present application;
[0018] FIG7 is a schematic diagram showing the effect of waking up a terminal device through a wake-up word voice in some embodiments of the present application;
[0019] FIG8 is a schematic flow chart of a method for waking up a dual-controller from standby mode according to some embodiments of the present application;
[0020] FIG9 is a flow chart of handshake protocol judgment logic according to some embodiments of the present application;
[0021] FIG10 is a schematic diagram of a change in the prompt lighting effect of an indicator light according to some embodiments of the present application;
[0022] FIG11 is a schematic diagram of another indicator light effect change effect of some embodiments of the present application;
[0023] FIG12 is a timing diagram of a terminal device when wake-up is successful in some embodiments of the present application;
[0024] FIG13 is a circuit diagram of a dual controller controlling an indicator light in some embodiments of the present application;
[0025] FIG14 is a timing diagram of a terminal device when wake-up fails in some embodiments of the present application;
[0026] FIG15 is a flowchart of a far-field voice standby wake-up method according to some embodiments of the present application;
[0027] FIG16 is a timing diagram of terminal state transitions according to some embodiments of the present application;
[0028] FIG17 is a diagram illustrating an operating system configuration of a display device according to some embodiments of the present application;
[0029] FIG18 is a flowchart of a display device adjusting the volume of a wake-up prompt tone according to the signal-to-noise ratio of a user's voice and the current system volume in some embodiments of the present application;
[0030] FIG19 is a flowchart showing a device identifying a wake-up word according to some embodiments of the present application;
[0031] FIG20 is a flowchart of calculating the wake-up value of a user's voice according to a display device usage model in some embodiments of the present application;
[0032] FIG21 is a schematic diagram of a model for calculating a wake-up value of a user's voice by a display device according to some embodiments of the present application;
[0033] FIG22 is a schematic diagram of a model for calculating a wake-up value of a user's voice by a display device according to some embodiments of the present application;
[0034] FIG23 is a flowchart illustrating a display device adjusting the amplitude of the wake-up prompt tone according to the signal-to-noise ratio of the user's voice and the current system volume in some embodiments of the present application;
[0035] FIG24 is a flowchart illustrating a display device adjusting the amplitude of the wake-up prompt tone according to the signal-to-noise ratio of the user's voice and the current system volume in some embodiments of the present application;
[0036] FIG25 is a logic determination diagram of a display device adjusting the amplitude of the wake-up prompt tone according to the signal-to-noise ratio of the user's voice and the current system volume in some embodiments of the present application;
[0037] FIG26 is a schematic diagram of a pop-up window containing information for prompting a user to adjust the system volume according to some embodiments of the present application;
[0038] FIG27 is a flowchart illustrating a display device adjusting the volume of a wake-up prompt tone according to a difference between a wake-up value and a wake-up value threshold and a current system volume in some embodiments of the present application;
[0039] FIG28 is a flowchart illustrating a display device adjusting the amplitude of an audio wake-up prompt tone according to a difference between a wake-up value and a wake-up value threshold and a current system volume in some embodiments of the present application;
[0040] FIG29 is a flowchart illustrating a display device adjusting the amplitude of an audio wake-up prompt tone according to a difference between a wake-up value and a wake-up value threshold and a current system volume in some embodiments of the present application;
[0041] FIG30 is a logic judgment diagram of a display device adjusting the amplitude of the wake-up prompt tone according to the difference between the wake-up value and the wake-up value threshold and the current system volume in some embodiments of the present application;
[0042] FIG31 is a schematic diagram of a first-style pop-up window and a second-style pop-up window of different sizes according to some embodiments of the present application;
[0043] FIG32 is a schematic diagram of a first-style pop-up window and a second-style pop-up window displayed in different positions according to some embodiments of the present application;
[0044] FIG33 is a schematic diagram of a first-style pop-up window and a second-style pop-up window of different sizes and display positions according to some embodiments of the present application;
[0045] FIG34 is a flowchart illustrating a display device adjusting the volume of a wake-up prompt tone according to a difference between a wake-up value and a wake-up value threshold, a signal-to-noise ratio of a user's voice, and a current system volume in some embodiments of the present application;
[0046] FIG35 is a flowchart illustrating a display device adjusting the amplitude of the wake-up prompt tone according to the difference between the wake-up value and the wake-up value threshold, the signal-to-noise ratio of the user's voice, and the current system volume in some embodiments of the present application;
[0047] FIG36 is a flowchart illustrating a display device adjusting the amplitude of the wake-up prompt tone according to the difference between the wake-up value and the wake-up value threshold, the signal-to-noise ratio of the user's voice, and the current system volume in some embodiments of the present application;
[0048] FIG37 is a diagram showing a logic decision of a display device adjusting the amplitude of the wake-up prompt tone according to the difference between the wake-up value and the wake-up value threshold, the signal-to-noise ratio of the user's voice, and the current system volume in some embodiments of the present application;
[0049] FIG38 is a flowchart of a display device performing a wake-up response when the display device is in a screen-on state and in a first state according to some embodiments of the present application;
[0050] FIG39 is a schematic diagram of a pop-up window containing first information according to some embodiments of the present application;
[0051] FIG40 is a flow chart showing second information according to some embodiments of the present application;
[0052] FIG41 is a schematic diagram of a pop-up window containing first information and second information according to some embodiments of the present application;
[0053] FIG42 is a flowchart showing the release of a silent mode of a display device according to some embodiments of the present application;
[0054] FIG43 is a flowchart showing the release of a silent mode of a display device according to some embodiments of the present application;
[0055] FIG44 is a schematic diagram of a pop-up window containing third information according to some embodiments of the present application;
[0056] FIG45 is a flowchart of a wake-up response when a display device is in a screen-on state and in a silent mode according to some embodiments of the present application;
[0057] FIG46 is a flowchart of a wake-up response when the display device is in a screen-on state, in a non-silent mode, and the current system volume is 0 according to some embodiments of the present application;
[0058] FIG47 is a flowchart illustrating how a display device responds to a wake-up word when in speaker mode and silent mode according to some embodiments of the present application;
[0059] FIG48 is a flowchart showing how a display device responds to a wake-up word when in speaker mode, in non-silent mode, and when the current system volume is 0 according to some embodiments of the present application;
[0060] FIG49 is a schematic diagram showing a display device playing a wake-up prompt tone and a first prompt message according to some embodiments of the present application;
[0061] FIG50 is a schematic diagram showing a display device playing a wake-up prompt tone, a first prompt message, and a second prompt message according to some embodiments of the present application;
[0062] FIG51 is a flowchart illustrating how a display device responds to a wake-up word when in speaker mode and in a first state according to some embodiments of the present application;
[0063] Figure 52 is a schematic diagram of the middle page and pop-up window in some embodiments of the present application. DETAILED DESCRIPTION
[0064] The following embodiments are described in detail, with examples illustrated in the accompanying drawings. When the following description refers to the drawings, identical numbers in different figures represent identical or similar elements unless otherwise indicated. The embodiments described in the following embodiments are not intended to represent all possible implementations consistent with the present application. They are merely examples of systems and methods consistent with certain aspects of the present application, as detailed in the claims.
[0065] It should be noted that the brief descriptions of terms in this application are only for the purpose of facilitating the understanding of the embodiments described below, and are not intended to limit the embodiments of this application. Unless otherwise specified, these terms should be understood according to their ordinary and usual meanings.
[0066] In the specification and claims of this application and the accompanying drawings, the terms "first," "second," "third," etc. are used to distinguish similar or similar objects or entities, and are not necessarily intended to limit a particular order or sequence, unless otherwise noted. It should be understood that the terms used in this manner are interchangeable under appropriate circumstances.
[0067] The terms "comprise," "include," and "have," and any variations thereof, are intended to cover but not exclude inclusion; for example, a product or device comprising a list of components is not necessarily limited to all the components expressly listed but may include other components not expressly listed or inherent to such product or device.
[0068] The display device of some embodiments of the present application may have a variety of implementation forms, and the display device may be hardware or software. When the display device is hardware, it may be various electronic devices with a sound collection function, including but not limited to televisions, laser projection equipment, monitors, electronic whiteboards (electronic bulletin boards), electronic desktops (electronic tables), e-book readers, etc., or it may be mobile phones, mobile terminals, smart speakers, tablet computers, watches, AI devices (such as smart educational equipment, etc.), robots, smart homes, car-mounted terminals and other devices with display screens. When the display device is software, it can be installed in the electronic devices listed above. It can be implemented as multiple software or software modules (for example, for providing sound collection services), or it can be implemented as a single software or software module. No specific limitation is made here. Figures 1A, 1B and 3 are some specific embodiments of the display device of the present application.
[0069] Figure 1A is a schematic diagram of a usage scenario of a display device according to some embodiments of the present application. As shown in Figure 1A, a user can operate a display device 200 through a control device 100, and the display device 200 can obtain network data through a server 300, or obtain a live broadcast signal through a satellite. The user can also control the display device 200 by voice. For example, the user wakes up the display device 200 by issuing a wake-up word, and the awakened display device 200 starts a far-field voice service and responds to the voice instructions issued by the user within a certain period of time. Figure 1B is a schematic diagram of an interactive scenario of a display device according to some embodiments. As shown in Figure 1B, 300 is a server, and 200 is a display device, for example, a TV 200a, a mobile device 200b, and a smart speaker 200c.
[0070] In some embodiments, the server 300 and the display device 200 communicate data via various communication methods. The display device 200 may be connected to a local area network (LAN), a wireless local area network (WLAN), or other networks. The server 300 may provide various content and interactions to the display device 200. In some embodiments, the display device 200 and the server 300 may send and receive information, as well as receive software program updates.
[0071] In some embodiments, the server 300 may be a server that provides various services, such as a background server that provides support for audio data collected by the display device 200. The background server may analyze and process the received audio data, and feed back the processing results (e.g., endpoint information) to the display device.
[0072] In some embodiments, the server 300 may be a server cluster or multiple server clusters, and may include one or more types of servers.
[0073] Figure 2 is a block diagram of a control device 100 according to some embodiments of the present application. As shown in Figure 2, the control device 100 includes at least one processor 110, a communication interface 130, a user input / output interface 140, a memory, and a power supply.
[0074] In some embodiments, the control device 100 may be a remote controller, and the communication between the remote controller and the display device 200 includes infrared protocol communication or Bluetooth protocol communication, and at least one of other short-range communication methods, to control the display device 200 wirelessly or wiredly.
[0075] The user can control the display device 200 by inputting user commands through buttons on the remote control, voice input, control panel input, etc. The control device 100 may include a mute button, which is used to activate and deactivate mute mode. In mute mode, the display device 200 does not change the amplitude of the audio data, that is, retains the current system volume, and after decoding the audio data, does not transmit the decoded audio data to the speaker for playback.
[0076] When the display device 200 is not in silent mode, the user inputs a control instruction to the display device 200 based on the mute key, and the display device 200 recognizes the control instruction as to turn on the silent mode, and in response to the control instruction, retains the current system volume, and after decoding the audio data, does not transmit the decoded audio data to the speaker.
[0077] When the display device 200 turns on the mute mode, the user inputs a control instruction to the display device 200 based on the mute key. The display device 200 recognizes the control instruction as to release the mute mode. In response to the control instruction, the decoded audio data is transmitted to the speaker and the audio is played at the retained current system volume.
[0078] The control device 100 may also include a volume key for adjusting the volume of the played audio. If the user inputs a volume-lowering instruction using the volume key, the display device 200, in response to the instruction, reduces the amplitude of the decoded audio data after decoding the audio data and transmits the reduced-amplitude audio data to the speaker for playback. If the user inputs a volume-up instruction using the volume key, the display device 200, in response to the instruction, increases the amplitude of the decoded audio data after decoding the audio data and transmits the increased-amplitude audio data to the speaker for playback.
[0079] In some embodiments, if the user adjusts the volume to 0 using the volume key, the display device 200 adjusts the amplitude of the decoded audio data to 0 after decoding the audio data, and transmits the audio data with amplitude 0 to the speaker for playback.
[0080] FIG3 is a block diagram of a configuration of a display device 200 according to some embodiments of the present application. In some embodiments, the display device 200 includes at least one of a tuner / demodulator 210, a communication device 220, a detector 230, an external device interface 240, at least one processor 250, a display 260, an audio output interface 270, a memory, a power supply, and a user interface 280.
[0081] In some embodiments, the communication device 220 is a component used to communicate with external devices or servers using various communication protocols. For example, the communication device may include at least one of a Wi-Fi module, a Bluetooth module, a wired Ethernet module, or other network communication protocol chip or a near-field communication protocol chip, as well as an infrared receiver. The display device 200 can use the communication device 220 to send and receive control signals and data signals with the control device 100 or the server 300.
[0082] In some embodiments, the detector 230 includes a sound collector. The display device 200 can collect user voice through the sound collector and transmit the collected user voice to the at least one processor 250 for processing.
[0083] In some embodiments, the sound collector can be a microphone, also known as a "microphone" or "microphone", which can be used to receive the user's voice and convert the sound signal into an electrical signal. The display device 200 can be provided with at least one microphone. In other embodiments, the display device 200 can be provided with two microphones, which can not only collect sound signals but also realize noise reduction function. In other embodiments, the display device 200 can also be provided with three, four or more microphones to realize sound signal collection, noise reduction, and identification of sound sources, and realize directional recording function, etc.
[0084] In addition, the microphone may be built into the display device 200, or the microphone may be connected to the display device 200 by wire or wireless means. Of course, the embodiment of the present application does not limit the position of the microphone on the display device 200. Alternatively, the display device 200 may not include a microphone, that is, the microphone is not provided in the display device 200. The display device 200 may be connected to an external microphone (also referred to as a microphone) through an interface (such as a USB interface). The external microphone may be fixed to the display device 200 by an external fixing member (such as a camera bracket with a clip).
[0085] In some embodiments, at least one processor 250 and the tuner / demodulator 210 may be located in different separate devices, that is, the tuner / demodulator 210 may also be located in an external device of the main device where the at least one processor 250 is located, such as an external set-top box.
[0086] In some embodiments, at least one processor 250 controls the operation of the display device and responds to user operations through various software control programs stored in the memory. At least one processor 250 controls the overall operation of the display device 200.
[0087] In some embodiments, at least one processor 250 includes at least one of a central processing unit (CPU), a video processor, an audio processor, a graphics processing unit (GPU), a random access memory (RAM), a read-only memory (ROM), a first interface to an nth interface for input / output, a communication bus, etc.
[0088] For example, at least one processor 250 can recognize the user's voice through the central processing unit and the audio processor. If the user's voice is recognized to contain a wake-up word, the audio of the wake-up prompt sound can be processed and the processed audio of the wake-up prompt sound can be transmitted to the audio output interface 270 for playback, such as playing through a speaker.
[0089] In some embodiments, the display 260 includes a display screen component for presenting images, and a driving component for driving image display, which is used to receive image signals output from at least one processor 250, and display video content, image content, and menu control interface components and user control UI interface.
[0090] In some embodiments, the display 260 may be a liquid crystal display, an OLED display, a projection display, or a projection device and a projection screen.
[0091] In some embodiments, a user may input a user command through a graphical user interface (GUI) displayed on display 260, and the user input interface receives the user input command through the GUI. Alternatively, a user may input a user command through a specific sound or gesture, and the user input interface receives the user input command by recognizing the sound or gesture through a sensor.
[0092] In some embodiments, a "user interface" is a medium interface for interaction and information exchange between an application or operating system and a user, which realizes the conversion between the internal form of information and the form acceptable to the user.
[0093] In some embodiments, the user interface 280 may be configured to receive external control signals.
[0094] In some embodiments, taking the operating system of the display device 200 as the Android system as an example, as shown in FIG4A , the display device 200 can be logically divided into an application layer (abbreviated as “application layer”) 21 , a kernel layer 22 and a hardware layer 23 .
[0095] As shown in Figure 4A, the hardware layer may include at least one processor 250, a communication device 220, a detector 230, and the like, as shown in Figure 3. The application layer 21 includes a speech recognition application, which can provide a speech interaction interface and services for connecting the television 200a to the server 300. The kernel layer 22, serving as software middleware between the hardware layer and the application layer 21, is used to manage and control hardware and software resources.
[0096] In some embodiments, the kernel layer 22 includes a detector driver, which is used to send the voice data (i.e., user voice) collected by the detector 230 to the voice recognition application. When the voice recognition application in the display device 200 is started and the display device 200 establishes a communication connection with the server 300, the detector driver is used to send the user voice input by the user collected by the detector 230 to the voice recognition application. Afterwards, the voice recognition application sends the query information containing the user voice to the intent recognition module in the server. The intent recognition module is used to input the user voice sent by the display device 200 into the intent recognition model to clarify the user's intention.
[0097] See Figure 4B, which is a schematic diagram of a voice interaction network architecture according to some embodiments of the present application. The smart device is used to receive input information and output the processing results of the information. The speech recognition module is deployed with a speech recognition service for recognizing audio as text; the semantic understanding module is deployed with a semantic understanding service for semantically parsing text; the business management module is deployed with a business instruction management service for providing business instructions; the language generation module is deployed with a language generation service (NLG) for converting instructions for the smart device to execute into text language; the speech synthesis module is deployed with a speech synthesis (TTS) service for processing the text language corresponding to the instruction and sending it to the speaker for broadcast.
[0098] In one embodiment, the architecture shown in FIG. 4B may include multiple physical service devices deployed with different business services, or one or more physical service devices may be integrated with one or more functional services.
[0099] In some embodiments, taking a smart device as an example, the following describes an example of a process for processing information input into the smart device based on the architecture shown in FIG4B , and takes the information input into the smart device as a query statement inputted via voice as an example:
[0100] [Speech Recognition]
[0101] After receiving a query statement input via voice, the smart device may perform noise reduction processing and feature extraction on the audio of the query statement. The noise reduction processing here may include steps such as removing echoes and ambient noise.
[0102] [Semantic Understanding]
[0103] Using acoustic and language models, the natural language understanding module performs natural language understanding on the identified candidate text and associated contextual information, parsing the text into structured, machine-readable information, including business domain, intent, word slots, and other information to express semantics. The executable intent is obtained and the intent confidence score is determined. The semantic understanding module selects one or more candidate executable intents based on the determined intent confidence score.
[0104] [Business Management]
[0105] Based on the semantic analysis results of the query statement text, the semantic understanding module sends query instructions to the corresponding business management module to obtain the query results given by the business service, and executes the actions required to "complete" the user's final request, and feeds back the device execution instructions corresponding to the query results.
[0106] It should be noted that the architecture shown in FIG4B is only an example and does not limit the scope of protection of this application. In this application, other architectures can also be used to implement similar functions. For example, all or part of the above process can be completed by the display device 200, which will not be described in detail here.
[0107] Based on the above-mentioned display device 200, the user can interact with the display device 200 through user voice. In some embodiments, the display device 200 can also receive user voice input by the user and parse the user voice when it is in standby state. When the user voice contains a wake-up word, the standby state of the display device 200 is switched to the running state, that is, the user can also wake up the display device 200 in the standby state through the wake-up word. The standby state can be a state in which the display device 200 is powered on but the processor is dormant. In this state, the power consumption of the display device 200 is the lowest. The running state is a state in which all modules of the display device 200 enter operation. In this state, the power consumption of the display device 200 is higher.
[0108] In some embodiments, the display device 200 recognizes user speech input using a built-in model, using the user speech as an input feature of the model. The model can then determine whether the user speech contains a wake-up word. For example, the model can calculate the similarity between the user speech and a speech feature template; when the calculated similarity is greater than or equal to a similarity threshold, the display device 200 is awakened to facilitate user interaction with the display device 200.
[0109] In related technologies, the standby wake-up function, that is, the function of turning on the display device through far-field voice commands when the display device is in standby mode, requires the device to go through the process of wake-up word recognition and system-on-chip (SOC) initialization from collecting user voice to waking up the display device. This process will take a certain amount of time and may currently be noticeable to the user. If the display device does not provide any feedback, the user will feel a little strange. Therefore, in some embodiments, the display device adds some status changes of the indicator light during this process to prompt the user until the display device successfully enters the operating state.
[0110] As shown in FIG5 , in some embodiments, the display device 200 can form a lighting effect control system with the indicator light 400. The lighting effect control system can be an independent lighting system or an auxiliary system that is built into or externally connected to a control device and constitutes a system together with the control device. For example, the lighting effect control system can be an auxiliary lighting effect system applied to the display device 200, that is, it can connect various lighting fixtures through the external device interface of the display device 200, thereby utilizing the processing power of the display device 200 to control the lighting effects of the various external lighting fixtures.
[0111] It should be noted that indicator lights generally refer to light-emitting components that can convert electrical energy into light energy, including warning lights, lighting lamps, landscape lights, signal lights, etc. This application does not limit the light-emitting principle of indicator lights, which can be traditional electric lamps, light-emitting diodes (LED) lamps, and lamps based on fluorescent principles, etc. Lamps can emit specific types of light according to different uses, such as specific colors, specific brightness, and specific frequencies. Multiple lamps can also be arranged and combined in a certain way to form indicator lights with special combination shapes. For example, a plurality of indicator lights of different colors, shapes, and light-emitting states are combined to form a scene with an artistic pattern effect, and a plurality of indicator lights of different colors, shapes, and light-emitting states flash in a specific cycle to create a stage effect.
[0112] In some embodiments, the indicator light 400 can be connected to a lighting effect control system, such as a lighting effect control board, through an interface such as a pin, so as to control the power supply status of each indicator light 400 and the light emission form of the indicator light 400 through the lighting effect control board.
[0113] In some embodiments, the power supply status of the indicator light 400 may include an on / off status and a power supply type status. By adjusting the power supply status, the corresponding indicator light can be turned on and off. For example, when the indicator light is required to be illuminated, the corresponding on / off status can be set to "on", and the indicator light can obtain power supply and illuminate. Similarly, when the indicator light is required not to be illuminated, the corresponding on / off status can be set to "off", and the light cannot obtain power supply and thus does not illuminate.
[0114] In some embodiments, the on / off state of the device is controlled by powering on and off the indicator light. For example, when the on / off state is "on," the indicator light is actually powered on; when the on / off state is "off," the indicator light is actually powered off. In other embodiments, the on / off state of the device is controlled by outputting high and low levels to the indicator light. For example, when the on / off state is "on," the indicator light is output a high level; when the on / off state is "off," the indicator light is output a low level. Of course, it is also possible to output alternating current to the indicator light to control the high and low levels according to a set pattern, so that the indicator light can flash at a set frequency, presenting different prompt lighting effects.
[0115] In some embodiments, the display device can display different prompt light effects to prompt the user of the current terminal status of the display device, wherein the terminal status may include standby state, pre-start state and running state.
[0116] Before the display device is officially operational, to maintain low power consumption, the display device includes a low-power first processor and a high-power second processor. For example, the first processor is the main chip of the display device, and the second processor is a sub-chip of the display device. The two processors can exist independently on the motherboard.
[0117] In other embodiments, the first processor may also be integrated into the second processor, that is, the sub-chip is integrated into the main chip as one of the sub-processing modules of the main chip. In this case, one chip includes two processors, namely the main processor and the sub-processor.
[0118] Of course, both the first processor and the sub-processor are relatively low-power modules that can run the wake-up word recognition module of the first computing power. Similarly, the second processor or the main processor is a relatively high-power module that can run the wake-up word recognition module of the second computing power. Because the computing power of the wake-up word recognition module is proportional to the power consumption of the processor, the first computing power is less than the second computing power.
[0119] As shown in Figure 6, a display device includes a main chip and a sub-chip. The main chip and the sub-chip can be two independent chips, i.e., they exist independently on the display device's motherboard; or, the main chip and the sub-chip can be a single chip, i.e., the sub-chip is integrated into the main chip, making the sub-chip a sub-module of the main chip.
[0120] When a display device is in standby mode, its sub-chip is powered on and its main chip is powered off, effectively dormant. A low-power sub-chip receives a power-on command from the user, which then switches the display device from standby to active mode via a wake-up command. This wake-up command is a control command used to switch the display device from standby to active mode. It can be input via a remote control supplied with the display device or by voice.
[0121] In some embodiments, when the display device is in standby mode, the display is off and the indicator light presents a first prompt light effect of the standby mode. The first prompt light effect can be a long off, a long on, or other light effect.
[0122] When the display device is in a pre-boot state, the sub-chip of the display device is turned on, and at least a portion of the main chip is turned on. During the boot process of the display device, the display remains off, and the indicator light displays a second pre-boot state prompt lighting effect. The second prompt lighting effect can be a flashing effect, etc.
[0123] When the display device is in operation, the sub-chip of the display device is turned on, and the main chip is turned on. At this time, the sub-chip is in an idle state, no longer performing any processing work, and only forwarding data. When the display device is in operation, the display is in a bright state, and the indicator light displays a third operating status indicator light effect. The third indicator light effect can be a constant on, a constant off, or a color change.
[0124] It is understood that the first, second, and third prompt lighting effects described in this embodiment may be multiple lighting effects. However, any two of the first, second, and third prompt lighting effects may be different lighting effects, allowing the user to perceive the changing state of the display device. For example, in some embodiments, the first prompt lighting effect is a long-on lighting effect, the second prompt lighting effect is a flashing lighting effect, and the third prompt lighting effect is a long-off lighting effect.
[0125] For example: the display device 200 is configured with a voice interaction function, assuming that the wake-up word is "Hi ABC". The display device is in standby mode, as shown in Figure 7, the indicator light of the display device is long-on, and the display is in off-screen mode. Among them, the long-on prompt light effect is used to remind the user that the current display device is in standby mode. At this time, the user says "Hi ABC" to the display device, and the display device collects the user's voice "Hi ABC" through the sound collector and recognizes it. After the display device recognizes the wake-up word "Hi ABC", it switches from standby mode to running mode, turns off the indicator light 400, and turns on the off-screen display to wake up the display device; the off indicator light effect is used to prompt that the current display device has been turned on and the display device has entered the running state.
[0126] Obviously, the display device 200 has certain restrictions on energy consumption when in standby mode, that is, the display device 200 in standby mode cannot exceed a certain energy consumption. The operating energy consumption of the main chip of the display device 200 is relatively large, such as the SOC (system-on-chip), so the wake-up word recognition through the main chip in standby mode will cause the display device 200 to consume a lot of energy. When the user input user voice does not contain the wake-up word, it will cause a large waste of resources and reduce the user experience.
[0127] Therefore, in some embodiments, the display device 200 uses a low-power chip to perform wake-up word recognition in standby mode. For example, a low-power digital signal processing (DSP) or ARM CORTEX-M4 single-chip microcomputer is used. That is, when the display device 200 is in standby mode, the main chip is turned off, the low-power chip is turned on, and the user voice input by the user is parsed by the low-power chip, and when it is recognized that the user voice input by the user includes the wake-up word, the main chip is notified to turn on; when it is recognized that the user voice input by the user does not include the wake-up word, the user voice is released and the main chip remains closed. By performing the work of wake-up word recognition in standby mode through the low-power chip, the display device 200 in standby mode can maintain a low energy consumption range.
[0128] However, wake-up word recognition requires calculating the characteristics of the user's voice, and the energy consumption of the chip is strongly correlated with its computing power. That is, at the same process level, the greater the chip's energy consumption, the stronger its computing power and the more accurate the calculated results; the lower the chip's energy consumption, the weaker its computing power and the greater the error in the calculated results. Therefore, when using a low-energy chip for wake-up word recognition, the display device 200 is prone to false wake-up or unsuccessful wake-up, resulting in low accuracy in the display device's 200 wake-up recognition and a reduced user experience.
[0129] Based on the above application scenarios, in order to enhance the user experience and improve the low accuracy of wake-up recognition of the display device 200 in standby mode, a voice wake-up method for a display device in some embodiments of the present application is mainly used for standby voice wake-up. At least one processor 250 of the display device 200 executes computer instructions stored in the memory of the display device 200 to enable the display device 200 to perform the aforementioned voice wake-up process. The at least one processor 250 includes a first processor 251 and a second processor 252. Among them, the first processor 251 can be a low-power chip, such as a DSP chip, and the second processor 252 is a system chip, such as a SOC chip. As shown in Figure 8, the method includes the following contents:
[0130] In the standby state, the first processor 251 is turned on and the second processor 252 is turned off to reduce the power consumption of the display device 200 in the standby state. At this time, the first level signal is sent to the indicator light 400 through the first processor 251, so that the indicator light 400 presents the first prompt light effect of the standby state. The first processor 251 in the standby state is used to preliminarily analyze the user voice input by the user, and then determine whether the user voice is the wake-up word voice. Among them, the first level signal can be a low level signal or a high level signal. The low level signal is used to control the indicator light 400 to present a long-off light effect, and the high level is used to control the indicator light 400 to present a long-on light effect.
[0131] In some embodiments, the first processor 251 also responds to the user voice and performs a first wake-up word recognition on the user voice. The first wake-up word recognition is used to detect whether the user voice includes a wake-up word. When it is recognized that the user voice includes a wake-up word, the first wake-up word recognition passes, and the display device is controlled to enter a pre-start state; when it is recognized that the user voice does not include a wake-up word, the first wake-up word recognition fails, and the user voice is released to filter the user voice and no longer process it. That is, when the display device 200 is in standby mode, the first processor 251 is configured to execute computer instructions stored in the memory of the display device 200 to enable the display device 200 to perform including but not limited to the following program steps:
[0132] S100: In response to a user voice, performing a first wake-up word recognition on the user voice.
[0133] When the display device 200 is in standby mode, the first processor 251 performs the first wake-up word recognition in response to the user's voice input. At this time, the first processor 251 is turned on and the second processor 252 is turned off. The display device 200 is controlled by two processors. The first processor 251 uses a low-power chip for wake-up word recognition in standby mode.
[0134] In some embodiments, in the standby state of the display device, the life states of the first processor 251 and the second processor 252 are different. When the display device 200 is in the standby state, the first processor 251 is turned on and the second processor 252 is turned off. When the user voice input by the user is detected, the first processor 251 starts to process the user voice and performs the first wake-up word recognition work; at this time, the second processor 252 is still turned off to reduce the energy consumption of the display device 200 in the standby state. It should be noted that when the display device 200 is in the standby state, the first processor 251 controls the indicator light to present the prompt light effect, that is, the first level signal is sent to the indicator light 400 through the first processor 251 to control the indicator light 400 to present the first prompt light effect of the standby state.
[0135] To provide a unified user experience, the display device 200 can display various status prompt lighting effects to the user through the indicator light 400. In some embodiments, the first processor 251 responds to a high-level signal when the display device 200 is powered on to detect the power-on status of the second processor 252. If the second processor 252 is on, the second processor 252 sends a second-level signal to the indicator light 400 to control the indicator light 400 to display the third prompt lighting effect. If the second processor 252 is off, the first processor 251 sends a first-level signal to the indicator light 400 to control the indicator light 400 to display the first prompt lighting effect of the standby state.
[0136] When the display device 200 is powered on, the first processor 251 is used as a determiner and the master controller of the indicator light 400 is selected based on the life state of the second processor 252. When the life state of the second processor 252 is off, the second processor 252 does not serve as the master controller of the indicator light 300, and the first processor 251 serves as the master controller of the indicator light 300.
[0137] For example, the first processor 251 is a DSP, the second processor 252 is an SOC, and the indicator light 400 is an LED. As shown in Figure 9, the display device 200 determines the master controller of the indicator light through a handshake protocol (S91). The handshake protocol logic judgment is performed when the display device 200 is just powered on, with the DSP acting as the master determiner. It determines whether the DSP is the master controller (S92). If the SOC is not started, that is, the DSP is the master controller, the DSP controls the loading of the LED control mode (S93). At this time, the DSP sends a high-level signal to the LED, and the LED is controlled by the DSP. In response to the high-level signal, the LED displays a long-on warning light effect to remind the user that the display device 200 is in standby mode. If the SOC also starts normally, that is, the DSP is not the master, the LED control mode is loaded through the SOC (S94). At this time, the SOC sends a low-level signal to the LED, and the LED is controlled by the SOC, and the LED displays a long-off warning light effect.
[0138] In some embodiments, when the display device 200 is in standby mode: if the user does not input a user voice, the first processor 251 is turned on and the second processor 252 is turned off; the first processor 251 controls the indicator light 400 to present a long-on prompt light effect. If the user inputs a user voice in standby mode, the first processor 251 performs a first wake-up word recognition on the user voice, and the second processor 252 remains off; the first processor 251 can control the indicator light 400 to present a flashing prompt light effect to indicate that the display device 200 has responded based on the user voice. If the user voice does not pass the first wake-up word recognition, the second processor 252 remains off; the first processor 251 controls the indicator light 400 to present a long-on prompt light effect.
[0139] In some embodiments, when the display device 200 is in a pre-boot state, if the user's voice is recognized by the first wake-up word and the first processor 251 has completed the first wake-up word recognition, the second processor 252 is triggered to start. After at least a portion of the second processor 252 is started, the activated second processor 252 performs the second wake-up word recognition. At this time, the first processor 251 can control the indicator light 400 to continue to display the flashing prompt light effect.
[0140] In some embodiments, when the display device 200 is in operation and the user's voice is recognized by the second wake-up word, the first processor 251 is turned on but enters an idle state, no longer recognizing the first wake-up word for the user's voice. The second processor 252 is turned on and begins responding to user interactions. At this point, the second processor 252 also gains control of the indicator light 400, controlling the indicator light 400 to display a long-off prompt lighting effect.
[0141] In addition, since the first processor 251 performs the process of the first wake-up word recognition on the user voice, it is the wake-up word recognition process to determine whether the user voice includes the wake-up word. In some embodiments, the display device 200 also parses the acoustic features of the user voice through the first processor 251, and controls the indicator light 400 to present a detection voice prompt light effect, such as a flashing prompt equivalent. If the user voice includes the wake-up word feature, the first wake-up word recognition of the user voice is performed; if the user voice does not include the wake-up word feature, the first wake-up word recognition of the user voice is not performed, and the control indicator light 400 presents the first prompt light effect of the standby state.
[0142] In other words, the first processor 251 can extract and analyze the acoustic features in the user's voice to determine whether the user's voice includes the wake-up word. If it is determined that the user's voice includes the wake-up word, the first wake-up word recognition is performed; if it is determined that the user's voice does not include the wake-up word, the first wake-up word recognition is not performed.
[0143] In some embodiments, to let the user know that the display device 200 has received and is processing the user's voice, the display device 200 can control the indicator light 400 to flash through the first processor 251 when parsing the user's voice. In this way, the user can perceive that the display device 200 is currently processing the user's voice based on the flashing indicator light effect, thus alleviating the long waiting process of the display device 200.
[0144] For example: Assume that the display device is equipped with a voice interaction function, the wake-up word is "Hi ABC", the first processor of the display device is a DSP, and the second processor is an SOC. As shown in Figure 10, the display device is in standby mode, and the indicator light 400 is a long-on prompt light effect. The long-on prompt light effect indicates that the display device is in standby mode and the display of the display device is off. The user speaks "Hello" near the display device, and the DSP obtains the user voice input by the user and performs the first wake-up word recognition on the user voice. Assuming that the first wake-up word recognition requires a waiting time of 1s, the DSP controls the indicator light 400 to flash multiple times within 1s of the first wake-up word recognition to prompt the user that the current user voice is being checked. After performing the first wake-up word recognition, the DSP determines that the user voice input by the user does not contain a wake-up word, and the DSP controls the indicator light 400 to return to a long-on prompt light effect without performing other processing.
[0145] It is understandable that, because the first wake-up word recognition process lasts for a short time, in some embodiments, the display device 200 may not present the voice detection prompt light effect. That is, when performing the first wake-up word recognition, the first prompt light effect in the standby state is maintained, and the second prompt light effect is presented only in the pre-start state. If the display device presents the voice detection prompt light effect, then the voice detection prompt light effect is different from the first prompt light effect, the second prompt light effect, and the third prompt light effect.
[0146] In the pre-boot state, the display device 200 has the first processor 251 turned on and at least a portion of the second processor 252 turned on. The first processor 251 controls the indicator light 400 to display the second prompt light effect of the pre-boot state to remind the user that the display device 200 is in the pre-boot state.
[0147] The second processor 252 in the pre-start state is used to secondary analyze the user voice input by the user to verify / check the result of the first wake-up word recognition. Since the second processor 252 is the main chip of the display device 200 and has stronger computing power, the wake-up word recognition result of its second wake-up word recognition is also more accurate. The second wake-up word recognition can be based on the same principle as the first wake-up word recognition, both of which detect whether the user voice includes the wake-up word. When the user voice includes the wake-up word, the second wake-up word recognition passes, and the second processor 252 is initialized, all the second processors 252 are turned on, and the display device 200 is controlled to enter the running state; when the user voice does not include the wake-up word, the second wake-up word recognition fails, and the user voice is released to filter the user voice and no longer process it.
[0148] In some embodiments, when the second wake-up word recognition fails, the second processor 252 is turned off and the first processor 251 is turned on to return the display device 200 to the standby state.
[0149] In some embodiments, the principle of the second wake-up word recognition may be different from that of the first wake-up word recognition, and the specific wake-up word recognition content may be set according to the purpose of the display device 200.
[0150] S200: When the user voice is recognized by the first wake-up word, a wake-up instruction and the user voice are sent to the second processor to control the display device to enter a pre-start state.
[0151] When the display device 200 completes the first wake-up word recognition through the first processor 251 and the wake-up word recognition result of the first wake-up word recognition is passed, the first processor 251 sends a wake-up instruction and the complete user voice to the second processor. Due to the limited computing power of the first processor, it is necessary to send the complete user voice to the second processor 252 with stronger computing power for another wake-up word recognition to ensure the accuracy of the wake-up word recognition.
[0152] Therefore, the wake-up instruction is used to power on at least a portion of the second processor 252 so that the second processor 252 can perform wake-up word recognition. At the same time, since the process of the second processor 252 from being turned off to starting up and performing calculations still requires a certain amount of time, in order to alleviate the long-term unchanged waiting process of the display device 200 during this stage, the first processor sends a pulse signal to the indicator light 400 during this process to control the indicator light 400 to present a second prompt light effect, such as flashing, to prompt the user that the display device 200 is currently in the pre-startup state.
[0153] To enhance the user's interactive experience with the display device 200, in some embodiments, the prompt light effect for detecting voice is a flashing light effect, and the second prompt light effect in the pre-start state is also a flashing light effect, but the flashing frequency in the voice detection stage is less than the flashing frequency in the pre-start stage. In some embodiments, the flashing frequency of the indicator light 400 can be changed by adjusting the period of the pulse signal. For example, the flashing frequency in the voice detection stage is 3 times / second, and the flashing frequency in the pre-start stage is 6 times / second. By flashing lights at different frequencies, the user can perceive the changes in each stage of the display device 200, thereby enhancing the user's sense of interaction.
[0154] After the first processor 251 of the display device 200 sends a wake-up command and user voice to the second processor 252, at least a portion of the second processor 252 is powered on. The second processor 252 of the display device 200 is configured to execute computer instructions stored in the memory of the display device 200 to enable the display device 200 to perform program steps including but not limited to the following:
[0155] S300: In response to the wake-up instruction, perform second wake-up word recognition on the user voice.
[0156] After the first processor 251 sends a wake-up instruction to the second processor 252, at least a part of the shut-down second processor 252 responds to the wake-up instruction and is powered on. At the same time, the first processor 251 also sends all the acquired user voices to the second processor 252. Due to the limited computing power of the first processor 251, in order to reduce errors in wake-up recognition, the second processor 252 immediately performs a second wake-up word recognition on the user voice after it is turned on. The content of the second wake-up word recognition may be the same as or different from the content of the first wake-up word recognition. The second wake-up word recognition is a secondary verification of the user voice. When both the first wake-up word recognition and the second wake-up word recognition are passed, the user voice is determined to be the wake-up word voice, and the display device 200 in the standby state is woken up.
[0157] To enhance the user's sense of interaction, the pre-startup state prompt lighting effect can also be refined, and different lighting effects can be set at different stages. In some embodiments, during the startup process of the second processor 252, the first processor 251 controls the indicator light 400 to present a prompt lighting effect indicating that the second processor 252 is partially started; when the second processor 252 performs the second wake-up word recognition, the first processor 251 can also control the indicator light 400 to present a second prompt lighting effect, refining the pre-startup state prompt lighting effect to prompt the user that the current display device 200 is in the stage of secondary wake-up word recognition.
[0158] In order to improve the accuracy of wake-up recognition, in some embodiments, when the second processor 252 performs the second wake-up word recognition on the user voice, it also calculates the wake-up score of the user voice through the model of the second wake-up word recognition. Among them, the complexity of the model of the second wake-up word recognition is greater than that of the first wake-up word recognition. Since the energy consumption of the first processor 251 is relatively small, only a simplified and optimized calculation model can be used, and the computing power is greatly reduced. The power consumption of the second processor 252 is much greater than that of the first processor 251, so a more complete and complex calculation model can be used to ensure the accuracy of the second processor 252. If the wake-up score is greater than or equal to the score threshold, the second wake-up word recognition of the user voice is passed; if the wake-up score is less than the score threshold, the second wake-up word recognition of the user voice is not passed.
[0159] In some embodiments, the wake-up score is calculated based on a registered wake-up word template. Specifically, if a wake-up word template is provided in the display device 200, and the second processor 252 calculates the wake-up score for the user's voice using the second wake-up word recognition model, it can score the user's voice by calculating the similarity between the user's voice and the registered wake-up word template. A higher similarity results in a higher score, while a lower similarity results in a lower score.
[0160] It should be noted that the first wake-up word recognition can also be performed using the above-mentioned wake-up score method, but the calculation model used for the first wake-up word recognition is different from that used for the second wake-up word recognition.
[0161] S400: When the user's voice is recognized by the second wake-up word, the display is turned on and the display device is controlled to enter an operating state.
[0162] When the display device 200 completes the second wake-up word recognition through the second processor 252 and the recognition result of the second wake-up word recognition is passed, the second processor 252 lights up the display 260 and controls the display device 200 to enter the running state. That is, when the user inputs the user voice containing only the wake-up word in the standby state, the display device 200 switches from the standby state to the running state to facilitate the user to perform subsequent interaction with the display device 200.
[0163] In this operating state, the first processor 251 is turned on, and the second processor 252 is turned on. The second processor 252 sends a second level signal to the indicator light 400, so that the indicator light 400 presents the third prompt light effect of the operating state. The second level signal can be a high level signal or a low level signal.
[0164] For example: As shown in Figure 11, it is still assumed that the display device is configured with a voice interaction function, and the wake-up word is "Hi ABC". After the user speaks the user voice of "Hi ABC" to the display device, the first processor of the display device performs a first wake-up word recognition on the user voice of "Hi ABC". After the user voice of "Hi ABC" passes the first wake-up word recognition, the first processor 251 sends a wake-up instruction to the second processor 252, and sends the complete user voice of "Hi ABC" to the second processor 252 of the display device 200, and sends a pulse signal to the indicator light to present the second prompt light effect. After receiving the user voice of "Hi ABC", the second processor 252 is powered on and performs a second wake-up word recognition on the user voice after it is turned on. After the user voice of "Hi ABC" passes the second wake-up word recognition, the second processor 252 lights up the display of the display device, sends a second level signal to the indicator light to control the indicator light to present the third prompt light effect, and enters the power-on procedure of the display device.
[0165] Based on the above embodiment, as shown in Figure 12, in the standby state, after the first processor 251 receives the user voice containing the wake-up word, it performs the first wake-up word recognition. After the first wake-up word recognition is passed, the second processor 252 is powered on to start at least a part of the second processor 252. The second processor 252 performs the second wake-up word recognition through the startup part, and after the second wake-up recognition is passed, the second processor 252 is fully turned on to enable the display device 200 to enter the running state. As can be seen from Figure 12, in the running state, the first processor 251 no longer performs the wake-up word recognition work and enters the idle mode. In the above process, the on and off states of the indicator light 400 and the display 260 are shown in Figure 12. Among them, in the on and off state of the indicator light 3400, the lighting effect presented by the solid line part is controlled by the first processor 251, and the lighting effect presented by the dotted line part is controlled by the second processor 252.
[0166] In addition, since the second wake-up word recognition also takes a certain amount of time, when the user's voice includes an obvious wake-up intention, the two wake-up word recognition processes will cause the display device 200 to be unable to respond quickly to the user's wake-up operation, which will cause the user to wait for the display device 200 to wake up for a long time, affecting the user's voice interaction experience.
[0167] Therefore, in some embodiments, when the first processor 251 performs the first wake-up word recognition on the user's voice, it also calculates the confidence level of the user's voice. When the confidence level is greater than or equal to a preset confidence threshold, it sends a recognition-free wake-up instruction to the second processor 252 and controls the indicator light 400 to display a quick start prompt light effect.
[0168] The recognition-free wake-up instruction is used to notify the second processor 252 to skip the second wake-up word recognition. When the confidence level of the user's voice is greater than or equal to a preset confidence threshold, i.e., when the confidence level is high, it indicates that the current user's voice contains a strong wake-up intention. In this case, the display device 200 needs to quickly respond to the user's wake-up operation, so the first processor 251 can send the recognition-free wake-up instruction to the second processor. On the one hand, it can wake up the second processor 252 to complete the display device 200 wake-up procedure; on the other hand, it can skip the second wake-up word recognition procedure, speeding up the display device 200 wake-up process. During this process, the first processor 251 can also adjust the pulse signal period so that the indicator light 400 displays a quick startup prompt light effect to notify the user that the display device 200 is currently in the wake-up phase. At the same time, the display device 200 is controlled to enter the operating state. The prompt light effect during the quick startup phase can be a very fast flashing light effect. This fast flashing light effect can provide users with timely voice interaction feedback, improving the user interaction experience.
[0169] Similarly, upon receiving the aforementioned recognition-free wake-up instruction, the second processor 252 of the display device 200 immediately responds to the user's voice instruction, thereby accelerating the response speed. That is, in some embodiments, the second processor 252 of the display device 200 further generates a voice instruction based on the user's voice and responds to the voice instruction in response to the recognition-free wake-up instruction.
[0170] To facilitate user interaction with the display device 300, in some embodiments, the user voice that wakes up the display device 200 is not limited to data including wake-up word features, but may also include other instructions, that is, the user voice includes at least a complete wake-up word feature.
[0171] For example: the wake-up word of the display device 200 is "Hi ABC". If the user wants to ask the display device 200 in standby mode about the time, he / she should say "Hi ABC, what day is today?" to the display device 200; after the user's voice is verified twice by the display device 200, the display device 200 can wake up the display device 200 according to the wake-up word based on the user's voice, and respond to the voice command "What day is today?" after waking up.
[0172] In some embodiments, the second processor 252 also monitors the pulse signal sent by the first processor 251; when the last pulse wave of the pulse signal is monitored, a second level signal is sent to the indicator light 400 to control the indicator light 400 to present a third prompt light effect of the operating status.
[0173] In some embodiments, after the user's voice is recognized by the second wake-up word, the display device 200 is awakened and enters the operating state. Before awakening, the display device 200 controls the indicator light 400 through the first processor 251. After awakening, the display device 200 needs to respond to subsequent operations through the core controller, that is, the second processor 252. Therefore, the second processor 252 can send a second level signal to the indicator light 400 to control the indicator light to present the third prompt lighting effect.
[0174] In other words, the control right of the indicator light 400 is determined through negotiation based on the life status of the first processor 251 and the second processor 252 during their life cycle. When the second processor 252 is turned on and the user's voice is recognized through the second wake-up word, the second processor 252 obtains the control right of the indicator light 400; when the second processor 252 is turned off, the first processor 251 obtains the control right of the indicator light 400.
[0175] To ensure consistency in user experience, in some embodiments, after obtaining control of the indicator light 400, the second processor 252 may also forward control instructions for the indicator light 400 through the first processor 251. The control instructions for the indicator light 400 include control parameter data.
[0176] For example, if the display device is in speaker mode, the first processor 251 is a DSP, the second processor 252 is a SOC, and the indicator light 400 is an LED. The SOC controls the LED by sending control instructions via the DSP. In speaker mode, if the display device is off and the SOC is active, the SOC transmits data to the DSP to control the LED, thereby ensuring a consistent user experience.
[0177] In addition, in the running state, the display device 200 can also receive the standby command input by the user after being awakened and return to the standby state. Among them, the standby command can be input by the user through the control device 100 of the display device 200, such as a remote control that is matched with the display device; the standby command can also be input by the user through voice commands, etc. After the display device 200 is awakened and enters the running state, the second processor 252 is turned on to work, and the first processor 251 is turned on but enters the idle state. Then, when the display device 200 switches from the running state to the standby state again, the second processor 252 needs to be turned off; and the first processor 251 needs to exit the idle state to perform the first wake-up word recognition work of the user's voice.
[0178] Therefore, in some embodiments, in the running state, the second processor 252 also receives a standby instruction input by the user. The standby instruction is used to control the display device 200 to enter the standby state. Then, in response to the standby instruction, the number of chip operating cores of the second processor 252 is reduced to shut down the second processor. That is, since the energy consumption of the display device 200 in the standby state is limited to a certain extent, it is necessary to reduce the number of chip operating cores of the second processor 252. The standby instruction is then forwarded to the first processor 251. After receiving the standby instruction and the control right instruction, the first processor 251 returns to the standby state and sends a first level signal to the indicator light 400 to control the indicator light 400 to present the first prompt light effect of the standby state.
[0179] In some embodiments, after the second processor 252 of the display device 200 successfully wakes up and obtains control of the indicator light 400, it can control the indicator light 400 to present a third prompt lighting effect of the operating state. For example, because the display screen is already lit during the operating phase to interact with the user, the prompt effect of the indicator light 400 is weakened at this time, and the prompt lighting effect of the display device during the operating phase can be a long off lighting effect.
[0180] It is understood that the above-mentioned different stages of the prompt lighting effects can also be distinguished by light color. For example, the light color of the display device 200 in the standby state is red, and the light color of the display device 200 in the working state can be blue. The prompt lighting effects in different stages can present different visual effects. This application does not impose any restrictions on this.
[0181] In some embodiments, when controlling the indicator light 400 to display the operating phase prompt lighting effect, the second processor 252 also detects lighting effect configuration information. The lighting effect configuration information is the configuration parameters stored in the display device 200. The lighting effect configuration information is then used to analyze the on / off state of the indicator light 400. If the indicator light 400 is on, at least one output port of the second processor 252 outputs a high level. If the indicator light 400 is off, at least one output port of the second processor 252 outputs a low level. In other words, when any output port of the second processor 252 is high, the indicator light 400 is on.
[0182] It is understandable that the first processor 251 and the second processor 252 may control the indicator lights 400 using the same control logic, and the first processor 251 and the second processor 252 may control the same indicator light 400 or the same group of indicator lights 400 .
[0183] In some embodiments, the first processor 251 and the second processor 252 control the same indicator light 400. When the output port of the first processor 251 and / or the second processor 252 is high, the indicator light 400 illuminates. If both the first processor 251 and the second processor 252 are within their lifecycles and both have control capabilities, the first processor 251 and the second processor 252 negotiate to determine who has control of the indicator light 400.
[0184] For example, assume that the first processor 251 of display device 200 is a DSP, the second processor 252 is a SOC, and the indicator light 400 is an LED. Figure 13 shows a circuit diagram for dual-controller control of the indicator lights. The DSP port represents the output IO port from a DSP chip, such as the TH1520 chip; the SOC port represents the output IO port from an SOC, such as the MT9652 chip. In the circuit shown in Figure 13, the transistor design ensures that when either the DSP or SOC IO port is high, the LED VD2 will illuminate. That is, when the SOC is low and the DSP is high, the transistor connected to the DSP will turn on, illuminating the LED. When the SOC is low and the DSP is low, both transistors will turn off, and the LED will not illuminate. Similarly, if either the SOC IO port is high, the corresponding transistor will turn on, illuminating the LED.
[0185] In addition, after the display device 200 performs the second wake-up word recognition through the second processor 252, if the second wake-up word recognition result is failed, the display device 200 needs to remain in the standby state. In some embodiments, if the user voice fails the second wake-up word recognition, the second processor 252 sends a recognition failure result to the first processor 251 to notify the first processor 251 of the final recognition result.
[0186] In some embodiments, when the user's voice fails to pass the second wake-up word recognition, the second processor 252 sends a recognition failure result of the second wake-up word recognition to the first processor 251 and turns off the enabled portion of the second processor 252. At this time, the second processor 252 returns to the off state and then controls the display device 200 to enter the standby state.
[0187] At the same time, the first processor 251 receives a recognition failure result of the second wake-up word recognition, and in response to the recognition failure result, sends a first level signal to the indicator light 400 to make the indicator light 400 present a first prompt light effect of the standby state.
[0188] That is, if the second wake-up word recognition fails, it means that the first processor 251 has misidentified the wake-up word when performing the wake-up word recognition. The second processor 252 needs to send the recognition failure result to the first processor 251, power off and shut down, and return to the off state. At the same time, after receiving the notification of the recognition failure result, the first processor 251 continues to maintain the standby state and performs the first wake-up word recognition of the user's voice. In this way, the display device 200 can not only ensure the low energy consumption requirement in the standby state, but also improve the accuracy of wake-up recognition and enhance the user experience.
[0189] In some embodiments, as shown in Figure 14, in the standby state, after the first processor 251 receives the user voice suspected of containing the wake-up word, it performs the first wake-up word recognition. After the first wake-up word recognition is passed, the second processor 252 is powered on to start at least a part of the second processor 252, and then the second wake-up word recognition is performed by the second processor 252 of the startup part. After the second wake-up word recognition fails, the second processor 252 is completely turned off. At this time, the display device 200 returns to the standby state. In the above process, the on and off states of the indicator light 400 and the display 260 are shown in Figure 13. Among them, in the on and off state of the indicator light 400, the lighting effects presented are all controlled and presented by the first processor 251.
[0190] Based on the above method, some embodiments of the present application include a display device 200, as shown in FIG15 , comprising: an indicator light 400, a sound collector 231, a display 260, a first processor 251, and a second processor 252. The indicator light 400 is configured to display a prompt light effect; the sound collector 231 is configured to collect user voice; and the terminal states of the display device 200 include a standby state, a pre-startup state, and an operating state.
[0191] As shown in FIG16 , in the standby state, the first processor 251 is turned on and the second processor 252 is turned off; the first processor 251 sends a first level signal to the indicator light 400 so that the indicator light 400 presents the first prompt light effect of the standby state.
[0192] In the pre-start state, the first processor 251 is turned on and at least a portion of the second processor 252 is turned on; a pulse signal is sent to the indicator light 400 through the first processor 251 to make the indicator light 400 present the second prompt light effect in the pre-start state.
[0193] In the running state, the first processor 251 is turned on, and the second processor 252 is turned on; the second level signal is sent to the indicator light 400 through the second processor 252, so that the indicator light 400 presents the third prompt light effect of the running state.
[0194] Among them, in the standby state, the first processor 251 responds to the user voice and performs a first wake-up word recognition on the user voice; when the user voice passes the first wake-up word recognition, a wake-up instruction and the user voice are sent to the first processor 251 to control the display device 200 to enter the pre-start state; the wake-up instruction is used to power on at least a part of the second processor 252;
[0195] The second processor 252 performs a second wake-up word recognition on the user voice in response to the wake-up instruction; when the user voice is recognized by the second wake-up word, the display 260 is turned on, and the display device 200 is controlled to enter an operating state.
[0196] In some embodiments, referring to FIG16 , the timing relationship of the terminal state transition may include: the display device 200 sends a first level signal to the indicator light 400 through the first processor 251 in the standby state (S161), so that the indicator light 400 presents the first prompt equivalent of the standby state (S162); after the user voice is collected by the sound collector 231, the user voice is sent to the first processor 251 (S163), and the first processor 251 performs a first wake-up word recognition on the user voice (S164). When the first wake-up word recognition passes, that is, when the first verification passes, the user voice is sent to the second processor 252 (S165-1), and the indicator light 400 is sent to the first processor 252 (S166-1). 0 sends a pulse signal (S165-2) to make the indicator light 400 present a second prompt light effect of the pre-start state (S166); perform a second wake-up word recognition on the user voice through the second processor 252 (S167), and when the second wake-up word recognition is passed, send a second level signal (S168) to the indicator light 400 to make the indicator light 400 present a third prompt light effect of the running state (S169), and light up the display 260 (S1610); when the second wake-up word recognition fails, that is, when the second verification fails, send a first electric frequency signal (S1611) to the indicator light 400 through the first processor 251 to make the indicator light 400 present the first prompt equivalent of the standby state.
[0197] In some embodiments, as shown in Figure 8, the first processor 251 is configured to execute computer instructions stored in the memory of the display device 200 so that the display device 200 executes including but not limited to the aforementioned program steps S100 to S200; the second processor 252 is configured to execute computer instructions stored in the memory of the display device 200 so that the display device 200 executes including but not limited to the aforementioned program steps S300 to S400. In this way, the user voice is recognized by the dual controllers, the accuracy of wake-up word recognition is improved, and different prompt light effects are presented in different terminal states, thereby improving the user experience.
[0198] In some embodiments, the display device 200 may indicate the wake-up process of the display device 200 to the user through the prompt lighting effect of the indicator light 400 . In addition, after the display device 200 is awakened, a wake-up prompt sound may be played to further enhance the user experience.
[0199] Referring to FIG17 , in some embodiments, the operating system of the display device 200 can be divided into four layers, namely, from top to bottom, the application layer (hereinafter referred to as the “application layer”), the application framework layer (hereinafter referred to as the “framework layer”), the Android runtime layer (hereinafter referred to as the “Android runtime layer”), and the kernel layer. In some embodiments, at least one application is running in the application layer. These applications can be window programs, system settings programs, or clock programs that come with the operating system, such as live broadcast applications, which are used to play live programs; or they can be applications developed by third-party developers. Compared with the architecture shown in FIG4A , the architecture shown in FIG17 subdivides the hardware layer 23 into a framework layer and a system runtime layer.
[0200] Based on the operating system shown in Figure 17, when the display device 200 plays audio, the audio management module (Audio Manager) in the framework layer determines the audio stream (Stream) type, decodes the audio pulse modulation coding (Pulse-Code Modulation, PCM) data, and passes the decoded audio to the audio track object created in the system runtime layer through the audio track (Audio Track) interface corresponding to the audio stream type, and processes and outputs the audio through the audio track object. When the audio track object processes the audio, the audio hardware device that outputs the audio is determined by the audio policy maker (Audio Policy Service) object, and the processed audio is transmitted to the audio hardware device through the audio policy executor (Audio Flinger) object, and the audio is input through the audio hardware device. The audio playback path is composed of the audio management module, the audio track object and the audio hardware device. It can be seen from the above process of playing audio that the audio playback path corresponds to the audio stream type of the audio.
[0201] The audio played by the display device 200 can be divided into (Voice Call), music (Music), ringtone (Ring), alarm (Alarm), Bluetooth physical link (Bluetooth Synchronous Connection Oriented), etc. according to the scene. In some embodiments, the display device 200 is configured with an audio stream type, such as the music type (Stream_Music). Therefore, the audio management module will uniformly determine the audio of different scenes as the Stream_Music type, and call the audio track object to process and output the audio through the audio track interface corresponding to the Stream_Music type. In other words, the audio of different scenes is played through the same playback path.
[0202] The display device 200 adjusts the volume through the audio management module, where the audio management module adjusts the volume of the corresponding playback channel based on the audio stream type. For example, the audio management module adjusts the volume of the playback channel corresponding to the audio stream type by calling the adjustStreamVolume method. When the display device 200 is configured with a single audio stream type, the display device 200 adjusts the volume of the playback channel corresponding to that audio stream type through the audio management module.
[0203] For example, if one of the audio stream types configured on the display device is Stream_Music, the audio management module raises the volume by calling adjustStreamVolume(AudioManager.STREAM_MUSIC, AudioManager.ADJUST_RAISE, AudioManager.FLAG), or lowers the volume by calling adjustStreamVolume(AudioManager.STREAM_MUSIC, AudioManager.ADJUST_DECREASE, AudioManager.FLAG). Since audio from different scenes is played through this playback path, the display device 200 does not adjust the volume of audio from different scenes separately, but adjusts the volume of audio from different scenes uniformly. At this time, the volume adjusted by the display device 200 is equivalent to adjusting the system volume.
[0204] The audio played by the display device 200 includes a wake-up prompt tone. The display device 200 is awakened after recognizing the user input of the wake-up word and plays the wake-up prompt tone to prompt the user to wake up. Based on the above correspondence between audio and playback path, and the correspondence between playback path and volume, it can be seen that the display device 200 plays the wake-up prompt tone according to the system volume. If the system volume is high, the volume of the wake-up prompt tone is also high, and if the system volume is low, the volume of the wake-up prompt tone is also low.
[0205] When the user wakes up the display device 200 by far-field voice, the display device 200 will be awakened in response to the wake-up word spoken by the user. The response method of the display device 200 after being awakened and the volume of the wake-up prompt sound will affect the user's experience of using the display device 200. In different scenarios, such as the noisiness of the environment in which the display device 200 is located, the working state of the display device 200, etc., the response method of the display device 200 after being awakened and the appropriate volume of the wake-up prompt sound are different. The display device 200 needs to adjust the response method after being awakened and the volume of the wake-up prompt sound according to different scenarios. Combined with the following embodiments, the response method of the display device 200 after being awakened and the volume of the wake-up prompt sound are explained.
[0206] After the display device 200 is awakened, the audio of the wake-up prompt sound is played. The effect of the wake-up prompt sound played by the display device 200 heard by the user will be affected by the noisiness of the environment in which the display device 200 is located and the current system volume of the display device 200. For example, when the environment in which the display device 200 is located is relatively noisy, if the current system volume of the display device 200 is relatively low, the volume of the wake-up prompt sound played by the display device 200 is also relatively low. At this time, the user is not easy to perceive the wake-up prompt sound, causing the user to mistakenly believe that the display device 200 is not awakened. When the environment in which the display device 200 is located is relatively quiet, if the current system volume of the display device 200 is relatively high, the volume of the wake-up prompt sound played by the display device 200 is also relatively high. At this time, the wake-up prompt sound with a relatively high volume is relatively abrupt in the environment and is likely to disturb the user.
[0207] In order to solve the above problems, the display device 200 of some embodiments of the present application may include: a sound collector, configured to: collect user voice; a speaker; a memory, configured to: store computer instructions and data associated with the display device; at least one processor 250, connected to the sound collector, the speaker and the memory, configured to execute the computer instructions so that the display device can follow the voice wake-up process for the display device shown in Figure 18 to adjust the volume of the wake-up prompt sound. It can be understood that in some embodiments, if at least one processor 250 includes a first processor 251 and a second processor 252, the processes in subsequent embodiments can be executed by the second processor 252; in other embodiments, they can also be executed collaboratively by the first processor 251 and the second processor 252, which is not specifically limited in this application. Referring to Figure 18, it may include:
[0208] S501: Collect user voice.
[0209] After the far-field voice function is activated, the display device 200 collects the user's voice through the sound collector.
[0210] S502: When it is recognized that the user voice contains the wake-up word, the signal-to-noise ratio of the user voice and the current system volume are obtained.
[0211] The display device 200 may identify whether the user's voice contains a wake-up word according to the process shown in FIG19 , which may include:
[0212] S601: Calculate a wake-up value corresponding to the user's voice.
[0213] The wake-up value is used to represent the degree of similarity between the user's voice and the standard voice of the wake-up word. The wake-up value can be a value between 0 and 1. The higher the wake-up value of the user's voice, the more similar the user's voice is to the standard voice of the wake-up word, and the higher the probability that the display device 200 will be awakened. The lower the wake-up value of the user's voice, the less similar the user's voice is to the standard voice of the wake-up word, and the lower the probability that the display device 200 will be awakened.
[0214] The display device 200 can calculate the wake-up value of the user's voice through a model. The model can be a neural network model, such as a model based on a convolutional neural network (CNN), a model based on a recurrent neural network (RNN), or a model based on a deep neural network (DNN). The model can be obtained in advance through standard voice training of user voice samples and wake-up words, and the model can be iteratively trained based on the accuracy of the user voice and wake-up results actually collected by the display device 200 to improve the accuracy of the model and its fit to the user.
[0215] The display device 200 may calculate the wake-up value of the user's voice using the model according to the process shown in FIG20 , which may include:
[0216] S701: Input the user's voice into the model to obtain the score of each syllable in the user's voice.
[0217] S702: Calculate the wake-up value corresponding to the user's voice based on the score of each syllable in the user's voice using the model.
[0218] Taking the model shown in FIG. 21 and using the wake-up word "Hisense Xiaoju" as an example, steps S601-S602 will be described by way of example. The display device 200 inputs the user voice to the model in real time, and the model sequentially recognizes each syllable according to the received user voice. Among them, when the model recognizes each syllable, a scoring network based on the phoneme transition probability calculates the score of this syllable. The score of each syllable is used to represent the similarity probability value, that is, the degree of similarity, with the syllable at the corresponding position in the standard voice of the wake-up word. For example, when the model recognizes that the first syllable in the user voice is "Hai", and the syllable ranked first in the standard voice of the wake-up word is "Hai", the model calculates the similarity probability value p00 between "Hai" in the user voice and "Hai" in the standard voice of the wake-up word, and obtains the score corresponding to "Hai" in the user voice, that is, p00.
[0219] The process of the model calculating the scores of "Xin", "Xiao", and "Ju" in the user voice is similar to the process of calculating the score of "Hai", and will not be elaborated here. Let the scores of "Xin", "Xiao", and "Ju" be p11, p22, and p33 respectively. The model calculates the wake-up value corresponding to the user voice according to the scores of each syllable in the user voice. For example, the model can calculate the forward and backward transition probability values between each syllable in the user voice and other syllables, and perform weighted calculation according to the score of each syllable and the forward and backward transition probability values between each syllable, to obtain a weighted value, and use this weighted value as the wake-up value of the user voice.
[0220] Taking the model shown in FIG. 22 and using the wake-up word "Hisense Xiaoju" as an example, steps S601-S602 will be described by way of example. This model includes an input layer, a hidden layer, and an output layer. The display device 200 inputs the user voice to the input layer of the model in real time, and the model divides the user voice into n voice segments in the input layer, such as voice segments x1-xn. Based on different training methods and training degrees of the model, the voice segments divided by the model for the user voice in the input layer are also different. For example, the input layer can divide the user voice into 4 voice segments according to syllables, where x1 corresponds to "Hai", x2 corresponds to "Xin", x3 corresponds to "Xiao", and x4 corresponds to "Ju". The model calculates the probability values of mapping the voice segments x1-xn to different targets based on a classification decoding method in the hidden layer. The targets of the hidden layer include "Hisense Xiaoju" and other voices, that is, through the hidden layer, the probability value P1 that the user voice is Hisense Xiaoju and the probability value P2 that the user voice is other voices can be obtained. The hidden layer transmits the calculated probability values to the output layer for output. Among them, the probability value P1 that the user voice is Hisense Xiaoju can be used as the wake-up value of the user voice.
[0221] The display device 200 can also use other models to calculate the wake-up value of the user voice, which is not limited here.
[0222] S602: Determine whether the user voice contains a wake-up word according to the wake-up value and the wake-up value threshold.
[0223] The wake-up value threshold is the minimum wake-up value for user voice to wake up display device 200. When the wake-up value of the user voice is greater than or equal to the wake-up value threshold, it is determined that the user voice contains the wake-up word, the display device 200 is awakened, and the wake-up prompt audio is played. When the wake-up value of the user voice is less than the wake-up value threshold, it is determined that the user voice does not contain the wake-up word, and the display device is not awakened.
[0224] When the wake-up value of the user voice is greater than or equal to the wake-up value threshold, that is, after the display device 200 determines to play the audio of the wake-up prompt sound, the display device 200 obtains the signal-to-noise ratio of the user voice and the current system volume to determine the volume of the wake-up prompt sound to be played based on the signal-to-noise ratio of the user voice and the current system volume.
[0225] The signal-to-noise ratio of the user's voice is used to indicate the noisiness of the environment in which the display device 200 is located. A higher signal-to-noise ratio indicates a lower noisiness level in the environment in which the display device 200 is located, i.e., a quieter environment in which the display device 200 is located. A lower signal-to-noise ratio indicates a higher noisiness level in the environment in which the display device 200 is located, i.e., a noisier environment in which the display device 200 is located.
[0226] The user voice acquired by the display device 200 includes a first voice and a second voice. The first voice is located before the second voice. The durations of the first voice and the second voice are equal. The first voice is the ambient sound that does not contain the wake-up word, and the second voice is the voice corresponding to the wake-up word. The display device 200 obtains the signal-to-noise ratio of the user voice by calculating the ratio of the volume or energy of the second voice and the first voice. The higher the ratio, the higher the signal-to-noise ratio, and the lower the ratio, the lower the signal-to-noise ratio. The display device 200 can obtain the signal-to-noise ratio of the user voice through a model. The model can belong to the same model as the model for calculating the wake-up value, or can belong to a different model from the model for calculating the wake-up value.
[0227] The display device 200 can obtain the current system volume through a system standard interface, for example, intcurrentVolume=mAudioManager.getStreamVolume(AudioManager.STREAM_MUSIC).
[0228] S503: Adjust the amplitude of the wake-up prompt tone audio according to the signal-to-noise ratio and the current system volume to obtain the to-be-played audio of the wake-up prompt tone.
[0229] The volume of the wake-up prompt tone is adjusted in combination with the noise level of the environment in which the display device 200 is located and the current system volume of the display device 200. Based on the above analysis of the playback channel of the audio played by the display device 200, in order not to affect the volume of other audio other than the audio of the wake-up prompt tone, the display device 200 does not adjust the volume of the playback channel (i.e., the system volume), but adjusts the amplitude of the audio of the wake-up prompt tone to achieve the effect of adjusting the volume of the wake-up prompt tone separately. When the system volume is the same, the smaller the amplitude of the audio of the wake-up prompt tone, the smaller the volume of the wake-up prompt tone output by the display device 200, and the larger the amplitude of the audio of the wake-up prompt tone, the larger the volume of the wake-up prompt tone output by the display device 200.
[0230] The display device 200 may adjust the amplitude of the wake-up prompt tone according to the process shown in FIG23 , which may include:
[0231] S1001: Obtain the audio of the wake-up prompt tone.
[0232] The display device 200 may pre-store a copy of the wake-up prompt tone audio locally, and the display device 200 may obtain the wake-up prompt tone audio from the local storage. The display device 200 may not pre-store the wake-up prompt tone audio, but obtain the wake-up prompt tone audio from the server 300.
[0233] S1002: Adjust the amplitude of the wake-up prompt tone audio according to the signal-to-noise ratio and the current system volume to obtain the audio to be played of the wake-up prompt tone.
[0234] The display device 200 dynamically adjusts the amplitude of the wake-up prompt tone audio according to the signal-to-noise ratio and the current system volume to obtain the wake-up prompt tone audio to be played. For example, the amplitude of the wake-up prompt tone audio is adjusted by adjusting the gain of the wake-up prompt tone audio.
[0235] Based on the above method of adjusting the amplitude of the wake-up prompt tone audio itself, it is convenient to set a finer amplitude adjustment granularity, so that the volume of the wake-up prompt tone can be more accurately tailored to the current environment.
[0236] The display device 200 may adjust the amplitude of the wake-up prompt tone according to the process shown in FIG24 , which may include:
[0237] S1101: Determine the amplitude of the audio to be played of the wake-up prompt tone according to the signal-to-noise ratio and the current system volume.
[0238] The display device 200 may pre-store multiple preset audios for the wake-up prompt sound locally, each preset audio corresponding to an amplitude, and the display device 200 may obtain pre-stored audios of the wake-up prompt sound with different amplitudes from the local storage. Instead of pre-stored audios for the wake-up prompt sound, the display device 200 may arrange multiple preset audios for the wake-up prompt sound on the server 300, each preset audio corresponding to an amplitude, and the display device 200 may obtain pre-stored audios of the wake-up prompt sound with different amplitudes from the server 300.
[0239] Different preset audios of the wake-up prompt tone can be distinguished according to the specific amplitude values, and the file name of the preset audio of the wake-up prompt tone can include the specific amplitude value, such as "wake-up prompt tone -3dB". Therefore, the display device 200 can identify the corresponding amplitude value based on the file name of the preset audio. The display device 200 can determine the specific amplitude value of the amplitude of the audio to be played of the wake-up prompt tone to be played based on the signal-to-noise ratio and the current system volume. For example, the signal-to-noise ratio and the current system volume have a corresponding relationship with the specific amplitude value of the audio to be played of the wake-up prompt tone, and the display device 200 can determine the specific amplitude value based on the corresponding relationship.
[0240] Different preset audios of the wake-up prompt tone can be distinguished according to the size of the amplitude, and the file name of the preset audio of the wake-up prompt tone may include the size of the amplitude, such as "wake-up prompt tone-small amplitude". Therefore, the display device 200 can identify the corresponding size of the amplitude based on the file name of the preset audio. The display device 200 can determine the size of the amplitude of the audio to be played of the wake-up prompt tone to be played based on the signal-to-noise ratio and the current system volume. For example, the signal-to-noise ratio and the current system volume have a corresponding relationship with the size of the amplitude of the audio to be played of the wake-up prompt tone, and the display device 200 can determine the size of the amplitude based on the corresponding relationship.
[0241] S1102: Obtain a preset audio of the wake-up prompt tone with a corresponding amplitude according to the amplitude of the audio to be played of the wake-up prompt tone.
[0242] After determining the amplitude of the to-be-played audio of the wake-up prompt tone, the display device 200 may obtain a preset audio of the wake-up prompt tone having a corresponding amplitude.
[0243] Based on the above method of directly obtaining the pre-stored audio based on the amplitude of the audio to be played for the wake-up prompt tone and adjusting the amplitude of the audio of the wake-up prompt tone, the audio to be played for the wake-up prompt tone can be obtained more quickly, thereby improving the efficiency of playing the wake-up prompt tone.
[0244] 25 , the process of the display device 200 adjusting the amplitude of the audio of the wake-up prompt tone according to the signal-to-noise ratio and the current system volume is described.
[0245] The display device 200 sets a first signal-to-noise ratio threshold and a second signal-to-noise ratio threshold, as well as a first volume threshold and a second volume threshold, wherein the first signal-to-noise ratio threshold is greater than the second signal-to-noise ratio threshold, and the first volume threshold is greater than the second volume threshold.
[0246] The display device 200 executes step S1201 to determine whether the signal-to-noise ratio of the user's voice is greater than or equal to a first signal-to-noise ratio threshold.
[0247] When the signal-to-noise ratio of the user voice is greater than or equal to the first signal-to-noise ratio threshold, the display device 200 performs step S1202 to determine whether the current system volume is greater than or equal to the first volume threshold.
[0248] When the signal-to-noise ratio of the user voice is greater than or equal to the first signal-to-noise ratio threshold and the current system volume is greater than or equal to the first volume threshold, the display device 200 executes step S1203 to reduce the amplitude of the audio of the wake-up prompt tone to obtain the audio to be played of the wake-up prompt tone.
[0249] The signal-to-noise ratio of the user's voice is greater than or equal to the first signal-to-noise ratio threshold, indicating that the environment in which the display device 200 is located is less noisy, that is, the environment in which the display device 200 is located is relatively quiet. The current system volume is greater than or equal to the first volume threshold, indicating that if the display device 200 uses the current system volume to directly play the audio of the wake-up prompt sound, the volume of the wake-up prompt sound will be relatively large. At this time, the display device 200 plays the wake-up prompt sound at a relatively high volume in a relatively quiet environment, which is likely to cause a strong sense of intrusion to the user. In order to solve this problem, the amplitude of the audio of the wake-up prompt sound is reduced to lower the volume of the wake-up prompt sound, thereby reducing the disturbance to the user.
[0250] When the signal-to-noise ratio of the user voice is greater than or equal to the first signal-to-noise ratio threshold and the current system volume is less than the first volume threshold, the display device 200 performs step S1204 and does not adjust the amplitude of the audio of the wake-up prompt tone.
[0251] The signal-to-noise ratio of the user's voice is greater than or equal to the first signal-to-noise ratio threshold, indicating that the environment in which the display device 200 is located is relatively quiet. The current system volume is less than the first volume threshold, indicating that if the display device 200 directly plays the audio of the wake-up prompt tone using the current system volume, the volume of the wake-up prompt tone will not be too loud. In this relatively quiet environment, the wake-up prompt tone at this volume will not cause excessive intrusion to the user. Therefore, the display device 200 does not need to adjust the volume of the wake-up prompt tone, and the display device 200 does not adjust the amplitude of the audio of the wake-up prompt tone.
[0252] When the signal-to-noise ratio of the user voice is less than the first signal-to-noise ratio threshold, the display device 200 performs step S1205 to determine whether the signal-to-noise ratio of the user voice is greater than or equal to the second signal-to-noise ratio threshold.
[0253] When the signal-to-noise ratio of the user voice is less than the first signal-to-noise ratio threshold and the signal-to-noise ratio of the user voice is greater than or equal to the second signal-to-noise ratio threshold, the display device 200 performs step S1204 and does not adjust the amplitude of the audio of the wake-up prompt tone.
[0254] If the signal-to-noise ratio of the user's voice is less than the first signal-to-noise ratio threshold and greater than or equal to the second signal-to-noise ratio threshold, the noise level of the environment in which display device 200 is located is moderate. Therefore, display device 200 is considered to directly play the wake-up reminder tone at the current system volume. In this environment, the volume of the wake-up reminder tone is sufficient to alert the user without causing excessive intrusion. Therefore, display device 200 does not need to adjust the volume of the wake-up reminder tone, nor does it adjust the amplitude of the wake-up reminder tone.
[0255] When the signal-to-noise ratio of the user voice is less than the second signal-to-noise ratio threshold, the display device 200 executes step S1206 to determine whether the current system volume is less than the second volume threshold.
[0256] When the signal-to-noise ratio of the user voice is less than the second signal-to-noise ratio threshold and the current system volume is greater than or equal to the second volume threshold, the display device 200 performs step S1204 and does not adjust the amplitude of the audio of the wake-up prompt tone.
[0257] The signal-to-noise ratio of the user's voice is less than the second signal-to-noise ratio threshold, indicating that the environment in which the display device 200 is located is relatively noisy, that is, the environment in which the display device 200 is located is relatively noisy. The current system volume is greater than or equal to the second volume threshold, indicating that if the display device 200 uses the current system volume to directly play the audio of the wake-up prompt tone, the volume of the wake-up prompt tone will not be too low. At this time, the display device 200 plays the wake-up prompt tone at a volume that is not too low in a noisy environment, which can also satisfy the user's perception of the wake-up prompt tone. Therefore, the display device 200 does not need to adjust the volume of the wake-up prompt tone, and the display device 200 does not adjust the amplitude of the audio of the wake-up prompt tone.
[0258] When the signal-to-noise ratio of the user voice is less than the second signal-to-noise ratio threshold and the current system volume is less than the second volume threshold, the display device 200 performs step S1207 to increase the amplitude of the audio of the wake-up prompt tone to obtain the audio to be played of the wake-up prompt tone.
[0259] The signal-to-noise ratio of the user's voice is less than the second signal-to-noise ratio threshold, indicating that the environment in which the display device 200 is located is relatively noisy, that is, the environment in which the display device 200 is located is relatively noisy. The current system volume is less than the second volume threshold, indicating that if the display device 200 uses the current system volume to directly play the audio of the wake-up prompt tone, the volume of the wake-up prompt tone will be relatively low. At this time, the display device 200 plays the wake-up prompt tone at a relatively low volume in a relatively noisy environment, which may easily cause the user to be unable to perceive the wake-up prompt tone, thereby mistakenly believing that the display device 200 has not been awakened. In order to solve this problem, the amplitude of the audio of the wake-up prompt tone is increased to increase the volume of the wake-up prompt tone, thereby improving the user's perception of the wake-up prompt tone.
[0260] S504: Play the waiting audio of the wake-up prompt tone at the current system volume.
[0261] After obtaining the audio to be played of the wake-up prompt sound based on step S503, the display device 200 plays the audio to be played of the wake-up prompt sound at the current system volume. The volume of the wake-up prompt sound is affected by the current system volume and the volume of the audio to be played of the wake-up prompt sound itself.
[0262] When the current system volume is less than the third volume threshold, the display device 200 displays a pop-up window and displays a prompt message within the pop-up window, which prompts the user that the current system volume is too low, resulting in the volume of the wake-up prompt tone being too low, and that the system volume can be increased, wherein the third volume threshold is less than or equal to the second volume threshold. Taking the display device 200 as a television as an example, when the current system volume is less than the third volume threshold, the display device 200 can display a pop-up window 1301 as shown in ① of Figure 26, which includes a prompt message such as "The system volume is too low, please increase the system volume."
[0263] When the current system volume is greater than the fourth volume threshold, the display device 200 displays a pop-up window and displays a prompt message within the pop-up window, which prompts the user that the current system volume is too high, causing the volume of the wake-up prompt tone to be too loud, and that the system volume can be lowered, wherein the fourth volume threshold is greater than or equal to the first volume threshold. Taking the display device 200 as an example of a television, when the current system volume is less than the third volume threshold, the display device 200 may display a pop-up window 1302 as shown in FIG. 26 ②, which includes a prompt message such as "The system volume is too loud, please lower the system volume."
[0264] After the display device 200 is awakened, the audio of the wake-up prompt sound is played. The effect of the wake-up prompt sound played by the display device 200 on the user will be affected by whether the user really wants to wake up the display device 200 and the current system volume of the display device 200. If the user wakes up the display device 200 by mistake with a high probability, and the current system volume of the display device 200 is high, the volume of the wake-up prompt sound played by the display device 200 will also be high. The high-volume wake-up prompt sound will make the user clearly aware that the display device 200 has been awakened. When the display device 200 is awakened by mistake, the user's normal use of the display device 200 is affected. If the user wakes up the display device 200 by mistake with a low probability, and the current system volume of the display device 200 is low, the volume of the wake-up prompt sound played by the display device 200 will also be low. The low-volume wake-up prompt sound will make it difficult for the user to perceive that the display device 200 has been awakened. When the display device 200 is not awakened by mistake, it is easy for the user to mistakenly believe that the display device 200 has not been awakened.
[0265] To solve the above problem, the display device 200 can adjust the volume of the wake-up prompt sound according to the process shown in FIG27 , which may include:
[0266] S1401: Collect user voice.
[0267] S1402: Calculate a wake-up value corresponding to the user's voice.
[0268] Steps S1401-S1402 are similar to steps S501 and S601, and steps S701-S702, and are not described in detail here.
[0269] S1403: When the wake-up value is greater than or equal to the wake-up value threshold, obtain the difference between the wake-up value and the wake-up value threshold and the current system volume.
[0270] The difference between the wake-up value and the wake-up value threshold is used to represent the probability of the display device 200 being falsely awakened. The larger the difference between the wake-up value and the wake-up value threshold, the smaller the probability of the display device 200 being falsely awakened, and the smaller the difference between the wake-up value and the wake-up value threshold, the greater the probability of the display device 200 being falsely awakened.
[0271] The process of the display device 200 obtaining the current system volume can be referred to step S502 and will not be described in detail here.
[0272] S1404: Adjust the amplitude of the wake-up prompt tone audio according to the difference between the wake-up value and the wake-up value threshold and the current system volume to obtain the audio to be played of the wake-up prompt tone.
[0273] The display device 200 may adjust the amplitude of the wake-up prompt tone according to the process shown in FIG28 , which may include:
[0274] S1501: Obtain the audio of the wake-up prompt tone.
[0275] Step S1501 may refer to step S1001 and will not be described in detail here.
[0276] S1502: Adjust the amplitude of the wake-up prompt tone audio according to the difference between the wake-up value and the wake-up value threshold and the current system volume to obtain the audio to be played of the wake-up prompt tone.
[0277] The display device 200 dynamically adjusts the amplitude of the wake-up prompt tone audio based on the difference between the wake-up value and the wake-up value threshold and the current system volume to obtain the wake-up prompt tone audio to be played. For example, the amplitude of the wake-up prompt tone audio is adjusted by adjusting the gain of the wake-up prompt tone audio.
[0278] Based on the above method of adjusting the amplitude of the wake-up prompt tone audio itself, it is convenient to set a finer amplitude adjustment granularity, so that the volume of the wake-up prompt tone can be more accurately tailored to the current environment.
[0279] The display device 200 may adjust the amplitude of the wake-up prompt tone according to the process shown in FIG29 , which may include:
[0280] S1601: Determine the amplitude of the audio to be played of the wake-up prompt tone according to the difference between the wake-up value and the wake-up value threshold and the current system volume.
[0281] The preset audio of the wake-up prompt sound and the preset method of the preset audio can refer to step S1001. The display device 200 can determine the amplitude of the audio to be played of the wake-up prompt sound to be played based on the difference between the wake-up value and the wake-up value threshold and the current system volume.
[0282] S1602: Obtain a preset audio of the wake-up prompt tone with a corresponding amplitude according to the amplitude of the to-be-played audio of the wake-up prompt tone.
[0283] Step S1602 may refer to step S1102 and will not be described in detail here.
[0284] 30 , the process of the display device 200 adjusting the amplitude of the audio of the wake-up prompt sound according to the difference between the wake-up value of the user voice and the wake-up value threshold and the current system volume is described.
[0285] The display device 200 sets a first difference threshold and a second difference threshold, as well as a first volume threshold and a second volume threshold, wherein the first difference threshold is smaller than the second difference threshold, and the first volume threshold is larger than the second volume threshold.
[0286] The display device 200 executes step S1701 to determine whether the difference between the wake-up value of the user voice and the wake-up value threshold is less than a first difference threshold.
[0287] When the difference is less than the first difference threshold, the display device 200 executes step S1702 to determine whether the current system volume is greater than the first volume threshold.
[0288] When the difference is less than the first difference threshold and the current system volume is greater than the first volume threshold, the display device 200 executes step S1703 to reduce the amplitude of the audio of the wake-up prompt tone to obtain the audio to be played of the wake-up prompt tone.
[0289] If the difference is less than the first difference threshold, it means that the display device 200 is more likely to be mistakenly awakened. If the current system volume is greater than the first volume threshold, it means that if the display device 200 uses the current system volume to directly play the audio of the wake-up prompt tone, the volume of the wake-up prompt tone will be higher. At this time, if the display device 200 is likely to be mistakenly awakened, playing the wake-up prompt tone at a higher volume can easily cause a strong sense of intrusion to the user. In order to solve this problem, the amplitude of the audio of the wake-up prompt tone is reduced to lower the volume of the wake-up prompt tone, thereby reducing the disturbance to the user.
[0290] When the difference is less than the first difference threshold and the current system volume is less than or equal to the first volume threshold, the display device 200 performs step S1704 and does not adjust the amplitude of the audio of the wake-up prompt tone.
[0291] The difference is less than the first volume threshold, indicating that the display device 200 is more likely to be mistakenly awakened. The current system volume is less than or equal to the first volume threshold, indicating that if the display device 200 directly plays the audio of the wake-up prompt tone using the current system volume, the volume of the wake-up prompt tone will not be too loud. At this time, when the display device 200 is likely to be mistakenly awakened, the wake-up prompt tone is played at a volume that is not too loud, and the wake-up prompt tone at this volume will not cause excessive intrusion to the user. Therefore, the display device 200 does not need to adjust the volume of the wake-up prompt tone, and the display device 200 does not adjust the amplitude of the audio of the wake-up prompt tone.
[0292] When the difference is greater than or equal to the first difference threshold, the display device 200 executes step S1705 to determine whether the difference is greater than or equal to the second difference threshold.
[0293] When the difference is greater than or equal to the first difference threshold and less than the second difference threshold, the display device 200 performs step S1704 and does not adjust the amplitude of the audio of the wake-up prompt tone.
[0294] If the difference is greater than or equal to the first difference threshold and less than the second difference threshold, it can be considered that within the threshold range, it is difficult for display device 200 to accurately determine whether the awakening of display device 200 is in accordance with the user's intention. In this case, if it is impossible to accurately determine whether the awakening of display device 200 is in accordance with the user's intention, display device 200 does not adjust the volume of the wake-up prompt tone, and the display device 200 does not adjust the audio amplitude of the wake-up prompt tone to reduce the data processing process.
[0295] When the difference is greater than or equal to the second difference threshold, the display device 200 executes step S1706 to determine whether the current system volume is less than the second volume threshold.
[0296] When the difference is greater than or equal to the second difference threshold and the current system volume is greater than or equal to the second volume threshold, the display device 200 performs step S1704 and does not adjust the amplitude of the audio of the wake-up prompt tone.
[0297] If the difference is greater than or equal to the second difference threshold, it means that the probability of the display device 200 being mistakenly awakened is low. If the current system volume is greater than or equal to the second volume threshold, it means that if the display device 200 uses the current system volume to directly play the audio of the wake-up prompt tone, the volume of the wake-up prompt tone will not be too low. When the display device 200 is mistakenly awakened with a small probability, the wake-up prompt tone is played at a volume that is not too low, and the wake-up prompt tone at this volume will not cause excessive intrusion to the user. Therefore, the display device 200 does not need to adjust the volume of the wake-up prompt tone, and the display device 200 does not adjust the amplitude of the audio of the wake-up prompt tone.
[0298] When the difference is greater than or equal to the second difference threshold and the current system volume is less than the second volume threshold, the display device 200 executes step S1707 to increase the amplitude of the audio of the wake-up prompt tone to obtain the audio to be played of the wake-up prompt tone.
[0299] If the difference is greater than or equal to the second difference threshold, it means that the probability of the display device 200 being mistakenly awakened is low. If the current system volume is less than the second volume threshold, it means that if the display device 200 uses the current system volume to directly play the audio of the wake-up prompt tone, the volume of the wake-up prompt tone will be relatively low. When the display device 200 is mistakenly awakened with a small probability, playing the wake-up prompt tone at a relatively low volume may easily cause the user to be unable to perceive the wake-up prompt tone, thereby mistakenly believing that the display device 200 has not been awakened. In order to solve this problem, the amplitude of the audio of the wake-up prompt tone is increased to increase the volume of the wake-up prompt tone, thereby improving the user's perception of the wake-up prompt tone.
[0300] S1405: Control the speaker to play the to-be-played audio of the wake-up prompt tone at the current system volume.
[0301] After obtaining the audio to be played of the wake-up prompt sound based on step S1404, the display device 200 plays the audio to be played of the wake-up prompt sound at the current system volume. The volume of the wake-up prompt sound is affected by the current system volume and the volume of the audio to be played of the wake-up prompt sound itself.
[0302] When the current system volume is less than the third volume threshold, the display device 200 displays a pop-up window and displays a prompt message in the pop-up window, which prompts the user that the current system volume is too low, resulting in the volume of the wake-up prompt tone being too low, and that the system volume can be increased. The third volume threshold is less than or equal to the second volume threshold. The pop-up window displayed by the display device 200 can refer to the pop-up window 1301 shown in ① of Figure 26, and is not repeated here.
[0303] When the current system volume is greater than the fourth volume threshold, the display device 200 displays a pop-up window and displays a prompt message in the pop-up window, which prompts the user that the current system volume is too high, causing the volume of the wake-up prompt tone to be too loud, and that the system volume can be lowered. The fourth volume threshold is greater than or equal to the first volume threshold. The pop-up window displayed by the display device 200 can be referred to as pop-up window 1302 shown in FIG. 26 ②, and is not described here in detail.
[0304] The display device 200 determines the style of the displayed pop-up window according to the difference between the wake-up value of the user voice and the wake-up value threshold.
[0305] When the difference is less than the first difference threshold, the display device 200 displays a pop-up window of the first style. When the difference is greater than or equal to the second difference threshold, the display device 200 displays a pop-up window of the second style. When the difference is greater than or equal to the first difference threshold and less than the second difference threshold, either the first style pop-up window or the second style pop-up window can be displayed. The second style pop-up window is easier for the user to perceive than the first style pop-up window.
[0306] For example, the second-style pop-up window is larger than the first-style pop-up window. Taking the display device 200 as a television and the current system volume being less than the third volume threshold as an example, the first-style pop-up window displayed by the display device 200 may be the pop-up window 1801 shown in ① of FIG31 , and the second-style pop-up window displayed by the display device 200 may be the pop-up window 1802 shown in ② of FIG31 . Both the pop-up window 1801 and the pop-up window 1802 include prompt information, such as "The system volume is too low, please increase the system volume." The size of the pop-up window 1802 is larger than that of the pop-up window 1801, so that the displayed pop-up window 1801 is less likely to be noticed by the user, and the displayed pop-up window 1802 is more likely to be noticed by the user.
[0307] For another example, the second-style pop-up window is displayed more obviously than the first-style pop-up window. Taking the display device 200 as a television and the current system volume being less than the third volume threshold as an example, the first-style pop-up window displayed by the display device 200 may be the pop-up window 1901 as shown in ① in FIG32, and the second-style pop-up window displayed by the display device 200 may be the pop-up window 1902 as shown in ② in FIG32. Both the pop-up window 1901 and the pop-up window 1902 include prompt information, such as "The system volume is too low, please increase the system volume." The size of the pop-up window 1901 and the pop-up window 1902 may be the same. The pop-up window 1901 is displayed in the lower right corner of the screen, and the pop-up window 1902 is displayed in the center of the screen, so that the displayed pop-up window 1901 is less likely to be noticed by the user, and the displayed pop-up window 1902 is more likely to be noticed by the user.
[0308] For another example, the second-style pop-up window is larger in size than the first-style pop-up window and is displayed in a more obvious position. Taking the display device 200 as a television and the current system volume being less than the third volume threshold as an example, the first-style pop-up window displayed by the display device 200 may be the pop-up window 2001 shown in ① of FIG33 , and the second-style pop-up window displayed by the display device 200 may be the pop-up window 2002 shown in ② of FIG33 . Both the pop-up window 2001 and the pop-up window 2002 include prompt information, such as “The system volume is too low, please increase the system volume.” The size of the pop-up window 2001 is smaller than that of the pop-up window 2002 and may be the same. The pop-up window 2001 is displayed in the lower right corner of the screen, and the pop-up window 2002 is displayed in the center of the screen, so that the displayed pop-up window 2001 is less likely to be noticed by the user and the displayed pop-up window 2002 is more likely to be noticed by the user.
[0309] The display device 200 may adjust the volume of the wake-up prompt tone based on the probability of being falsely awakened, the current system volume, and the noise level of the surrounding environment.
[0310] The display device 200 may adjust the volume of the wake-up prompt tone according to the process shown in FIG34 , which may include:
[0311] S2101, collecting user voice.
[0312] S2102: Calculate a wake-up value corresponding to the user's voice.
[0313] Steps S2101-S2102 are similar to steps S501 and S601, and steps S701-S702, and are not described in detail here.
[0314] S2103: When the wake-up value is greater than or equal to the wake-up value threshold, obtain the difference between the wake-up value and the wake-up value threshold, the signal-to-noise ratio of the user voice, and the current system volume.
[0315] The difference between the wake-up value and the wake-up value threshold is used to represent the probability of the display device 200 being falsely awakened. The larger the difference between the wake-up value and the wake-up value threshold, the smaller the probability of the display device 200 being falsely awakened, and the smaller the difference between the wake-up value and the wake-up value threshold, the greater the probability of the display device 200 being falsely awakened.
[0316] The process of the display device 200 obtaining the signal-to-noise ratio of the user voice and the current system volume can be referred to step S502 and will not be described in detail here.
[0317] S2104 : Adjust the amplitude of the wake-up prompt tone audio according to the difference between the wake-up value and the wake-up value threshold, the signal-to-noise ratio of the user voice, and the current system volume to obtain the audio to be played of the wake-up prompt tone.
[0318] The display device 200 may adjust the amplitude of the wake-up prompt tone according to the process shown in FIG35 , which may include:
[0319] S2201: Obtain the audio of the wake-up prompt tone.
[0320] Step S2201 can refer to step S1001 and will not be repeated here.
[0321] S2202: Adjust the amplitude of the wake-up prompt tone audio according to the difference between the wake-up value and the wake-up value threshold, the signal-to-noise ratio of the user voice, and the current system volume to obtain the audio to be played of the wake-up prompt tone.
[0322] The display device 200 dynamically adjusts the amplitude of the wake-up prompt tone based on the difference between the wake-up value and the wake-up value threshold, the signal-to-noise ratio of the user's voice, and the current system volume to obtain the wake-up prompt tone audio to be played. For example, the amplitude of the wake-up prompt tone audio is adjusted by adjusting the gain of the wake-up prompt tone audio.
[0323] Based on the above method of adjusting the amplitude of the wake-up prompt tone audio itself, it is convenient to set a finer amplitude adjustment granularity, so that the volume of the wake-up prompt tone can be more accurately tailored to the current environment.
[0324] The display device 200 may adjust the amplitude of the wake-up prompt tone according to the process shown in FIG36 , which may include:
[0325] S2301: Determine the amplitude of the to-be-played audio of the wake-up prompt tone according to the difference between the wake-up value and the wake-up value threshold, the signal-to-noise ratio of the user voice, and the current system volume.
[0326] The preset audio of the wake-up prompt sound and the preset method of the preset audio can refer to step S1101. The display device 200 can determine the amplitude of the audio to be played of the wake-up prompt sound to be played based on the difference between the wake-up value and the wake-up value threshold, the signal-to-noise ratio of the user voice and the current system volume.
[0327] S2302: Obtain a preset audio of the wake-up prompt tone with a corresponding amplitude according to the amplitude of the to-be-played audio of the wake-up prompt tone.
[0328] Step S2302 may refer to step S1102 and will not be described in detail here.
[0329] 37 , the process of the display device 200 adjusting the amplitude of the audio of the wake-up prompt sound according to the difference between the wake-up value of the user voice and the wake-up value threshold, the current system volume, and the signal-to-noise ratio of the user voice is described.
[0330] The display device 200 sets a first difference threshold and a second difference threshold, a first signal-to-noise ratio threshold and a second signal-to-noise ratio threshold, and a first volume threshold and a second volume threshold, wherein the first difference threshold is smaller than the second difference threshold, the first signal-to-noise ratio threshold is greater than the second signal-to-noise ratio threshold, and the first volume threshold is greater than the second volume threshold.
[0331] The display device 200 executes step S2401 to determine whether the difference between the wake-up value of the user voice and the wake-up value threshold is less than a first difference threshold.
[0332] When the difference is less than the first difference threshold, the display device 200 performs step S2402 to determine whether the signal-to-noise ratio of the user voice is greater than or equal to the first signal-to-noise ratio threshold.
[0333] When the signal-to-noise ratio of the user voice is greater than or equal to the first signal-to-noise ratio threshold, the display device 200 executes step S2403 to determine whether the current system volume is greater than or equal to the first volume threshold.
[0334] When the signal-to-noise ratio of the user voice is greater than or equal to the first signal-to-noise ratio threshold and the current system volume is greater than or equal to the first volume threshold, the display device 200 executes step S2404 to reduce the amplitude of the audio of the wake-up prompt tone to obtain the audio to be played of the wake-up prompt tone.
[0335] The difference is less than the first difference threshold, indicating that the probability of the display device 200 being mistakenly awakened is high. The signal-to-noise ratio of the user's voice is greater than or equal to the first signal-to-noise ratio threshold, indicating that the environment in which the display device 200 is located is relatively quiet. The current system volume is greater than or equal to the first volume threshold, indicating that if the display device 200 uses the current system volume to directly play the audio of the wake-up prompt sound, the volume of the wake-up prompt sound will be relatively large. In a relatively quiet environment, when the display device 200 is likely to be mistakenly awakened, playing the wake-up prompt sound at a relatively large volume can easily cause a strong sense of intrusion to the user. In order to solve this problem, the amplitude of the audio of the wake-up prompt sound is reduced to lower the volume of the wake-up prompt sound, thereby reducing disturbance to the user.
[0336] When the signal-to-noise ratio of the user voice is greater than or equal to the first signal-to-noise ratio threshold and the current system volume is less than the first volume threshold, the display device 200 performs step S2405 and does not adjust the amplitude of the audio of the wake-up prompt tone.
[0337] If the difference is less than the first difference threshold, it means that the probability of the display device 200 being mistakenly awakened is high. The signal-to-noise ratio of the user's voice is greater than or equal to the first signal-to-noise ratio threshold, which means that the environment in which the display device 200 is located is relatively quiet. The current system volume is less than the first volume threshold, which means that if the display device 200 uses the current system volume to directly play the audio of the wake-up prompt sound, the volume of the wake-up prompt sound will not be too loud. In a relatively quiet environment, when the display device 200 is likely to be mistakenly awakened, it plays the wake-up prompt sound at a volume that is not too loud, and the sense of intrusion caused to the user will not be too strong. The display device 200 does not need to adjust the volume of the wake-up prompt sound.
[0338] When the signal-to-noise ratio of the user voice is less than the first signal-to-noise ratio threshold, the display device 200 performs step S2405 and does not adjust the amplitude of the audio of the wake-up prompt tone.
[0339] The signal-to-noise ratio of the user's voice is less than the first signal-to-noise ratio threshold, indicating that the environment in which the display device 200 is located is not too quiet. In an environment that is not too quiet, if the display device 200 is in a high probability of being mistakenly awakened, if the audio of the wake-up prompt sound is played directly at a higher volume, the sense of intrusion caused to the user will not be too strong. Therefore, the display device 200 does not need to lower the volume of the wake-up prompt sound. If the audio of the wake-up prompt sound is played directly at a lower volume, due to the high probability of being mistakenly awakened, even if the user's perception of the wake-up prompt sound is weak, the probability of affecting the user is also low. Therefore, the display device 200 does not need to increase the volume of the wake-up prompt sound. Therefore, the display device 200 does not need to adjust the volume of the wake-up prompt sound.
[0340] When the difference is greater than or equal to the first difference threshold, the display device 200 executes step S2406 to determine whether the difference is less than the second difference threshold.
[0341] When the difference is less than the second difference threshold, the display device 200 executes step S2405 and does not adjust the amplitude of the audio of the wake-up prompt tone.
[0342] If the difference is greater than or equal to the first difference threshold and less than the second difference threshold, it can be considered that within the threshold range, it is difficult for display device 200 to accurately determine whether the awakening of display device 200 is in accordance with the user's intention. In this case, if it is impossible to accurately determine whether the awakening of display device 200 is in accordance with the user's intention, display device 200 does not adjust the volume of the wake-up prompt tone, and the display device 200 does not adjust the audio amplitude of the wake-up prompt tone to reduce the data processing process.
[0343] When the difference is greater than or equal to the second difference threshold, the display device 200 executes step S2407 to determine whether the signal-to-noise ratio of the user voice is greater than or equal to the second signal-to-noise ratio threshold.
[0344] When the signal-to-noise ratio of the user voice is less than the second signal-to-noise ratio threshold, the display device 200 executes step S2408 to determine whether the current system volume is less than the second volume threshold.
[0345] When the signal-to-noise ratio of the user voice is less than the second signal-to-noise ratio threshold and the current system volume is less than the second volume threshold, the display device 200 executes step S2409 to increase the amplitude of the audio of the wake-up prompt tone to obtain the audio to be played of the wake-up prompt tone.
[0346] The difference is greater than or equal to the second difference threshold, indicating that the probability of the display device 200 being mistakenly awakened is low. The signal-to-noise ratio of the user's voice is less than the second signal-to-noise ratio threshold, indicating that the noise level of the environment in which the display device 200 is located is relatively high, that is, the environment in which the display device 200 is located is relatively noisy. The current system volume is less than the second volume threshold, indicating that if the display device 200 uses the current system volume to directly play the audio of the wake-up prompt tone, the volume of the wake-up prompt tone will be relatively low. In a noisy environment, when the display device 200 is mistakenly awakened with a small probability, playing the wake-up prompt tone at a relatively low volume can easily cause the user to be unable to perceive the wake-up prompt tone, thereby mistakenly believing that the display device 200 has not been awakened. In order to solve this problem, the amplitude of the audio of the wake-up prompt tone is increased to increase the volume of the wake-up prompt tone, thereby improving the user's perception of the wake-up prompt tone.
[0347] When the signal-to-noise ratio of the user voice is less than the second signal-to-noise ratio threshold and the current system volume is greater than or equal to the second volume threshold, the display device 200 performs step S2405 and does not adjust the amplitude of the audio of the wake-up prompt tone.
[0348] The difference is greater than or equal to the second difference threshold, indicating that the probability of the display device 200 being mistakenly awakened is low. The signal-to-noise ratio of the user's voice is less than the second signal-to-noise ratio threshold, indicating that the noise level of the environment in which the display device 200 is located is relatively high, that is, the environment in which the display device 200 is located is relatively noisy. The current system volume is greater than or equal to the second volume threshold, indicating that if the display device 200 uses the current system volume to directly play the audio of the wake-up prompt sound, the volume of the wake-up prompt sound will not be too small. In a noisy environment, when there is a small probability of being mistakenly awakened, the display device 200 plays the wake-up prompt sound at a volume that is not too small, and it is believed that the user can also perceive the wake-up prompt sound. Therefore, the display device 200 does not need to adjust the volume of the wake-up prompt sound, and the display device 200 does not adjust the amplitude of the audio of the wake-up prompt sound.
[0349] When the signal-to-noise ratio of the user voice is greater than or equal to the second signal-to-noise ratio threshold, the display device 200 performs step S2405 and does not adjust the amplitude of the audio of the wake-up prompt tone.
[0350] The signal-to-noise ratio of the user's voice is greater than or equal to the second signal-to-noise ratio threshold, indicating that the environment in which the display device 200 is located is not too noisy. In an environment that is not too noisy, when the display device 200 is mistakenly awakened with a small probability, if the audio of the wake-up prompt sound is directly played at a higher volume, the user's perception of the wake-up prompt sound can be enhanced due to the small probability of being mistakenly awakened. Therefore, the display device 200 does not need to lower the volume of the wake-up prompt sound. If the audio of the wake-up prompt sound is directly played at a lower volume, the user can also perceive it because the environment is not too noisy. Therefore, the display device 200 does not need to increase the volume of the wake-up prompt sound. Therefore, the display device 200 does not need to adjust the volume of the wake-up prompt sound.
[0351] S2105: Control the speaker to play the to-be-played audio of the wake-up prompt tone at the current system volume.
[0352] After obtaining the audio to be played of the wake-up prompt sound based on step S2104, the display device 200 plays the audio to be played of the wake-up prompt sound at the current system volume. The volume of the wake-up prompt sound is affected by the current system volume and the volume of the audio to be played of the wake-up prompt sound itself.
[0353] When the current system volume is less than the third volume threshold, the display device 200 displays a pop-up window and displays a prompt message in the pop-up window, which prompts the user that the current system volume is too low, resulting in the volume of the wake-up prompt tone being too low, and that the system volume can be increased. The third volume threshold is less than or equal to the second volume threshold. The pop-up window displayed by the display device 200 can refer to the pop-up window 1301 shown in ① of Figure 26, and is not repeated here.
[0354] When the current system volume is greater than the fourth volume threshold, the display device 200 displays a pop-up window and displays a prompt message in the pop-up window, which prompts the user that the current system volume is too high, causing the volume of the wake-up prompt tone to be too loud, and that the system volume can be lowered. The fourth volume threshold is greater than or equal to the first volume threshold. The pop-up window displayed by the display device 200 can be referred to as pop-up window 1302 shown in FIG. 26 ②, and is not described here in detail.
[0355] The display device 200 determines the style of the displayed pop-up window according to the difference between the wake-up value of the user voice and the wake-up value threshold.
[0356] When the difference is less than the first difference threshold, the display device 200 displays a pop-up window of the first style. When the difference is greater than or equal to the second difference threshold, the display device 200 displays a pop-up window of the second style. When the difference is greater than or equal to the first difference threshold and less than the second difference threshold, either the pop-up window of the first style or the pop-up window of the second style can be displayed. Among them, the pop-up window of the second style is easier for the user to perceive than the pop-up window of the first style. The pop-up window of the first style and the pop-up window of the second style can be referred to Figures 31, 32 and 33, which are not described here.
[0357] The working state of the display device 200 includes a first state, which includes the display device 200 being in silent mode and the display device 200 being in non-silent mode and the current system volume being 0. When the display device 200 is in the first state, if the display device 200 is awakened, due to the limitations of the first state, when the display device 200 plays the wake-up prompt tone, the speaker does not make any sound. Since the user cannot perceive the wake-up prompt tone, it is easy for the user to mistakenly believe that the display device 200 is not responding. Even if the user perceives that the display device 200 is awakened, it cannot quickly determine whether the speaker is silent because it is in silent mode or the current system volume is 0, making it impossible to quickly make the speaker sound.
[0358] To solve the above problem, the display device 200 may perform a wake-up response according to the process shown in FIG38 , which may include:
[0359] S2501, when the screen is in the bright screen state and in the first state, collecting user voice.
[0360] S2502: When it is recognized that the user voice contains a wake-up word, a pop-up window is displayed according to the first state.
[0361] Step S2501 is similar to step S501, and the process of identifying the wake-up word in step S2502 is similar to steps S601-S602, which will not be repeated here.
[0362] The task manager of the display device 200 stores a volume status table, which is used to record the current system volume, whether the silent mode is enabled, etc. The first flag bit in the volume status table is used to indicate whether the silent mode is enabled. For example, if the value of the first flag bit is 1, the silent mode is enabled; if the value of the first flag bit is 0, the silent mode is disabled. By identifying the value of the first flag bit, it can be determined whether the display is in silent mode. For example, if the value of the first flag bit is 1, the display is in silent mode; if the value of the first flag bit is 0, the display is in non-silent mode.
[0363] The display device 200 performs a process of obtaining the audio to be played of the wake-up prompt sound when the wake-up value is greater than or equal to the wake-up value threshold.
[0364] The display device 200 may identify whether it is in silent mode before obtaining the audio to be played for the wake-up prompt tone. If the display device 200 identifies that it is in silent mode, it does not obtain the audio to be played for the wake-up prompt tone; if the display device 200 identifies that it is in non-silent mode, it obtains the audio to be played for the wake-up prompt tone.
[0365] The display device 200 can identify whether it is in silent mode during the process of obtaining the audio to be played for the wake-up prompt tone. For example, obtaining the audio to be played for the wake-up prompt tone and identifying whether it is in silent mode are respectively executed in two parallel processes / threads. If the display device 200 identifies that it is in silent mode, the execution of obtaining the audio to be played for the wake-up prompt tone is terminated; if the display device 200 identifies that it is in non-silent mode, the execution of obtaining the audio to be played for the wake-up prompt tone is executed. In this way, when it is identified that it is in silent mode, the computing resources of the display device 200 can be avoided from being continuously occupied. It is also possible to quickly obtain the audio to be played for the wake-up prompt tone when it is identified that it is in non-silent mode.
[0366] After the display device 200 completes the process of obtaining the audio to be played for the wake-up prompt tone, it can identify whether it is in silent mode. Therefore, when the display device 200 identifies that it is in silent mode, once the silent mode is released, it can directly use the audio to be played for the wake-up prompt tone after being awakened, thereby improving the efficiency of playing the wake-up prompt tone after releasing the silent mode.
[0367] If the display device 200 recognizes the wake-up word, a pop-up window is displayed to prompt the user that he has been awakened.
[0368] In some embodiments, when the display device 200 is in silent mode, the display device 200 responds to the wake-up word and is in the awake state, displays a pop-up window, does not release silent mode, and does not adjust the system volume. While in the awake state, the display device 200 can respond to other far-field voice commands from the user and interact with the user. This pop-up window is used to notify the user that they have been awakened and are in silent mode.
[0369] The pop-up window includes prompt information, which includes information for prompting the display device 200 to be awakened and in silent mode (for ease of distinction, referred to as the first information). Taking the display device 200 as a TV as an example, the display device 200 can display a pop-up window 2601 as shown in Figure 39, and the pop-up window 2601 includes the first information, such as "awakened, currently in silent mode". As a result, the display device 200 is awakened when it is in silent mode. Although the wake-up prompt tone played is silent, the user can be prompted through a pop-up window that he has been awakened and that he is currently in silent mode, so that the user knows that the reason for being awakened but no wake-up prompt tone is that he is in silent mode.
[0370] The display device 200 determines the style of the pop-up window displayed based on the difference between the wake-up value of the user voice and the wake-up value threshold. When the difference is less than the first difference threshold, the display device 200 displays a pop-up window of the first style. When the difference is greater than or equal to the second difference threshold, the display device 200 displays a pop-up window of the second style. When the difference is greater than or equal to the first difference threshold and less than the second difference threshold, the display device 200 can display a pop-up window of the first style or a pop-up window of the second style. The pop-up window of the second style is more easily perceived by the user than the pop-up window of the first style. The pop-up window of the first style and the pop-up window of the second style can be referred to Figures 31, 32 and 33. The difference is that the first information is displayed in the pop-up window, which is not repeated here.
[0371] The display device 200 may also display a prompt message in a pop-up window based on the current system volume, and the prompt message may also include information for prompting the user to adjust the system volume (for ease of distinction, referred to as the second information). The display device 200 may display the second information according to the process shown in FIG40 , which may include:
[0372] S2701: Get the current system volume.
[0373] The current system volume is the system volume before the display device 200 turns on the silent mode. The process of the display device 200 obtaining the current system volume can be referred to step S502 and will not be described in detail here.
[0374] S2702: Display second information in a pop-up window according to the current system volume.
[0375] The display device 200 sets a first volume threshold and a second volume threshold, wherein the first volume threshold is greater than the second volume threshold.
[0376] When the current system volume is greater than or equal to the first volume threshold, the second information is used to prompt the user to lower the system volume.
[0377] The previous system volume is greater than or equal to the first volume threshold, indicating that the system volume of the display device 200 is relatively high. After unmuting, the display device 200 will play audio at the relatively high volume, which can easily disturb the user. In order to solve this problem, the display device 200 displays a second message to prompt the user that the current system volume is relatively high and the system volume can be lowered. Taking the display device 200 as a TV as an example, the display device 200 can display a pop-up window 2801 as shown in ① in Figure 41. The pop-up window 2801 includes the first message, such as "Awake, currently in silent mode", and the pop-up window 2801 also includes the second message, such as "The volume is relatively high, it is recommended to lower it".
[0378] When the current system volume is less than the second volume threshold, the second information is used to prompt the user to increase the system volume.
[0379] The previous system volume is less than the second volume threshold, indicating that the system volume of the display device 200 is low. After unmuting, the display device 200 will play audio at the lower volume, which may not be easily perceived by the user. In order to solve this problem, the display device 200 displays a second message to prompt the user that the current system volume is low and the system volume can be increased. Taking the display device 200 as a TV as an example, the display device 200 can display a pop-up window 2802 as shown in ② in Figure 41. The pop-up window 2802 includes the first information, such as "Awake, currently in silent mode", and the pop-up window 2802 also includes the second information, such as "The volume is low, it is recommended to increase it".
[0380] After displaying the pop-up window, the display device 200 can play the audio sound after canceling the silent mode.
[0381] The display device 200 may cancel the mute mode according to the process shown in FIG42 , which may include:
[0382] S2901 , in response to a volume adjustment instruction from a user, canceling the mute mode and adjusting the current system volume to obtain an adjusted system volume.
[0383] The volume adjustment instruction may be a volume increase instruction or a volume decrease instruction. Taking the control device 100 as a remote controller as an example, the user may input the volume adjustment instruction via the volume key on the remote controller.
[0384] In response to the volume adjustment instruction, the display device 200 transmits the decoded audio data to the speaker for playback, thereby releasing the mute mode. In response to the volume adjustment instruction, the display device 200 also adjusts the current system volume. For example, if the volume adjustment instruction is a volume increase instruction, the system volume is increased. If the volume adjustment instruction is a volume decrease instruction, the system volume is decreased.
[0385] S2902: Play audio at the adjusted system volume.
[0386] After the display device 200 is released from mute mode, the system volume is the adjusted system volume, and the display device 200 plays audio at the adjusted system volume. For example, if the current system volume is 2 and the volume adjustment instruction is a volume increase instruction, such as increasing the volume by 1, the display device 200 responds to the volume increase instruction and releases mute mode. After the mute mode is released, the audio is played at the volume of 3.
[0387] After the display device 200 cancels the mute mode, the system volume is adjusted in response to a volume adjustment instruction input again by the user.
[0388] Based on the above method of canceling the mute mode, the user does not need to enter an additional mute cancellation command. The mute mode can be canceled directly while adjusting the system volume, which makes the user operation more convenient and smoother.
[0389] The display device 200 may cancel the mute mode according to the process shown in FIG43 , which may include:
[0390] S3001: In response to a release instruction input by a user, release the silent mode.
[0391] The release command is used to indicate the end of mute mode. For example, if control device 100 is a remote control, a user can input a control command using the mute button on the remote control. Display device 200 responds to the control command and, upon recognizing that the device is in mute mode, determines that the control command is a release command. In response to the release command, display device 200 transmits the decoded audio data to the speaker for playback, thereby ending mute mode.
[0392] S3002: Play audio at the current system volume.
[0393] After the display device 200 cancels the mute mode, it plays the audio at the current system volume. After the display device 200 cancels the mute mode, it can adjust the system volume in response to a volume adjustment instruction input again by the user.
[0394] The display device 200 can also adjust the system volume in response to the user's volume adjustment instruction before step S3001 to obtain the adjusted system volume. At this time, the display device 200 is still in silent mode. After the user completes adjusting the system volume, the user inputs a release instruction. After the display device 200 is unmuted, it plays the audio at the current system volume, which is actually the adjusted system volume. Therefore, if the current system volume is too loud or too low, the user can adjust the system volume to an appropriate volume before actually playing the audio, so that after the display device 200 is unmuted, the audio can be played directly at the appropriate volume to ensure the user's listening quality.
[0395] In other embodiments, when the display device 200 is in silent mode, the display device 200 responds to the wake-up word and is in an incompletely awakened state, displays a pop-up window, does not cancel the silent mode, and does not adjust the system volume. The incompletely awakened state means that the display device 200 responds to the wake-up word, starts the pop-up window function, and after displaying the pop-up window, exits the interaction with the user and does not respond to other far-field voice commands of the user. The pop-up window is used to prompt the user that the user is currently in silent mode and needs to be woken up again after the silent mode is canceled. In this way, it is possible to avoid destroying the silent mode set by the user, and to prompt the user through a less intrusive pop-up window, and to avoid the impact of the silent mode on the display device 200 responding to the user's far-field voice commands by requiring it to wake up again after canceling the silent mode, thereby improving the far-field voice interaction effect between the display device 200 and the user.
[0396] In other embodiments, when the display device 200 is in silent mode, the display device 200 responds to the wake-up word, is in a wake-up state, automatically cancels the silent mode, displays a pop-up window, and adjusts the system volume to a preset volume. The display device 200 is in a wake-up state and can interact with the user in response to other far-field voice commands of the user. The pop-up window is used to prompt the user that he has been awakened and is in silent mode, but the silent mode has been canceled. The preset volume can be the default system volume or the system volume saved when the silent mode is started. In this way, the display device 200 does not play the wake-up prompt sound, but uses a pop-up window as a transition to prompt the user that he has been awakened to avoid disturbing the user. Moreover, the display device 200 can continue to interact with the user without the user waking up again.
[0397] In some embodiments, when the display device 200 is not in a muted state and the current system volume is 0, the display device 200 responds to the wake-up word and enters an awake state, displays a pop-up window, and does not adjust the system volume. The pop-up window serves to notify the user that the device has been awakened and that the system volume is 0. At this point, the display device 200 is awake and can respond to other far-field voice commands from the user to interact with the user.
[0398] The pop-up window includes prompt information, which includes information for prompting that the display device 200 is awakened and the current system volume is 0 (for ease of distinction, referred to as the third information). Taking the display device 200 as a television as an example, the display device 200 can display a pop-up window 3101 as shown in Figure 44, and the pop-up window 3101 includes the third information, such as "Awake, current system volume is 0".
[0399] In other embodiments, when the display device 200 is in a non-muted state and the current system volume is 0, the display device 200 responds to the wake-up word and is in an incomplete wake-up state, displays a pop-up window, and does not adjust the system volume. The incomplete wake-up state means that the display device 200 responds to the wake-up word, starts the pop-up window function, and after displaying the pop-up window, exits the interaction with the user and does not respond to other far-field voice commands of the user. The pop-up window is used to prompt the user that the system volume is 0 and that it needs to be woken up again after increasing the volume. In this way, the user can be prompted through a less intrusive pop-up window, and by needing to wake up again after increasing the volume, the impact of the system volume of 0 on the display device 200 responding to the user's far-field voice commands can be avoided, thereby improving the far-field voice interaction effect between the display device 200 and the user.
[0400] In other embodiments, when the display device 200 is in a non-mute state and the current system volume is 0, the display device 200 responds to the wake-up word, is in an awake state, automatically increases the system volume to a preset volume, and displays a pop-up window. The pop-up window is used to prompt the user that he has been awakened, and the system volume is 0, but it has currently been increased to the first volume. The first volume can be the default system volume, or it can be the system volume saved when the silent mode is started. At this point, the display device 200 is in an awake state and can interact with the user in response to other far-field voice commands of the user. In this way, the display device 200 does not play the wake-up prompt sound, but uses a pop-up window as a transition to prompt the user that he has been awakened, so as not to disturb the user. Moreover, the display device 200 can continue to interact with the user without the user waking up again.
[0401] The display device 200 may also perform a wake-up response according to the process shown in FIG45 , which may include:
[0402] S3201: When the screen is on and in silent mode, collect user voice.
[0403] S3202: When it is recognized that the user voice contains the wake-up word, the silent mode is released, and after the silent mode is released, the to-be-played audio or prompt voice of the wake-up prompt sound is played at the first volume.
[0404] Step S3201 is similar to step S501, and the process of identifying the wake-up word in step S3202 is similar to steps S601-S602, which will not be repeated here.
[0405] In some embodiments, when the display device 200 is in silent mode, the display device 200 responds to the wake-up word, enters the awake state, automatically exits silent mode, and plays the wake-up prompt tone at a first volume. The first volume can be the default system volume or the system volume value saved when silent mode is activated. In this way, the display device 200 can interact well with the user through the wake-up prompt tone, and the display device 200 can continue to interact with the user without the user waking up again.
[0406] In other embodiments, when the display device 200 is in silent mode, the display device 200 responds to the wake-up word and is in an incomplete wake-up state, cancels the silent mode, and after canceling the silent mode, plays the prompt voice at a preset volume, and after playing the first prompt voice, restores the silent mode to maintain the silent mode. The incomplete wake-up state means that the display device 200 responds to the wake-up word, starts the first prompt voice playback function, and after playing the first prompt voice, exits the interaction with the user and does not respond to other far-field voice commands of the user. The preset volume value can be the default system volume or the system volume value saved when the silent mode is started. The prompt voice is used to remind the user that it is currently in silent mode and needs to be woken up again after canceling the silent mode. In this way, it is possible to avoid destroying the silent mode set by the user, and to prompt the user through the first prompt voice with a stronger sense of interaction, and to avoid the impact of the silent mode on the display device 200 responding to the user's far-field voice commands by requiring it to wake up again after canceling the silent mode, thereby improving the far-field voice interaction effect between the display device 200 and the user.
[0407] The display device 200 can increase the system volume based on the signal-to-noise ratio (SNR) of the user's voice. If the SNR of the user's voice is greater than a specified SNR threshold, the system volume is increased by a first amount, resulting in a lower volume. If the SNR of the user's voice is less than the specified SNR threshold, the system volume is increased by a second amount, which is greater than the first amount, resulting in a higher volume. The display device 200 can also increase the system volume based on the difference between the wake-up value of the user's voice and the wake-up value threshold. If the difference is greater than a specified difference threshold, the system volume is increased by the first amount, resulting in a higher volume. If the SNR of the user's voice is less than a specified difference threshold, the system volume is increased by a second amount, which is less than the first amount, resulting in a lower volume. The display device 200 can also increase the system volume based on the SNR of the user's voice and the difference between the wake-up value of the user's voice and the wake-up value threshold. If the difference is greater than the specified difference threshold and the SNR of the user's voice is less than the specified SNR threshold, the system volume is increased by the first amount, resulting in a higher volume. If the difference is less than the specified difference threshold and the signal-to-noise ratio of the user voice is greater than the specified signal-to-noise ratio threshold, the system volume is increased by a second amount, the second amount is less than the first amount, and the first volume is relatively low.
[0408] The display device 200 may also perform a wake-up response according to the process shown in FIG46 , which may include:
[0409] S3301: When the screen is on, in non-mute mode, and the current system volume is 0, collect user voice.
[0410] S3302: When it is recognized that the user voice contains the wake-up word, the system volume is increased to a preset volume, and the to-be-played audio or prompt voice of the wake-up prompt sound is played at the preset volume.
[0411] Step S3301 is similar to step S501, and the process of identifying the wake-up word in step S3302 is similar to steps S601-S602, which will not be repeated here.
[0412] In some embodiments, when the display device 200 is not in a muted state and the current system volume is 0, the display device 200 responds to the wake-up word and enters a wake-up state, automatically increases the system volume to a preset volume, and plays the wake-up prompt tone at the preset volume. The third volume can be the default system volume. In this way, the display device 200 can interact well with the user through the wake-up prompt tone, and the display device 200 can continue to interact with the user without the user waking up again.
[0413] In other embodiments, when the display device 200 is in a non-muted state and the current system volume is 0, the display device 200 is in an incompletely awakened state in response to the wake-up word, automatically increases the system volume to a preset volume, and plays the prompt voice at the preset volume. The incompletely awakened state means that the display device 200 responds to the wake-up word, starts the prompt voice playback function, and after playing the prompt voice, exits the interaction with the user and does not respond to other far-field voice commands of the user. The preset volume can be the default system volume. The prompt voice is used to prompt the user that the system volume is 0 and that it needs to be woken up again after increasing the volume. In this way, the user can be prompted by a first prompt voice with a stronger sense of interaction, and by needing to wake up again after increasing the system volume, the influence of the system volume of 0 on the display device 200 responding to the user's far-field voice commands can be avoided, thereby improving the far-field voice interaction effect between the display device 200 and the user.
[0414] The operating modes of the display device 200 include a speaker mode. When the display device 200 is in speaker mode, the display is in an off state, but sounds can be played. When the display device 200 is in speaker mode, if the display device 200 is in the first state when it is awakened, the wake-up prompt sound is not played due to the limitation of the first state, and the user may mistakenly believe that the display device 200 has not been awakened.
[0415] To solve the above problem, the display device 200 may respond to the wake-up word according to the process shown in FIG47 , which may include:
[0416] S3401: When the screen is off and in silent mode, collect user voice.
[0417] S3402: When the user's voice is recognized to include a wake-up word, detect whether the device is in speaker mode.
[0418] Step S3401 is similar to step S501, and the process of identifying the wake-up word in step S3402 is similar to steps S601-S602, which will not be repeated here.
[0419] The display device 200 can determine whether it is in speaker mode by identifying the status of the audio service. If the audio service is started, it is in audio mode. If the audio service is turned off, it is in non-speaker mode. The display device 200 can also identify whether it is in speaker mode based on device status information, such as the display device 200 obtains device status information through the device management module, and the device status information includes a flag bit for the speaker mode. When the flag bit carries the number 1, the display device 200 is in speaker mode. When the flag bit is empty or carries the number 0, the display device 200 is in non-speaker mode. The display device 200 can also identify whether it is in speaker mode based on other methods, which are not limited here.
[0420] The display device 200 performs a process of obtaining the audio to be played of the wake-up prompt sound when the wake-up value is greater than or equal to the wake-up value threshold.
[0421] The display device 200 can identify whether it is in speaker mode before obtaining the audio to be played for the wake-up prompt tone. If the display device 200 identifies that it is in non-speaker mode, it does not obtain the audio to be played for the wake-up prompt tone. If the display device 200 identifies that it is in speaker mode, it obtains the audio to be played for the wake-up prompt tone. This can avoid occupying the computing resources of the display device 200 when it is in non-speaker mode.
[0422] The display device 200 can identify whether it is in speaker mode during the process of obtaining the audio to be played for the wake-up prompt tone. For example, obtaining the audio to be played for the wake-up prompt tone and identifying whether it is in speaker mode are respectively executed in two parallel processes / threads. If the display device 200 identifies that it is in non-speaker mode, the execution of obtaining the audio to be played for the wake-up prompt tone is terminated; if the display device 200 identifies that it is in speaker mode, the execution of obtaining the audio to be played for the wake-up prompt tone is executed. In this way, when it is identified that it is in non-speaker mode, the computing resources of the display device 200 can be avoided from being continuously occupied. It is also possible to quickly obtain the audio to be played for the wake-up prompt tone when it is identified that it is in speaker mode.
[0423] After the display device 200 completes the process of obtaining the audio to be played for the wake-up prompt tone, it can identify whether it is in speaker mode. Therefore, when the display device 200 identifies that it is not in speaker mode, once it enters a working mode capable of playing the wake-up prompt tone, it can directly use the audio to be played for the wake-up prompt tone after being awakened, thereby improving the efficiency of playing the wake-up prompt tone.
[0424] S3403: If the device is in speaker mode, keep the screen off and cancel the mute mode. After canceling the mute mode, play the waiting audio of the wake-up prompt tone at the first volume.
[0425] The display device 200 remains in the screen-off state and responds to the user's voice by playing a wake-up prompt sound to prompt the user that he has been awakened.
[0426] Before playing the audio to be played for the wake-up prompt sound, the display device 200 obtains the audio to be played for the wake-up prompt sound.
[0427] The display device 200 can obtain the audio to be played of the wake-up prompt sound with a specified amplitude, and the specified amplitude is less than the amplitude threshold. The display device 200 plays the audio to be played of the wake-up prompt sound at the current system volume or the default system volume. Since the amplitude of the audio to be played of the wake-up prompt sound is less than the amplitude threshold, the volume of the audio to be played of the wake-up prompt sound played at the current system volume or the default system volume is also relatively small. The display device 200 directly obtains the audio to be played of the wake-up prompt sound with a specified amplitude, which can effectively improve the efficiency of obtaining the audio to be played of the wake-up prompt sound, and thereby improve the efficiency of playing the audio to be played of the wake-up prompt sound.
[0428] The display device 200 can adjust the amplitude of the audio of the wake-up prompt sound according to the first volume and the relevant parameters of the user's voice to obtain the audio to be played of the wake-up prompt sound. Among them, the relevant parameters of the user's voice include the signal-to-noise ratio of the user's voice and / or the difference between the wake-up value of the user's voice and the wake-up value threshold. The display device 200 responds to the user's wake-up while reducing the sense of intrusion to the user. The display device 200 dynamically obtains the audio to be played of the wake-up prompt sound according to the first volume and the relevant parameters of the user's voice, so that the volume of the played wake-up prompt sound is more in line with the user's needs and improves the user's experience.
[0429] The display device 200 may also respond to the wake-up word according to the process shown in FIG48 , which may include:
[0430] S3501: When the screen is off, in non-mute mode, and the current system volume is 0, collect user voice.
[0431] S3502: When the user's voice is recognized to contain a wake-up word, detect whether the device is in speaker mode.
[0432] Steps S3501 and S3502 are similar to steps S3401 and S3402 and are not described in detail here.
[0433] S3503: If the device is in speaker mode, keep the screen off and increase the system volume to the second volume to play the waiting audio of the wake-up prompt tone at the second volume.
[0434] If it is in the speaker mode, the display device 200 keeps the screen off and responds to the user's voice by playing a wake-up prompt sound to prompt the user that he has been awakened.
[0435] The display device 200 first increases the system volume to a second volume, which may be a default system volume.
[0436] In some embodiments, the second volume can be less than a preset volume threshold. The display device 200 can determine the second volume based on relevant parameters of the user's voice, including the signal-to-noise ratio of the user's voice and / or the difference between the wake-up value of the user's voice and the wake-up value threshold. Because the second volume is less than the volume threshold, the system volume of the audio of the wake-up prompt sound played by the display device 200 is also lower. The display device 200 can respond to the user's wake-up while reducing the sense of intrusion to the user.
[0437] Based on steps S3401-S3403, or based on steps S3501-S3503, after playing the audio of the wake-up prompt tone, the display device 200 also plays a prompt message through the speaker based on the current system volume, where the prompt message includes information for prompting the user to adjust the system volume (for ease of distinction, referred to as the first prompt message). The current system volume is the system volume of the audio of the wake-up prompt tone.
[0438] In which, when the current system volume is greater than or equal to the first volume threshold, the first prompt message is used to prompt the user to lower the system volume. When the current system volume is less than the second volume threshold, the first prompt message is used to prompt the user to increase the system volume. In which, the first volume threshold is greater than the second volume threshold. Taking the display device 200 as a TV as an example, the display device 200 can play a wake-up prompt sound when it is awakened in the screen-off state and the first state as shown in ① in Figure 49. If the current system volume is greater than or equal to the first volume threshold, the first prompt message is also played, such as "The system volume is too high, it is recommended to lower the system volume." The display device 200 can play a wake-up prompt sound when it is awakened in the screen-off state and the first state as shown in ② in Figure 49. If the current system volume is less than the second volume threshold, the first prompt message is also played, such as "The system volume is too low, it is recommended to increase the system volume."
[0439] After playing the wake-up prompt sound, the display device 200 may also play a prompt message, which includes information for reminding the user that the user is currently in speaker mode (for ease of distinction, referred to as the second prompt message). The prompt message played by the display device 200 may include the first prompt message and the second prompt message.
[0440] Taking the display device 200 as a television as an example, the display device 200 can, as shown in ① in Figure 50, play a wake-up prompt sound when it is awakened when in the screen-off state and the first state. If the current system volume is greater than or equal to the first volume threshold, a first prompt message is also played, such as "The system volume is too high, it is recommended to lower the system volume." The display device 200 can also play a second prompt message, such as "Currently in speaker mode." The display device 200 can, as shown in ② in Figure 50, play a wake-up prompt sound when it is awakened when in the screen-off state and the first state. If the current system volume is less than the second volume threshold, the first prompt message is also played, such as "The system volume is too low, it is recommended to increase the system volume." The display device 200 can also play a second prompt message, such as "Currently in speaker mode." In this way, the user can quickly determine the reason why the display device 200 is currently in the screen-off state.
[0441] In some embodiments, if the display device 200 is in silent mode, the display device 200 is in an incompletely awakened state in response to the wake-up word, maintains the screen-off state, cancels the silent mode, and plays the prompt voice at the first volume after canceling the silent mode, and after playing the prompt voice, restores the silent mode. The incompletely awakened state means that the display device 200 responds to the wake-up word, starts the prompt voice playback function, and after playing the prompt voice playback function, exits the interaction with the user and does not respond to other far-field voice commands of the user. The first volume can be the default system volume or the system volume saved when the silent mode is started. The prompt voice is used to prompt the user that he is in speaker mode, in silent mode, and needs to be woken up again after canceling the silent mode.
[0442] In other embodiments, if the display device 200 is in a non-silent mode and the current system volume is 0, the display device 200 is in an incomplete awakening state in response to the wake-up word, maintains the screen-off state, increases the system volume to the first volume, and plays the prompt voice at the first volume, and after playing the prompt voice, readjusts the system volume to 0. The incomplete awakening state means that the display device 200 responds to the wake-up word, starts the prompt voice playback function, and after playing the prompt voice playback function, exits the interaction with the user and does not respond to other far-field voice commands of the user. The first volume can be the default system volume. The prompt voice is used to prompt the user that the user is in speaker mode, is in non-silent mode but the current system volume is 0, and needs to be woken up again after increasing the volume.
[0443] The display device 200 may also respond to the wake-up word according to the process shown in FIG51 , which may include:
[0444] S3801, when the screen is in the off state and in the first state, collecting user voice.
[0445] S3802: When the user's voice is recognized to contain a wake-up word, detect whether the device is in speaker mode.
[0446] Step S3801 is similar to step S501, and the process of identifying the wake-up word in step S3802 is similar to steps S601-S602, which will not be repeated here.
[0447] S3803: If the device is in speaker mode, the display is adjusted to a bright screen state, the middle page is displayed, and a pop-up window is displayed on the middle page according to the first state.
[0448] The display device 200 is in the awake state in response to the wake-up word. The display device 200 powers on the display and adjusts the display to the bright screen state. The display device 200 does not exit the speaker mode. In order to simulate the screen-off state when in the speaker mode in the bright screen state, the display device 200 displays the middle page. Referring to Figure 52, the display device 200 displays the middle page 3901. The middle page 3901 is black. When the middle page 3901 is displayed, it looks like the display device 200 is in the screen-off state. At this time, a pop-up window is displayed on the middle page. As shown in Figure 52, a pop-up window 3902 is displayed on the middle page 3901. The pop-up window is used to prompt the user that he has been awakened and is in the speaker mode and the first state. The pop-up window can be small in size and displayed in non-center positions such as the top corner and frame of the display to reduce the sense of intrusion to the user on the basis of prompting the user.
[0449] In some embodiments, if the first state is that the display device 200 is in silent mode, the display device 200 is adjusted to a bright screen state, silent mode is released, the system volume is adjusted to a preset volume, the intermediate page is displayed, and a pop-up window is displayed on the intermediate page. The preset volume can be the default system volume or the system volume saved when silent mode is turned on. The pop-up window is used to prompt the user that the current state is speaker mode and silent mode, but that silent mode has been released.
[0450] In other embodiments, if the first state is that the display device 200 is in non-mute mode and the current system volume is 0, the display device 200 is adjusted to a bright screen state, the system volume is adjusted to a preset volume, the intermediate page is displayed, and a pop-up window is displayed on the intermediate page. The preset volume can be the default system volume. The pop-up window is used to prompt the user that the current state is speaker mode, the system volume is 0, but the volume has been increased.
[0451] When the intermediate page is displayed after the display device 200 is awakened, the display device 200 may respond to other far-field voice commands of the user.
[0452] In some embodiments, the display device 200 exits the intermediate page and displays the video page in response to a user's far-field voice command to play a video. For example, if the far-field voice command is "Play Movie A," the display device 200 exits the intermediate page and displays the video of Movie A in response to the far-field voice command.
[0453] In other embodiments, the display device 200 maintains the display of the middle page and plays the audio in response to a far-field voice command from the user instructing the user to play the audio. For example, if the far-field voice command is "Play Song B," the display device 200 maintains the display of the middle page and plays the audio of Song B in response to the far-field voice command.
[0454] In other embodiments, the display device 200, in response to a user's far-field voice command instructing the user to play audio, adjusts the display to a screen-off state and plays the audio. For example, if the far-field voice command is "today's weather," the display device 200, in response to the far-field voice command, adjusts the display to a screen-off state and plays weather-related audio.
[0455] In other embodiments, the display device 200 is in an incompletely awake state in response to the wake-up word.
[0456] If the first state is that the display device 200 is in silent mode, the display device 200 is in an incompletely awakened state in response to the wake-up word, adjusts to a bright screen state, does not cancel silent mode, does not adjust the system volume, displays an intermediate page, and displays a pop-up window on the intermediate page. The incompletely awakened state means that the display device 200 activates the pop-up window function in response to the wake-up word, and after displaying the pop-up window, exits interaction with the user and does not respond to other far-field voice commands of the user. The pop-up window is used to remind the user that the current state is speaker mode and silent mode, and needs to be awakened again after canceling the silent mode.
[0457] If the first state is that the display device 200 is in non-silent mode and the system volume is 0, the display device 200 is in an incomplete wake-up state in response to the wake-up word, adjusts to a bright screen state, does not adjust the system volume, displays an intermediate page, and displays a pop-up window on the intermediate page. The incomplete wake-up state means that the display device 200 activates the pop-up window function in response to the wake-up word, and after displaying the pop-up window, exits interaction with the user and does not respond to other far-field voice commands of the user. The pop-up window is used to remind the user that the current state is speaker mode and the system volume is 0, and that it needs to be woken up again after increasing the volume.
[0458] For ease of explanation, the above description has been made in conjunction with specific embodiments. However, the above discussion of some embodiments is not intended to be exhaustive or to limit the embodiments to the specific forms disclosed above. Based on the above teachings, various modifications and variations can be obtained. The above embodiments are selected and described to better explain the content of this application, thereby enabling those skilled in the art to better use the embodiments.
Claims
1. A display device, include: The sound collector is configured to: collect user voice; speaker; A memory configured to: store computer instructions and data associated with the display device; At least one processor is connected to the sound collector, the speaker and the memory, and is configured to execute the computer instructions to enable the display device to execute: When it is recognized that the user voice contains a wake-up word, a signal-to-noise ratio of the user voice and a current system volume are obtained, where the signal-to-noise ratio is used to characterize the noisiness of the environment in which the display device is located; According to the signal-to-noise ratio and the current system volume, adjusting the amplitude of the audio of the wake-up prompt tone to obtain the audio to be played of the wake-up prompt tone; The speaker is controlled to play the to-be-played audio of the wake-up prompt tone at the current system volume.
2. The display device according to claim 1, wherein the at least one processor adjusts the amplitude of the audio of the wake-up prompt sound according to the signal-to-noise ratio and the current system volume to obtain the audio to be played of the wake-up prompt sound, and is configured to execute the computer instruction to enable the display device to execute: Obtaining the audio of the wake-up prompt tone; According to the signal-to-noise ratio and the current system volume, the amplitude of the audio of the wake-up prompt tone is adjusted to obtain the audio to be played of the wake-up prompt tone.
3. The display device according to claim 1, wherein the at least one processor adjusts the amplitude of the audio of the wake-up prompt sound according to the signal-to-noise ratio and the current system volume to obtain the audio to be played of the wake-up prompt sound, and is configured to execute the computer instruction to enable the display device to execute: Determining, according to the signal-to-noise ratio and the current system volume, an amplitude of the to-be-played audio of the wake-up prompt tone; According to the amplitude of the to-be-played audio of the wake-up prompt sound, a preset audio of the wake-up prompt sound with a corresponding amplitude is obtained.
4. The display device according to any one of claims 1 to 3, wherein the at least one processor adjusts the amplitude of the audio of the wake-up prompt sound according to the signal-to-noise ratio and the current system volume to obtain the audio to be played of the wake-up prompt sound, and is configured to execute the computer instruction to enable the display device to execute: When the signal-to-noise ratio is greater than or equal to a first signal-to-noise ratio threshold, and the current system volume is greater than or equal to a first volume threshold, the amplitude of the audio of the wake-up prompt sound is reduced to obtain the audio to be played of the wake-up prompt sound.
5. The display device according to claim 4, wherein the at least one processor adjusts the amplitude of the audio of the wake-up prompt sound according to the signal-to-noise ratio and the current system volume to obtain the audio to be played of the wake-up prompt sound, and is configured to execute the computer instruction to enable the display device to execute: When the signal-to-noise ratio is less than a second signal-to-noise ratio threshold and the current system volume is less than a second volume threshold, increasing the amplitude of the audio of the wake-up prompt sound to obtain the audio to be played of the wake-up prompt sound, in, The first signal-to-noise ratio threshold is greater than the second signal-to-noise ratio threshold, and the first volume threshold is greater than the second volume threshold.
6. The display device according to claim 5, wherein the at least one processor adjusts the amplitude of the audio of the wake-up prompt sound according to the signal-to-noise ratio and the current system volume to obtain the audio to be played of the wake-up prompt sound, and is further configured to execute the computer instruction to enable the display device to execute: When the signal-to-noise ratio is greater than or equal to the first signal-to-noise ratio threshold and the current system volume is less than the first volume threshold, or when the signal-to-noise ratio is less than the first signal-to-noise ratio threshold and the signal-to-noise ratio is greater than or equal to the second signal-to-noise ratio threshold, or when the signal-to-noise ratio is less than the second signal-to-noise ratio threshold and the current system volume is greater than or equal to the second volume threshold, the amplitude of the audio of the wake-up prompt sound is not adjusted, and the first signal-to-noise ratio threshold is greater than the second signal-to-noise ratio threshold.
7. The display device according to claim 6, further comprising: include: The display is configured to be: in a bright screen state; The at least one processor is further configured to execute the computer instructions to cause the display device to perform: According to the current system volume, controlling the display to display a pop-up window, the pop-up window including prompt information, the prompt information being used to prompt the user to adjust the volume of the wake-up prompt tone by adjusting the system volume; Wherein, when the current system volume is less than a third volume threshold, the prompt information is used to prompt to increase the system volume to increase the volume of the wake-up prompt sound, and the third volume threshold is less than or equal to the second volume threshold; When the current system volume is greater than a fourth volume threshold, the prompt information is used to prompt to lower the system volume to lower the volume of the wake-up prompt sound, and the fourth volume threshold is greater than or equal to the first volume threshold.
8. The display device according to claim 1, wherein the at least one processor is further configured to execute the computer instructions to cause the display device to perform: When it is recognized that the user voice contains a wake-up word, obtaining a difference between the wake-up value corresponding to the user voice and the wake-up value threshold and a current system volume, wherein the difference is used to characterize a probability that the display device is mistakenly awakened; According to the difference between the wake-up value and the wake-up value threshold and the current system volume, adjusting the amplitude of the audio of the wake-up prompt sound to obtain the audio to be played of the wake-up prompt sound; The speaker is controlled to play the to-be-played audio of the wake-up prompt tone at the current system volume.
9. The display device according to claim 8, wherein the at least one processor adjusts the amplitude of the audio of the wake-up prompt sound according to the difference between the wake-up value and the wake-up value threshold and the current system volume to obtain the audio to be played of the wake-up prompt sound, and is configured to execute the computer instruction to enable the display device to execute: Obtaining the audio of the wake-up prompt tone; According to the difference between the wake-up value and the wake-up value threshold and the current system volume, the amplitude of the audio of the wake-up prompt sound is adjusted to obtain the audio to be played of the wake-up prompt sound.
10. The display device according to claim 8, wherein the at least one processor adjusts the amplitude of the audio of the wake-up prompt sound according to the difference between the wake-up value and the wake-up value threshold and the current system volume to obtain the to-be-played audio of the wake-up prompt sound, and is configured to execute the computer instruction to enable the display device to execute: Determining the amplitude of the audio of the wake-up prompt tone according to the difference between the wake-up value and the wake-up value threshold and the current system volume; According to the amplitude of the audio of the wake-up prompt tone, a preset audio of the wake-up prompt tone with a corresponding amplitude is obtained.
11. The display device according to any one of claims 8 to 10, wherein the at least one processor adjusts the amplitude of the audio of the wake-up prompt sound according to the difference between the wake-up value and the wake-up value threshold and the current system volume to obtain the audio to be played of the wake-up prompt sound, and is configured to execute the computer instruction to enable the display device to execute: When the difference is less than a first difference threshold and the system volume is greater than a first volume threshold, the amplitude of the audio of the wake-up prompt sound is reduced to obtain the audio to be played of the wake-up prompt sound.
12. The display device according to claim 11, wherein the at least one processor adjusts the amplitude of the audio of the wake-up prompt sound according to the difference between the wake-up value and the wake-up value threshold and the current system volume to obtain the to-be-played audio of the wake-up prompt sound, and is further configured to execute the computer instruction to enable the display device to execute: When the difference is greater than or equal to a second difference threshold and the system volume is less than a second volume threshold, increasing the amplitude of the audio of the wake-up prompt sound to obtain the audio to be played of the wake-up prompt sound, in, The second difference threshold is greater than the first difference threshold, and the second volume threshold is less than the first volume threshold.
13. The display device according to claim 12, wherein the at least one processor adjusts the amplitude of the audio of the wake-up prompt sound according to the difference between the wake-up value and the wake-up value threshold and the current system volume to obtain the to-be-played audio of the wake-up prompt sound, and is further configured to execute the computer instruction to enable the display device to execute: When the difference is less than the first difference threshold and the system volume is less than or equal to the first volume threshold, or when the difference is greater than or equal to the first difference threshold and the difference is less than the second difference threshold, or when the difference is greater than or equal to the second difference threshold and the current system volume is greater than or equal to the second volume threshold, the amplitude of the audio of the wake-up prompt sound is not adjusted.
14. The display device according to claim 13, further comprising: include: The display is configured to be: in a bright screen state; The at least one processor is further configured to execute the computer instructions to enable the display device to execute: according to the current system volume, control the display to display a pop-up window, the pop-up window including prompt information, the prompt information is used to prompt the user to adjust the volume of the wake-up prompt sound by adjusting the system volume; Wherein, when the current system volume is less than a third volume threshold, the prompt information is used to prompt to increase the system volume to increase the volume of the wake-up prompt sound, and the third volume threshold is less than or equal to the second volume threshold; When the current system volume is greater than a fourth volume threshold, the prompt information is used to prompt to lower the system volume to lower the volume of the wake-up prompt sound, and the fourth volume threshold is greater than or equal to the first volume threshold.
15. The display device according to claim 14, wherein the at least one processor controls the display to display a pop-up window according to the current system volume, and is further configured to execute the computer instruction to enable the display device to execute: Displaying the pop-up window according to the difference; in, When the difference is less than the first difference threshold, a pop-up window of a first style is displayed; when the difference is greater than or equal to the second difference threshold, a pop-up window of a second style is displayed.
16. The display device according to claim 1, further comprising: include: The display is configured to be: in a bright screen state; The speaker is configured to be in a first state, wherein the display device is in a silent mode or the display device is in a non-silent mode and the current system volume is 0; The at least one processor is configured to execute the computer instructions to cause the display device to perform: When it is recognized that the user voice contains a wake-up word, the display is controlled to display a pop-up window according to the first state, and the pop-up window includes first information, and the first information is used to prompt that the display device is awakened and is in the silent mode.
17. The display device according to claim 16, wherein when the at least one processor recognizes that the user voice contains a wake-up word, the at least one processor controls the display to display a pop-up window according to the first state, and is configured to execute the computer instruction to enable the display device to execute: Controlling the display to display the pop-up window according to the difference between the wake-up value corresponding to the user voice and the wake-up value threshold; in, When the difference is less than a first difference threshold, controlling the display to display a pop-up window of a first style; When the difference is greater than or equal to a second difference threshold, the display is controlled to display a pop-up window of a second style, and the first difference threshold is less than the second difference threshold.
18. The display device according to claim 16 or 17, wherein when in the silent mode, the at least one processor controls the display to display a pop-up window, and is further configured to execute the computer instructions to enable the display device to execute: Get the current system volume; According to the current system volume, the display is controlled to display second information in the pop-up window, where the second information is used to prompt the user to adjust the system volume.
19. The display device according to claim 18, wherein the at least one processor controls the display to display the second information in the pop-up window according to the current system volume, and is configured to execute the computer instruction to enable the display device to execute: When the current system volume is greater than or equal to the first volume threshold, the second information is used to prompt the user to turn down the system volume; When the current system volume is less than a second volume threshold, the second information is used to prompt the user to increase the system volume, and the first volume threshold is greater than the second volume threshold.
20. The display device according to claim 16, wherein the at least one processor is further configured to execute the computer instructions to cause the display device to perform: When the display device is in a non-mute mode and the current system volume is 0, the display is controlled to display prompt information, where the prompt information is used to prompt that the current system volume is 0.
21. The display device according to claim 1, further comprising: include: The display is configured to be: in a bright screen state; The speaker is configured to be in a silent mode; The at least one processor is configured to execute the computer instructions to cause the display device to perform: When it is recognized that the user voice contains the wake-up word, the silent mode is released, and after the silent mode is released, the to-be-played audio of the wake-up prompt sound is played at a first volume.
22. The display device according to claim 1, further comprising: include: The display is configured to be: in a bright screen state; The speaker is configured to be in a silent mode; The at least one processor is configured to execute the computer instructions to cause the display device to perform: When it is recognized that the user voice contains a wake-up word, the display device is adjusted to an incomplete wake-up state and the silent mode is released. After the silent mode is released, a prompt voice is played at a first volume, and after playing the prompt voice, the display device is readjusted to the silent mode, wherein the prompt voice is used to prompt the user to release the silent mode and wake up the display device again. In the incomplete wake-up state, the display device supports playing the prompt voice but does not support responding to other far-field voice commands of the user.
23. The display device according to claim 1, further comprising: include: The display is configured to be: in a bright screen state; The speaker is configured to be in a non-mute mode and the current system volume is 0; The at least one processor is configured to execute the computer instructions to cause the display device to perform: When it is recognized that the user voice contains the wake-up word, the system volume is increased to a preset volume, and the to-be-played audio of the wake-up prompt sound is played at the preset volume.
24. The display device according to claim 1, further comprising: include: The display is configured to be: in a bright screen state; The speaker is configured to be in a non-mute mode and the current system volume is 0; The at least one processor is configured to execute the computer instructions to cause the display device to perform: When it is recognized that the user voice contains a wake-up word, it is adjusted to an incomplete wake-up state, and the system volume is increased to a preset volume, a prompt voice is played at the preset volume, and after playing the prompt voice, the system volume is adjusted to 0, and the prompt voice is used to prompt the user to increase the system volume and wake up the display device again, wherein, in the incomplete wake-up state, the display device supports playing the prompt voice, but does not support responding to other far-field voice commands of the user.
25. The display device according to claim 1, further comprising: include: The display is configured to be: in a screen-off state; The speaker is configured to be in a silent mode or the display device is in a non-silent mode and the current system volume is 0; The at least one processor is configured to execute the computer instructions to cause the display device to perform: When recognizing that the user voice includes a wake-up word, detecting whether the user is in a speaker mode; If it is in the speaker mode, the display is controlled to maintain the screen-off state and the silent mode is released, and after the silent mode is released, the speaker is controlled to play the audio to be played of the wake-up prompt sound at the first volume. 26 . The display device according to claim 25 , wherein the first volume is a default system volume or a system volume in the silent mode.
27. The display device according to claim 25, if in the speaker mode, the at least one processor is further configured to execute the computer instructions so that the display device executes: Obtaining the to-be-played audio of the wake-up prompt tone with a specified amplitude, where the specified amplitude is less than an amplitude threshold; The speaker is controlled to play the to-be-played audio of the wake-up prompt tone with the specified amplitude.
28. The display device according to claim 25, if in the speaker mode, the at least one processor is further configured to execute the computer instructions so that the display device executes: According to the first volume and the relevant parameters of the user voice, the amplitude of the audio of the wake-up prompt sound is adjusted to obtain the audio to be played of the wake-up prompt sound, in, The parameters related to the user voice include a signal-to-noise ratio of the user voice, and / or a difference between the wake-up value and the wake-up value threshold.
29. The display device according to any one of claims 25 to 28, if in the speaker mode, the at least one processor is further configured to execute the computer instructions so that the display device executes: Controlling the speaker to play a first prompt message, where the first prompt message is used to prompt a user to adjust the system volume; in, When the current system volume is greater than or equal to a first volume threshold, the first prompt information is used to prompt the user to lower the system volume; When the current system volume is less than a second volume threshold, the first prompt information is used to prompt the user to increase the system volume, and the second volume threshold is less than the first volume threshold.
30. The display device according to claim 1, further comprising: include: The display is configured to be in a screen-off state; The speaker is configured as: in non-mute mode and the current system volume is 0; The at least one processor is further configured to execute the computer instructions to cause the display device to perform: When recognizing that the user voice includes a wake-up word, detecting whether the user is in a speaker mode; If it is in the speaker mode, the display is controlled to maintain the screen-off state, and the system volume is increased to the second volume, and the audio to be played of the wake-up prompt sound is played at the second volume.
31. The display device according to claim 1, further comprising: include: The display is configured to be in a screen-off state; The speaker is configured to be in a first state, wherein the display device is in a silent mode or the display device is in a non-silent mode and the current system volume is 0; The at least one processor is further configured to execute the computer instructions to cause the display device to perform: When recognizing that the user voice contains a wake-up word, detecting whether the user is in speaker mode; If it is in the speaker mode, the display is adjusted to the bright screen state, and the display is controlled to display the middle page, and a pop-up window is displayed on the middle page according to the first state, wherein the middle page is black, and the pop-up window is used to prompt the user that the display device has been awakened.
32. The display device according to claim 31, wherein the at least one processor is further configured to execute the computer instructions to cause the display device to perform: In response to a far-field voice command from a user indicating to play a video, controlling the display to exit the intermediate page and display a page of the video; Alternatively, in response to a far-field voice instruction from a user to play audio, the display is controlled to keep displaying the intermediate page, or the display is adjusted to the screen-off state, and the speaker is controlled to play the audio.
33. The display device according to any one of claims 1-3, 8-10, 16, 17, 21-28, 30-32, wherein the at least one processor is further configured to execute the computer instructions to cause the display device to execute: Calculating a wake-up value corresponding to the user voice, where the wake-up value is used to represent the degree of similarity between the user voice and a standard voice of the wake-up word; Determining whether the user voice contains a wake-up word according to the wake-up value and the wake-up value threshold; in, If the wake-up value is greater than or equal to the wake-up value threshold, it is determined that the user voice contains the wake-up word; if the wake-up value is less than the wake-up value threshold, it is determined that the user voice does not contain the wake-up word.
34. The display device according to claim 33, wherein the at least one processor is configured to execute the computer instructions to cause the display device to perform: Input the user voice into the model to obtain a score for each syllable in the user voice, wherein the model is used to calculate a wake-up value, and the score for each syllable is used to represent a degree of similarity to a corresponding syllable in the standard voice of the wake-up word; The model is used to calculate the wake-up value corresponding to the user voice according to the score of each syllable in the user voice.
35. The display device according to claim 1, further comprising: include: The indicator light is configured to: present a plurality of prompt light effects to indicate that the display device is in different terminal states, wherein the terminal state is one of a standby state, a pre-start state, and a running state; monitor; The at least one processor includes a first processor and a second processor; In the standby state, the first processor is turned on and the second processor is turned off; the first processor is configured to execute the computer instructions to enable the display device to execute: Sending a first level signal to the indicator light so that the indicator light presents a first prompt light effect of a standby state; In response to the user voice, performing a first wake-up word recognition on the user voice; when the user voice passes the first wake-up word recognition, sending a wake-up instruction and the user voice to the second processor to control the display device to enter the pre-start state; the wake-up instruction is used to power on at least a portion of the second processor; In the pre-start state, the first processor is turned on, and at least a portion of the second processor is turned on; The first processor is configured to execute the computer instructions to cause the display device to perform: Sending a pulse signal to the indicator light so that the indicator light presents a second prompt light effect in a pre-start state; In the running state, the first processor is turned on, and the second processor is turned on; The second processor is configured to execute the computer instructions to cause the display device to perform: Sending a second level signal to the indicator light so that the indicator light presents a third prompt light effect of the operating state; in response to the wake-up instruction, performing a second wake-up word recognition on the user voice; when the user voice is recognized by the second wake-up word, lighting up the display, and controlling the display device to enter the operating state.
36. The display device of claim 35, wherein the second processor is configured to execute the computer instructions to cause the display device to: Monitor the pulse signal sent by the first processor; when the last pulse wave of the pulse signal is monitored, send the second level signal to the indicator light to control the indicator light to present a third prompt light effect of the operating status.
37. The display device of claim 35, wherein the first processor is configured to execute the computer instructions to cause the display device to: In response to a high level signal when the display device is powered on, detecting a power-on state of the second processor; If the second processor is turned on, the second processor is configured to execute the computer instructions to cause the display device to perform: Sending the second level signal to the indicator light to control the indicator light to present the third prompt light effect; If the second processor is turned off, the first processor is configured to execute the computer instructions to cause the display device to perform: The first level signal is sent to the indicator light to control the indicator light to present the first prompt lighting effect.
38. The display device according to claim 35, wherein in the operating state, the second processor is further configured to execute the computer instructions to enable the display device to execute: receiving a standby instruction input by a user, wherein the standby instruction is used to control the display device to enter a standby state; In response to the standby instruction, reducing the number of chip operating cores of the second processor to shut down the second processor; The standby instruction is forwarded to the first processor.
39. The display device according to claim 35, wherein the first processor performs a first wake-up word recognition on the user voice, and is configured to execute the computer instructions to enable the display device to perform: Analyze the acoustic features of the user's voice, and control the indicator light to present a detection voice prompt light effect; If the user voice includes a wake-up word feature, identifying the first wake-up word of the user voice; If the user voice does not include a wake-up word feature, the first wake-up word recognition of the user voice is not performed, and the indicator light is controlled to present the first prompt light effect.
40. The display device according to claim 35, wherein the first processor performs a first wake-up word recognition on the user voice, and is further configured to execute the computer instructions to enable the display device to perform: Calculating the confidence of the user's voice; When the confidence level is greater than or equal to a preset confidence level threshold, a recognition-free wake-up instruction is sent to the second processor, and the period of the pulse signal is adjusted so that the indicator light presents a prompt light effect of quick startup. ; The recognition-free wake-up instruction is used to notify the second processor to skip the second wake-up word recognition; Control the display device to enter an operating state.
41. The display device of claim 40, wherein the second processor is configured to execute the computer instructions to cause the display device to: In response to the recognition-free wake-up instruction, a voice instruction is generated according to the user voice and the voice instruction is responded to.
42. The display device according to claim 35, wherein the second processor performs a second wake-up word recognition on the user voice, and is further configured to execute the computer instructions to enable the display device to perform: Calculating a wake-up score of the user's voice by using a second wake-up word recognition model, where the complexity of the second wake-up word recognition model is greater than that of the first wake-up word recognition model; If the wake-up score is greater than or equal to the score threshold, recognizing the second wake-up word of the user voice; If the wake-up score is less than the score threshold, the second wake-up word recognition of the user voice is not performed.
43. The display device according to claim 35, when the user voice is not recognized by the second wake-up word, the second processor is further configured to execute the computer instruction to enable the display device to execute: Sending a recognition failure result of the second wake-up word recognition to the first processor, and shutting down the enabled part of the second processor; Controlling the display device to enter a standby state; The first processor is further configured to execute the computer instructions to cause the display device to perform: receiving the recognition failure result; In response to the recognition failure result, a first level signal is sent to the indicator light, so that the indicator light presents the first prompt lighting effect.
44. A voice wake-up method for a display device, include: Collect user voice; When it is recognized that the user voice contains a wake-up word, a signal-to-noise ratio of the user voice and a current system volume are obtained, where the signal-to-noise ratio is used to characterize the noisiness of the environment in which the display device is located; According to the signal-to-noise ratio and the current system volume, adjusting the amplitude of the audio of the wake-up prompt tone to obtain the audio to be played of the wake-up prompt tone; The to-be-played audio of the wake-up prompt tone is played at the current system volume.
