Display device, control device and device awakening method

Abstract

The invention provides a display device, a control device and a device awakening method, and the method comprises the steps: responding to a distance measurement instruction input by the control device, sending a feedback signal to the control device, receiving a distance parameter sent by the control device, determining the amplitude of an audio array according to the distance parameter, and awakening the audio array according to the distance parameter. And adjusting configuration parameters of the sound collector according to the audio array amplitude, when the display device is in a standby mode, obtaining wake-up audio data, performing wake-up voice recognition on the wake-up audio data, and waking up the display device. The distance parameter between the user and the display device is acquired through the control device, and the audio array amplitude of the sound collector is dynamically adjusted according to the distance parameter, so that the sound receiving effect of the sound collector on different distance positions is improved, and the accuracy of awakening the display device by voice is improved.

Description

Technical Field

[0001] The present application relates to the technical field of display devices, and in particular to a display device, a control device, and a device wake-up method. Background Art

[0002] When the display device does not receive a control command for a period of time, it will enter standby mode to save power consumption. In standby mode, the user can wake up the display device by saying a wake-up word. The display device will obtain the user's wake-up word audio through the microphone array and perform wake-up word recognition on the wake-up word audio. If the recognition is successful, the display device will be woken up to switch from standby mode to working mode.

[0003] The user can say the wake-up word at any location in the area where the display device is located, so the distance between the sound source and the microphone array is not fixed. When the user is far away from the display device, if the microphone array has a small sound amplitude, the wake-up word audio received by the display device will be unclear, thus affecting the voice wake-up effect of the display device. Summary of the invention

[0004] The present application provides a display device, a control device, and a device wake-up method to solve the problem that the distance between a user and a display device is not fixed, resulting in unclear received wake-up audio data.

[0005] In a first aspect, some embodiments of the present application provide a display device, including a display, a sound collector, and a controller, wherein the display is configured to display a user interface, the sound collector is configured to acquire audio data, and the controller is configured to:

[0006] In response to a distance measurement instruction input by a user through a control device, sending a feedback signal to the control device;

[0007] Receiving a distance parameter sent by the control device, where the distance parameter is calculated by the control device according to a signal time difference, where the signal time difference is a time difference between a time point when the control device sends the distance measurement instruction and a time point when the control device receives the feedback signal;

[0008] Determining an audio array amplitude according to the distance parameter, and adjusting a configuration parameter of the sound collector according to the audio array amplitude;

[0009] When the display device is in a standby mode, acquiring wake-up audio data through the adjusted sound collector, and calculating a similarity value between the wake-up audio data and a preset wake-up word;

[0010] If the similarity value is greater than or equal to the similarity value threshold, the display is controlled to display the user interface.

[0011] In some embodiments, the feedback signal includes a signal light speed parameter and a distance parameter acquisition request, and the controller executes sending the feedback signal to the control device, and is configured to:

[0012] Sending a signal light speed parameter and a distance parameter acquisition request to the control device, so that the control device responds to the distance parameter acquisition request and calculates the distance parameter according to the signal light speed parameter and the signal time difference;

[0013] Obtain the distance parameter sent by the control device.

[0014] In some embodiments, the distance measurement instruction is an instruction generated by the display device in response to a boot process, and before the controller executes the request to send a signal light speed parameter and a distance parameter acquisition request to the control device, it is further configured to:

[0015] Get the boot time;

[0016] Generate a delay parameter according to the boot time;

[0017] A feedback interval duration of the distance parameter acquisition request is generated according to the delay parameter, so that the control device sends the distance parameter to the display device after the feedback interval duration has passed.

[0018] In some embodiments, the feedback signal further includes a termination count signal, and the controller executes sending the feedback signal to the control device and is configured to:

[0019] The termination counting signal is sent to the control device so that the control device outputs the current number of pulses of the counter and calculates the signal time difference based on the number of pulses. The counter is created when the control device sends the ranging instruction, and the number of pulses is the number of counting pulses input by the clock oscillator to the counter.

[0020] In some embodiments, the controller executes acquiring the distance parameter sent by the control device and is configured to:

[0021] Acquire a distance analog signal obtained by the control device performing signal encoding on the distance parameter;

[0022] Performing signal amplification on the distance analog signal to obtain a level coded signal;

[0023] Binary decoding is performed on the level-coded signal to obtain the distance parameter.

[0024] In some embodiments, the controller performs determining the audio array amplitude according to the distance parameter, and is configured to:

[0025] Get the distance parameter range;

[0026] Determining a target parameter range within the distance parameter range according to the distance parameter;

[0027] The audio array amplitude is determined according to the target parameter range.

[0028] In some embodiments, the controller is further configured to:

[0029] Acquire sample audio data played within a distance parameter range, and acquire audio energy amplitude of the sample audio data;

[0030] The audio array amplitude within the distance parameter range is adjusted according to the amplitude difference between the audio energy amplitude and the energy amplitude threshold.

[0031] In some embodiments, a memory is further included, wherein the memory stores a speech recognition model, and the controller calculates a similarity value between the wake-up audio data and a preset wake-up word, and is configured as follows:

[0032] Inputting the wake-up audio data into the speech recognition model to obtain wake-up text data;

[0033] Extracting text features of the wake-up text data;

[0034] Calculating the cosine similarity between the text feature and a reference text feature, where the reference text feature is the text feature of the preset wake-up word;

[0035] The similarity value is calculated according to the cosine similarity.

[0036] In a second aspect, the present application provides a control device for sending a control instruction to the display device according to the first aspect to control the display device, the control device comprising a key component, including at least one control key for generating a control instruction, a communication interface, configured to send the control instruction to the display device, and a processor, configured to:

[0037] generating a distance measurement instruction in response to a click event of a distance measurement button, and sending the distance measurement instruction to a display device through the communication interface;

[0038] receiving a feedback signal generated by the display device in response to the distance measurement instruction, and calculating a signal time difference according to a time point of sending the distance measurement instruction and a time point of receiving the feedback signal;

[0039] Calculate the distance parameter according to the signal time difference;

[0040] The distance parameter is sent to the display device so that the display device determines the audio array amplitude according to the distance parameter, and adjusts the configuration parameters of the sound collector according to the audio array amplitude, and when the display device is in standby mode, the wake-up audio data is obtained through the adjusted sound collector, and the similarity value between the wake-up audio data and the preset wake-up word is calculated, and when the similarity value is greater than or equal to the similarity value threshold, the display is controlled to display the user interface.

[0041] In a third aspect, the present application provides a device wake-up method, which is applied to the display device described in the first aspect, and the method includes:

[0042] In response to a distance measurement instruction input by a user through a control device, sending a feedback signal to the control device;

[0043] Receiving a distance parameter sent by the control device, where the distance parameter is calculated by the control device according to a signal time difference, where the signal time difference is a time difference between a time point when the control device sends the distance measurement instruction and a time point when the control device receives the feedback signal;

[0044] Determining an audio array amplitude according to the distance parameter, and adjusting a configuration parameter of the sound collector according to the audio array amplitude;

[0045] When the display device is in a standby mode, acquiring wake-up audio data through the adjusted sound collector, and calculating a similarity value between the wake-up audio data and a preset wake-up word;

[0046] If the similarity value is greater than or equal to the similarity value threshold, the display is controlled to display the user interface.

[0047] It can be seen from the above technical solutions that the present application provides a display device, a control device and a device wake-up method, wherein the method sends a feedback signal to the control device in response to the distance measurement instruction input by the control device, receives the distance parameter sent by the control device, determines the audio array amplitude according to the distance parameter, and adjusts the configuration parameters of the sound collector according to the audio array amplitude. When the display device is in standby mode, the wake-up audio data is obtained, and the wake-up voice recognition is performed on the wake-up audio data to wake up the display device. The present application obtains the distance parameter between the user and the display device through the control device, and dynamically adjusts the audio array amplitude of the sound collector according to the distance parameter, thereby improving the sound collection effect of the sound collector at different distance positions, thereby improving the accuracy of voice wake-up of the display device. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0049] Figure 1 A schematic diagram of an operation scenario between a display device and a control device provided in some embodiments of the present application;

[0050] Figure 2 A schematic diagram of the hardware configuration of a display device provided in some embodiments of the present application;

[0051] Figure 3 A schematic diagram of the hardware configuration of a control device provided in some embodiments of the present application;

[0052] Figure 4 A schematic diagram of software configuration of a display device provided in some embodiments of the present application;

[0053] Figure 5 A schematic diagram of a process of performing device wake-up on a display device provided in some embodiments of the present application;

[0054] Figure 6 A flowchart of sending a control instruction to a display device in some embodiments of the present application;

[0055] Figure 7 A flowchart of a display device receiving a distance parameter sent by a control device provided in some embodiments of the present application;

[0056] Figure 8 A flowchart of a control device calculating a distance parameter provided in some embodiments of the present application;

[0057] Fig. 9 A flowchart of a display device receiving a distance parameter sent with a delay by a control device provided in some embodiments of the present application;

[0058] Fig.10 A timing diagram of a display device and a control device sending distance parameters provided in some embodiments of the present application;

[0059] Fig.11 A flow chart of calculating audio array amplitude for a display device provided in some embodiments of the present application;

[0060] Fig.12 A flowchart of a control device calculating a signal time difference provided in some embodiments of the present application;

[0061] Fig.13A schematic diagram of adjusting configuration parameters when a user is away from a display device in some embodiments of the present application;

[0062] Fig.14 This is a schematic diagram of configuration parameter adjustment when a user approaches a display device in some embodiments of the present application. DETAILED DESCRIPTION

[0063] The following embodiments are described in detail, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following embodiments do not represent all implementations consistent with the present application. They are only examples of systems and methods consistent with some aspects of the present application as detailed in the claims.

[0064] It should be noted that the brief description of terms in this application is only for the convenience of understanding the embodiments described below, and is not intended to limit the embodiments of this application. Unless otherwise specified, these terms should be understood according to their common and usual meanings.

[0065] The terms "first", "second", "third", etc. in the specification and claims of this application and the above drawings are used to distinguish similar or similar objects or entities, and do not necessarily mean to limit a specific order or sequence, unless otherwise noted. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances.

[0066] The terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover but not exclude inclusion, for example, a product or device comprising a list of components is not necessarily limited to all the components expressly listed but may include other components not expressly listed or inherent to such product or device.

[0067] The term "module" refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code that is capable of performing the functions associated with that element.

[0068] In the embodiment of the present application, the display device 200 generally refers to a device with image display and data processing capabilities. For example, the display device 200 includes but is not limited to a smart TV, a mobile terminal, a computer, a monitor, an advertising screen, a wearable device, a virtual reality device, an augmented reality device, etc.

[0069] Figure 1 This is a schematic diagram of an operation scenario between a display device and a control device provided in some embodiments of the present application. Figure 1As shown in FIG. 1 , a user can operate the display device 200 through touch operation, the mobile terminal 300 and the control device 100. The control device 100 is used to receive operation instructions input by the user and convert the operation instructions into control instructions that can be recognized and responded to by the display device 200. For example, the control device 100 can be a remote controller, a stylus pen, a handle, etc.

[0070] The mobile terminal 300 can be used as a control device for performing human-computer interaction between the user and the display device 200. The mobile terminal 300 can also be used as a communication device for establishing a communication connection with the display device 200 and performing data interaction. In some embodiments, the mobile terminal 300 can install software applications with the display device 200, and achieve connection and communication through a network communication protocol to achieve the purpose of one-to-one control operation and data communication. The audio and video content displayed on the mobile terminal 300 can also be transmitted to the display device 200 to achieve a synchronous display function.

[0071] In some embodiments, the mobile terminal 300 or other electronic devices may also simulate the functions of the control device 100 by running an application program for controlling the display device 200 .

[0072] like Figure 1 As also shown in FIG. 4 , the display device 200 also communicates data with the server 400 through various communication methods. The display device 200 may be allowed to communicate and connect through a local area network (LAN), a wireless local area network (WLAN), and other networks.

[0073] The display device 200 may provide a broadcast receiving television function, and may also additionally provide an intelligent network television function with a computer support function, including but not limited to network television, smart television, Internet Protocol television (IPTV), and the like.

[0074] Figure 2 Some embodiments of the present application provide Figure 1 2 is a block diagram of the hardware configuration of the display device 200.

[0075] In some embodiments, the display device 200 may include at least one of a tuner 210, a communication device 220, a detector 230, a device interface 240, a controller 250, a display 260, an audio output device 270, a memory, a power supply, and a user input interface.

[0076] In some embodiments, the detector 230 is used to collect signals of the external environment or external interaction. For example, the detector 230 includes a light receiver, a sensor for collecting the intensity of ambient light; or, the detector 230 includes an image collector, such as a camera, which can be used to collect external environment scenes, user attributes or user interaction gestures; or, the detector 230 includes a sound collector, such as a microphone, etc., for receiving external sounds.

[0077] In some embodiments, the display 260 includes a display function component for presenting a picture, and a driving component for driving an image display. The display 260 is used to receive an image signal output from the controller 250 for display. For example, the display 260 can be used to display video content, image content, and components of a menu control interface and a user control UI interface.

[0078] In some embodiments, the communication device 220 is a component for communicating with an external device or server 400 according to various communication protocol types. The display device 200 may be provided with a plurality of communication devices 220 according to different supported communication modes. For example, when the display device 200 supports wireless network communication, the display device 200 may be provided with a communication device 220 including a WiFi function. When the display device 200 supports Bluetooth connection communication, the display device 200 needs to be provided with a communication device 220 including a Bluetooth function.

[0079] The communication device 220 can enable the display device 200 to communicate with the external device or server 400 by wireless or wired connection. Among them, the wired connection can connect the display device 200 with the external device through components such as data cables and interfaces. The wireless connection can connect the display device 200 with the external device through wireless signals or wireless networks. The display device 200 can establish a connection relationship with the external device directly, or indirectly establish a connection relationship through a gateway, a router, a connection device, etc.

[0080] In some embodiments, the controller 250 may include at least one of a central processing unit, a video processor, an audio processor, a graphics processor, and a power processor, and a first interface to an nth interface for input / output. The controller 250 controls the operation of the display device and responds to the user's operation through various software control programs stored in the memory. The controller 250 controls the overall operation of the display device 200.

[0081] In some embodiments, the controller 250 and the tuner-demodulator 210 may be located in different separate devices, that is, the tuner-demodulator 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box.

[0082] In some embodiments, the user may input a user command through a graphical user interface (GUI) displayed on the display 260 , and the user input interface receives the user input command through the graphical user interface (GUI).

[0083] In some embodiments, the audio output device 270 may be a local speaker of the display device 200, or may be an external audio output device of the display device 200. In particular, for the external audio output device of the display device 200, the display device 200 may also be provided with an external audio output terminal, and the audio output device may be connected to the display device 200 through the external audio output terminal to output the sound of the display device 200.

[0084] In some embodiments, the user input interface 280 may be used to receive instructions from a user.

[0085] Figure 3 Some embodiments of the present application provide Figure 1 The hardware configuration diagram of the control device in the figure. Figure 3 As shown, the control device 100 may include: a controller 110, a communication interface 130, a user input / output interface, a memory, and a power supply.

[0086] The control device 100 is configured to control the display device 200 , and can receive user input operation instructions, and convert the operation instructions into instructions that the display device 200 can recognize and respond to, playing the role of an interactive intermediary between the user and the display device 200 .

[0087] In some embodiments, the control device 100 may be a smart device, for example, the control device 100 may be installed with various applications for controlling the display device 200 according to user needs.

[0088] In some embodiments, Figure 1 As shown, the mobile terminal 300 or other intelligent electronic devices can play a similar function as the control device 100 after installing the application for controlling the display device 200 .

[0089] The controller 110 includes a processor 112, a RAM 113, a ROM 114, a communication interface 130, and a communication bus. The controller 110 is used to control the operation and operation of the control device 100, as well as the communication and cooperation between the internal components and the external and internal data processing functions.

[0090] The communication interface 130 implements communication of control signals and data signals with the display device 200 under the control of the controller 110. The communication interface 130 may include at least one of other near field communication modules such as a WiFi chip 131, a Bluetooth module 132, and an NFC module 133.

[0091] The user input / output interface 140 , wherein the input interface includes at least one of other input interfaces such as a microphone 141 , a touch panel 142 , a sensor 143 , and a button 144 .

[0092] In some embodiments, the control device 100 includes at least one of a communication interface 130 and an input / output interface 140. The control device 100 is configured with a communication interface 130, such as a WiFi, Bluetooth, NFC or other module, and can encode the user input command through the WiFi protocol, Bluetooth protocol, or NFC protocol and send it to the display device 200.

[0093] The memory 190 is used to store various operating programs, data and applications for driving and controlling the control device 100 under the control of the controller. The memory 190 can store various control signal instructions input by the user.

[0094] The power supply 180 is used to provide operating power support for each component of the control device 100 under the control of the controller.

[0095] In order to perform user interaction, in some embodiments, the display device 200 may run an operating system. The operating system is a computer program for managing and controlling hardware resources and software resources in the display device 200. The operating system can control the display device to provide a user interface, for example, the operating system can directly control the display device to provide a user interface, or can provide a user interface by running an application program. The operating system also allows the user to interact with the display device 200.

[0096] It should be noted that the operating system may be a native operating system based on a specific operating platform, or a third-party operating system deeply customized based on a specific operating platform, or an independent operating system specially developed for the display device.

[0097] The operating system can be divided into different modules or layers according to the functions implemented, such as Figure 4 As shown, in some embodiments, the system is divided into four layers, from top to bottom, namely, the application layer (Applications) layer (referred to as "application layer"), the application framework layer (Application Framework) layer (referred to as "framework layer"), the system library layer and the kernel layer.

[0098] In some embodiments, the application layer is used to provide services and interfaces for applications so that the display device 200 can run applications and interact with users based on the applications. At least one application can be run in the application layer, and these applications can be window programs, system settings programs, clock programs, etc. that come with the operating system; they can also be applications developed by third-party developers. In specific implementations, the application packages in the application layer are not limited to the above examples.

[0099] The framework layer provides application programming interfaces (APIs) and programming frameworks for applications. The application framework layer includes some predefined functions. The application framework layer is equivalent to a processing center that determines the actions that applications in the application layer take. Through the API interface, applications can access system resources and obtain system services during execution.

[0100] like Figure 4 As shown, Figure 4 A software configuration diagram of a display device provided for some embodiments of the present application. In some embodiments, the system of the display device 200 can be divided into three layers, namely, an application layer, a middleware layer, and a hardware layer from top to bottom.

[0101] The application layer mainly includes applications on the TV and the application framework. Among them, applications are mainly applications developed based on the browser, such as HTML5 APPs and native applications.

[0102] The Application Framework is a complete program model that has all the basic functions required by standard application software, such as file access, data exchange, etc., as well as the user interfaces of these functions (toolbars, status bars, menus, dialog boxes).

[0103] Native apps can support online or offline, message push or local resource access.

[0104] The middleware layer includes various TV protocols, multimedia protocols, system components and other middleware. The middleware can use the basic services (functions) provided by the system software to connect various parts of the application system or different applications on the network, and can achieve the purpose of resource sharing and function sharing.

[0105] The hardware layer mainly includes HAL interface, hardware and driver. Among them, HAL interface is the unified interface for all TV chips to connect, and the specific logic is implemented by each chip. Drivers mainly include: audio driver, display driver, Bluetooth driver, camera driver, WIFI driver, USB driver, HDMI driver, sensor driver (such as fingerprint sensor, temperature sensor, pressure sensor, etc.), and power driver, etc.

[0106] It should be noted that the above example is only a simple division of the operating system functions and does not constitute a limitation on the specific operating system form of the display device 200 in the embodiment of the present application. Depending on factors such as the function of the display device and the type of operating system, the number of levels and specific level types contained in the operating system may be expressed in other forms.

[0107] In some embodiments, the display device 200 can enter the standby mode after a preset period of time in the non-operating state to suspend the executing application program, so that the display device 200 is in a low-power operation state, and the power consumption of the display device 200 is reduced. When the display device 200 is in the standby state, the user can send a wake-up instruction to switch the display device 200 from the standby mode to the working mode. The wake-up instruction can be any control instruction. For example, when the user clicks a button of the control device 100, the control device 100 can generate a wake-up instruction according to the corresponding button, and send the wake-up instruction to the display device 200 to wake up the display device 200. After receiving the wake-up instruction, the display device 200 in the standby mode switches to the working mode and continues to run the application program at the program progress before suspension.

[0108] The display device 200 may also have a voice interaction function. For this purpose, the wake-up instruction may also be a voice instruction. The user may use the voice wake-up function (KWS, KeyWord Spotting) to speak the wake-up word preset by the display device 200 to generate a wake-up instruction to wake up the display device 200.

[0109] In some embodiments, the wake-up instruction may also be wake-up audio data. In the process of waking up the display device 200, the controller 250 needs to perform voice recognition on the wake-up audio data, obtain the voice recognition result, and calculate the matching degree between the voice recognition result and the wake-up word text preset by the display device 200. When the matching degree is greater than or equal to the preset matching degree threshold, the controller 250 may wake up the display device 200 according to the wake-up audio data. If the matching degree is less than the preset matching degree threshold, it means that the user said the wrong wake-up audio data and failed to wake up the display device 200.

[0110] In standby mode, the display device 200 can obtain the wake-up audio data input by the user through the sound collector. The sound collector has multiple microphone arrays built in to receive sound sources from different directions and improve the audio acquisition range of the display device 200. In the scenario where the user is watching the display device 200, the distance between the user and the display device 200 is not fixed. When the distance between the user and the display device 200 changes, the array amplitude of the microphone array in the sound collector remains unchanged, resulting in poor sound reception of the sound collector. For example, when the user is relatively far away from the TV, the sound waves emitted by the user will decay during the propagation process, resulting in a decrease in the sound reception energy value of the microphone array. When the user is relatively close to the TV, the emitted sound waves are directly received by the microphone array, which may cause the clipping phenomenon of the microphone array due to the high volume. Both of the above situations will cause the sound collector to have a poor sound reception effect, resulting in unclear wake-up audio data input by the user, affecting the wake-up recognition of the display device 200.

[0111] In order to solve the problem of unclear received wake-up audio data due to the uncertain distance between the user and the display device, the present application provides a display device 200, which should include at least a display 260, a sound collector and a controller 250. The display 260 is configured to display a user interface, the sound collector is used to obtain audio data, and the controller 250 is configured to execute a device wake-up method.

[0112] Figure 5 A schematic diagram of a process of waking up a display device provided in some embodiments of the present application. Figure 5 , the method comprises the following contents:

[0113] S100: In response to a distance measurement instruction input by a user through a control device, a feedback signal is sent to the control device.

[0114] like Figure 6 As shown, the control device 100 is provided with a button component, which includes multiple control buttons, such as an "OK" button, an exit "button", and direction buttons, such as an "↑" button, a "↓" button, a "←" button, a "→" button, etc. The control device 100 can monitor the click events of the control buttons and generate different control instructions according to the click events of the corresponding control buttons. For example, when the user moves the focus to select a video media asset in the display device 200, the focus movement instruction can be generated by clicking the direction button on the control device 100 to move the focus media asset in the display device 200.

[0115] In some embodiments, the button assembly may include a distance measuring button, which is used to generate a distance measuring instruction to measure the distance between the control device 100 and the display device 200. Since the user holds the control device 100 when using the control device 100, the position of the control device 100 can represent the position of the user relative to the display device 200, thereby determining the distance between the user and the display device 200.

[0116] The distance measurement button can be a control button separately set in the button component, or it can be other control buttons in the button component, that is, the control button has the corresponding multimedia function and also has the distance measurement function. For example, the "OK" button can be a distance measurement button. After the user clicks the "OK" button, a selection instruction can be generated based on the "OK" button to execute the interactive effect corresponding to the "OK" button, and a distance measurement instruction is generated at the same time and sent to the display device 200 via an infrared signal.

[0117] After receiving the distance measurement instruction, the display device 200 sends a feedback signal to the control device 100, and the feedback signal is used to indicate that the distance measurement instruction has been sent to the display device 200. The feedback signal can be generated synchronously after the distance measurement instruction is sent to the display device 200, and sent to the control device 100 through signal reflection.

[0118] S200: Receive a distance parameter sent by the control device.

[0119] After receiving the feedback signal through the communication interface 130, the control device 100 can calculate the distance parameter according to the signal time difference, and the distance parameter is used to represent the distance between the control device 100 and the display device 200. The signal time difference is the time difference between the time point when the control device 100 sends the distance measurement instruction and the time point when the feedback signal is received. Figure 7 A flow chart of calculating a signal time difference for a display device provided in some embodiments of the present application. Figure 7 , the time when the control device 100 sends the distance measurement instruction is a1, and the time when the feedback signal is received is a2, then the signal time difference T is a1-a2, and the control device 100 can calculate the distance parameter through the signal time difference.

[0120] The distance parameter needs to be calculated based on the signal light speed. For this purpose, Figure 8 As shown, the feedback signal includes a signal light speed parameter and a distance parameter acquisition request. After receiving the distance measurement instruction, the controller 250 can detect the signal light speed parameter of the distance measurement instruction and generate a distance parameter acquisition request to request the control device 100 to calculate the distance parameter.

[0121] In some embodiments, the controller 250 may send a request to obtain the signal light speed parameter and the distance parameter to the control device 100. After the control device 100 calculates the signal time difference according to the time point of receiving the feedback signal, the distance parameter may be calculated according to the signal time difference and the following formula:

[0122] D = 1 / 2 CT;

[0123] Among them, D is the distance parameter, C is the signal light speed parameter, and T is the signal time difference.

[0124] After calculating the distance parameter, the control device 100 may send the distance parameter to the display device 200 in response to a distance parameter acquisition request, so that the controller 250 receives the distance parameter sent by the control device 100 .

[0125] In some embodiments, the display device 200 can complete the distance measurement process in the powered-on state, that is, receive the distance measurement instruction sent by the control device 100 in the powered-on state, and obtain the distance parameter calculated by the control device 100 according to the feedback signal. The display device 200 can also complete the distance process in the powered-on state. The powered-on state is the state when the display device 200 is responding to the power-on process, and the user can generate a power-on instruction through the power-on button of the control device 100. The display device 200 can respond to the power-on instruction and run the power-on process. At this time, the distance measurement instruction is the instruction generated by the display device 200 in response to the power-on process. During the running of the power-on process, the display device 200 can send the feedback signal generated according to the distance measurement instruction to the control device 100.

[0126] Since the boot process of the display device 200 needs to run for a preset time, if the control device 100 sends the distance parameter to the display device 200 immediately after obtaining the distance parameter, the display device 200 will not be in the boot state, resulting in failure to receive the distance parameter.

[0127] To alleviate the above problems, see Fig. 9 In some embodiments, the controller 250 may obtain the power-on duration of the display device 200, which is the duration from when the display device 200 responds to the power-on instruction and starts to run the power-on process to when the power-on process ends, and the display device 200 is in the powered-on state. The controller 250 may generate a delay parameter according to the power-on duration, and the delay parameter is used to indicate the duration for which the control device 100 delays sending the distance parameter. The delay parameter may be greater than or equal to the power-on duration to ensure that the display device 200 is in the powered-on state.

[0128] The controller 250 can generate a feedback interval duration of a distance parameter acquisition request according to the delay parameter, for example, Fig.10As shown, when the power-on time is 5 seconds, the delay parameter can be set to be greater than or equal to 5 seconds, for example, 5 seconds, 6 seconds or 7 seconds, etc. When the display device 200 generates a distance parameter acquisition request, the feedback interval duration can be set according to the delay parameter. After responding to the distance parameter acquisition request, the control device 100 can send the distance parameter to the display device 200 after the feedback interval duration has passed. For example, the feedback interval duration of the distance parameter acquisition request is 6 seconds. At this time, the display device 200 is running the power-on program. The control device 100 calculates the distance parameter according to the signal time difference while the display device 200 is running the power-on program. After acquiring the distance parameter, the control device 100 responds to the distance parameter acquisition request and executes the delay program. The delay duration of the delay program is the feedback interval duration. When the control device 100 delays for 5 seconds, the power-on program of the display device 200 ends and is in the powered-on state. At this time, the display device 200 can receive the parameter data sent by the control device 100. When the control device 100 delays for 6 seconds, the distance parameter can be sent to the display device 200 to alleviate the problem of failure to receive the distance parameter due to the display device 200 not being fully powered on, thereby improving the efficiency of the display device 200 in receiving the distance parameter.

[0129] S300: Determine an audio array amplitude according to the distance parameter, and adjust configuration parameters of the sound collector according to the audio array amplitude.

[0130] After receiving the distance parameter, the display device 200 can determine the audio array amplitude according to the distance parameter. The audio array amplitude is used to adjust the configuration parameters of the microphone array in the sound collector to improve the microphone array's ability to collect sound sources at different distances. For a relatively large distance parameter, the audio array amplitude can be increased, and for a relatively small distance parameter, the audio array amplitude can be reduced to dynamically adjust the microphone array's ability to collect sound.

[0131] In order to improve the accuracy of determining the audio array amplitude, the controller 250 can divide the distance parameter ranges into multiple distance parameter ranges to configure the audio array amplitudes of different distance parameter ranges. Exemplarily, the controller 250 can divide the distance parameter range into d1: D ≤ 1 meter, d2: 1 meter < D ≤ 2 meters, d3: 2 meters < D ≤ 3 meters, d4: 3 meters < D ≤ 4 meters, d5: D > 4 meters, a total of d1, d2, d3, d4, d5, 5 distance parameter ranges. The controller 250 can determine the target parameter range in the distance parameter range according to the acquired distance parameter. For example, when the distance parameter is 2.3 meters, the distance parameter range d3 can be determined as the target parameter range.

[0132] Each distance parameter range includes an audio array amplitude, corresponding to d1, d2, d3, d4, and d5, and the audio array amplitude may include g1, g2, g3, g4, and g5, wherein g1 is the audio array amplitude of the distance parameter range d1, g2 is the audio array amplitude of the distance parameter range d2, g3 is the audio array amplitude of the distance parameter range d3, g4 is the audio array amplitude of the distance parameter range d4, and g5 is the audio array amplitude of the distance parameter range d5. Since the sound waves emitted by the user will attenuate during the propagation process, resulting in a decrease in the sound receiving energy value of the microphone array, the audio matrix amplitude of the relatively far distance parameter range is larger. When the user is relatively close to the TV, the sound waves emitted will cause the clipping phenomenon of the microphone array due to the high volume, so the audio matrix amplitude of the relatively close distance parameter range is smaller, therefore, the audio matrix amplitudes of g1, g2, g3, g4, and g5 should be increased in sequence, that is, g1 < g2, g2 < g3, g3 < g4, and g4 < g5.

[0133] Continuing with the above example, when the distance parameter is 2.3 meters, the target parameter range is d3, and therefore the audio array amplitude is the audio array amplitude corresponding to the distance parameter range d3. The controller 250 can adjust the configuration parameters of the microphone array of the sound collector according to the audio array amplitude corresponding to the distance parameter range d3 to improve the sound collection effect of the microphone array.

[0134] In some embodiments, to determine the audio array amplitude for each distance parameter range, such as Fig.11 As shown, the controller 250 can also obtain sample audio data, and the sample audio data is used to play in different distance parameter ranges. The display device 200 can be provided with an initial array amplitude. After the display device 200 receives the sample audio data with the initial array amplitude, an audio energy analysis can be performed on the sample audio data to obtain the audio energy amplitude of the sample audio data.

[0135] In order to facilitate the adjustment of the initial array amplitude, the controller 250 can set an energy amplitude threshold, and calculate the amplitude difference between the audio energy amplitude and the energy amplitude threshold, and adjust the audio array amplitude of the distance parameter range according to the amplitude difference. The amplitude difference can be a positive value or a negative value. A positive value is used to indicate that the audio energy amplitude is greater than the energy amplitude threshold, indicating that the sample audio data received within the distance parameter range is prone to cause clipping of the microphone array. Therefore, the initial array amplitude of the distance parameter range can be reduced according to the amplitude difference to obtain the audio array amplitude. A negative value is used to indicate that the audio energy amplitude is less than the energy amplitude threshold, indicating that the distance parameter range is far from the display device 200, and the sample audio data received within the distance parameter range is attenuated, resulting in unclear audio. Therefore, the initial array amplitude of the distance parameter range can be increased according to the amplitude difference to obtain the audio array amplitude.

[0136] S400: When the display device is in a standby mode, the wake-up audio data is obtained through the adjusted sound collector, and a similarity value between the wake-up audio data and a preset wake-up word is calculated.

[0137] When the display device 200 is in standby mode, the controller 250 controls the display 260 to display a standby interface. At this time, the user can wake up the display device 200 by saying a preset wake-up word. At this time, the display device 200 can obtain the wake-up audio data input by the user through the adjusted sound collector, and calculate the similarity value between the wake-up audio data and the preset wake-up word.

[0138] In some embodiments, the display device 200 also includes a memory, and the memory stores a speech recognition model. The controller 250 can input the wake-up audio data into the speech recognition model, and perform preprocessing on the wake-up audio data in sequence through the speech recognition model to obtain the MFCC coefficients of the wake-up audio data. Among them, the preprocessing may include pre-emphasis, framing, windowing, fast Fourier transform (FFT), calculation of energy spectrum, passing through a Mel filter bank, taking logarithmic energy, and performing discrete cosine transform (DCT) and other processing processes to improve the accuracy of MFCC coefficient extraction. After extracting the MFCC coefficients, the speech recognition model can output a text sequence according to the MFCC coefficients to obtain the wake-up text data, and extract the text features of the wake-up text data. After obtaining the text features, the controller 250 can calculate the cosine similarity between the text features and the reference text features, thereby calculating the similarity value between the wake-up audio data and the preset wake-up word according to the cosine similarity, wherein the reference text features are the text features of the preset wake-up words.

[0139] S500: If the similarity value is greater than or equal to the similarity value threshold, control the display to display the user interface.

[0140] When the similarity value is greater than or equal to the similarity value threshold, it means that the wake-up audio data successfully wakes up the display device 200, and the controller 250 can control the display 260 to display the user interface. When the similarity value is less than the similarity value threshold, it means that the wake-up audio data cannot wake up the display device 200, and the controller 250 can generate prompt information based on the wake-up audio data, and control the display device 200 to display the prompt information on the standby interface to prompt the user that the wake-up failed.

[0141] In some embodiments, Fig.12As shown, the control device 100 can also create a counter, which is used to record the time difference. After sending the ranging instruction, the control device 100 starts the counter and inputs counting pulses to the counter through the clock oscillator. The feedback signal also includes a termination counting signal. After the display device 200 sends a feedback signal to the control device 100, the control device 100 can respond to the termination counting signal and stop the counting process of the counter. After the counting process stops, the clock oscillator stops inputting counting pulses to the counter. The control device 100 can output the current number of pulses of the counter and calculate the signal time difference according to the following formula:

[0142] T = N * (1 / F);

[0143] Where N is the current number of pulses in the counter, and F is the operating frequency of the clock oscillator.

[0144] In order to improve the transmission speed of parameter data, wireless transmission can be adopted between the control device 100 and the display device 200, that is, data is transmitted between the control device 100 and the display device 200 by infrared signals. Therefore, before the control device 100 sends the distance parameter to the display device 200, the distance parameter can also be encoded to convert the distance parameter into an infrared light signal and send it to the display device 200. In this process, the processor 112 of the control device 100 encodes the distance parameter by an infrared communication protocol, such as the NEC communication protocol, to obtain a distance analog signal to distinguish the data bits between the distance parameters, and performs error checking by the inverse code of the address code and the command code, which effectively reduces the bit error rate during data transmission.

[0145] The processor 112 can send the distance analog signal after signal encoding to the display device 200 through the communication interface 130. The display device 200 can receive the distance analog signal through the communication device 220, and perform operations such as signal amplification and signal detection on the distance analog signal to obtain a level encoding signal of TTL level, thereby improving the signal-to-noise ratio and clarity of the distance analog signal received by the display device 200. After obtaining the level encoding signal, the controller 250 performs binary decoding on the level encoding signal to obtain the distance parameter, thereby ensuring that the data of the distance parameter is accurately decoded and restored.

[0146] In some embodiments, since the distance between the user and the display device 200 is not fixed, after the display device 200 completes the configuration parameter adjustment of the sound collector according to the distance parameter sent by the control device 100, the user can move a second time, at which time the configuration parameter may not be applicable to the position where the user moves for the second time. Therefore, the controller 250 can also send a feedback signal to the control device 100 again according to the new control instruction sent by the control device 100 to update the distance parameter.

[0147] Fig.13 This is a schematic diagram of parameter adjustment when the user is away from the display device in the embodiment of the present application. Fig.13 , when the user is located in the distance parameter range d3 for the first time, after adjusting the configuration parameters of the sound collector, he immediately moves to the distance parameter range d4, that is, moves in the direction away from the display device 200. At this time, the user sends a control instruction to the display device 200 through the control device 100. After receiving the control instruction, the display device 200 can send a feedback signal to the control device 200 again according to the control instruction. The control device 100 can update the signal time difference according to the time point of sending the control instruction and the time point of receiving the second feedback signal. Since the user moves in the direction away from the display device 200, the distance traveled by the signal increases, and the corresponding signal time difference also increases accordingly. Therefore, the controller 250 can calculate that the user is located in the distance parameter range d4 at this time according to the signal time difference and the signal light speed parameter, thereby improving the configuration parameters of the sound collector of the display device 200 according to the audio array parameters of the distance parameter range d4, so that the sound collector can receive sound data from a farther sound source.

[0148] Fig.14 Schematic diagram of parameter adjustment when a user approaches a display device in an embodiment of the present application. Fig.14 If the user moves to the distance parameter range d2 for the second time, that is, moves in a direction close to the display device 200, the control device 100 can update the signal time difference according to the time point of sending the control instruction and the time point of receiving the second feedback signal, and calculate that the user is in the distance parameter range d2 at this time according to the signal time difference and the signal light speed parameter, thereby reducing the configuration parameters of the sound collector of the display device 200 according to the audio array parameters of the distance parameter range d2, so that the sound collector can alleviate the amplitude phenomenon caused by the sound source approaching the display device 200.

[0149] In some embodiments, if the user moves to another direction of the distance parameter range d3 for the second time, since the distance parameter range remains unchanged, the configuration parameters of the sound collector remain unchanged. The controller 250 can adjust the direction of the microphone array in the sound collector according to the receiving direction of the control instruction.

[0150] It should be noted that this application is only exemplified by two user movements. In the actual application of the display device 200, each time the user changes the distance parameter range and uses the control device 100 to send a new control instruction, the configuration parameters of the sound collector can be adjusted according to the distance parameter range of the control device 100 after the movement. The configuration parameters of the sound collector can be adjusted with reference to the above embodiment, which will not be described in detail in this embodiment.

[0151] Some embodiments of the present application provide a control device 100, which is used to send a control instruction to the display device 200 described above. The control device 100 includes a button component, a communication interface 130, and a processor 112, wherein the button component includes at least one control button, and a user can generate a control instruction for the display device 200 to respond by clicking the control button, and send the control instruction to the display device 200 through the communication interface 130. The processor is configured to generate a distance measurement instruction in response to a click event of a distance measurement button, and send the distance measurement instruction to the display device 200 through the communication interface 130. After receiving the distance measurement instruction, the display device 200 sends a feedback signal to the control device 100, and the processor 112 can receive the feedback signal generated by the display device 200 in response to the distance measurement instruction, and calculate the signal time difference according to the time point of sending the distance measurement instruction and the time point of receiving the feedback signal, and calculate the distance parameter according to the signal time difference.

[0152] After calculating the distance parameter, the processor 112 sends the distance parameter to the display device 200, so that the display device 200 determines the audio array amplitude according to the distance parameter, and adjusts the configuration parameters of the sound collector according to the audio array amplitude, and when the display device 200 is in standby mode, obtains the wake-up audio data through the adjusted sound collector, and calculates the similarity value between the wake-up audio data and the preset wake-up word, and when the similarity value is greater than or equal to the similarity value threshold, controls the display 260 to display the user interface.

[0153] The control device 100 provided in the present application sends a distance measurement instruction to the display device 200, receives a feedback signal from the display device 200, calculates the signal time difference, and determines the distance parameter of the user relative to the display device 200 according to the signal time difference, so that the display device 200 adjusts the configuration parameters of the sound collector according to the distance parameter, improves the clarity of the wake-up audio data obtained, and improves the accuracy of wake-up recognition.

[0154] Some embodiments of the present application provide a device wake-up method, which is applied to the display device 200 described above. The display device 200 should at least include a display 260, a sound collector and a controller 250. The display 260 is configured to display a user interface, and the sound collector is configured to obtain audio data. The method includes:

[0155] S100: In response to a distance measurement instruction input by a user through a control device, sending a feedback signal to the control device;

[0156] S200: receiving a distance parameter sent by the control device, where the distance parameter is calculated by the control device according to a signal time difference, where the signal time difference is a time difference between a time point when the control device sends the distance measurement instruction and a time point when the control device receives the feedback signal;

[0157] S300: determining an audio array amplitude according to the distance parameter, and adjusting configuration parameters of the sound collector according to the audio array amplitude;

[0158] S400: when the display device is in a standby mode, obtaining wake-up audio data through the adjusted sound collector, and calculating a similarity value between the wake-up audio data and a preset wake-up word;

[0159] S500: If the similarity value is greater than or equal to the similarity value threshold, control the display to display the user interface.

[0160] It can be seen from the above technical solutions that the present application provides a display device, a control device and a device wake-up method, wherein the method sends a feedback signal to the control device in response to the distance measurement instruction input by the control device, receives the distance parameter sent by the control device, determines the audio array amplitude according to the distance parameter, and adjusts the configuration parameters of the sound collector according to the audio array amplitude. When the display device is in standby mode, the wake-up audio data is obtained, and the wake-up voice recognition is performed on the wake-up audio data to wake up the display device. The present application obtains the distance parameter between the user and the display device through the control device, and dynamically adjusts the audio array amplitude of the sound collector according to the distance parameter, thereby improving the sound collection effect of the sound collector at different distance positions, thereby improving the accuracy of voice wake-up of the display device.

[0161] The same and similar parts between the various embodiments in this specification can be referenced to each other and will not be described again here.

[0162] Those skilled in the art can clearly understand that the technology in the embodiments of the present invention can be implemented by means of software plus a necessary general hardware platform. Based on this understanding, the technical solution in the embodiments of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a disk, an optical disk, etc., and includes a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of various embodiments of the present invention or certain parts of the embodiments.

[0163] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

[0164] For the convenience of explanation, the above description has been made in conjunction with specific embodiments. However, the above exemplary discussion is not intended to be exhaustive or limit the embodiments to the specific forms disclosed above. Based on the above teachings, various modifications and variations can be obtained. The selection and description of the above embodiments are to better explain the principles and practical applications, so that those skilled in the art can better use the embodiments and various different variations of the embodiments suitable for specific use considerations.

Claims

1. A display device, characterized in that: include: a display configured to display a user interface; A sound collector is configured to obtain audio data; The controller is configured as: In response to a distance measurement instruction input by a user through a control device, sending a feedback signal to the control device; Receiving a distance parameter sent by the control device, where the distance parameter is calculated by the control device according to a signal time difference, where the signal time difference is a time difference between a time point when the control device sends the distance measurement instruction and a time point when the control device receives the feedback signal; Determining an audio array amplitude according to the distance parameter, and adjusting a configuration parameter of the sound collector according to the audio array amplitude; When the display device is in a standby mode, acquiring wake-up audio data through the adjusted sound collector, and calculating a similarity value between the wake-up audio data and a preset wake-up word; If the similarity value is greater than or equal to the similarity value threshold, the display is controlled to display the user interface.

2. The display device according to claim 1, characterized in that The feedback signal includes a signal light speed parameter and a distance parameter acquisition request, and the controller executes sending the feedback signal to the control device, and is configured to: Sending a signal light speed parameter and a distance parameter acquisition request to the control device, so that the control device responds to the distance parameter acquisition request and calculates the distance parameter according to the signal light speed parameter and the signal time difference; Obtain the distance parameter sent by the control device.

3. The display device according to claim 2, characterized in that The distance measurement instruction is an instruction generated by the display device in response to the boot process. Before the controller executes the request to send the signal light speed parameter and the distance parameter acquisition request to the control device, it is also configured as follows: Get the boot time; Generate a delay parameter according to the boot time; A feedback interval duration of the distance parameter acquisition request is generated according to the delay parameter, so that the control device sends the distance parameter to the display device after the feedback interval duration has passed.

4. The display device according to claim 2, characterized in that The feedback signal also includes a termination count signal, and the controller executes sending the feedback signal to the control device, and is configured to: The termination counting signal is sent to the control device so that the control device outputs the current number of pulses of the counter and calculates the signal time difference based on the number of pulses. The counter is created when the control device sends the ranging instruction, and the number of pulses is the number of counting pulses input by the clock oscillator to the counter.

5. The display device according to claim 2, characterized in that The controller executes acquiring the distance parameter sent by the control device, and is configured to: Acquire a distance analog signal obtained by the control device performing signal encoding on the distance parameter; Performing signal amplification on the distance analog signal to obtain a level coded signal; Binary decoding is performed on the level-coded signal to obtain the distance parameter.

6. The display device according to claim 1, characterized in that The controller determines the audio array amplitude according to the distance parameter and is configured to: Get the distance parameter range; Determining a target parameter range within the distance parameter range according to the distance parameter; The audio array amplitude is determined according to the target parameter range.

7. The display device according to claim 6, characterized in that The controller is also configured to: Acquire sample audio data played within a distance parameter range, and acquire audio energy amplitude of the sample audio data; The audio array amplitude within the distance parameter range is adjusted according to the amplitude difference between the audio energy amplitude and the energy amplitude threshold.

8. The display device according to claim 1, characterized in that The system further includes a memory storing a speech recognition model, wherein the controller calculates a similarity value between the wake-up audio data and a preset wake-up word, and is configured as follows: Inputting the wake-up audio data into the speech recognition model to obtain wake-up text data; Extracting text features of the wake-up text data; Calculating the cosine similarity between the text feature and a reference text feature, where the reference text feature is the text feature of the preset wake-up word; The similarity value is calculated according to the cosine similarity.

9. A control device, characterized in that: include: A button assembly, comprising at least one control button, wherein the control button is used to generate a control instruction; A communication interface configured to send control instructions to a display device The processor is configured as: generating a distance measurement instruction in response to a click event of a distance measurement button, and sending the distance measurement instruction to a display device through the communication interface; receiving a feedback signal generated by the display device in response to the distance measurement instruction, and calculating a signal time difference according to a time point of sending the distance measurement instruction and a time point of receiving the feedback signal; Calculate the distance parameter according to the signal time difference; The distance parameter is sent to the display device so that the display device determines the audio array amplitude according to the distance parameter, and adjusts the configuration parameters of the sound collector according to the audio array amplitude, and when the display device is in standby mode, the wake-up audio data is obtained through the adjusted sound collector, and the similarity value between the wake-up audio data and the preset wake-up word is calculated, and when the similarity value is greater than or equal to the similarity value threshold, the display is controlled to display the user interface.

10. A device wake-up method, characterized in that: The display device according to any one of claims 1 to 8 comprises a display, a sound collector and a controller, the display is configured to display a user interface, the sound collector is configured to acquire audio data, and the method comprises: In response to a distance measurement instruction input by a user through a control device, sending a feedback signal to the control device; Receiving a distance parameter sent by the control device, where the distance parameter is calculated by the control device according to a signal time difference, where the signal time difference is a time difference between a time point when the control device sends the distance measurement instruction and a time point when the control device receives the feedback signal; Determining an audio array amplitude according to the distance parameter, and adjusting a configuration parameter of the sound collector according to the audio array amplitude; When the display device is in a standby mode, acquiring wake-up audio data through the adjusted sound collector, and calculating a similarity value between the wake-up audio data and a preset wake-up word; If the similarity value is greater than or equal to the similarity value threshold, the display is controlled to display the user interface.

Images