Display device and sound collector control method

Abstract

The invention provides a display device and a sound collector control method. The display device analyzes and judges whether a user exists in the preset area or not by receiving the user data sent by the millimeter wave radar, and then controls the operation state of the sound collector to form a multi-sensor linkage control mechanism, thereby facilitating the reduction of the energy consumption of the device. Moreover, in the scene that the user exists in the preset area, the gain value can be dynamically adjusted according to the specific position of the user, the accuracy of sound collection is ensured, the fluency and naturalness of voice interaction are further improved, and the user experience is optimized.

Description

Technical Field

[0001] The present application relates to the technical field of display devices, and in particular to a method for controlling a display device and a sound collector. Background Art

[0002] With the rapid development of display devices, multimodal perception systems equipped with millimeter-wave radar and sound collectors have become an important technical direction. By combining the spatial perception capabilities of millimeter-wave radar with the acoustic collection characteristics of sound collectors, these display devices can achieve complex functions such as voice recognition, gesture interaction, and environmental perception.

[0003] In practical applications, the millimeter-wave radar and sound collector on display devices operate independently. For example, the millimeter-wave radar continuously scans the space to detect gestures, while the sound collector performs sound collection. These two lack effective coordination mechanisms for signal acquisition, data processing, and function triggering. This discrete operating architecture results in low sensor resource utilization and is prone to perception redundancy and energy waste in complex acoustic environments or dynamic spatial scenarios, limiting the overall performance optimization of multimodal interaction systems.

[0004] In a discrete architecture, the lack of radar-assisted spatial positioning results in insufficient far-field voice recognition sensitivity, while near-field voice recognition is susceptible to interference from ambient noise, leading to significant differences in wake-up success rates between far and near distances. Furthermore, a wake-up mechanism that relies solely on acoustic features carries a low probability of false triggering at startup and false wake-ups during standby mode. Summary of the invention

[0005] The present application provides a method for controlling a display device and a sound collector to solve the problem of poor sensor operation due to insufficient utilization of the synergistic relationship between sensors.

[0006] In a first aspect, the present application provides a display device comprising: a display; a millimeter-wave radar configured to collect user data within a preset area; a sound collector configured to receive voice commands input by the user; and a controller configured to:

[0007] Parsing the user data to obtain a first identifier or a second identifier contained in the user data; the first identifier is used to indicate that the user exists in the preset area; the second identifier is used to indicate that the user does not exist in the preset area;

[0008] When the first identifier is obtained, a first control strategy is determined according to the human-machine distance, and the operating state of the sound collector is controlled according to the first control strategy; the first control strategy is used to control the sound collector to be in an activated state and to control the gain value of the sound collector;

[0009] When the second identifier is obtained, control the operating state of the sound collector according to the second control strategy; the second control strategy is used to control the sound collector to be in a shutdown state.

[0010] In this way, the controller identifies whether there is a user in the preset area based on the user data collected by the millimeter-wave radar. When there is no user in the preset area, the sound collector can be controlled to be turned off to save power consumption. When there is a user in the preset area, the sound collector can be controlled to operate, and the gain of the sound collector can be adjusted to provide the user with a better voice interaction experience.

[0011] In some feasible embodiments, the controller executes controlling the operating state of the sound collector according to the first control strategy, and is specifically configured to:

[0012] Obtain the human-machine distance from the user data; the human-machine distance is the distance between the user and the display device;

[0013] Adjust the gain value of the sound collector to a first target gain value according to the human-machine distance, or adjust the gain value of the sound collector to a secondary target gain value according to the human-machine distance and generate a prompt message; the prompt type of the prompt message includes a voice prompt or a text prompt.

[0014] In this way, the controller can control the gain value of the sound collector according to the human-machine distance between the user and the display device, so that the sound collector is in an optimal state during operation, which is conducive to improving the voice interaction experience between the user and the display device.

[0015] In some feasible embodiments, after the controller executes obtaining the human-machine distance from the user data, it is further configured to:

[0016] Obtain a gain mapping table; the gain mapping table includes the mapping relationship between the human-machine distance and the gain value of the sound collector;

[0017] Search for the first target gain value according to the human-machine distance;

[0018] Adjust the gain value of the sound collector to the first target gain value.

[0019] In this way, based on obtaining the gain mapping table, the controller can perform a look-up operation in combination with the human-machine distance to find the first target gain value corresponding to the current human-machine distance in the gain mapping table, thereby improving the adjustment efficiency of the gain value of the sound collector.

[0020] In some feasible embodiments, after the controller executes obtaining the human-machine distance from the user data, it is further configured to:

[0021] Calculate the first target gain value based on the initial gain value of the sound collector, the sound gain coefficient, and the human-machine distance;

[0022] Adjust the gain value of the sound collector to the first target gain value.

[0023] In this way, the controller can calculate the first target gain value by combining the initial gain value and the human-machine distance, which is beneficial to saving the storage space occupied by the gain mapping table and meeting the control requirements of the sound collector through real-time operation.

[0024] In some feasible embodiments, the secondary target gain value includes a second target gain value and a third target gain value; the second target gain value is less than the first target gain value; the third target gain value is greater than the first target gain value; the controller is configured to execute adjusting the gain value of the sound collector to the second target gain value according to the human-machine distance and generating a prompt message, specifically configured as:

[0025] Compare the initial gain value of the sound collector with the first target gain value corresponding to the human-machine distance;

[0026] If the initial gain value of the sound collector is less than the first target gain value, then increase the gain value of the sound collector

[0027] to the second target gain value, and generate a first prompt message; the first prompt message is used to prompt the user to increase the input volume of the voice command or to prompt the user to move a first preset distance in the direction closer to the display device;

[0028] If the initial gain value of the sound collector is greater than the first target gain value, then adjust the gain value of the sound collector to the third target gain value, and generate a second prompt message; the second prompt message is used to prompt the user to decrease the input volume of the voice command or to prompt the user to move a second preset distance away from the display device.

[0029] In this way, by comparing the initial gain value and the first target gain value, the controller can prompt the user to change the relative distance from the display device in combination with the user's position, so as to obtain a better sound collection effect of the sound collector without adjusting the gain of the sound collector, and avoid introducing more noise interference during the gain adjustment process, resulting in a decrease in the sound collection effect of the sound collector.

[0030] In some feasible embodiments, the preset area includes a first preset area and a second preset area; the controller is further configured to execute adjusting the gain value of the sound collector to the secondary target gain value according to the human-machine distance and generating a prompt message,

[0031] Obtain the positioning information in the user data; the positioning information is used to characterize the relative orientation between the user and the display device;

[0032] If it is determined based on the positioning information that the user is located in a first preset area, control the display to display a text prompt message;

[0033] If it is determined based on the positioning information that the user is located in a second preset area, play a voice prompt message.

[0034] In this way, the controller can determine which type of prompt message the user is more convenient to receive according to the relative orientation between the user and the display device. To improve the user experience during the interaction with the display device.

[0035] In some feasible embodiments, after the controller executes to obtain the positioning information in the user data, it is further configured to:

[0036] Based on the relative orientation between the user and the display device, control the pickup beam of the sound collector to shift towards the user, so as to enhance the sound collection effect of the sound collector facing the user.

[0037] In this way, the controller can control the pickup beam by identifying the orientation of the user relative to the display device, so that the sound collector can better receive the voice commands issued by the user based on the adjusted pickup beam.

[0038] In some feasible embodiments, the controller is further configured to:

[0039] According to the non-user data collected by the millimeter wave radar, determine the relative orientation between the noise source in the preset area and the display device;

[0040] Control the sound collector to generate a suppression beam in the direction towards the noise source, so as to reduce the influence of the noise generated by the noise source on the sound collection effect of the sound collector.

[0041] In this way, the controller can identify the noise source in the preset area and reduce the influence of the noise generated by the noise source on the normal voice collection effect by generating a suppression beam, thereby improving the sound collection effect of the sound collector.

[0042] In some feasible embodiments, the controller is further configured to:

[0043] Obtain the system time;

[0044] Determine the current running scenario according to the system time; the running scenario includes a high-frequency usage scenario and a low-frequency usage scenario;

[0045] When the operating scenario is a high-frequency usage scenario, control the millimeter-wave radar to scan the preset area at a first frequency;

[0046] When the operating scenario is a low-frequency usage scenario, control the millimeter-wave radar to scan the preset area at a second frequency; the second frequency is lower than the first frequency; the power consumption of the millimeter-wave radar operating at the second frequency is less than the power consumption of the millimeter-wave radar operating at the first frequency.

[0047] In this way, the controller can combine the system time to determine the current operating scenario of the display device. That is, in a scenario where the user has a high usage frequency, the millimeter-wave radar can be controlled to operate at a higher scanning frequency to timely detect user data in the preset area, and then timely control the operating state of the sound collector. In a scenario where the user has a low usage frequency, the millimeter-wave radar can be controlled to operate at a lower scanning frequency to save power.

[0048] In a second aspect, an embodiment of the present application provides a method for controlling a sound collector, which can be applied to the display device in the first aspect. The method includes:

[0049] Analyze the user data to obtain a first identifier or a second identifier included in the user data; the first identifier is used to represent that there is a user in the preset area; the second identifier is used to indicate that there is no user in the preset area;

[0050] When the first identifier is obtained, determine a first control strategy according to the human-machine distance, and control the operating state of the sound collector according to the first control strategy; the first control strategy is used to control the sound collector to be in a startup state and to control the gain value of the sound collector;

[0051] When the second identifier is obtained, control the operating state of the sound collector according to a second control strategy; the second control strategy is used to control the sound collector to be in a shutdown state. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the technical solutions of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, for those of ordinary skill in the art, other drawings can also be obtained based on these drawings without creative efforts.

[0053] Figure 1 A schematic diagram of an operating scenario of a display device provided by some embodiments of the present application;

[0054] Figure 2 A schematic diagram of the hardware configuration of a display device provided by some embodiments of the present application;

[0055] Figure 3 Schematic diagram of the software configuration of the display device provided in some embodiments of the present application;

[0056] Figure 4 Timing diagram of the display device provided in some embodiments of the present application for linkage control of the millimeter-wave radar and the sound collector;

[0057] Figure 5 Timing diagram of the display device provided in some embodiments of the present application for linkage control of the millimeter-wave radar and the sound collector based on middleware;

[0058] Figure 6 Schematic diagram of adjusting the gain value of the sound collector according to the human-machine distance provided in some embodiments of the present application;

[0059] Figure 7 Schematic diagram of the first method for determining the display mode of the prompt information according to the relative orientation between the user and the display device provided in some embodiments of the present application;

[0060] Figure 8 Schematic diagram of the second method for determining the display mode of the prompt information according to the relative orientation between the user and the display device provided in some embodiments of the present application;

[0061] Figure 9 Schematic diagram of the first method for adjusting the sound pickup beam of the sound collector according to the relative orientation between the user and the display device provided in some embodiments of the present application;

[0062] Figure 10 Schematic diagram of the second method for adjusting the sound pickup beam of the sound collector according to the relative orientation between the user and the display device provided in some embodiments of the present application;

[0063] Figure 11 Flow chart of controlling the scanning frequency of the millimeter-wave radar according to different scenarios provided in some embodiments of the present application. Detailed implementation manners

[0064] The embodiments will be described in detail below, and the examples are shown in the drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementation manners described in the following embodiments do not represent all the implementation manners consistent with the present application. They are only examples of the systems and methods consistent with some aspects of the present application detailed in the claims.

[0065] Embodiments will be described in detail below, and examples thereof are shown in the accompanying drawings. When the following description refers to the accompanying drawings, unless otherwise indicated, the same numerals in different drawings represent the same or similar elements. The embodiments described in the following examples do not represent all embodiments consistent with the present application. They are merely examples of systems and methods consistent with some aspects of the present application as detailed in the claims.

[0066] It should be noted that the brief description of the terms in this application is only for the convenience of understanding the embodiments described next, and is not intended to limit the embodiments of the present application. Unless otherwise specified, these terms should be understood in their ordinary and general meanings.

[0067] In this application, terms such as "first", "second", "third", etc. in the specification, claims, and the above-mentioned drawings are used to distinguish similar or like objects or entities, and do not necessarily mean to limit a specific order or sequence, unless otherwise noted. It should be understood that such terms can be interchanged under appropriate circumstances.

[0068] The terms "comprising" and "having" and any variations thereof are intended to cover but not exclude inclusion. For example, a product or device comprising a series of components does not necessarily have to be limited to all the components clearly listed, but may include other components not clearly listed or inherent to these products or devices.

[0069] The term "module" refers to any known or later-developed hardware, software, firmware, artificial intelligence, fuzzy logic, or a combination of hardware or / and software code that can perform functions related to that element.

[0070] In the embodiments of the present application, the display device 200 generally refers to a device with the ability to display images and process data. For example, the display device 200 includes, but is not limited to, smart TVs, mobile terminals, computers, monitors, advertising screens, wearable devices, virtual reality devices, augmented reality devices, etc.

[0071] Figure 1 It is a schematic diagram of the operation scenario between the display device and the control device provided for some embodiments of the present application. As Figure 1 shown, the user can operate the display device 200 through touch operations, the mobile terminal 300, and the control device 100. For example, the control device 100 can be a remote control, a stylus, a gamepad, etc.

[0072] The mobile terminal 300 can be used as a control device to perform human-computer interaction between the user and the display device 200. The mobile terminal 300 can also be used as a communication device to establish a communication connection with the display device 200 for data interaction. In some embodiments, software applications can be installed on the mobile terminal 300 and the display device 200, and the connection communication can be achieved through network communication protocols to achieve the purpose of one-to-one control operations and data communication. It is also possible to transmit the audio and video content displayed on the mobile terminal 300 to the display device 200 to achieve the synchronous display function.

[0073] As Figure 1 also shown in, the display device 200 also communicates with the server 400 for data communication through various communication methods. The display device 200 is allowed to establish a communication connection through a local area network (LAN), a wireless local area network (WLAN), and other networks.

[0074] The display device 200 can provide a broadcast receiving television function, and can also additionally provide an intelligent network television function with computer support functions, including but not limited to, network television, smart television, Internet Protocol Television (IPTV), etc.

[0075] Figure 2 For some embodiments of this application Figure 1 The hardware configuration block diagram of the display device 200 in

[0076] In some embodiments, the display device 200 may include at least one of a tuner demodulator 210, a communication device 220, a detector 230, a device interface 240, a controller 250, a display 260, an audio output device 270, a memory, a power supply, and a user input interface.

[0077] In some embodiments, the detector 230 is used to collect signals from the external environment or for external interaction. For example, the detector 230 may include a millimeter-wave radar, and the millimeter-wave radar can be used to detect whether a user is included within a preset range. The detector 230 may also include a sound collector to collect voice commands input by the user through the sound collector.

[0078] In some embodiments, the display 260 includes a display function component for presenting a picture and a driving component for driving the image display. The display 260 is used to receive the image signal output from the controller 250 for display. For example, the display 260 can be used to display video content, image content, components of a menu manipulation interface, and a user manipulation UI interface, etc.

[0079] In some embodiments, the communication device 220 is a component for communicating with external devices or servers 400 according to various communication protocol types. The display device 200 may be provided with a plurality of communication devices 220 according to different supported communication methods. For example, when the display device 200 supports wireless network communication, the display device 200 may be provided with a communication device 220 including WiFi function. When the display device 200 supports Bluetooth connection communication, the display device 200 needs to be provided with a communication device 220 including Bluetooth function.

[0080] The communication device 220 can enable the display device 200 to communicate with external devices or servers 400 through wireless or wired connection. Among them, the wired connection can connect the display device 200 with external devices through components such as data lines and interfaces. The wireless connection can connect the display device 200 with external devices through wireless signals or wireless networks. The display device 200 can directly establish a connection relationship with external devices, or can indirectly establish a connection relationship through gateways, routers, connection devices, etc.

[0081] In some embodiments, the controller 250 may include at least one of a central processing unit, a video processor, an audio processor, a graphics processor, and a power processor, and the first interface to the nth interface for input / output. The controller 250 controls the operation of the display device and responds to user operations through various software control programs stored in the memory. The controller 250 controls the overall operation of the display device 200.

[0082] In some embodiments, the controller 250 and the tuner demodulator 210 may be located in different split devices, that is, the tuner demodulator 210 may also be in an external device of the main device where the controller 250 is located, such as an external set-top box, etc.

[0083] In some embodiments, the user can input a user command on the graphical user interface (Graphical User Interface, GUI) displayed on the display 260, and then the user input interface receives the user input command through the graphical user interface (GUI).

[0084] In some embodiments, the audio output device 270 may be the native speaker of the display device 200, or may be an external audio output device connected to the display device 200. Among them, for the external audio output device connected to the display device 200, the display device 200 may also be provided with an external audio output terminal, and the audio output device can be connected to the display device 200 through the external audio output terminal to output the sound of the display device 200.

[0085] In some embodiments, the user input interface 280 can be used to receive instructions from user input. The user input interface 280 can include at least one of a microphone, a touchpad, a sensor, a remote control, etc. Further, the display device 200 can receive instructions input by the user based on the user input interface 280 to perform an interactive function with the user.

[0086] To perform user interaction, in some embodiments, the display device 200 can run an operating system. The operating system is a computer program for managing and controlling the hardware resources and software resources in the display device 200. The operating system can control the display device to provide a user interface. For example, the operating system can directly control the display device to provide a user interface, or can provide a user interface by running an application program. The operating system also allows the user to interact with the display device 200.

[0087] It should be noted that the operating system can be a native operating system based on a specific operating platform, a third-party operating system deeply customized based on a specific operating platform, or an independent operating system specially developed for the display device.

[0088] The operating system can be divided into different modules or levels according to the functions implemented. For example, as Figure 3 shown, in some embodiments, the system is divided into four layers, from top to bottom are the application layer (abbreviation: "application layer"), the application framework layer (abbreviation: "framework layer"), the system library layer, and the kernel layer.

[0089] In some embodiments, the application layer is used to provide services and interfaces for application programs so that the display device 200 can run application programs and interact with the user based on the application programs. At least one application program can run in the application layer. These application programs can be window programs, system setting programs, or clock programs, etc. that come with the operating system; they can also be application programs developed by third-party developers. In specific implementation, the application program packages in the application layer are not limited to the above examples.

[0090] The framework layer provides application programming interfaces (APIs) and programming frameworks for application programs. The application framework layer includes some predefined functions. The application framework layer is equivalent to a processing center, and this center determines the actions of the application programs in the application layer. Application programs can access the resources in the system and obtain system services through the API interfaces during execution.

[0091] As Figure 3As shown, in the embodiments of the present application, the application framework layer includes a view system, managers, a content provider, etc. Among them, the view system can design and implement the interface and interaction of the application. The view system includes lists, grids, text boxes, buttons, etc. The managers include at least one of the following modules: The Activity Manager is used to interact with all the activities running in the system; the Location Manager is used to provide access to the system location service for system services or applications; the Package Manager is used to retrieve various information related to the application packages currently installed on the device; the Notification Manager is used to control the display and clearing of notification messages; the Window Manager is used to manage icons, windows, toolbars, wallpapers, and desktop widgets on the user interface.

[0092] In some embodiments, the Activity Manager is used to manage the life cycle of each application and the general navigation back function, such as controlling the exit, opening, and back of the application. The Window Manager is used to manage all window programs, such as obtaining the display screen size, determining whether there is a status bar, locking the screen, capturing the screen, and controlling the change of the display window. For example, shrinking the display window, jittering the display, distorting the display, etc.

[0093] In some embodiments, the system runtime layer can provide support for the framework layer. When the framework layer is used, the operating system will run the instruction library contained in the system runtime layer, such as the C / C++ instruction library, to implement the functions that the framework layer needs to achieve.

[0094] In some embodiments, the kernel layer is a functional layer between the hardware and software of the display device 200. The kernel layer can implement functions such as hardware abstraction, multitasking, and memory management. For example, as Figure 3 shown, hardware drivers can be configured in the kernel layer. The drivers included in the kernel layer can be at least one of the following drivers: audio driver, display driver, Bluetooth driver, camera driver, WIFI driver, USB driver, HDMI driver, sensor drivers (such as fingerprint sensors, temperature sensors, pressure sensors, etc.), and power drivers, etc.

[0095] In some embodiments, the display device 200 may be equipped with a variety of detectors 230 to adapt to richer application scenarios. For example, the display device 200 may be equipped with a sound collector to facilitate voice interaction with the user, reduce the number of times the user operates the remote control, and thus improve the user experience.

[0096] However, during the use of the sound collector, being in the on state for a long time will additionally increase the power consumption of the display device. Even when in an operating state that requires user wake-up, it may be accidentally woken up due to noise sources such as pets, thus affecting the normal operation of the display device 200.

[0097] To solve the above problems, an embodiment of the present application provides a display device 200, which includes a display 260, a millimeter-wave radar, a sound collector, and a controller 250. Among them, the millimeter-wave radar can be used to detect user data in a preset area, and the user data may include, but is not limited to, the human-machine distance between the user and the display device 200, the presence information of the user, and the relative orientation between the user and the display device 200. Then, the controller 250 can control the operating state of the sound collector according to the user data.

[0098] As Figure 4 shown, the controller 250 is configured to:

[0099] Analyze the user data to obtain a first identifier or a second identifier included in the user data.

[0100] When the first identifier is obtained, control the operating state of the sound collector according to a first control strategy. The first control strategy is used to control the sound collector to be in a startup state and to control the gain value of the sound collector;

[0101] When the second identifier is obtained, control the operating state of the sound collector according to a second control strategy. The second control strategy is used to control the sound collector to be in a closed state.

[0102] It should be noted that the first identifier is used to represent the presence of a user in the preset area, and the second identifier is used to indicate the absence of a user in the preset area. The first control strategy includes adjusting the startup state of the sound collector and adjusting the gain value of the sound collector.

[0103] In some embodiments, the millimeter-wave radar continuously scans the preset area. When a reflected signal conforming to human body characteristics is detected, user data containing the first identifier is generated. After receiving the user data, the controller 250 extracts the first identifier in the user data through a parsing algorithm, and then confirms the presence of a user in the preset area. The controller 250 then sends a startup instruction to the sound collector to make it enter the working state and start receiving voice signals.

[0104] As Figure 5 shown, the millimeter-wave radar can form user data from the data obtained by scanning a preset area according to a preset protocol, and report the user data to a pre-configured middleware. Then, the middleware receives the user data and sends various identifiers (the first identifier, the second identifier, the human-machine distance, and the relative azimuth) included in the user data to an application or component that needs the user data to run a specific function. For example, various identifiers in the user data are fed back to an application for controlling the operating state of the voice collector, and the application can control the start / stop and gain of the voice collector based on these identifiers.

[0105] During the scanning phase, the millimeter-wave radar can detect whether there is a user in the preset area based on the method of transmitting continuous linear frequency modulation pulses and receiving echoes. For example, based on the Doppler effect, when the linear frequency modulation pulses emitted by the millimeter-wave radar encounter fluctuations similar to the human chest cavity, frequency shift and phase change will occur. Then, the millimeter-wave radar can determine whether there are movements similar to the fluctuations of the human chest cavity in the preset area by monitoring the phase and frequency of the echo, and further determine whether there is a user in the preset area, and further report relevant information to the controller 250.

[0106] Moreover, the millimeter-wave radar can also determine the human-machine distance between the user and the display device 200 by combining the detection of the user's breathing (chest cavity fluctuations) and heartbeat. During this kind of detection, the human body's breathing is manifested as a periodic signal with a lower frequency, such as 0.1 - 0.5 Hz; the heartbeat is manifested as a periodic signal with a higher frequency, such as 1 - 2 Hz. Then, the millimeter-wave radar can calculate the human-machine distance between the user and the display device 200 by analyzing the frequency difference between the echo and the transmitted wave.

[0107] For example, the following formula can be used to calculate the human-machine distance:

[0108]

[0109] where d 人体 is the human-machine distance, Δf is the frequency difference between the echo and the transmitted wave, S is the frequency modulation slope, and c is the speed of light.

[0110] In this way, the millimeter-wave radar can detect and obtain multiple pieces of information related to the user's state during the scanning process and report them to the controller 250. Then, the controller 250 performs linkage control based on this information related to the user's state.

[0111] In addition, on the basis of starting the voice collector, the controller 250 can also adjust the gain value of the voice collector so that the gain value of the voice collector is in the state most suitable for receiving the user's voice command, thereby improving the voice collection effect of the voice collector and being beneficial to improving the quality of voice interaction.

[0112] It can be understood that various parameters of the sound collector can be adjusted in combination with user data. For example, the gain value can be adjusted according to the relative distance between the user and the display device 200, and the pickup beam range of the sound collector can also be adjusted according to the relative orientation between the user and the display device 200 to obtain a better sound pickup effect.

[0113] In some other embodiments, when the millimeter-wave radar continuously fails to detect the human body feature signal, the controller 250 analyzes the user data to obtain a second identifier and determines that there is no user in the preset area. The controller 250 immediately sends a shutdown instruction to the sound collector to cut off its power supply and stop the voice signal receiving function.

[0114] In this way, through the linkage control of the millimeter-wave radar and the sound collector, the energy consumption of the sound collector can be reduced in the scenario without users, so as to save the overall energy consumption of the display device 200. Moreover, it can also prevent the sound collector from being accidentally awakened by noise caused by pets or other noise sources, resulting in abnormal operation of the display device 200 and other situations.

[0115] In some embodiments, when the controller 250 determines that there is a user in the preset area, it can also adjust the gain of the sound collector according to the voice interaction state of the user. For example, when the sound collector is in the startup state but no valid voice command is received, the controller 250 triggers the gain value attenuation logic according to the preset silence threshold. If the silence duration exceeds the threshold, the gain value is gradually reduced step by step until the lowest working threshold is reached; if voice activity is detected, the initial gain value is immediately restored.

[0116] In this way, on the premise of maintaining the operation of the sound collector, the circuit power consumption is reduced through progressive gain attenuation, and at the same time, it is ensured that the best sensitivity can be quickly restored when the user initiates a command at any time, achieving a balance between energy saving and response efficiency.

[0117] It should be noted that the millimeter-wave radar can be in a continuous scanning state, and thus can feedback user data to the controller 250 in real time. The controller 250 can determine whether there is a user in the preset area according to the user data feedback by the millimeter-wave radar, and then timely adjust the operation state of the sound collector. Among them, the scanning frequency of the millimeter-wave radar can be adjusted in real time according to the state of the user in the preset area. For example, when the user stays in the preset area for a long time, the controller 250 can determine that the current scenario is a long-time viewing scenario of the user. Therefore, the controller 250 can reduce the scanning frequency of the millimeter-wave radar to save power consumption and is beneficial to extending the service life of the millimeter-wave radar.

[0118] Such as Figure 6As shown, in some embodiments, the controller 250 may also adjust the gain value of the sound collector according to the human-machine distance between the user and the display device 200, so that the sound collector is in the best sound pickup state, thereby ensuring the quality of voice interaction between the user and the display device 200. That is, the controller 250 executes to control the operating state of the sound collector according to the first control strategy, and is specifically configured as:

[0119] Obtain the human-machine distance from the user data; the human-machine distance is the distance between the user and the display device.

[0120] Adjust the gain value of the sound collector to the first target gain value according to the human-machine distance, or adjust the gain value of the sound collector to the secondary target gain value according to the human-machine distance and generate a prompt message.

[0121] In some embodiments, the prompt message may include information presented in a variety of multimedia manners, such as text information, voice information, and backlight transformation. Furthermore, through the prompt message, the user can be prompted to adjust the human-machine distance from the display device 200 to cooperate with the gain adjustment of the sound collector, which is beneficial to keeping the sound pickup effect of the sound collector in the best state.

[0122] In some embodiments, when a user is scanned in the preset area of the millimeter-wave radar, the human-machine distance data between the user and the display device 200 may also be added to the user data, so that the controller 250 can identify the human-machine distance between the user and the display device 200, and then control the sound collector according to the human-machine distance.

[0123] In some embodiments, the sound collector may have an initial gain value to adapt to a wide range of application scenarios. In order to enable the sound collector to obtain a better sound pickup effect in a voice interaction scenario, the controller may adjust the gain value of the sound collector according to the human-machine distance. For example, the greater the human-machine distance, the greater the gain value of the sound collector can be adjusted to obtain a better sound pickup effect. Conversely, the smaller the human-machine distance, the smaller the gain value of the sound collector can be adjusted.

[0124] In this way, even when the user is far from the display device 200, the sound pickup effect of the sound collector can be ensured by increasing the gain; when the user is close to the display device 200, the problem of signal distortion caused by the excessive intensity of the voice signal input by the user exceeding the processing range of circuit elements such as amplifiers can also be avoided by reducing the gain.

[0125] In some other embodiments, taking the first gain value as an example, the first gain value is the maximum gain value or the minimum gain value corresponding to the current human-machine distance. The controller 250 can also adjust the gain value to a secondary gain value that is only slightly lower than the first gain value, and display a prompt message to the user to prompt the user to reduce the distance from the display device 200.

[0126] In this way, by combining the adjustment of the gain value of the sound collector with the adjustment of the human-machine distance, it is possible to not only ensure the sound pickup effect of the sound collector, but also prevent the problem that the noise is amplified synchronously due to adjusting the gain value of the sound collector to an excessive value, which in turn affects the sound pickup effect.

[0127] It can be understood that the controller 250 can form a diversified prompt effect by increasing the richness of the prompt message, so that the user can timely know how to adjust the human-machine distance. For example, the controller 250 can control the display 260 to display a prompt text, such as "Please take a step forward", so that the user can clearly know the adjustment range. For another example, the controller 250 can control a voice playback device such as a speaker to play "Please take a step forward", which can also enable the user to clearly know the adjustment range.

[0128] In addition, the controller 250 can also cooperate with other components carried by the display device 200 to map the adjustment result of the gain value of the sound collector. For example, by adjusting the backlight brightness of the backlight component to represent the adjustment result of the gain value, when the gain value is increased, the backlight can be controlled to flash step by step in an enhanced manner, and when the gain value is decreased, the backlight can be controlled to flash step by step in a weakened manner. Thus, a rich prompt effect is provided.

[0129] In some embodiments, the controller 250 can determine the first target gain value by calling the gain mapping table in the storage structure, so as to dynamically adjust the gain value of the sound collector to the first target gain value. That is, after the controller 250 obtains the human-machine distance from the user data, it is further configured to:

[0130] Obtain the gain mapping table. Search for the first target gain value according to the human-machine distance. Adjust the gain value of the sound collector to the first target gain value.

[0131] Among them, the gain mapping table can include the mapping relationship between the human-machine distance and the gain value of the sound collector. Then, after the controller 250 identifies the human-machine distance from the user data, it can perform a look-up operation in the gain mapping table based on the human-machine distance to determine the first target gain value. By presetting the gain mapping table, the efficiency of adjusting the gain value of the sound collector can be improved.

[0132] In some other embodiments, the gain mapping table may also form a mapping relationship in combination with the azimuth parameter, that is, the gain mapping table may include the mapping relationship among azimuth, human-machine distance, and gain value. And an identification identifier regarding the azimuth may be added in the table header. Correspondingly, the relative azimuth data between the user and the display device 200 may also be added to the user data fed back by the millimeter-wave radar to the controller 250. In this way, the controller 250 can identify the corresponding gain mapping table according to the azimuth / human-machine distance, and further perform a look-up in the identified gain mapping table to obtain an appropriate gain value, and adjust the gain value of the sound collector to the appropriate gain value obtained by the look-up.

[0133] In this way, adding the azimuth parameter in the stage of adjusting the gain value can make the adjustment of the gain value of the sound collector more precise. For example, when the user is on the left side of the display device, the controller 250 can identify that the azimuth information is "left", and then look up the gain value corresponding to the "left" azimuth and the current human-machine distance in the gain mapping table, and adjust the gain value of the sound collector to this gain value. Similarly, when the user is in other azimuths of the display device, a similar gain value adjustment logic can also be implemented.

[0134] In some embodiments, in order to relieve the pressure on the storage space, the controller 250 may adjust the gain value of the sound collector to the target gain value by means of real-time calculation of the gain value. That is, after the controller 250 obtains the human-machine distance from the user data, it is further configured to:

[0135] Calculate the first target gain value based on the initial gain value of the sound collector, the sound gain coefficient, and the human-machine distance. Adjust the gain value of the sound collector to the first target gain value.

[0136] In some embodiments, the controller 250 may calculate the first target gain value according to a gain calculation formula. The gain calculation formula is as follows:

[0137] mic gain = A + Kd 人体 ;

[0138] where A is the initial gain value of the sound collector, K is the influence coefficient of the human-machine distance on the sound collector, and d 人体 is the human body distance.

[0139] In this way, during the process of real-time calculating the gain value, by reasonably setting the sound gain coefficient K, it can be ensured that the gain value of the sound collector adjusts smoothly with the change of the human-machine distance, thereby avoiding the problem of unstable sound pickup effect caused by the sudden change of the gain value. For example, when the human-machine distance increases, the calculated gain value will also increase accordingly to ensure that the sound collector can still clearly pick up the user's voice command at a long distance. On the contrary, when the human-machine distance decreases, the calculated gain value will also decrease accordingly to avoid the problem of signal distortion caused by close-range sound pickup.

[0140] In addition, considering that different users may have different preferences for the sensitivity of the sound collector, the setting menu of the display device 200 may also include an option control for adjusting the sound gain coefficient K. Based on the setting of the option control, it is also possible to allow the user to customize the initial gain value A of the sound collector and the sound gain coefficient K. In this way, the user can adjust the gain value of the sound collector according to their own usage habits and needs to obtain the best sound pickup effect.

[0141] In some embodiments, during the process of adjusting the gain value of the sound collector, the controller 250 can balance the relationship between the gain value and the human-machine distance. For example, the controller 250 can adjust the gain value of the sound collector to a second target gain value or a third target gain value. Among them, the second target gain value is less than the first target gain value, and the third target gain value is greater than the first target gain value. That is, the controller 250 is configured to adjust the gain value of the sound collector to a secondary target gain value according to the human-machine distance and generate a prompt message, specifically configured as:

[0142] Compare the initial gain value of the sound collector with the first target gain value corresponding to the human-machine distance.

[0143] If the initial gain value of the sound collector is less than the first target gain value, adjust the gain value of the sound collector to the second target gain value and generate a first prompt message. The first prompt message is used to prompt the user to increase the input volume of the voice command or to prompt the user to move a first preset distance in the direction close to the display device.

[0144] If the initial gain value of the sound collector is greater than the first target gain value, adjust the gain value of the sound collector to the third target gain value and generate a second prompt message. The second prompt message is used to prompt the user to decrease the input volume of the voice command or to prompt the user to move a second preset distance away from the display device.

[0145] In some embodiments, the controller 250 may determine a gain value adjustment strategy by comparing the initial gain value of the sound collector with the first target gain value corresponding to the current human-machine distance. For example, when the first target gain value is greater than the initial gain value, the controller 250 may determine that it is necessary to adjust the initial gain value of the sound collector to a larger gain value. And at this time, the controller 250 may only adjust the initial gain value of the sound collector to the second target gain value. The second target gain value is slightly less than the first target gain value. Moreover, the controller 250 may prompt the user to actively move closer to the display device 200 by displaying a prompt message to the user.

[0146] In this way, by adjusting the initial gain value to the second target gain value, on the one hand, although the sound pickup effect of the sound collector is reduced to a certain extent, the influence of noise on the sound pickup effect of the sound collector is also relatively reduced. On the other hand, by prompting the user to move closer to the display device 200, the sound pickup effect of the sound collector at a relatively low gain value can be compensated.

[0147] In some embodiments, the millimeter-wave radar continuously detects the human-machine distance, but the gain value of the sound collector has an upper limit, and signal distortion will occur when the gain value exceeds the upper limit. Therefore, when the sound collector is in the maximum gain value state, even if the human-machine distance increases, the controller 250 will not further increase the gain value, but generate a prompt message to prompt the user that the sound collector has reached the maximum gain value.

[0148] It can be understood that the first prompt message and the second prompt message are used to distinguish the content of the prompt messages generated by the controller 250 in different scenarios, rather than a limitation on the number and order of the prompt messages.

[0149] In some other embodiments, the controller 250 determines by comparison that the initial gain value of the sound collector is greater than the first target gain value. At this time, the controller 250 may adjust the gain value of the sound collector from the initial gain value to the third target gain value. At the same time, the controller 250 may also display a prompt message to the user to prompt the user to increase the human-machine distance from the display device 200. Furthermore, by combining the change in the human-machine distance and the adjustment of the gain value, the sound collector has a better sound pickup effect. The prompt message may be, for example, "Please move slightly away from the display device" or "Please take a step back", so as to clearly inform the user how to adjust the human-machine distance from the display device 200. In this way, the controller 250 can not only effectively optimize the sound pickup effect of the sound collector, but also ensure that the user obtains intuitive and easy-to-understand guidance during the operation, thereby improving the overall human-machine interaction experience.

[0150] Based on the linkage adjustment method of the human-machine distance and the gain value in the above embodiments, this intelligent adjustment mechanism can also dynamically adjust the gain value in combination with the change of environmental noise to ensure the best sound pickup effect in different usage scenarios. The user adjusts the distance according to the prompt, which not only avoids signal distortion but also improves the accuracy of speech recognition, further enhancing the intelligence and practicality of the system.

[0151] As Figure 7 and Figure 8 shown, the controller 250 can also control the state of the sound collector in combination with the relative orientation between the user and the display device 200, that is, the controller 250 is further configured to:

[0152] Obtain the positioning information in the user data; the positioning information is used to characterize the relative orientation between the user and the display device.

[0153] If it is determined based on the positioning information that the user is located in the first preset area, control the display to display a text prompt message.

[0154] If it is determined based on the positioning information that the user is located in the second preset area, play a voice prompt message.

[0155] In some embodiments, the millimeter-wave radar can detect the relative orientation between the user and the display device 200, and then adjust the display mode of the prompt information according to the relative orientation.

[0156] It can be understood that the controller 250 can divide the area that the millimeter-wave radar can scan into multiple sub-areas, that is, divide the preset area into multiple sub-areas, such as the first preset area and the second preset area. Then the controller 250 can select the way to display the prompt information to the user by judging the area where the user is located.

[0157] In some embodiments, when the user is in the first preset area of the display device 200, that is, the area facing the display device 200, the controller 250 can control the display 260 to display a prompt text to prompt the user to adjust the human-machine distance between the user and the display device 200.

[0158] In this way, through a prompt method that is convenient for the user to observe and understand, the user can quickly know how to adjust the position, so as to effectively optimize the sound collection effect, improve the accuracy of speech recognition, and make the human-machine interaction more fluent.

[0159] In some other embodiments, when the user is in the second preset area of the display device 200, that is, in the side area of the display device 200, the controller 250 can control the speaker mounted on the display device 200 to play a voice prompt, and guide the user to adjust to the optimal position through the content of the voice prompt, ensuring that the pick-up effect of the sound collector can be guaranteed by combining the adjustment of the human-machine distance and the gain value adjustment, so as to achieve more accurate speech recognition and improve the user experience.

[0160] In this way, based on the display mode of multi-dimensional prompt information, the controller 250 can dynamically adjust the prompt strategy according to the real-time position of the user, ensuring that the user can receive the most appropriate prompt information at any position, and further optimizing the human-machine interaction process.

[0161] On the basis that the prompt information can be richly displayed, the controller 250 can better prompt the user to reach the target position to cooperate with the gain value adjustment strategy of the sound collector. In this way, in a noisy scenario, the problem that the pick-up effect is poor due to the single adjustment of the gain value of sound collection, which is likely to increase the proportion of noise in the collected signal, can be alleviated. And through the increased interaction process between the display device 200 and the user, the user experience is improved.

[0162] It can be understood that in the embodiments of the present application, there is no limitation on how to divide the preset area. For example, the scanning angle of the millimeter-wave radar can be divided into several segments, and then the preset area can be divided based on the scanning angle, ensuring that each sub-area covers a specific angle range, so as to achieve accurate judgment of the user's position.

[0163] As Figure 9 and Figure 10 shown, in some embodiments, the controller 250 can also control the pick-up beam of the sound collector according to the relative orientation between the user and the display device 200, so that the sound collector has a better pick-up effect. That is, after the controller executes to obtain the positioning information in the user data, it is further configured to:

[0164] Based on the relative orientation between the user and the display device, control the pick-up beam of the sound collector to shift towards the user to enhance the sound collection effect of the sound collector facing the user.

[0165] In some embodiments, the controller 250 can determine the relative orientation between the user and the display device 200 by parsing the relative orientation identifier in the user data. For example, if the controller 250 determines that the user is on the left side of the display device, it can control the sound collector to enhance the pick-up beam in the left range to better receive the voice command issued by the user. Another example is that if the controller 250 determines that the user is on the right side of the display device, it can control the sound collector to enhance the pick-up beam in the right range to better receive the voice command issued by the user.

[0166] In some other embodiments, the controller 250 can further refine the ability to identify the relative orientation of the user, thereby synchronously improving the control accuracy of the sound pickup beam of the sound collector. For example, by identifying the orientation information in the user data, the controller 250 determines that the user is within a preset scanning range of 0° - 30°, and then can enhance the sound pickup beam of the sound collector at 0° - 30° to ensure the best sound collection effect within this angular range. Similarly, if the user is within the range of 30° - 60°, the sound pickup beam of the sound collector at 30° - 60° is correspondingly enhanced.

[0167] In this way, on the one hand, the sound pickup effect of the sound collector can be enhanced, and on the other hand, noise interference can be reduced, which is beneficial to improving the signal-to-noise ratio and further optimizing the process and effect of voice interaction.

[0168] In some embodiments, the controller 250 can also judge the noise source through the non-user data fed back by the millimeter-wave radar, and then weaken the sound pickup beam in the direction of the noise source to avoid the interference of the noise on the sound pickup effect. That is, the controller 250 is further configured to:

[0169] According to the non-user data collected by the millimeter-wave radar, determine the relative orientation of the noise source in the preset area and the display device.

[0170] Control the sound collector to generate an inhibition beam in the direction towards the noise source to reduce the influence of the noise generated by the noise source on the sound collection effect of the sound collector.

[0171] In some embodiments, the non-user data can refer to noise sources in the scene. The noise sources can include but are not limited to pet noises, natural noises (wind sounds or other environmental sounds). The noise generated by the noise sources will cause greater interference to the voice interaction process. Therefore, the controller 250 can combine the detection ability of the millimeter-wave radar to reduce the influence of such noises on voice interaction.

[0172] In some embodiments, the millimeter-wave radar can provide the action trajectory of the noise source to the controller 250. Then, the controller 250 can judge the presence of a pet through the comparison of the action trajectories, and then control the sound collector to generate an inhibition beam in the pet activity area to reduce the influence of the noise generated by the pet activity on the voice interaction process.

[0173] It can be understood that the judgment method of the action trajectory is not limited to pets, and can also be extended to other objects that generate sounds during movement. For example, the vibration sounds or friction sounds generated by moving objects will also interfere with the sound pickup effect of the sound collector. Then, the controller 250 can effectively generate an inhibition beam by combining the judgment of the action trajectory to prevent noise interference, thereby effectively improving the signal-to-noise ratio and being beneficial to improving the quality of voice interaction.

[0174] In some other embodiments, the millimeter-wave radar can identify environmental noises such as the sound of wind near an open window. Then, after receiving the user data fed back by the millimeter-wave radar, the controller 250 can determine the environmental noise source and further suppress the sound collection beam in the direction of the environmental noise source. For example, the method of generating a suppression beam can be adopted to cancel the noise generated by the noise source.

[0175] It can be understood that when the controller 250 generates a suppression beam, it can also determine the angle range corresponding to the suppression beam according to the angle range where the noise source is located, thereby effectively suppressing the interference of the noise on the voice interaction process. In addition, the living room scene where the display device 200 is located belongs to a relatively stable scene, and the relative position between the display device 200 and noise sources such as windows is relatively fixed. Therefore, the controller 250 can combine historical data to perform more accurate positioning and prediction of the noise source, dynamically adjust the suppression strategy, ensure the accuracy of voice recognition, and further guarantee the user experience during the voice interaction process.

[0176] In addition, in some embodiments, the controller 250 needs to verify the authenticity of the user data in the millimeter-wave radar. For example, a signal quantity threshold can be preset, that is, when the controller 250 continuously identifies the first identifier from the user data in the millimeter-wave radar, it is considered that the user enters the preset area, and then the operating state of the sound collector is adjusted.

[0177] In some other embodiments, a verification strategy can also be set to adapt to the situation where the user briefly leaves the preset area and then returns to the preset area, thereby avoiding the problem that the repeated start and stop of the sound collector caused by the user briefly leaving the preset area affects the normal operation of the display device 200. It can also extend the service life of the sound collector.

[0178] As Figure 11 shown, the millimeter-wave radar needs to continuously operate to detect the user / noise source in the preset area in a timely manner. Therefore, the power consumption of the millimeter-wave radar also needs to be reasonably controlled to ensure that while it efficiently detects, it will not be over-consumed, and it is beneficial to extend the service life of the millimeter-wave radar. That is, the controller is further configured to:

[0179] Obtain the system time.

[0180] Determine the current operating scenario according to the system time. The operating scenario includes a high-frequency usage scenario and a low-frequency usage scenario;

[0181] When the operating scenario is a high-frequency usage scenario, control the millimeter-wave radar to scan the preset area at a first frequency.

[0182] When the operating scenario is a low-frequency usage scenario, control the millimeter-wave radar to scan the preset area at a second frequency. The second frequency is lower than the first frequency; the power consumption of the millimeter-wave radar operating at the second frequency is less than the power consumption of the millimeter-wave radar operating at the first frequency.

[0183] In some embodiments, the system time refers to the time that the controller 250 can currently obtain. The controller 250 can obtain accurate system time through methods such as networking, and then match it with the preset operating scenario.

[0184] The operating scenario here refers to a scenario constructed in advance according to the user's usage habits, which can be divided according to the user's working hours, rest hours, and activity patterns to ensure that the scanning frequency of the millimeter-wave radar matches the user's actual needs during different time periods.

[0185] For example, between 17:00 and 18:00 on weekend evenings, users often cook in the kitchen. At this time, the controller 250 can set the scenario corresponding to this time period as a low-frequency usage scenario, and then control the millimeter-wave radar to scan the preset area at a second frequency.

[0186] Another example is that between 19:00 and 21:00 on weekend evenings, users often use the display device 200 in the living room, and during the process of using the display device 200, there are often voice interactions, operations such as picking up and placing drinks and food. Then the controller 250 can set the scenario corresponding to this time period as a high-frequency usage scenario, and then control the millimeter-wave radar to scan the preset area at the first frequency to timely control the sound collector to cooperate with the user's daily use.

[0187] In this way, by adjusting the scanning frequency of the millimeter-wave radar, not only can energy consumption be effectively reduced, but also the user experience can be significantly improved, ensuring that the device can operate efficiently in different scenarios and extending the overall service life.

[0188] It can be understood that even in the high-frequency usage scenario or the low-frequency usage scenario, the usage scenario can still be further divided to optimize the control of the scanning frequency of the millimeter-wave radar at a deeper level.

[0189] For example, in the high-frequency usage scenario, if it is detected that the user has not had voice interactions or activity signs for a long time, the controller 250 can control the millimeter-wave radar to reduce the scanning frequency to further save energy. Similarly, in the low-frequency usage scenario, if it is detected that the user's activity frequency suddenly increases, the controller 250 can immediately increase the scanning frequency to ensure timely response to the user's needs.

[0190] Through refined scenario division and dynamic frequency adjustment, the system can not only accurately capture user behavior, but also maximize energy conservation and consumption reduction while ensuring functionality, as well as extend the service life of the millimeter-wave radar.

[0191] In some embodiments, a method for controlling a sound collector is further provided. The method can be applied to a display device 200. The steps of the method include:

[0192] Analyze the user data to obtain a first identifier or a second identifier included in the user data; the first identifier is used to represent that there is a user in the preset area; the second identifier is used to indicate that there is no user in the preset area.

[0193] When the first identifier is obtained, determine a first control strategy according to the human-machine distance, and control the operating state of the sound collector according to the first control strategy; the first control strategy is used to control the sound collector to be in a startup state and to control the gain value of the sound collector.

[0194] When the second identifier is obtained, control the operating state of the sound collector according to a second control strategy; the second control strategy is used to control the sound collector to be in a shutdown state.

[0195] In some embodiments, the method for controlling a sound collector provided by the embodiments of the present application can be applied to a display device 200 configured with a millimeter-wave radar and a sound collector. The display device 200 can construct a linkage control architecture between the millimeter-wave radar and the sound collector, and then optimize the control method of the sound collector based on the detection information provided by the millimeter-wave radar. On the one hand, the operating mode of the sound collector can be optimized, and on the other hand, the voice interaction process with the user can be optimized. And it can also extend the service life of related components such as the millimeter-wave radar while ensuring the voice interaction quality. Through this linkage mechanism, the device can more intelligently identify the user state.

[0196] As can be seen from the above technical content, the present application provides a method for controlling a display device and a sound collector. The display device receives user data sent by the millimeter-wave radar, analyzes and judges whether there is a user in the preset area, and then controls the operating state of the sound collector to form a multi-sensor linkage control mechanism, which is beneficial to reducing the device energy consumption. And in the scenario where there is a user in the preset area, the gain value can also be dynamically adjusted according to the specific position of the user to ensure the accuracy of sound collection, further improving the fluency and naturalness of the voice interaction and optimizing the user experience.

[0197] For the similar parts between the embodiments provided by the present application, reference can be made to each other. The specific embodiments provided above are only several examples under the general concept of the present application, and do not constitute a limitation on the protection scope of the present application. For those skilled in the art, any other implementation manner extended based on the solution of the present application without creative efforts belongs to the protection scope of the present application.

Claims

1. A display device, characterized in that, Including: A display; A millimeter-wave radar configured to collect user data within a preset area; A sound collector configured to receive voice commands input by a user; A controller configured to: Analyze the user data to obtain a first identifier or a second identifier included in the user data; the first identifier is used to characterize the presence of a user within the preset area; the second identifier is used to indicate the absence of a user within the preset area; When the first identifier is obtained, determine a first control strategy according to the human-machine distance, and control the operating state of the sound collector according to the first control strategy; the first control strategy is used to control the sound collector to be in a startup state and to control the gain value of the sound collector; When the second identifier is obtained, control the operating state of the sound collector according to a second control strategy; the second control strategy is used to control the sound collector to be in a shutdown state.

2. The display device according to claim 1, wherein The controller executes controlling the operating state of the sound collector according to the first control strategy, and is specifically configured to: Obtain the human-machine distance from the user data; the human-machine distance is the distance between the user and the display device; Adjust the gain value of the sound collector to a first target gain value according to the human-machine distance, or adjust the gain value of the sound collector to a secondary target gain value according to the human-machine distance and generate a prompt message; the prompt type of the prompt message includes a voice prompt or a text prompt.

3. The display device according to claim 2, wherein After the controller executes obtaining the human-machine distance from the user data, it is further configured to: Obtain a gain mapping table; the gain mapping table includes the mapping relationship between the human-machine distance and the gain value of the sound collector; Look up the first target gain value according to the human-machine distance; Adjust the gain value of the sound collector to the first target gain value.

4. The display device according to claim 2, wherein After the controller executes obtaining the human-machine distance from the user data, it is further configured to: Calculate the first target gain value based on the initial gain value of the sound collector, the sound gain coefficient, and the human-machine distance; Adjust the gain value of the sound collector to the first target gain value.

5. The display device according to claim 2, characterized in that, The secondary target gain value includes a second target gain value and a third target gain value; the second target gain value is less than the first target gain value; the third target gain value is greater than the first target gain value; the controller executes adjusting the gain value of the sound collector to the second target gain value according to the human-machine distance and generating a prompt message, and is specifically configured to: Compare the initial gain value of the sound collector with the first target gain value corresponding to the human-machine distance; If the initial gain value of the sound collector is less than the first target gain value, adjust the gain value of the sound collector to the second target gain value and generate a first prompt message; The first prompt message is used to prompt the user to increase the input volume of the voice command or to prompt the user to move a first preset distance in the direction of approaching the display device; If the initial gain value of the sound collector is greater than the first target gain value, adjust the gain value of the sound collector to the third target gain value and generate a second prompt message; The second prompt message is used to prompt the user to reduce the input volume of the voice command or to prompt the user to move a second preset distance away from the display device.

6. The display device according to claim 2, wherein The preset area includes a first preset area and a second preset area; the controller is configured to adjust the gain value of the sound collector to a secondary target gain value according to the human-machine distance and generate a prompt message, and is further configured to: Obtain the positioning information in the user data; the positioning information is used to characterize the relative orientation of the user and the display device; If it is determined based on the positioning information that the user is located in the first preset area, control the display to display a text prompt message; If it is determined based on the positioning information that the user is located in the second preset area, play a voice prompt message.

7. The display device according to claim 6, wherein After the controller obtains the positioning information in the user data, it is further configured to: Based on the relative orientation of the user and the display device, control the pickup beam of the sound collector to shift towards the user to enhance the sound collection effect of the sound collector facing the user.

8. The display device according to claim 2, wherein The controller is further configured to: According to the non-user data collected by the millimeter-wave radar, determine the relative orientation of the noise source in the preset area and the display device; Control the sound collector to generate a suppression beam in the direction of the noise source to reduce the influence of the noise generated by the noise source on the sound collection effect of the sound collector.

9. The display device according to claim 1, wherein The controller is further configured to: Obtain the system time; Determine the current operating scenario according to the system time; the operating scenario includes a high-frequency usage scenario and a low-frequency usage scenario; When the operating scenario is a high-frequency usage scenario, control the millimeter-wave radar to scan the preset area at a first frequency; When the operating scenario is a low-frequency usage scenario, control the millimeter-wave radar to scan the preset area at a second frequency; the second frequency is lower than the first frequency; the power consumption of the millimeter-wave radar operating at the second frequency is less than the power consumption of the millimeter-wave radar operating at the first frequency.

10. A method for controlling a sound collector, characterized in that, Applied to the display device according to any one of claims 1-9, the method includes: Parse the user data to obtain the first identifier or the second identifier included in the user data; the first identifier is used to characterize the presence of a user in the preset area; the second identifier is used to indicate the absence of a user in the preset area; When the first identifier is obtained, determine a first control strategy according to the human-machine distance and control the operating state of the sound collector according to the first control strategy; the first control strategy is used to control the sound collector to be in a startup state and to control the gain value of the sound collector; When the second identifier is obtained, control the operating state of the sound collector according to a second control strategy; the second control strategy is used to control the sound collector to be in a shutdown state.

Images