Display device, sound field adjustment method, storage medium and program product
By detecting user voice data and adjusting the sound field of the display device, the sound imbalance caused by the user's viewing position is not in the center of the sound field, and the improvement of the balanced sound experience is achieved.
Patent Information
- Application Number
- CN202210498140.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-05-09
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2042-05-09
AI Technical Summary
When a user is watching a display device, the viewing position selected is usually not in the center of the sound field, resulting in uneven sound and reducing the user experience.
By detecting the target voice data of the user, determining the incident angle, and adjusting the sound field of the display device based on this, ensuring that the user is always at the center of the sound field of the sound source, and using voice interaction to adjust the center of the sound field in real time.
It realizes a balanced sound experience when users watch content, improving users' viewing experience.
Smart Images

Figure CN115022716B_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of display technology, and in particular to a display device, a sound field adjustment method, a storage medium, and a program product. Background Art
[0002] The display device can display some multimedia contents to the user for the user to browse, thereby enriching the user's spare time.
[0003] When users watch a display device, they usually choose a viewing position based on their personal viewing needs. However, the viewing position selected by the user may not be the center of the sound field of the display device. Especially when the user's viewing position is far away from the center of the sound field of the display device, it will affect the sound emitted by the display device felt by the user's left and right ears, causing imbalance, thereby greatly reducing the user experience. Summary of the invention
[0004] In order to solve the above technical problems or at least partially solve the above technical problems, the present disclosure provides a display device, a sound field adjustment method, a storage medium and a program product, which can perform voice interaction with the display device in real time when the user is watching the playback content of the display device, so that the display device can adjust the sound field center position of the playback source based on the voice interaction content to always be consistent with the user, so that the user can feel a balanced sound and improve the user experience.
[0005] In a first aspect, the present disclosure provides a display device, the display device comprising:
[0006] The detector is configured to: collect target voice data of the user, where the target voice data is a target wake-up word for sound field calibration of the display device;
[0007] a controller configured to: when the target wake-up word is a preset wake-up word, determine an incident angle of the target voice data based on direction information of the target voice data, where the incident angle of the target voice data is a relative angle between the user and the display device;
[0008] Based on the incident angle of the target voice data, the sound field of the display device is adjusted.
[0009] In some embodiments of the present disclosure, the controller is specifically configured as follows:
[0010] Determining an equalization parameter of a sound equalizer of the display device based on an incident angle of the target voice data, wherein the equalization parameter is used to control a gain value of each channel of the display device;
[0011] Based on the equalization parameters of the sound equalizer in the display device, the sound field of the display device is adjusted.
[0012] In some embodiments of the present disclosure, the controller is further configured to:
[0013] Obtaining a preset value corresponding to the equalization parameter, where the preset value is used to describe a maximum channel gain value corresponding to the equalization parameter;
[0014] The controller is specifically configured as follows:
[0015] Determining a first value according to a preset value corresponding to the equalization parameter, an incident angle of the target voice data, and a preset angle value;
[0016] Based on the first value and a preset value corresponding to the equalization parameter, an equalization parameter of a sound equalizer in the display device is determined.
[0017] In some embodiments of the present disclosure, the controller is specifically configured as follows:
[0018] Determine a second value based on the first value and a preset value corresponding to the equalization parameter, where the second value is used to describe an actual value corresponding to the equalization parameter;
[0019] The second value is rounded to an integer to obtain an equalization parameter of a sound equalizer in the display device.
[0020] In some embodiments of the present disclosure, the controller is specifically configured as follows:
[0021] Adjusting a gain value of a first sound channel in the display device based on an equalization parameter of a sound equalizer in the display device;
[0022] And / or, based on an equalization parameter of a sound equalizer in the display device, adjusting a gain value of a second sound channel in the display device.
[0023] In some embodiments of the present disclosure, the controller is specifically configured as follows:
[0024] If the equalization parameter of the sound equalizer in the display device is within a first preset parameter range, increasing the gain value of the first channel and decreasing the gain value of the second channel, the first preset parameter range corresponding to the first channel;
[0025] Alternatively, if the equalization parameter of the sound equalizer in the display device is within a second preset parameter range, the gain value of the second channel is increased and the gain value of the first channel is decreased, and the second preset parameter range corresponds to the second channel.
[0026] In some embodiments of the present disclosure, the detector is further configured as:
[0027] When detecting that the relative angle between the user and the display device changes, generating prompt information, the prompt information is used to instruct the user to perform sound field calibration on the display device;
[0028] The controller is further configured to: output the prompt information.
[0029] In a second aspect, the present disclosure provides a sound field adjustment method, comprising:
[0030] Collecting target voice data of the user, where the target voice data is a target wake-up word for sound field calibration of the display device;
[0031] When the target wake-up word is a preset wake-up word, determining an incident angle of the target voice data based on the direction information of the target voice data, where the incident angle of the target voice data is a relative angle between the user and the display device;
[0032] Based on the incident angle of the target voice data, the sound field of the display device is adjusted.
[0033] In some embodiments of the present disclosure, adjusting the sound field of the display device based on the incident angle of the target voice data includes:
[0034] Determining an equalization parameter of a sound equalizer of the display device based on an incident angle of the target voice data, wherein the equalization parameter is used to control a gain value of each channel of the display device;
[0035] Based on the equalization parameters of the sound equalizer in the display device, the sound field of the display device is adjusted.
[0036] In some embodiments of the present disclosure, it also includes:
[0037] Obtaining a preset value corresponding to the equalization parameter, where the preset value is used to describe a maximum channel gain value corresponding to the equalization parameter;
[0038] The step of determining an equalization parameter of a sound equalizer of the display device based on the incident angle of the target voice data includes:
[0039] Determining a first value according to a preset value corresponding to the equalization parameter, an incident angle of the target voice data, and a preset angle value;
[0040] Based on the first value and a preset value corresponding to the equalization parameter, an equalization parameter of a sound equalizer in the display device is determined.
[0041] In some embodiments of the present disclosure, determining the equalization parameter of the sound equalizer in the display device based on the first value and the preset value corresponding to the equalization parameter includes:
[0042] Determine a second value based on the first value and a preset value corresponding to the equalization parameter, where the second value is used to describe an actual value corresponding to the equalization parameter;
[0043] The second value is rounded to an integer to obtain an equalization parameter of a sound equalizer in the display device.
[0044] In some embodiments of the present disclosure, adjusting the sound field of the display device based on the equalization parameters of the sound equalizer in the display device includes:
[0045] Adjusting a gain value of a first sound channel in the display device based on an equalization parameter of a sound equalizer in the display device;
[0046] And / or, based on an equalization parameter of a sound equalizer in the display device, adjusting a gain value of a second sound channel in the display device.
[0047] In some embodiments of the present disclosure, adjusting the gain value of the first channel in the display device based on the equalization parameter of the sound equalizer in the display device; and / or adjusting the gain value of the second channel in the display device based on the equalization parameter of the sound equalizer in the display device, includes:
[0048] If the equalization parameter of the sound equalizer in the display device is within a first preset parameter range, increasing the gain value of the first channel and decreasing the gain value of the second channel, the first preset parameter range corresponding to the first channel;
[0049] Alternatively, if the equalization parameter of the sound equalizer in the display device is within a second preset parameter range, the gain value of the second channel is increased and the gain value of the first channel is decreased, and the second preset parameter range corresponds to the second channel.
[0050] In some embodiments of the present disclosure, the method further includes:
[0051] When detecting that the relative angle between the user and the display device changes, generating prompt information, the prompt information is used to instruct the user to perform sound field calibration on the display device;
[0052] The prompt information is output.
[0053] In a third aspect, the present disclosure provides a computer-readable storage medium, including: a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the sound field adjustment method shown in the second aspect is implemented.
[0054] In a fourth aspect, the present disclosure provides a computer program product, characterized in that it includes: when the computer program product runs on a computer, it enables the computer to implement the sound field adjustment method shown in the second aspect.
[0055] Compared with the prior art, the technical solution provided by the embodiments of the present disclosure has the following advantages: when a user interacts with a display device, target voice data of the user is collected, wherein the target voice data is a target wake-up word for sound field calibration of the display device, and when the target wake-up word and the preset wake-up word are determined, the incident angle corresponding to the target voice data is determined based on the direction information of the target voice data to adjust the sound field of the sound source emitted by the display device to the user. The above method can ensure that the user is always at the center of the sound field of the sound source of the display device when watching the displayed content, ensure the balance of the sound heard by the user, and effectively improve the user experience. BRIEF DESCRIPTION OF THE DRAWINGS
[0056] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.
[0057] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.
[0058] Figure 1 A schematic diagram of an operation scenario between a display device and a control apparatus according to one or more embodiments of the present disclosure;
[0059] Figure 2 is a hardware configuration block diagram of a control device 100 according to one or more embodiments of the present disclosure;
[0060] Figure 3 is a hardware configuration block diagram of a display device 200 according to one or more embodiments of the present disclosure;
[0061] Figure 4 is a schematic diagram of software configuration in a display device 200 according to one or more embodiments of the present disclosure;
[0062] Figure 5 A schematic diagram showing an icon control interface of an application in a display device 200 according to one or more embodiments of the present disclosure;
[0063] Fig. 6A A system framework diagram for performing sound field adjustment according to one or more embodiments of the present disclosure;
[0064] Figure 6B A diagram of an architecture for adjusting a sound field according to one or more embodiments of the present disclosure;
[0065] Fig. 7A A flow chart of a sound field adjustment method provided by an embodiment of the present disclosure;
[0066] Figure 7B A schematic diagram of the structure of a microphone array sound pickup assembly provided in an embodiment of the present disclosure;
[0067] Figure 7C A schematic diagram of the operation of a microphone array sound pickup assembly provided by an embodiment of the present disclosure;
[0068] Fig.7D A schematic diagram of sound source localization provided in an embodiment of the present disclosure;
[0069] Fig. 7E A schematic diagram of a position relationship between a user and a display device provided in an embodiment of the present disclosure;
[0070] Figure 7F A schematic diagram of a user's location and a sound field range of a display device provided in an embodiment of the present disclosure;
[0071] Figure 7G A schematic diagram of another user location area and a sound field range of a display device provided in an embodiment of the present disclosure;
[0072] Fig. 8A Another schematic diagram of the position relationship between a user and a display device provided in an embodiment of the present disclosure;
[0073] Figure 8B A schematic diagram of a position relationship between a user and a display device provided in another embodiment of the present disclosure;
[0074] Figure 8C A schematic diagram of a position relationship between a user and a display device provided in another embodiment of the present disclosure;
[0075] Fig.8D A schematic diagram of adjusting the gain value of a channel provided by an embodiment of the present disclosure;
[0076] Fig. 8E A schematic diagram of adjusting the gain value of another channel provided by an embodiment of the present disclosure;
[0077] Fig.8F A schematic diagram of the position of the user's optimal viewing vision provided by an embodiment of the present disclosure. DETAILED DESCRIPTION
[0078] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.
[0079] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.
[0080] Different display contents can be played on the display device. The different playback channel gain values of each display content will make the playback sound field of the display device different, and the sound effects heard by the user will also be different. Specifically, the most basic configuration of the current display device mainly includes 2.0 dual-channel (left channel and right channel) stereo, and different gain values can be set for the left channel and the right channel to facilitate user listening.
[0081] Generally speaking, if the display device is configured with balanced left and right channels, the sound field of the display device is in the middle position, and the user's best listening position is also in the middle position of the display device. If the user is in other positions, the sounds of the left and right channels of the display device will interfere with each other. At this time, for the user, the sound field of the display device is still in the middle position, and the user's listening effect will be worse.
[0082] In the related art, a user can manually control the display device to adjust the gain values of the left and right channels through the adjustment buttons on the controller / display device, so that the playback sound of the displayed content of the display device adapts to his or her viewing position. Therefore, the manual adjustment method will increase the tediousness of the user's viewing of the displayed content and reduce the user's viewing experience.
[0083] In order to solve the above problems, in the embodiment of the present disclosure, when a user interacts with a display device, a detector of the display device collects target voice data spoken by the user, wherein the target voice data is a target wake-up word for sound field calibration of the display device. When the target wake-up word and the preset wake-up word are determined, the incident angle corresponding to the target voice data is determined based on the direction information of the target voice data to adjust the sound field of the sound source emitted by the display device to the user. The above method can ensure that the user is always at the center of the sound field of the sound source of the display device when watching the displayed content, thereby ensuring the balance of the sound heard by the user and effectively improving the user experience.
[0084] Figure 1 Schematic diagram of an operation scenario between a display device and a control device according to one or more embodiments of the present disclosure. Figure 1As shown, the user can operate the display device 200 through the smart device 300 (such as a mobile phone) or the control device 100 (such as a remote control) to make the display device 200 play the corresponding multimedia content, wherein the display device 200 can collect the sound emitted by the user in real time through the microphone array, and plan the current sound field of the display device in real time based on the propagation direction of the sound, and adjust the current sound field of the display device to be consistent with the user's location.
[0085] In some embodiments, when a user turns on a display device through a remote control and controls the display content played on the display device, the display device can adjust the sound field of the display device in real time based on the incident angle of the sound source emitted by the user.
[0086] Exemplarily, the range of the sound field of the display device may be a range area where a scene image can be captured by a shooting device in the display device, such as including: a first area, a second area, a third area, a fourth area and a fifth area, wherein, by default, the sound field direction of the display device may correspond to the third area; when it is determined that the sound field of the display device corresponds to the first area according to the incident angle of the sound source emitted by the user, the display area corresponding to the current sound field of the display device may be adjusted from the third area to the first area, or, when it is determined that the sound field of the display device corresponds to the second area according to the incident angle of the sound source emitted by the user, the display area corresponding to the current sound field of the display device may be adjusted from the third area to the second area, or Alternatively, when it is determined that the sound field of the display device corresponds to the fourth area based on the incident angle of the sound source emitted by the user, the display area corresponding to the current sound field of the display device can be adjusted from the third area to the fourth area. Alternatively, when it is determined that the sound field of the display device corresponds to the fifth area based on the incident angle of the sound source emitted by the user, the display area corresponding to the current sound field of the display device can be adjusted from the third area to the fifth area. Thus, the problems in the prior art are solved, and voice interaction can be performed with the display device in real time while the user is watching the content played on the display device, so that the display device can adjust the center position of the sound field of the playback source based on the voice interaction content to be always consistent with the user, so that the user can feel a balanced sound and improve the user experience.
[0087] It should be noted that the first area, the second area, the third area, the fourth area and the fifth area may be display areas on the same horizontal plane and without overlapping, or the first area, the second area, the third area, the fourth area and the fifth area may be display areas on different horizontal planes and without overlapping / overlapping, and the present disclosure does not make any specific limitations on this.
[0088] In some embodiments, the control device 100 may be a remote controller, and the communication between the remote controller and the display device includes infrared protocol communication, Bluetooth protocol communication, wireless or other wired methods to control the display device 200. The user may input user commands through buttons on the remote controller, voice input, and control panel input to control the display device 200. In some embodiments, a mobile terminal, a tablet computer, a computer, a laptop computer, and other smart devices may also be used to control the display device 200.
[0089] In some embodiments, the display device 200 may not use the above-mentioned smart device or control device to receive instructions, but may receive user control through touch or gestures.
[0090] In some embodiments, the display device 200 can also be controlled in a manner other than the control device 100 and the smart device 300. For example, the user's voice command control can be directly received through a module for obtaining voice commands configured inside the display device 200, or the user's voice command control can be received through a voice control device set outside the display device 200.
[0091] In some embodiments, the smart device 300 can be connected and communicated with the software application installed in the display device 200 through a network communication protocol to achieve the purpose of one-to-one control operation and data communication. The audio and video content displayed on the smart device 300 can also be transmitted to the display device 200 to achieve a synchronous display function. The display device 200 also communicates data with the server 400 through a variety of communication methods, allowing the display device 200 to communicate and connect through a local area network (LAN), a wireless local area network (WLAN) and other networks. The server 400 can be a cluster or multiple clusters, and can include one or more types of servers. The server 400 can provide various content and interactions to the display device 200. The display device 200 can be a liquid crystal display, an OLED display or a projection display device, etc. In addition to providing a broadcast receiving television function, the display device 200 can also provide an additional intelligent network TV function that provides a computer support function.
[0092] Figure 2 Schematically shows a block diagram of a configuration of the control device 100 according to an exemplary embodiment. Figure 2As shown, the control device 100 includes a controller 110, a communication interface 130, a user input / output interface 140, a memory, and a power supply. The control device 100 can receive the user's input operation instructions, and convert the operation instructions into instructions that the display device 200 can recognize and respond to, and play the role of an interactive intermediary between the user and the display device 200. The communication interface 130 is used to communicate with the outside, and includes at least one of a WIFI chip, a Bluetooth module, NFC or an alternative module. The user input / output interface 140 includes at least one of a microphone, a touchpad, a sensor, a button or an alternative module.
[0093] Figure 3 FIG. 2 shows a hardware configuration block diagram of a display device 200 according to an exemplary embodiment. Figure 3 The display device 200 shown includes at least one of a tuner 210, a communicator 220, a detector 230, an external device interface 240, a controller 250, a display 260, an audio output interface 270, a memory, a power supply, and a user interface (i.e., a user input interface) 280. The controller 250 includes a central processing unit, a video processor, an audio processor, a graphics processor, a RAM, a ROM, and a first interface to an nth interface for input / output. The display 260 can be at least one of a liquid crystal display, an OLED display, a touch display, and a projection display, and can also be a projection device and a projection screen. The tuner 210 receives broadcast television signals by wired or wireless reception, and demodulates audio and video signals from multiple wireless or wired broadcast television signals, such as and EPG data signals. The communicator 220 is a component for communicating with an external device or server according to various communication protocol types. For example: the communicator can include other network communication protocol chips or near field communication protocol chips such as Wifi modules, Bluetooth modules, wired Ethernet modules, and at least one of infrared receivers. The display device 200 can establish the transmission and reception of control signals and data signals with the external control device 100 or the server 400 through the communicator 220. The detector 230 is used to collect signals of the external environment or the external interaction. The controller 250 and the tuner-demodulator 210 can be located in different split devices, that is, the tuner-demodulator 210 can also be in an external device of the main device where the controller 250 is located, such as an external set-top box. The user interface 280 can be used to receive control signals of the control device 100 (such as an infrared remote controller, etc.).
[0094] In some embodiments, the controller 250 controls the operation of the display device and responds to the user's operation through various software control programs stored in the memory. The controller 250 controls the overall operation of the display device 200. The user can input a user command through a graphical user interface (GUI) displayed on the display 260, and the user input interface receives the user input command through the graphical user interface (GUI). Alternatively, the user can input a user command by inputting a specific sound or gesture, and the user input interface recognizes the sound or gesture through a sensor to receive the user input command.
[0095] In some embodiments, "user interface" is a medium interface for interaction and information exchange between an application or operating system and a user, which realizes the conversion between the internal form of information and the form acceptable to the user. The commonly used form of user interface is the graphical user interface (GUI), which refers to a user interface related to computer operation displayed in a graphical manner. It can be an interface element such as an icon, window, and control displayed on the display screen of an electronic device, where the control can include at least one of the visual interface elements such as icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, widgets, etc.
[0096] Figure 4 FIG. 1 is a schematic diagram of software configuration in a display device 200 according to one or more embodiments of the present disclosure. Figure 4 As shown in the figure, the system is divided into four layers, from top to bottom, namely the application layer (Applications) layer (referred to as "application layer"), the application framework layer (Application Framework) layer (referred to as "framework layer"), the Android runtime (Android runtime) and system library layer (referred to as "system runtime library layer"), and the kernel layer.
[0097] In some embodiments, at least one application is running in the application layer, and these applications may be window programs, system settings programs, clock programs, etc. provided by the operating system, or applications developed by third-party developers. In specific implementations, the applications in the application layer include but are not limited to the above examples.
[0098] In some embodiments, the system runtime layer provides support for the upper layer, namely the framework layer. When the framework layer is used, the Android operating system will run the C / C++ library contained in the system runtime layer to implement the functions to be implemented by the framework layer.
[0099] In some embodiments, the kernel layer is a layer between hardware and software, and includes at least one of the following drivers: audio driver, display driver, Bluetooth driver, camera driver, WIFI driver, USB driver, HDMI driver, sensor driver (such as fingerprint sensor, temperature sensor, pressure sensor, etc.), and power driver, etc.
[0100] Figure 5 FIG. 1 is a schematic diagram showing an icon control interface of an application in a display device 200 according to one or more embodiments of the present disclosure. Figure 5 As shown in , the application layer includes at least one application that can display corresponding icon controls in the display, such as: live TV application icon control, video on demand application icon control, media center application icon control, application center icon control, game application icon control, etc. Live TV application can provide live TV through different signal sources. Video on demand application can provide videos from different storage sources. Unlike live TV application, video on demand provides video display from certain storage sources. Media center application can provide applications for playing various multimedia content. Application center can provide storage for various applications.
[0101] In some embodiments, the display device is a terminal device with a display function, such as a television or a flat-panel TV. In the display device:
[0102] In some embodiments, the controller 250 controls the operation of the display device and responds to the user's operation through various software control programs stored in the memory. The controller 250 controls the overall operation of the display device 200. The user can input a user command through a graphical user interface (GUI) displayed on the display 260, and the user input interface receives the user input command through the graphical user interface (GUI). Alternatively, the user can input a user command by inputting a specific sound or gesture, and the user input interface recognizes the sound or gesture through a sensor to receive the user input command.
[0103] An output interface (display 260 and / or audio output interface 270), configured to output user interaction information;
[0104] The communicator 220 is used to communicate with the server 400;
[0105] The detector 230 is configured to: collect target voice data of the user, where the target voice data is a target wake-up word for sound field calibration of the display device;
[0106] The controller 250 is configured to: when the target wake-up word is a preset wake-up word, determine the incident angle of the target voice data based on the direction information of the target voice data, where the incident angle of the target voice data is a relative angle between the user and the display device;
[0107] Based on the incident angle of the target voice data, the sound field of the display device is adjusted.
[0108] In some embodiments, the controller 250 is specifically configured to: determine the equalization parameters of the sound equalizer of the display device based on the incident angle of the target voice data, and the equalization parameters are used to control the gain value of each channel of the display device; and adjust the sound field of the display device based on the equalization parameters of the sound equalizer in the display device.
[0109] In some embodiments, the controller 250 is further configured to: obtain a preset value corresponding to the equalization parameter, wherein the preset value is used to describe the maximum channel gain value corresponding to the equalization parameter; the controller 250 is specifically configured to: determine a first value based on the preset value corresponding to the equalization parameter, the incident angle of the target voice data and the preset angle value; and determine the equalization parameter of the sound equalizer in the display device based on the first value and the preset value corresponding to the equalization parameter.
[0110] In some embodiments, the controller 250 is specifically configured to: determine a second value based on the first value and a preset value corresponding to the equalization parameter, the second value being used to describe the actual value corresponding to the equalization parameter; and round the second value to obtain the equalization parameter of the sound equalizer in the display device.
[0111] In some embodiments, the controller 250 is specifically configured to: adjust the gain value of the first channel in the display device based on the equalization parameters of the sound equalizer in the display device; and / or adjust the gain value of the second channel in the display device based on the equalization parameters of the sound equalizer in the display device.
[0112] In some embodiments, the controller 250 is specifically configured as follows: if the equalization parameter of the sound equalizer in the display device is within a first preset parameter range, the gain value of the first channel is increased and the gain value of the second channel is decreased, and the first preset parameter range corresponds to the first channel; or, if the equalization parameter of the sound equalizer in the display device is within a second preset parameter range, the gain value of the second channel is increased and the gain value of the first channel is decreased, and the second preset parameter range corresponds to the second channel.
[0113] In some embodiments, the detector 230 is further configured to: generate prompt information when it detects that the relative angle between the user and the display device changes, and the prompt information is used to instruct the user to calibrate the sound field of the display device; the controller 250 is further configured to: output the prompt information.
[0114] To sum up, in the embodiment of the present disclosure, when a user interacts with a display device, a detector of the display device collects target voice data spoken by the user, wherein the target voice data is a target wake-up word for sound field calibration of the display device. When the target wake-up word and the preset wake-up word are determined, the incident angle corresponding to the target voice data is determined based on the direction information of the target voice data to adjust the sound field of the sound source emitted by the display device to the user. The above method can ensure that the user is always at the center of the sound field of the sound source of the display device when watching the displayed content, thereby ensuring the balance of the sound heard by the user and effectively improving the user experience.
[0115] Fig. 6A A system framework diagram for adjusting the sound field according to one or more embodiments of the present disclosure is shown in FIG. Fig. 6A As shown, the system may include a voice data acquisition module 601, an incident angle determination module 602, and a sound field adjustment module 603. After the system collects the user's target voice data through the voice data acquisition module 601, where the target voice data is the target wake-up word for sound field calibration of the display device, the incident angle determination module 602 determines the incident angle of the target voice data based on the direction information of the target voice data when the target wake-up word is a preset wake-up word, and the incident angle of the target voice data is the relative angle between the user and the display device, and the sound field adjustment module 603 adjusts the sound field of the display device based on the incident angle of the target voice data, so that when the user watches the playback content of the display device, the display device can interact with the display device in real time by voice, so that the display device can adjust the sound field center position of the playback source based on the voice interaction content to always be consistent with the user, so that the user can feel the balanced sound and improve the user experience.
[0116] Figure 6B This is an architecture diagram for adjusting the sound field according to one or more embodiments of the present disclosure. Based on the above system framework, the present disclosure is implemented in the Android system as follows: Figure 6B As shown, the Android system mainly includes the application layer, framework layer, system runtime layer and kernel layer. The implementation logic is mainly reflected in the application layer, which includes a voice data acquisition module, an incident angle determination module and a sound field adjustment module.
[0117] The sound field adjustment method provided in the embodiments of the present disclosure detects the user's voice behavior in real time. When the user is detected speaking, the voice data of the user in a speaking period is collected and matched with a pre-set wake-up word, wherein the pre-set wake-up word is a start condition for adjusting the sound field of the display device. When it is determined that the degree of match between the voice data spoken by the user and the pre-set wake-up word is higher than a certain threshold, the sound incident angle between the user and the display device is measured based on the propagation direction of the voice data emitted by the user, and the corresponding area of the sound field of the display device is corrected in real time according to the sound incident angle between the user and the display device, to ensure that the corresponding area of the sound field of the display device is consistent with the area where the user is located, thereby facilitating the user to enjoy consistent sound sensations in the left and right ears at different viewing positions, effectively improving the sound experience when watching displayed content.
[0118] In order to explain this scheme in more detail, the following will be combined in an exemplary manner Fig. 7A To explain, it is understandable that Fig. 7A The steps involved may include more steps or fewer steps in actual implementation, and the order of these steps may also be different, so long as the sound field adjustment method provided in the embodiment of the present disclosure can be implemented. The embodiment of the present disclosure does not limit this.
[0119] like Fig. 7A As shown, Fig. 7A A flow chart of a sound field adjustment method provided in an embodiment of the present disclosure is provided. The sound field adjustment method specifically includes the following steps:
[0120] S110. Collect target voice data of the user, where the target voice data is a target wake-up word for sound field calibration of a display device.
[0121] The user may be a group of people viewing the display device, or the user may be a group of people appearing in a scene image that can be captured by a camera device of the display device.
[0122] For example, when person A watches multimedia content through a display device, person A can be referred to as the user mentioned in this disclosure; alternatively, when person B, person C, and person D appear simultaneously in a scene image captured by a camera device of the display device, person B / person C / person D can be referred to as the users mentioned in this disclosure.
[0123] Among them, when it is determined that there are multiple users corresponding to the display device, that is, when there are multiple people appearing simultaneously in the scene image that can be captured by the camera device of the display device, the user referred to in this disclosure can be determined by the voice priority of the person / whether he or she is watching the display device.
[0124] In combination with the above example, person B, person C and person D appear simultaneously in the scene image that can be captured by the camera device of the display device. Person B, person C and person D are taken as candidate users, and it is determined whether there is a person watching the display device among person B, person C and person D. If only person B watches the display device, person B can be regarded as the user referred to in this disclosure, or, if only person C watches the display device, person C can be regarded as the user referred to in this disclosure, or, if only person D watches the display device, person D can be regarded as the user referred to in this disclosure.
[0125] If there are two persons viewing the display device at the same time, the user may be determined based on the voice priorities of the two persons, and the voice priority may be the person who is speaking.
[0126] In combination with the above example, person B, person C and person D appear simultaneously in the scene image that can be captured by the camera device of the display device. Person B, person C and person D are taken as candidate users. If person B and person C watch the display device at the same time, and person B speaks, person B can be considered as the user referred to in the present disclosure, and the target voice data of the user is collected, which may be the speech content of person B. Alternatively, if person B and person D watch the display device at the same time, and person D speaks, person D can be considered as the user referred to in the present disclosure, and the target voice data of the user is collected, which may be the speech content of person D.
[0127] Among them, if there are at least two persons viewing the display device at the same time, and both of these two persons make sounds, it is possible to determine who is the user based on the directions in which the sounds of the two persons are propagated.
[0128] In combination with the above example, person B, person C and person D appear at the same time in the scene image that can be captured by the camera device of the display device. Person B, person C and person D are taken as candidate users. If person B, person C and person D watch the display device at the same time, and person B and person C are both speaking, but the propagation direction of the sound emitted by person B is not the direction of the display device, and the propagation direction of the sound emitted by person C is the direction of the display device, person C can be considered as the user referred to in this disclosure, and the target voice data of the user can be collected, which can be the speech content of person C.
[0129] Among them, the target voice data can be a word or a sentence composed of multiple words used by the user to interact with the display device, which can be used as the target wake-up word for the user to calibrate the sound field of the display device.
[0130] For example, the target wake-up word may be a word, such as the user may say "turn on" to the display device, or the user may say "open" to the display device, or the user may say "calibrate" to the display device, etc.
[0131] For another example, the target wake-up word can be a sentence composed of multiple words, such as the user can say to the display device "turn on sound field correction", or the user can say to the display device "perform sound field correction", or the user can say to the display device "I want to calibrate the sound field", etc.
[0132] It should be noted that the user can send out the target voice data at any position corresponding to the display device, that is, when the display device detects that the user makes a sound, it can collect the sound as the target wake-up word.
[0133] In addition, when the display device is collecting the user's target voice data, the display device is turned on, that is, when the display device is turned on, the target wake-up word issued by the user is used to correct and control the sound field of the display device.
[0134] Among them, a sound field correction control button can be set in the display device in advance. When the control button is turned on, the display device can perform subsequent operations by collecting the user's target wake-up word. When the control button is turned off, the display device can trigger sound field correction, indicating that the display device cannot perform subsequent operations by collecting the user's target wake-up word.
[0135] Specifically, when the control button is turned off, after the display device collects the target wake-up word issued by the user, a prompt window can pop up in the display interface and ask the user whether to turn on sound field correction. After obtaining user confirmation, subsequent operations can be performed.
[0136] Alternatively, when the control button is turned off, the display device can automatically turn on sound field correction after collecting the target wake-up word issued by the user, continue to perform subsequent operations, and display the message of turning on sound field correction to the user on the display interface, so that the user can understand the correction status.
[0137] It should be noted that the control button set on the display device can be a virtual button or a physical button. If the control button is a virtual button, it can support the user to turn on / off in the display device through a remote control or other control methods. If the control button is a physical button, the user can turn on / off on the display device through triggering methods such as pressing or clicking.
[0138] Among them, before collecting the user's target voice data, the present invention can integrate a microphone array pickup component in the display device, so that when collecting the user's target voice data, the user's target voice data can be collected through the microphone array pickup component. The microphone array pickup component can be a dual-microphone far-field pickup board or a four-microphone or more far-field pickup board, which can support the collection of 2 or 4 or more original audio.
[0139] like Figure 7BAs shown in the example, the microphone array sound pickup assembly is integrated in the display device through interface 1 to communicate with the display device, wherein the microphone array sound pickup assembly may also include two microphones, namely mic2 and mic3, which can be used to collect audio data emitted by one or more users respectively.
[0140] like Figure 7C As shown in the example, mic2 is used to collect audio data sent by a first user, and mic3 is used to collect audio data sent by a second user, thereby obtaining two channels of audio data.
[0141] S120. When the target wake-up word is a preset wake-up word, determine the incident angle of the target voice data based on the direction information of the target voice data.
[0142] Among them, the degree of match between the target wake-up word and the preset wake-up word can be used to determine whether the target wake-up word is the preset wake-up word, and a critical threshold can be set to measure the degree of match between the target wake-up word and the preset wake-up word.
[0143] Among them, the preset wake-up word can be "turn on / turn on / perform sound field correction".
[0144] In combination with the above examples, when the target wake-up word is "turn on", at this time, the display device is already turned on, and the user has not clearly stated which function or application is turned on, it can be considered that the target wake-up word said by the user is used to turn on the sound field correction of the display device, and the target wake-up word can be considered to be the preset wake-up word; or, when the target wake-up word is "turn on", at this time, the display device is already turned on, and the user has not clearly stated which function or application is turned on, it can be considered that the target wake-up word said by the user is used to turn on the sound field correction of the display device, and the target wake-up word can be considered to be the preset wake-up word; or, when the target wake-up word is "correction", at this time, the user has not clearly stated that it is the correction object, it can be considered that the target wake-up word said by the user is used to calibrate the sound field correction of the display device, and the target wake-up word can be considered to be the preset wake-up word.
[0145] For another example, when the target wake-up word is "turn on sound field correction", the target wake-up word can be matched with the preset wake-up word and the degree of match is higher than the critical threshold, then the target wake-up word can be considered to be the preset wake-up word; or, when the target wake-up word is "perform sound field correction", the target wake-up word can be matched with the preset wake-up word and the degree of match is higher than the critical threshold, then the target wake-up word can be considered to be the preset wake-up word; or, when the target wake-up word is "I want to correct the sound field", the target wake-up word can be matched with the preset wake-up word and the degree of match is higher than the critical threshold, then the target wake-up word can be considered to be the preset wake-up word.
[0146] The direction information of the target voice data can be obtained based on locating the sound source of the target voice data. Specifically, the sound source of the target voice data (ie, the user referred to above) can be located by utilizing the phase difference between the collected multi-channel target voice data.
[0147] The direction information of the audio data sent by the first user collected by mic2 is as follows Figure 7C As shown in the example, mic3 collects the direction information of the audio data sent by the second user as follows Figure 7C As shown in the example, the sound source of the target speech data can be determined as follows Fig.7D The position U shown in the example, correspondingly, the incident angle of the target speech data is θ.
[0148] The incident angle of the target voice data is the relative angle between the user and the display device, that is, the incident angle of the target voice data may be the angle between the user and the display device on a horizontal plane with the long side of the display device being the horizontal plane.
[0149] like Fig. 7E As shown in the example, the relative angle between the user and the display device is as follows Fig. 7E The angle θ shown as an example, the relative angle between the user and the display device ranges from [0°, 180°].
[0150] In addition, when it is determined that the target wake-up word issued by the user is a preset wake-up word, a non-receiving period can be set to avoid receiving voice data multiple times in a short period of time, which affects the processing efficiency of the display device.
[0151] For example, within the first time period after determining that the target wake-up word sent by user A is the preset wake-up word, voice data sent by user A or other user B is received, and the first time period is within the set non-receiving time period, then the received voice data sent by user A or other user B can be blocked.
[0152] S130: Adjust the sound field of the display device based on the incident angle of the target voice data.
[0153] Among them, adjusting the sound field of the display device based on the incident angle of the target voice data may include: adjusting the sound field of the display device to the area where the user is located based on the incident angle of the target voice data, or adjusting the sound field of the display device to overlap / highly overlap with the area where the user is located based on the incident angle of the target voice data.
[0154] For example, the area where user A is located is area a, the area corresponding to the current sound field of the display device is area b, area a is not equal to area b, or the overlapping area between area a and area b is small / no overlapping area, then the current sound field of the display device can be adjusted from area b to area a, or the current sound field of the display device can be adjusted from area b to area c which has a higher degree of overlap with area a.
[0155] Specifically, Figure 7F As shown in the example, the area corresponding to the current sound field of the display device is as follows Figure 7F The area b in the figure shows an exemplary range, and the area where user A is located is as follows Figure 7F The range area shown in the exemplary area a in the middle can adjust the sound field of the display device to be as follows according to the incident angle of the target voice data. Figure 7F The range area shown in area b is an example.
[0156] like Figure 7G As shown in the example, the area corresponding to the current sound field of the display device is as follows Figure 7G The area b in the figure shows an exemplary range, and the area where user A is located is as follows Figure 7F The range area shown in the exemplary area a in the middle can adjust the sound field of the display device to be as follows according to the incident angle of the target voice data. Figure 7F The range area is shown as an example in region c, wherein region c nearly overlaps with region a and / or overlaps with region a in most areas.
[0157] Among them, the parameter value of the sound equalizer used to adjust the sound field range in the display device can be calculated through the incident angle of the target voice data, so that the display device automatically adjusts the gain values of the left and right channels of the display device based on the calculated parameter value of the sound equalizer used to adjust the sound field range, so as to quickly adjust the sound field range.
[0158] It should be noted that the method of adjusting the sound field disclosed in the present invention includes but is not limited to adjusting the left and right channels. Specifically, according to the multi-channel setting of the display device, the gain value of the multi-channel can be adjusted to meet the user's viewing needs, such as adjusting the gain value of the upper and lower channels.
[0159] Based on the description of the above embodiments, when adjusting the sound field of the display device, the gain value of each channel of the sound equalizer in the display device can be determined by first determining the relative angle between the user and the display device, so that the display device can adjust the playback volume of each channel based on the gain value of each channel, so that the sound field of the display device corresponds to the area where the user is located.
[0160] The gain value of each channel is used to control the volume of the sound emitted by the channel.
[0161] Specifically, when the user is in the middle of the display device, Fig. 8A In the exemplary position shown, the left and right channels of the display device can use the same gain value to ensure that the volume values heard by the left and right ears of the user are uniform.
[0162] When the user is not in the center of the display device, Figure 8B In the exemplary position shown, at this time, the left and right channels of the display device can use different gain values to avoid the problem of inconsistent volume heard by the user.
[0163] When the user is not in the center of the display device, such as Figure 8C In the exemplary position shown, at this time, the left and right channels of the display device can use different gain values to avoid the problem of inconsistent volume heard by the user.
[0164] In some embodiments, when determining the equalization parameter of the sound equalizer in the display device, it can be implemented in combination with a preset value, wherein the preset value can be used to describe the maximum channel gain value corresponding to the equalization parameter, and specifically, can be set to 9.
[0165] Among them, a first value can be determined according to the preset value corresponding to the equalization parameter, combined with the incident angle of the target voice data and the preset angle value, and then based on the first value and the preset value, the equalization parameter of the sound equalizer in the display device is obtained.
[0166] Specifically, a second value can be determined based on the first value and the preset value, wherein the second value can be used to describe the actual value calculated / measured based on the equalization parameter, and the second value is rounded to the nearest integer to obtain the equalization parameter of the sound equalizer in the display device.
[0167] For example, the equalization parameters may be implemented by referring to the following formula.
[0168]
[0169] in, corresponds to the first value, N corresponds to the preset value, which is the maximum value of the equalization parameter, and 180° corresponds to the preset angle value. corresponds to the second value.
[0170] Among them, when N=9, the value range of W is [-9,9], where W=-9 can indicate that the sound emitted by the display device is all controlled by the gain value of the left channel, and the gain value of the right channel is 0, and W=9 can indicate that the sound emitted by the display device is all controlled by the gain value of the right channel, and the gain value of the left channel is 0.
[0171] Normally, the default value of W of a display device is 0, indicating that the gain values of the left and right channels of the display device are the same, that is, the volume values of the sounds emitted by the left and right speakers are the same.
[0172] The value range of θ is [0°, 180°], and W changes with the incident angle θ of the target speech data.
[0173] In some embodiments, when the display device has a first sound channel and a second sound channel, gain values of the first sound channel and the second sound channel may be adjusted separately or simultaneously based on different equalization parameters.
[0174] Among them, the gain value of the first channel of the display device and the gain value of the second channel of the display device can be adjusted at the same time according to the determined equalization parameters, or the gain value of the first channel of the display device can be adjusted according to the determined equalization parameters, or the gain value of the second channel of the display device can be adjusted.
[0175] Specifically, the specific range of the value corresponding to the equalization parameter may be determined to determine the adjustment strategy of the first channel and the second channel.
[0176] Among them, two parameter ranges can be set, namely a first preset parameter range and a second preset parameter range. The first preset parameter range can be (-∞, 0), and the second preset parameter range can be (0, +∞).
[0177] Combined with the above example, N = 9, θ = 60°, we get W = -3. The gain value adjustment of the channel of the display device can be found in Fig.8D As shown in the example, the sound field of the display device can be ensured to be within the area where the user is located by increasing the gain value of the left channel and decreasing the gain value of the right channel, wherein the equalization parameter W=-3 can be written into the sound playback parameters of the display device so that the display device automatically performs the increase and decrease operations of the gain values of the left and right channels.
[0178] Combined with the above example, N = 9, θ = 120°, we get W = 3. The gain value adjustment of the channel of the display device can be found in Fig. 8E As shown in the example, the sound field of the display device can be ensured to be within the area where the user is located by increasing the gain value of the right channel and reducing the gain value of the left channel, wherein the equalization parameter W=3 can be written into the sound playback parameters of the display device so that the display device automatically performs the increase and decrease operations of the gain values of the left and right channels.
[0179] Wherein, corresponding to N = 9, θ = 90°, W = 0 can be obtained, and the gain value of the channel of the display device does not need to be adjusted. For details, see Fig.8F As shown in the example, it can be seen that the user is in the middle position of the display device and in the center of the sound field, and can achieve the best auditory effect.
[0180] In some embodiments, after determining that the sound field of the display device has been adjusted successfully, it is possible to detect in real time whether the relative angle between the user and the display device has changed. When it is detected that the relative angle between the user and the display device has changed, a prompt message is generated and output to the display of the display device, prompting the user whether to request to adjust the sound field, so that the user can effectively understand the visual effect of the current viewing position when watching the display device and make timely processing.
[0181] Among them, detecting whether the relative angle between the user and the display device changes can be achieved through image detection or voice detection.
[0182] For example, it is detected whether the user makes a sound. If it is detected that the user makes a sound, the sound made by the user can be collected to locate the user's current position. If it is detected that the user's current position changes, it is determined that the relative angle between the user and the display device changes.
[0183] Alternatively, the original relative angle value between the user and the display device when the user last adjusted the sound field is recorded, and when the user is detected making a sound, the user's current position is located by collecting the sound made by the user to obtain a new relative angle value between the user's current position and the display device; when the new relative angle value is not equal to the original relative angle value / the difference between the new relative angle value and the original relative angle value is greater than a certain value, it can be determined that the relative angle between the user and the display device has changed.
[0184] For another example, the user's position is located in real time through the camera device of the display device (embedded camera or external camera). If it is detected that the user's current position changes, it is determined that the relative angle between the user and the display device changes.
[0185] Alternatively, the original relative angle value between the user and the display device when the user last adjusted the sound field is recorded, and when a change in the user's position is detected, a new relative angle value between the user's current position and the display device is calculated, and when the new relative angle value is not equal to the original relative angle value / the difference between the new relative angle value and the original relative angle value is greater than a certain value, it can be determined that the relative angle between the user and the display device has changed.
[0186] In summary, the present disclosure collects the user's target voice data when the user interacts with the display device by executing the above-mentioned sound field adjustment method on the display device, wherein the target voice data is the target wake-up word for sound field calibration of the display device, and when the target wake-up word and the preset wake-up word are determined, the incident angle corresponding to the target voice data is determined based on the direction information of the target voice data to adjust the sound field of the sound source emitted by the display device to the user. The above-mentioned method can ensure that the user is always at the center of the sound field of the sound source of the display device when watching the displayed content, ensure the balance of the sound heard by the user, and effectively improve the user experience.
[0187] The embodiment of the present disclosure provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the various processes of the above-mentioned sound field adjustment method are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.
[0188] The computer readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.
[0189] The present disclosure provides a computer program product, characterized in that it includes: when the computer program product is run on a computer, the computer is enabled to implement the above-mentioned sound field adjustment method.
[0190] For the convenience of explanation, the above description has been made in conjunction with specific embodiments. However, the above discussion in some embodiments is not intended to be exhaustive or limit the embodiments to the specific forms disclosed above. According to the above teachings, various modifications and variations can be obtained. The selection and description of the above embodiments are to better explain the principles and practical applications, so that those skilled in the art can better use the embodiments and various different variations of the embodiments suitable for specific use considerations.
Claims
1. A display device, It is characterized in that include: The detector is configured to: collect target voice data of the user, where the target voice data is a target wake-up word for sound field calibration of the display device; a controller configured to: when the target wake-up word is a preset wake-up word, determine an incident angle of the target voice data based on direction information of the target voice data, where the incident angle of the target voice data is a relative angle between the user and the display device; Obtaining a preset value corresponding to an equalization parameter, wherein the preset value is used to describe a maximum channel gain value corresponding to the equalization parameter; The controller is specifically configured as follows: Determining a first value according to a preset value corresponding to the equalization parameter, an incident angle of the target voice data, and a preset angle value; Determining an equalization parameter of a sound equalizer in the display device based on the first value and a preset value corresponding to the equalization parameter, wherein the equalization parameter is used to control a gain value of each channel of the display device; Based on the equalization parameters of the sound equalizer in the display device, the sound field of the display device is adjusted.
2. The display device according to claim 1, It is characterized in that The controller is specifically configured as follows: Determine a second value based on the first value and a preset value corresponding to the equalization parameter, where the second value is used to describe an actual value corresponding to the equalization parameter; The second value is rounded to an integer to obtain an equalization parameter of a sound equalizer in the display device.
3. The display device according to claim 1, It is characterized in that The controller is specifically configured as follows: Adjusting a gain value of a first sound channel in the display device based on an equalization parameter of a sound equalizer in the display device; And / or, based on an equalization parameter of a sound equalizer in the display device, adjusting a gain value of a second sound channel in the display device.
4. The display device according to claim 3, It is characterized in that The controller is specifically configured as follows: If the equalization parameter of the sound equalizer in the display device is within a first preset parameter range, increasing the gain value of the first channel and decreasing the gain value of the second channel, the first preset parameter range corresponding to the first channel; Alternatively, if the equalization parameter of the sound equalizer in the display device is within a second preset parameter range, the gain value of the second channel is increased and the gain value of the first channel is decreased, and the second preset parameter range corresponds to the second channel.
5. The display device according to claim 1, It is characterized in that The detector is further configured as: When detecting that the relative angle between the user and the display device changes, generating prompt information, the prompt information is used to instruct the user to perform sound field calibration on the display device; The controller is further configured to: output the prompt information.
6. A sound field adjustment method, It is characterized in that include: Collecting the user's target voice data, where the target voice data is a target wake-up word for sound field calibration of the display device; When the target wake-up word is a preset wake-up word, determining an incident angle of the target voice data based on the direction information of the target voice data, where the incident angle of the target voice data is a relative angle between the user and the display device; Obtaining a preset value corresponding to an equalization parameter, wherein the preset value is used to describe a maximum channel gain value corresponding to the equalization parameter; Determining a first value according to a preset value corresponding to the equalization parameter, an incident angle of the target voice data, and a preset angle value; Determining an equalization parameter of a sound equalizer in the display device based on the first value and a preset value corresponding to the equalization parameter, wherein the equalization parameter is used to control a gain value of each channel of the display device; Based on the equalization parameters of the sound equalizer in the display device, the sound field of the display device is adjusted.
7. A computer-readable storage medium, It is characterized in that include: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the sound field adjustment method according to claim 6 is implemented.
8. A computer program product, It is characterized in that include: When the computer program product is run on a computer, the computer is enabled to implement the sound field adjustment method according to claim 6 .
