Individual listening zone for consumer audio devices using ultrasonic speakers

Abstract

A method and an electronic consumer device for creating an individual spatial listening zone for a user are provided. The method includes receiving and processing an input audio signal to generate a first ultrasonic electrical audio signal and a second ultrasonic electrical audio signal. The ultrasonic signal is provided to a first ultrasonic transducer and a second ultrasonic transducer, which convert them into a directional ultrasonic acoustic signal. The signals are directed to a spatial listening zone in which the user is located. An audible sound signal for the user is generated by a non-linear interaction of the ultrasound signal in the air within the spatial listening zone, the audible sound signal corresponding to desired audio based on the input signal. The audible sound is substantially confined to the spatial listening zone and is not audible outside it. Various techniques are disclosed for dynamically adjusting a spatial listening zone based on user location. The system allows a personalized audio experience to be created while minimizing disturbances to nearby others.

Description

Technical Field

[0001] The various examples of this disclosure generally relate to audio reproduction in electronic consumer devices. More specifically, the various examples of this disclosure relate to methods and apparatus for creating an individual spatial listening zone using directional ultrasonic speakers in portable and stationary consumer audio devices. Background Technology

[0002] Audio played back from consumer electronics devices in shared spaces often presents a challenge in balancing personal listening experience with the need to avoid disturbing others. Conventional consumer electronics devices, such as televisions and portable speakers, typically emit sound over a wide angle, making it difficult to confine the audio to a specific area. This can lead to conflicts in multi-user environments or public spaces. Current solutions to this problem, such as personal headphones, offer a private listening experience, but can be uncomfortable for extended periods and isolate the user from their surroundings. Some users find wearing such devices unpleasant, especially during physical activity or when they desire a more natural listening experience. Directional speaker technology, similar to that used in museums or specialized installations, creates localized sound zones, but they typically require large, expensive hardware and are usually fixed in place. Such solutions are not practical for everyday consumer use, especially in home environments or with portable devices. Furthermore, existing directional audio solutions may not easily adapt to changing listener locations or multiple users. They often require complex setup procedures and lack the flexibility needed for dynamic real-world scenarios. Summary of the Invention

[0003] Therefore, advanced technologies are needed to render sound through electronic consumer devices, which would mitigate or alleviate at least some of the aforementioned limitations and drawbacks.

[0004] The features of the independent claims satisfy this requirement. The features of the dependent claims define further advantageous examples.

[0005] The solutions according to this disclosure will now be described with respect to the claimed methods and the claimed audio apparatus and audio system, wherein features, advantages, or alternative embodiments may be assigned to other claimed objects, and vice versa. In other words, the claims relating to the apparatus and system may be improved by features described in the context of the method, and the method may be improved by features described in the context of the system.

[0006] A method is provided for creating an individual spatial listening zone for a user or listener of an electronic consumer device. The method includes several steps, which can be performed at least partially on the electronic consumer device. However, it should be understood that in some examples, at least some of these steps can also be performed remotely, such as on a cloud server or a separate processing unit.

[0007] In this step, an input audio signal is received. This input audio signal serves as the basis for outputting the desired audible sound signal to the user. In this step, the input audio signal is processed to generate a first ultrasonic audio signal and a second ultrasonic audio signal. The first and second ultrasonic audio signals are electrical signals, which may, for example, be fed to an ultrasonic transducer to generate ultrasound, i.e., an ultrasonic acoustic signal. This processing includes modulating one of the first and second ultrasonic electrical audio signals as a carrier signal using a signal derived from the input audio signal. In this step, the first ultrasonic electrical audio signal is provided to the first ultrasonic transducer. The second ultrasonic electrical audio signal is provided to the second ultrasonic transducer. Both transducers are included in electronic consumer devices. For example, in portable devices such as smartphones or tablets, the transducers may be positioned along an edge or on a rear panel. For larger fixed devices (such as portable speakers or televisions), the transducers may be arranged in an array configuration, potentially hidden behind an acoustically transparent grille or integrated into the device's bezel. In this step, the first ultrasonic transducer converts the first ultrasonic electrical audio signal into a first directional ultrasonic acoustic signal, and the second ultrasonic transducer converts the second ultrasonic audio signal into a second directional ultrasonic acoustic signal. In this step, a first directional ultrasonic acoustic signal and a second directional ultrasonic acoustic signal are directed to a spatial listening zone. The spatial listening zone is a restricted area within the user's space. In this step, an audible sound signal for the user is generated within the spatial listening zone. The audible sound signal substantially corresponds to the desired audible sound signal based on the input audio signal. The audible sound signal is generated through the nonlinear interaction of the first and second directional ultrasonic acoustic signals in the air within the spatial listening zone (i.e., wherever the two directional ultrasonic acoustic signals interfere).

[0008] An input audio signal can be understood as an electronic representation of sound data intended for playback. It can be a digital or analog signal, containing information about the amplitude and frequency of the sound wave as it changes over time. Input audio signals can come from various sources, such as stored audio files, streaming media, or real-time audio input.

[0009] Ultrasonic acoustic signals can be understood as high-frequency sound waves above the upper limit of human hearing, typically above 20 kHz. These signals can be used as carrier waves for audio information. Ultrasonic transducers are devices capable of converting electrical signals into ultrasonic sound waves. They operate at frequencies beyond the range of human hearing and can produce highly directional sound beams. Expected audible sound signals refer to the anticipated audio output that a user should hear within their spatial listening area and are designed to closely match or represent the input audio signal.

[0010] The spatial listening zone can be understood as a constrained region in three-dimensional space where the expected audio can be clearly heard. This zone is created by the focusing interaction of ultrasound waves, resulting in a localized area of ​​audible sound. Nonlinear interaction in the air refers to the phenomenon where high-intensity ultrasound waves interact nonlinearly, producing lower frequency components that fall within the range of human hearing.

[0011] Corresponding electronic consumer devices are provided. These devices may include one or more products with audio capabilities, including but not limited to stationary televisions, Bluetooth speakers, smartphones, tablets, laptops, desktop computers, smart home speakers, or soundbars. These devices may incorporate ultrasonic transducer technology to create personalized listening zones.

[0012] An electronic consumer device is configured to create a personal listening zone for a user. The electronic consumer device includes a first ultrasonic transducer and a second ultrasonic transducer. A conversion circuit is operatively coupled to the first and second ultrasonic transducers.

[0013] The conversion circuit is configured to receive an input audio signal. The conversion circuit processes the input audio signal to generate a first ultrasonic electro-audio signal and a second ultrasonic electro-audio signal. The processing includes modulating one of the first and second ultrasonic electro-audio signals as a carrier signal using a signal derived from the input audio signal. The conversion circuit provides the first ultrasonic electro-audio signal to a first ultrasonic transducer and the second ultrasonic electro-audio signal to a second ultrasonic transducer.

[0014] A first ultrasonic transducer and a second ultrasonic transducer are respectively configured to convert a first ultrasonic electro-audio signal and a second ultrasonic electro-audio signal into a first directional ultrasonic acoustic signal and a second directional ultrasonic acoustic signal. An electronic consumer device directs the first directional ultrasonic acoustic signal and the second directional ultrasonic acoustic signal to a spatial listening area. The spatial listening area is a restricted area within the user's space.

[0015] Due to the spatial overlap of ultrasonic signals within the spatial listening zone, the first and second directional ultrasonic acoustic signals interact nonlinearly with the air within the spatial listening zone. This results in the demodulation of an audible sound signal for the user. The audible sound signal essentially corresponds to the desired audible sound signal based on the input audio signal. This audible sound signal is essentially confined to the spatial listening zone and is essentially inaudible outside of it.

[0016] Electronic consumer devices are configured to perform methods according to any method or any combination of methods pursuant to this disclosure.

[0017] Electronic consumer devices can be understood as devices designed for personal use that process audio signals to create a customized listening experience. Electronic consumer devices can include portable or handheld devices. They can include various audio processing components and transducers to manipulate and output sound. Such electronic consumer devices can be portable or stationary, and they can be designed for entertainment and / or communication, and / or other consumer applications involving audio output.

[0018] A transducer circuit can be understood as an electronic component or system, such as a digital signal processor (DSP), that modifies the input signal by running a computer program within the DSP. In this context, it processes the audio signal and generates an ultrasonic signal for the transducer. The circuit may include a digital signal processor, a modulator, and other components necessary for manipulating complex audio signals. An ultrasonic transducer can be understood as a device that converts an electrical signal into ultrasonic sound waves. They operate at frequencies above the range of human hearing, typically exceeding 20 kHz. Due to the short wavelength of ultrasound, these transducers can produce highly directional ultrasonic beams. Generally, the ultrasonic acoustic signal, or the sound signal from the transducer, can be referred to as an ultrasonic beam, which may originate from an ultrasonic loudspeaker.

[0019] The disclosed technology could potentially create personalized audio experiences for users of electronic user devices. By confining audible sound to a specific spatial area, this method can reduce sound pollution in the surrounding area. Compared to traditional speaker systems, the use of ultrasound allows for precise control of the sound field, thereby improving audio clarity and directionality. The nonlinear interaction of ultrasound in the air enables the generation of audible sound without the need for large, visible speaker arrays, potentially allowing for more compact and aesthetically pleasing audio devices.

[0020] It should be understood that the features described above and those explained below can be used not only in the indicated combinations, but also in other combinations or in isolation, without departing from the scope of this disclosure. In particular, the features of the disclosed embodiments can be combined with each other in other embodiments.

[0021] It should be understood that the features described above and those to be explained below can be used not only in the indicated combinations, but also in other combinations or in isolation without departing from the scope of this disclosure. Attached Figure Description

[0022] Those skilled in the art will understand and appreciate these and other objects of the invention from the detailed description of the preferred embodiments and the following drawings, in which the same reference numerals refer to the same elements.

[0023] Figure 1 The illustrations schematically depict the direct positioning of ultrasonic beams from a television device to create individual listening zones, according to various examples.

[0024] Figure 2 The illustration schematically shows the indirect positioning from various examples. Figure 1 The ultrasonic beam of the television device is used to create a further individual listening zone.

[0025] Figure 3 A block diagram of a television device with an integrated ultrasonic speaker and head-tracking camera for creating a personalized listening area is shown schematically according to various examples.

[0026] Figure 4 Portable wireless speakers with integrated ultrasonic speakers mounted on a rotatable platform are illustrated schematically according to various examples.

[0027] Figure 5 The diagram schematically illustrates methods for automatically positioning an ultrasonic beam from a portable wireless speaker on a horizontal plane, according to various examples.

[0028] Figure 6 The methods for automatically and simultaneously positioning ultrasonic beams from a portable wireless speaker located in both a vertical and horizontal plane are illustrated schematically according to various examples.

[0029] Figure 7 The schematic illustration shows the configuration of a directional microphone on a rotatable platform of a portable wireless speaker according to various examples, for differential control of beam positioning.

[0030] Figure 8 A block diagram of a closed-loop control system for automatically locating an ultrasonic beam using differential microphone signals, based on various examples, is shown schematically.

[0031] Figure 9 The steps for creating individual listening zones are illustrated schematically according to various examples.

[0032] Figure 10 Audio devices are illustrated schematically according to various examples. Detailed Implementation

[0033] Embodiments of the invention will now be described in detail with reference to the accompanying drawings. It should be understood that the following description of the embodiments should not be construed as limiting. The scope of the invention is not intended to be limited by the embodiments described below or by the drawings, which are considered illustrative examples of the overall inventive concept. Unless otherwise expressly stated, features of various embodiments may be combined with each other.

[0034] The accompanying drawings should be considered as schematic representations of the elements shown, and they are not necessarily shown to scale. Rather, the various elements are shown such that their function and general purpose will become apparent to those skilled in the art. Any connection or coupling between functional blocks, devices, elements, or other physical or functional units shown in the drawings or described herein may also be achieved through indirect connection or coupling.

[0035] The following describes technologies related to audio reproduction in consumer electronic devices, with a particular focus on wireless speakers, televisions, and other stationary and portable audio devices capable of reproducing audio signals in indoor or outdoor environments. These technologies address the challenge of creating individual listening zones using directional sound technologies, particularly ultrasonic speakers. The aim of these technologies is to confine audible sound to a restricted area where the target listener is located, thereby minimizing disturbance to others nearby. This approach aims to enhance the user's listening experience while reducing overall noise pollution, potentially enabling features such as a "night mode" for watching or listening late at night without disturbing others. By leveraging the directional nature of ultrasonic technology, these technologies strive to create dedicated listening spaces that meet the needs of modern consumers in a variety of environments, from home entertainment systems to portable audio devices used in public spaces.

[0036] Some examples of this disclosure typically provide multiple processors, sensors, transducers, speakers, or other electrical processing devices. All references to circuits and other electrical devices and the functions they each provide are not intended to be limited to what is illustrated and described herein. It should be understood that any audio system, speaker, or other processing device disclosed herein may include any number of microcontrollers, general-purpose processor units (CPUs), graphics processing units (GPUs), integrated circuits, memory devices (e.g., flash memory, random access memory (RAM), read-only memory (ROM), electrically programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or other suitable variations thereof), and software that cooperates with each other to perform the operations disclosed herein. Additionally, any one or more of the electrical devices may be configured to execute program code embodied in a non-transitory computer-readable medium programmed to perform any number of the disclosed functions. In various examples, the processing device may be embodied as a remote or cloud computing device. It should be understood that other sensors may be used to detect vibrations in a solid-state body, including sensor devices with optical, mechanical, electromagnetic, or capacitive structures, which can be used to detect vibrations in the solid-state body of the sound transducer element.

[0037] Using loudspeakers (such as portable wireless speakers) to play audio in your home or public areas can disturb others nearby. For example, if someone is exercising on the beach and uses a portable wireless speaker to play audio from their smartphone, it can be uncomfortable for other visitors. Another example is watching television at night when others might want to be in a quiet environment. However, the sound waves emitted by a conventional loudspeaker are typically distributed in all directions due to their physical properties, which cannot be overcome when using a conventional transducer.

[0038] Current solutions to this problem have limitations. One way to listen to audio without disturbing others is to wear personal headphones or headsets. This method delivers acoustic signals personally and makes them inaudible (or barely audible) to others nearby. However, for some people, wearing headphones or headsets can be uncomfortable, especially during physical activity. Some individuals find it unpleasant to have devices in their ears or to have audio sources very close to their ears.

[0039] Another method for creating individual listening spaces is to use directional loudspeakers, such as electrostatic loudspeakers or loudspeaker arrays. This technique is commonly used in museums to play audio information about exhibits back only to the sweet spot in a specific space where the listener is located. Other visitors standing at a distance, in front of other exhibits, or moving around will not hear the audio from the loudspeaker. There may only be low-intensity reflections from the floor and clothing, which may allow other visitors to hear it slightly, but will not significantly disturb them.

[0040] However, this solution assumes a fixed optimal listening point (i.e., the area where the sound reproduced by the speaker is clearly audible) and that the loudspeaker is fixed in a certain position, usually above the listener's head. This method is also quite expensive because it requires either an electrostatic loudspeaker, which is expensive, large, heavy, and requires special equipment to generate a high-voltage signal, or a loudspeaker array, which is expensive due to the need for digital signal processing algorithms to generate directionality. The loudspeaker panel is also quite large.

[0041] These existing methods do not allow the use of portable speakers capable of producing omnidirectional sound and, when necessary, switch to a "limited optimal listening point" mode, where the generated sound can only be heard in a dedicated, relatively small space. While it might be possible to use directional speakers in televisions by mounting them in the ceiling, this solution significantly increases the overall cost of the television system due to the high cost of directional speakers (whether electrostatic or array-based). Furthermore, this approach requires professional installers to mount the speakers in the ceiling, provide the cables, and conceal these cables in conduits for aesthetic purposes, further increasing the total cost. Therefore, this method is not suitable for television products targeting an economy system.

[0042] The disclosed technology aims to address these limitations by providing a method for creating an individual spatial listening zone for users of electronic consumer devices, which can be implemented by hardware components included in the electronic consumer device. The method uses an ultrasonic transducer to generate a directional ultrasonic acoustic signal that nonlinearly interacts with the air within the spatial listening zone, producing an audible sound signal that is substantially confined to that zone and inaudible outside of it.

[0043] In various examples, the disclosed technology provides methods and apparatus for creating individual spatial listening zones by supplying a directional and compact ultrasonic loudspeaker mounted on a rotatable platform to a portable wireless loudspeaker and a television. An ultrasonic loudspeaker can refer to a device for generating directional audible sound waves. A single ultrasonic loudspeaker may include two ultrasonic transducers connected to a DSP via two digital-to-analog converters (DACs), which convert digital input audio signals into ultrasonic digital audio signals. The digital input audio signals may be applied directly to the input of the ultrasonic loudspeaker or obtained from analog input audio signals via an analog-to-digital converter (ADC).

[0044] When the system is activated by a user using a remote control (such as a smartphone) in "Restricted Optimal Point" mode, the system deactivates the conventional speakers, activates the ultrasonic speakers, and allows the user to set up a personal listening zone by adjusting the spatial orientation of the ultrasonic speakers. The aim of this technology is to minimize areas where audio signals from mobile or fixed devices might disturb others nearby. By focusing sound reproduction on the specific area where the target listener is located, these methods aim to create a dedicated listening space. This targeted approach seeks to confine the audible output to the user's immediate area, thereby reducing sound spillover and potential disturbance to others nearby. This technology may prove particularly useful in shared living spaces, open-plan offices, or public areas where traditional audio playback could be disruptive. By creating these individualized listening zones, the technology aims to enhance the user's audio experience while respecting the acoustic environment of those around them.

[0045] Figure 1 The illustration schematically shows the direct positioning of an ultrasonic beam 120 from a television device 101 to create an individual listening zone 103, according to various examples.

[0046] If available Figure 1 As seen, the listening environment 100 is a room containing various components of an audio system and a listener 102. A television device 101, a typical electronic consumer device 101, is equipped with an ultrasonic speaker 110 built into its hardware structure. This ultrasonic speaker 110 is configured to emit a highly directional ultrasonic beam 120, which is visualized as a conical projection from the television device toward the user 102. The user or listener 102 is positioned within the room, seated at a distance from the television device 101. The ultrasonic beam 120 is directed toward the user to create a focused audio experience only within spatial listening areas 103, 104.

[0047] The spatial listening area 103, which directly surrounds the user's head, has high audio quality and can be considered the optimal listening point 103. This optimal listening point 103 represents the optimal listening area, where ultrasonic signals interact through nonlinear demodulation in the air to produce the desired audible sound.

[0048] Surrounding the optimal listening point 103 is a low-intensity zone 104. Some sound is experienced in this area, but at a reduced intensity compared to the optimal listening point. The low-intensity zone demonstrates the spatial constraint of the audio signal. The rest of the room (indicated as 105) is designated as a silent zone. In this area, the audio signal is essentially inaudible. Directly positioning the ultrasonic beam 120 towards the user 102 allows for precise control of audio delivery, thereby enhancing the user experience while minimizing potential disturbance to others in the room.

[0049] Ultrasonic loudspeakers are known in audio engineering. Their operation is based on the partial nonlinearity of the medium (air and surface) used for ultrasound. This nonlinearity can be demonstrated by emitting two ultrasound waves at frequencies f1 and f2 (f1≠f2, f1, f2 = 40…55 kHz). As they interfere with each other in the medium, parasitic sum-frequency harmonics and difference-frequency harmonics appear: f1+f2 and f1-f2, with amplitudes significantly lower than the original wave. For example, if f1 = 49000 Hz and f2 = 48000 Hz, the corresponding sum-frequency and difference-frequency harmonics are 97000 Hz and 1000 Hz, respectively. When f1 and f2 are emitted into the air, three of the aforementioned frequencies—f1, f2, and f1+f2—are inaudible to the human ear because they are beyond its hearing capacity, while f1-f2 is audible. Ultrasonic loudspeakers utilize the presence of these parasitic difference-frequency harmonics.

[0050] The system receives an input audio signal and outputs the desired audible sound signal to the user based on that input audio signal. The input audio signal is processed to generate a first ultrasonic electro-audio signal and a second ultrasonic electro-audio signal, wherein the processing includes modulating one of the first and second ultrasonic electro-audio signals into a carrier signal using a signal obtained from the input audio signal. Each ultrasonic loudspeaker is a box containing circuitry with a digital signal processor (DSP), in which a signal processing algorithm for forming two ultrasonic channels is operating. The conversion circuitry can be referred to as an audio processing unit that generates ultrasonic signals or a digital signal processor (DSP). The purpose of the algorithm is to form an amplitude modulation on one of these frequencies such that the difference frequency wave f1(t)-f2(t) contains the audio signal to be played back at each time point t. The DSP is electrically connected to an analog-to-digital converter (ADC), which forms two sine waves with frequencies f1(t) and modulated f2(t), which are fed to two ultrasonic transmitters (also called ultrasonic transducers). The ultrasonic transmitters convert the electrical signals into acoustic ultrasonic waves, which will propagate in the air, interfere with the air, and thus form the aforementioned sum-frequency side effect wave and difference-frequency side effect wave. Since the difference frequency component is audible to the human ear, it is used as the target (desired) wave. Although its amplitude is significantly lower than that of the two main ultrasonic waves, sin(f1(t)) and sin(f2(t)), it is clearly audible.

[0051] A first ultrasonic electro-audio signal is provided to a first ultrasonic transducer, and a second ultrasonic electro-audio signal is provided to a second ultrasonic transducer, both of which are included in the electronic consumer device. The first and second ultrasonic electro-audio signals are converted into a first directional ultrasonic acoustic signal and a second directional ultrasonic acoustic signal, respectively, by the first and second ultrasonic transducers. The first and second directional ultrasonic acoustic signals are directed to a spatial listening area, which is a restricted area within the user's space.

[0052] Ultrasonic loudspeakers are known to have strong (narrow beam) directivity. This means that when an ultrasonic loudspeaker is directed toward a point in space, the resulting desired difference-frequency acoustic signal will only be audible in the space around the line connecting the ultrasonic loudspeaker to that point, thus forming an acoustic "beam." In the rest of the space, there will be no audible signal. Furthermore, ultrasonic loudspeakers are small and lightweight. This property makes it possible to use them as supplementary speakers in portable loudspeakers and televisions, which can be used in situations where a personal listening space must be created. If the played-back audio will not disturb others, a regular omnidirectional loudspeaker can be used.

[0053] Therefore, an audible sound signal for the user is generated within the spatial listening zone, wherein the audible sound signal substantially corresponds to a desired audible sound signal based on the input audio signal, and wherein the audible sound signal is generated by the nonlinear interaction of a first directional ultrasonic acoustic signal and a second directional ultrasonic acoustic signal in the air within the spatial listening zone.

[0054] Figure 2 The illustration schematically shows the indirect positioning from various examples. Figure 1 The ultrasonic beam of the television device 101 is used to create further different shapes of individual listening areas 203, 204.

[0055] Indirect positioning assumes that the ultrasonic beam is first routed to a surface, such as the ceiling or wall of a room, and then reflected towards the listener's head, as can be seen in... Figure 2 As seen in the text.

[0056] Ultrasound is known to have excellent reflective properties, which makes it possible to use an ultrasonic loudspeaker to create a personal listening space outside the loudspeaker's direct "visible" area. For example, if a loudspeaker is located around a corner where a personal listening space is to be created, if the beam is directed against a wall and then reflected from the wall, it will change its direction and may enter an area that the ultrasonic loudspeaker cannot directly see, thus creating that personal listening space.

[0057] exist Figure 2 The image illustrates the use of reflectivity, where an ultrasonic beam 120 emitted by the integrated ultrasonic speaker 110 of the television unit 101 is directed upwards and then reflected from the ceiling toward the user at a reflection point 105, as shown in the image. Figure 2 As shown. The reflected ultrasonic beam is guided toward user 102, creating spatially confined listening zones 203, 204. The spatial listening zones 203, 204 are larger than those in the horizontal plane. Figure 1 Stronger restrictions in.

[0058] The ultrasonic beams depicted in the diagram correspond to or include two ultrasonic beams, each emitted from one of two ultrasonic transducers. These beams represent a first directional ultrasonic acoustic signal and a second directional ultrasonic acoustic signal. When these ultrasonic signals interact in the air within the spatial listening area, they produce an audible sound signal through nonlinear demodulation.

[0059] The disclosed technology allows for flexible beam path configurations. Two ultrasonic beams from the first and second transducers can be guided along different paths. For example, one beam can follow a direct path to the spatial listening area, while the other beam can take an indirect path, such as reflecting off a surface like a wall or ceiling. It is at least possible to guide the two ultrasonic beams along different reflection paths. Each beam from the first and second transducers can be independently aimed to reflect off different surfaces in the environment. For example, one beam might reflect off a sidewall, while the other might reflect off a ceiling.

[0060] The characteristics of direct and indirect methods of ultrasonic beam localization can be listed in the table below:

[0061]

[0062] In various examples, a television set may be equipped with both: a set of conventional speakers (e.g., a soundbar) for normal playback, and ultrasonic speakers for use in cases where the conventional speakers might disturb other occupants in the same room. Conventional speakers can refer to loudspeakers that emit sound or acoustic signals in the frequency range audible to the human ear.

[0063] Figure 3 Schematic illustrations of US1 to US with integrated ultrasonic loudspeakers according to various examples. N and head tracking camera Cam1 to Cam N A block diagram of a television device 300 used to create personalized listening areas.

[0064] In addition to targeting Figure 1 and Figure 2 In addition to the description, the television device 300 may also be equipped with at least one head-tracking camera that continuously detects and monitors the position of the head. Based on the video stream obtained by the camera, a video image processing algorithm can generate commands to change the angle position of the beam according to the user's movement during playback, thereby providing the user with optimal sound quality. Furthermore, several ultrasonic speakers US1 to US2 may also be provided. N Each speaker is used to create a personalized optimal listening point for each listener / viewer. Meanwhile, a single tracking camera is sufficient to track the head position of all users. The tracking camera can be an integral part of the television set or a separate device connected via a wired or wireless interface. The idea of ​​using a tracking camera to follow the listener's head position along with a corresponding tracking algorithm is not new, but the idea of ​​combining ultrasonic speakers to control their angular position, thereby guiding the ultrasonic beam and creating an optimal listening point for each listener, is novel. Angular position can refer to the orientation or posture of the ultrasonic transducer or ultrasonic beam.

[0065] The television device 300 includes a processing unit 310, which serves as the central computing core of the system. Within this processing unit, a control unit 320 manages and coordinates various components and processes. An audio signal processing unit 340 is responsible for processing the input audio signals and generating the ultrasonic electro-audio signals required to create a spatial listening area.

[0066] Television unit 300 includes K conventional built-in loudspeakers LS1...LS K And N ultrasonic amplifiers US1...US N The system is mounted on N rotatable platforms, each with two angular degrees of freedom: one in the horizontal plane and one in the vertical plane. To rotate these platforms in each plane, two servo motors are used: SrvMH... n - Used for rotations in the horizontal plane, and SrvMV n - For rotation in the vertical plane, n=1……N. These servo motors are controlled by a control unit, providing power to the ultrasonic speakers US1......US mounted on the rotatable platforms 330-1 to 330-N. N The emitted ultrasonic beam is positioned in the front half of the television unit 300. The control unit 320, which can physically operate within a microcontroller or another processor, is responsible for several sub-functions. One of its functions is to process data from the head-tracking cameras Cam1...Cam... MThe video stream is obtained by M≥3 cameras. These cameras are equipped with infrared sensors to operate in dark spaces, delivering images to a video processing algorithm that identifies the position of the user's head in each video frame and calculates the coordinates of the center of each listener's head relative to the camera at each discrete moment. To calculate not only the angular position of the head relative to each ultrasonic speaker but also the three-dimensional coordinates, the number of cameras should be three or more. Therefore, three images of the same space taken from different positions are used, and a stereo-based measurement method is applied. For better accuracy in head coordinate calculation, the cameras can be positioned at a distance from each other, for example, 2…5 cm. A second function of the control unit 320 is to perform angular positioning of the ultrasonic beams to ensure that each beam reaches its target head. Various control algorithms can be applied for this purpose. The simplest method is direct control without feedback, since the positions of all ultrasonic speakers relative to the cameras are known. Direct control is the simplest and fastest method for positioning ultrasonic beams. However, the accuracy of direct control may be affected, especially when there is an error between the actual angular position of the moving platform and the desired angular position. Other causes of error may exist, such as slight angular displacement of the ultrasonic loudspeaker (which may occur during system utilization) or aging effects, leading to a mismatch between the expected and actual optimal listening points. To eliminate this problem, closed-loop control can be used. For this reason, infrared lasers IRLas1...IRLas can be installed on all rotatable platforms 330-1 to 330-N. N This involves emitting laser beams that are guided to the same position as the ultrasonic beams. These laser beams are invisible to the human eye but visible to an infrared camera, which can then assess the error in the actual angular position of each rotating platform, thus providing input to the closed-loop control system to correct for the error. Closed-loop control may be slightly slower, but more precise.

[0067] The safety of using low-intensity invisible infrared laser beams, either directly or through reflection, on a person's face (especially if the eyes are accidentally affected) is not discussed here. However, we assume that an acceptable radiation intensity and duration can be found after additional medical investigation. For example, the laser beam may only be activated if significant user head movement (e.g., exceeding 7-10 cm) is detected, so that the angular position of the corresponding moving platform needs to be corrected. The duration of the laser beam radiation could be a few milliseconds, just enough for the camera to detect the actual angular position of the corresponding rotating platform.

[0068] Listeners can decide whether to use direct or any type of indirect beam positioning by changing settings (e.g., via a remote control). The reflective surface used for indirect positioning (e.g., a ceiling or any wall) can also be specified via settings.

[0069] A combination of feedforward (direct) and closed-loop control can also be considered, where direct control is used for normal tracking mode, while closed-loop control is used for calibration mode, which will be activated in rare cases to eliminate possible aging effects and other reasons for the mismatch between the assumed angular position of the camera and its actual angular position. The detected deviation (mismatch) is then stored as a correction factor in memory (e.g., in EEPROM).

[0070] The third function of control unit 320 is to search for the user's head and define its three-dimensional coordinates before the control system begins tracking the user's head. This step includes image analysis and human head recognition. Image analysis algorithms are well-known and have been used for many years. We will not discuss them in detail here.

[0071] The fourth function of the control unit 320 is sound management. Specifically, it activates the individual listening zone mode by sending corresponding commands to the audio processing unit, thereby enabling the ultrasonic speakers US1...US N And disable the regular speakers LS1...LS K The individual listening zone mode is also deactivated by the control unit.

[0072] The audio processing unit 340 is responsible for receiving audio from audio sources ASrc1...ASrc L The audio processing unit can process any signal and execute audio processing algorithms, such as volume, bass, treble, balance, multi-channel audio processing, surround sound algorithms, etc., as well as execute control commands sent from the control unit. The audio processing unit can be physically implemented on a DSP (Digital Signal Processor), which may be the same as or different from the processor used to implement the control unit.

[0073] Another application of ultrasonic loudspeakers in creating individual listening zones is in portable wireless loudspeakers, such as those mentioned above. Figure 4 As shown. For this purpose, the portable speaker can be supplied with an additional ultrasonic speaker mounted on a mobile platform, which will allow adjustment of the beam position. If the portable speaker does not appear to disturb others, the user can activate the normal mode, in which a conventional (e.g., electrodynamic) loudspeaker will play back the audio content. However, if the played-back audio may disturb others, an individual listening zone mode based on the integrated ultrasonic speaker can be activated.

[0074] Figure 4 Portable wireless speakers with integrated ultrasonic speakers mounted on a rotatable platform are illustrated schematically according to various examples.

[0075] If available Figure 4 As seen, the portable wireless speaker includes an integrated ultrasonic speaker mounted on a rotatable platform, which can perform any method or combination of methods according to this disclosure.

[0076] The portable wireless speaker 400 has Bluetooth functionality, as indicated by the symbol on its front. On the top of the speaker, position 410 is designated for mounting a rotating platform with an attached ultrasonic speaker.

[0077] Figure 4 The right side provides a detailed view of the rotatable platform 420, including mechanical and electrical aspects. This platform is designed to control the orientation of the spatial listening area, as shown below. A small box 450 comprising two ultrasonic transmitters is mounted on the platform. These transmitters correspond to the first and second ultrasonic transducers described in the claims. Arrow 430 indicates the direction of the emitted ultrasonic sound, which can be adjusted by rotating the platform. The platform allows rotation in the horizontal plane 460, enabling the ultrasonic beam to be directed at different angles around the speaker. Furthermore, the platform can tilt in the vertical plane 440, thereby allowing adjustment of the beam height.

[0078] The direction of the ultrasonic beam can be adjusted manually or by a servo motor. The mini-box with the ultrasonic transmitter can be manually positioned in a horizontal plane by rotating the rotating platform clockwise or counterclockwise. To adjust the beam in the vertical plane, the mini-box can be mounted on shaft 424. To fix the tilt in the vertical plane, a special holding mechanism is anticipated, which can fix the angular position of the mini-box, for example, a clamp holding the mini-box from the side. Figure 4 (Not shown above). A more convenient method for changing the angular position of the beam is to supply one or two servo motors to the rotating platform: one servo motor will rotate the platform around gear shaft 423. Rotation in the vertical plane can be accomplished by the other servo motor. It allows the mini-box to tilt to the desired angle via gear 422 and half gear 421. Both servo motors can be remotely controlled, for example, using a smartphone connected to the speaker via Bluetooth or other wireless channels.

[0079] In control applications, users can change the azimuth and vertical angles through an intuitive graphical user interface, such as requesting clockwise or counterclockwise rotation until the user stops, for example, after hearing a sound or reaching a desired angle. Furthermore, if the device used for remotely controlling the angle positioning of the ultrasonic beam has an integrated microphone (e.g., a smartphone or tablet PC), automatic positioning is possible, such as by adjusting the position of the ultrasonic beam. Figure 5 Described.

[0080] This rotatable platform design allows for control of the direction of the ultrasonic beam, with dual-axis rotation (horizontal and vertical) allowing for adaptation to different user locations and room configurations for individual spatial listening areas.

[0081] Figure 5The diagram schematically illustrates methods for automatically positioning an ultrasonic beam from a portable wireless speaker on a horizontal plane, according to various examples.

[0082] If available Figure 5 As seen in the graph, the curve is plotted with time on the x-axis and sound intensity I (in dBA) on the y-axis. Above the curve, there is a graph showing the angular position of the ultrasonic beam relative to the mobile device at different time points. This graph represents the microphone signal received at the mobile device based on test signals from the first and / or second transducers; in other words, the audio feedback signal. Based on the feedback signal, the characteristics of the microphone signal (intensity in this example) can be determined to determine and send angle correction control signals for adjusting the directions of the first and second directional ultrasonic acoustic signals.

[0083] As will be described, the optimal angular position of the transducer is identified based on the monitored microphone signal corresponding to the user's position. Furthermore, the directions of the first and second directional ultrasonic acoustic signals emitted by the ultrasonic transducer can be adjusted to align with the identified optimal angular position.

[0084] In general, the process can be divided into several stages, marked by reference time points t1 to t6. At time point t1, the beam is not directed towards the moving device, resulting in no sound detection. At time point t2, as the beam begins to rotate, a low-intensity sound is detected. At time point t3, the beam approaches its optimal position, resulting in a high-intensity sound. At time point t4, a stop command is issued. At time point t5, the beam rotation stops. At time point t6, the beam reaches the position of maximum sound intensity.

[0085] The angular orientation of the beam relative to the mobile device is expressed as follows: arrive ,in This indicates the optimal orientation for maximum sound intensity. The adjustment process is broken down into several stages. In stage 510, an initial counter-clockwise rotation is performed at a high angular velocity. In stage 511, the counter-clockwise rotation slows to a lower speed. Stage 512 corresponds to the interruption stage when the system detects high intensity. In stage 513, a low-speed clockwise rotation is performed, entirely operated by the wireless speaker without any additional commands from the control application. In stage 514, the rotation stops, where a "stop" command is issued at time points t4 and t5. The diagram at top 500 illustrates the various angular positions / orientations of the beam relative to the mobile device throughout the process. This automated method allows the ultrasonic beam to find the optimal position to deliver maximum sound intensity to the target mobile device, thereby improving the effectiveness of the individual listening area.

[0086] The process can be described in more detail below for the aforementioned time points and stages. We assume that the angular height (vertical plane) is fixed at a predefined position (e.g., +20 degrees), and that an optimal azimuth orientation (horizontal plane) should be found where the maximum intensity of the audio signal is perceived. For example, after coupling with a portable wireless speaker, for instance, via Bluetooth, the smartphone can initiate the playback of a periodic test signal (e.g., a pure tone at 440 Hz) through the portable wireless speaker. At the start time t1, the smartphone's integrated microphone does not detect the test tone because the ultrasonic speaker's angular position in the horizontal plane is away from the direction of the microphone.

[0087] Then, a specialized control application sends a command to the wireless speaker to begin rotating in the horizontal plane, for example, counterclockwise (the initial direction of rotation is not important). During rotation, the wireless speaker maintains a transmission of the instantaneous angular position of the rotating platform relative to a predefined "zero" position. These values, transmitted with a certain discreteness (e.g., a sampling frequency of 50 Hz), can be used by the application as confirmation that rotation has begun and for detecting the actual instantaneous speed. These values ​​will later be used to precisely position the rotating platform at maximum intensity. At a certain time t2, the microphone begins detecting a test tone of a (low) intensity. After the rotating platform continues to rotate in the same direction but at a lower speed, it passes through the direction of maximum intensity at a certain point t3. However, the application still cannot determine whether this direction represents maximum intensity. Therefore, the rotating platform continues to rotate in the same direction and at the same speed until a decrease in the intensity of the test signal is detected (point t4). At this point, the control application sends a command to stop rotating. Simultaneously, the angular position of maximum intensity is defined and memorized. Defining this position is easy: it's the angular value received from the wireless speaker, where the maximum intensity is detected. Due to some inertia, the rotating platform cannot stop rotating immediately, but continues to rotate until time t5 (the actual distance between points t4 and t5 is small, but for better understanding, we need to consider the time t5). Figure 5 (This time distance was artificially magnified). Afterward, the application can send commands to the wireless speaker to rotate to the desired angular position. Instead of controlling the entire process, the user only needs to confirm that the requested angle position has been reached and the rotation has stopped. If, for some reason, the user needs to correct the optimal angle position of the beam (for example, if there are several listeners whose positions may be slightly different from those of the listeners with control over the application), he can do this manually within the same application, which will provide a corresponding GUI.

[0088] In other examples, the algorithms used in control applications can automatically and simultaneously search for optimal angular positions in both the vertical and horizontal degrees of freedom. For this reason, rotation in the horizontal plane should be slower, such that during a period when the horizontal angular motion is equivalent to half the angular width of the beam in the horizontal plane, the vertical motion should cover the entire angular range from the highest to the lowest vertical position, or vice versa. This will ensure that even in the worst-case scenario, the beam will reach the angular position relative to the microphone of the moving device used to control the beam position. Figure 6 The image shows a graphical representation of the beam's angle positioning when searching for the optimal point.

[0089] Figure 6 The methods for automatically and simultaneously positioning ultrasonic beams from a portable wireless speaker located in both a vertical and horizontal plane are illustrated schematically according to various examples.

[0090] This graph illustrates the acoustic search pattern, representing the movement of the ultrasonic beam in both the vertical and horizontal planes. The vertical axis 610 represents the vertical angular position, with 611 indicating the highest vertical angular position and 612 indicating the lowest. The horizontal axis 620 represents the horizontal angular position, with 621 marking the starting position in the horizontal plane.

[0091] The oscillation movement begins at the top 611 and progresses downwards and horizontally, searching for the optimal beam position. To avoid missing the user's angular position detected by the microphone of the small mechanism, one cycle of the oscillation should not exceed the beam width. The width of the oscillation pattern is indicated by 630, representing half the beam width.

[0092] In summary, the key points in the optimization process are marked as follows: Point 631 indicates the point where the signal was first detected. At this point, horizontal angular movement stops, and only vertical movement continues. From point 631 to 632, the signal intensity changes, first increasing and then decreasing. At point 632, vertical angular movement stops because the signal intensity has decreased. At point 633, the system returns to the point of maximum intensity in the vertical plane. From point 633 to 634, the search continues in the horizontal plane. At point 634, horizontal angular movement stops because the signal intensity has decreased again. At point 635, the system returns to the point of maximum intensity in both the horizontal and vertical planes, which is the optimal position for the ultrasonic beam.

[0093] The circular area marked 637 represents the angular space in which the microphone receives the acoustic signal. This area encompasses the optimal beam position and the surrounding region where the signal can be detected.

[0094] This method allows for precise two-dimensional positioning of the ultrasonic beam (angular position in the vertical plane and angular position in the horizontal plane) to create the most effective individual listening zone for the user, as described in more detail below. Angle correction control signals can be sent as commands to adjust the direction of the ultrasonic beam.

[0095] When according to Figure 6 When the sinusoidal pattern described is used to filter space, the ultrasonic beam will reach point 631, where the microphone first detects the test signal. At this point, horizontal rotation should stop, and only rotation in the vertical plane should remain to find position 3, where the maximum intensity of the test signal is detected. Point 633 cannot be found immediately, but is found by comparing the intensity at each point during vertical movement. Therefore, during vertical movement, the beam will first pass through point 633, where the maximum intensity is, and reach a point 632, where the intensity is lower than at angle position 633. The optimal vertical position with the highest intensity can be found only by comparing the intensity at different points. After stopping vertical movement at point 632 and having information that the maximum intensity is at point 633, the application returns the beam to point 633, always maintaining the horizontal position. After finding the vertical position with the highest intensity, movement in the horizontal plane begins, keeping the vertical angle position fixed. During this horizontal movement, the beam will pass through point 635, where the maximum intensity received by the microphone controlled by the moving device will be detected. However, just like with vertical motion, the maximum value might only be detected by comparing the intensity at each point. Therefore, horizontal motion will only stop when the test signal intensity begins to decrease as it approaches point 634. The control algorithm will then return the beam to point 635. Thus, the optimal position will be found automatically.

[0096] The control application can give users the ability to manually adjust the angle beam position after finding the optimal point. For example, this might be necessary if there are several users whose angular positions relative to a portable wireless speaker may differ from those of someone holding a mobile device during a screening of a space.

[0097] Figure 6 The curve depicted is time-invariant. It shows the relative proportions between angular motion in the horizontal and vertical planes. Maintaining rotation in the vertical plane at the servo motor's maximum speed is meaningful, while the rotational speed in the horizontal plane should be selected in a way that maintains the aforementioned proportions.

[0098] In addition to the above, in order to obtain a clear reference for the beam being guided, light with a narrow beam can be supplied to the mini-box to show the current angular position of the mini-box.

[0099] Another method for setting the angular position is to use a person's voice, which will serve as a reference signal for automatically defining the angular position of the human head. For this purpose, the rotating plate should be supplied with one or more directional microphones. In the case of using a single microphone, a search algorithm is used to define the direction, assuming the rotation of the rotating plate and mini-box until the sound intensity is maximized. However, this method can take a relatively long time, during which the user should continue speaking, keeping the loudness of their voice at a consistent level. Accuracy may also be affected. A smarter and more accurate beamforming method is to use differential control by installing four directional microphones: two in the vertical plane and two more in the horizontal plane, as shown in the diagram. Figure 7 As stated above.

[0100] Figure 7 The schematic illustration shows the configuration of a directional microphone on a rotatable platform of a portable wireless speaker according to various examples, for differential control of beam positioning.

[0101] Figure 7 A diagram showing a microphone with two directional microphones (labeled C) is shown. L (Left) and C R (Right) Top view of the loudspeaker system. These microphones are positioned symmetrically with respect to the center of the ultrasonic transmitter.

[0102] The directional patterns of the microphones are depicted by dashed lines for the left microphone and the right microphone. These patterns indicate how sensitive the microphones are to sound from certain directions, with the center being the most sensitive. Two points, 701 and 702, are marked on the directional patterns. These represent the signal strength received by each microphone. The left (dashed line) microphone 701 receives a stronger signal than the right (dashed line) microphone 702. The intensity difference between these two points is shown as... This represents the difference in intensity 701 and 702 of the speech received by the two microphones. This difference is crucial for determining the optimal angular position of the ultrasonic loudspeaker. The different angular positions of each pair of microphones allow the system to utilize the effects of different directions. This results in different electrical signal intensities received by the two microphones. The symmetrical positioning of the microphones relative to the ultrasonic transmitter allows the system to accurately determine the positional error in the horizontal plane based on the difference in signal amplitude. Through analysis... The system can automatically adjust the angle and position of the ultrasonic loudspeaker to optimize sound delivery to the target listening area. This method allows for precise aiming of the ultrasonic beam towards the user's location, thereby improving the effectiveness of individual listening area creation. A similar method using two microphones can be introduced to find the optimal orientation in the vertical plane. Figure 8As shown, the control circuit will begin to compensate for the difference by applying a corresponding execution signal to the servo motor, which will define the direction (clockwise or counterclockwise) and rotation speed.

[0103] Figure 8 A block diagram of a closed-loop control system for automatically locating an ultrasonic beam using differential microphone signals, based on various examples, is shown schematically.

[0104] At each subsequent moment, as the servo motor rotates, the amplitudes of the electrical signals from each pair of microphones are compared. Since this difference should be small, the amplitude of the execution signal (proportional to the error) will decrease, thus reducing the rotational speed of the corresponding servo motor. When the difference is zero, the corresponding servo motor should stop rotating, meaning that both microphones in the pair receive the same intensity of the user's voice. This implies that the beam's angular position in the corresponding plane is correct. The servo motor is an inertial mechanism with a transfer function in the Laplace transform domain:

[0105] ,

[0106] in – Transfer gain, which indicates the power intensity of the servo motor. – Inertial time constant, s – Complex-valued variable used in the Laplace transform. The multiplier s in the denominator refers to the pure integrator. In fact, the angular rotational speed is proportional to the applied voltage. Angular position relative to the initial angular position. It will be the integral of the angular velocity.

[0107] Selectable control gain k in the feedback chain c Adjust the control speed. The higher the speed, the faster the control system will respond. However, if k is chosen too high... c This can lead to overcontrol, causing oscillations. In the worst case, it can cause system instability. This parameter should be carefully set when tuning the system. It can also be obtained by calculating using an optimization criterion, for example, a combination of assuming the fastest speed and avoiding oscillations. The optimal value of this gain can also be obtained by modeling the entire system. In automatic control theory, methods for finding the optimal value of the control gain are well-known. They have been used for decades.

[0108] It should be noted that if the device is used in a room, the beam can be directed towards the ceiling, and sound will reflect from there. In this case, angular positioning using voice is only possible in the horizontal plane (azimuth). The angular position of the beam in the vertical plane should be manually corrected.

[0109] The above-mentioned automatic closed-loop control system only works when the user is located at each of the two microphones C. L and C RThe system only operates when the user's voice is between the two center lines of the directional pattern. If this is not the case, a search should be performed. During the search process, the servo motor will begin rotating until it reaches a position where the user's voice is located between the aforementioned center lines. Physically, this state can be detected by fixing a state in which the signal strength in the left channel begins to decrease, while the signal strength in the right channel remains increasing; or vice versa. Once this state is fixed, the aforementioned automatic high-precision control loop will begin operating to compensate for any remaining errors.

[0110] To ensure the above method works in noisy environments, where various voices and ambient noise may exist, the user's voice must be distinguishable from other voices and noise. Therefore, using a specific signal is meaningful, such as the long vowel "AAA" or whistling. A special algorithm will filter out all other noise signals besides the user-generated reference signal.

[0111] The "Normal Mode" and "Personal Listening Space Mode" can be selected using a remote control. In Normal Mode, the ultrasonic speaker is not used; the sound is produced by a conventional (e.g., electrodynamic) speaker. In Personal Listening Space Mode, the conventional speaker is inactive, and the sound is produced solely by the ultrasonic speaker. Modes can also be selected using voice commands. For this purpose, the digital signal processing unit of the portable speaker should be equipped with a voice recognition algorithm to recognize the command used to switch modes. This algorithm can be tuned to accept command phrases in several languages. It can also be trained to accept only the owner's voice and reject any commands from other occupants of the listening space.

[0112] Figure 9 The steps for creating individual listening zones are illustrated schematically according to various examples.

[0113] The method begins at step S10. In step S20, an input audio signal is received, and a desired audible sound signal is output to the user based on the input audio signal. In step S30, the input audio signal is processed to generate a first ultrasonic electro-audio signal and a second ultrasonic electro-audio signal, wherein the processing includes modulating one of the first and second ultrasonic electro-audio signals as a carrier signal using a signal obtained from the input audio signal. In step S40, the first ultrasonic electro-audio signal is provided to a first ultrasonic transducer, and the second ultrasonic electro-audio signal is provided to a second ultrasonic transducer, the first and second ultrasonic transducers being included in an electronic consumer device. In step S50, the first and second ultrasonic transducers convert the first and second ultrasonic electro-audio signals into a first directional ultrasonic acoustic signal and a second directional ultrasonic acoustic signal, respectively. In step S60, the first and second directional ultrasonic acoustic signals are directed to a spatial listening area, which is a restricted area in the user's space. In step S70, an audible sound signal for the user is generated within the spatial listening area, wherein the audible sound signal substantially corresponds to a desired audible sound signal based on the input audio signal, and wherein the audible sound signal is generated by the nonlinear interaction of a first directional ultrasonic acoustic signal and a second directional ultrasonic acoustic signal in the air within the spatial listening area. The method terminates in step S80.

[0114] Figure 10 An audio device 101 according to various examples is shown schematically.

[0115] The audio device 101 may be, for example, an electronic consumer device described throughout this disclosure and is configured to create a personal spatial listening zone for the user. The audio device includes two ultrasonic transducers 111 and 112, positioned to emit directional ultrasonic acoustic signals, indicated by a dashed cone marked 101 above them. The audio device includes a control unit 2, which contains a processor 4 and a memory 5. The memory includes instructions that, when executed by the processor, cause the control unit to perform any method or combination of methods according to this disclosure.

[0116] The dashed cone extending from transducers 111 and 112 represents a listening zone 103 generated by directional ultrasonic acoustic signals, where audible sound is generated through nonlinear interactions in the air. This configuration allows electronic consumer devices to create localized listening zones for users, providing personalized audio without disturbing others nearby, as described in this disclosure.

[0117] From the above, the following general conclusions can be drawn:

[0118] In various examples, the first and second ultrasonic transducers can be mounted on a rotatable platform of the electronic consumer device. The rotatable platform can be configured to adjust the direction of the first and second directional ultrasonic acoustic signals emitted by the first and second ultrasonic transducers in at least one angular degree of freedom. By mounting the ultrasonic transducers on the rotatable platform, the electronic consumer device can dynamically adjust the direction of the ultrasonic acoustic signals. This allows for precise aiming at a spatial listening zone, thereby enhancing the user's listening experience. The rotatable platform enables the device to adapt to changes in the user's position or create multiple listening zones in different directions. Adjustment of at least one angular degree of freedom provides flexibility in guiding the ultrasonic signals, potentially allowing both horizontal and vertical adjustments to optimize the formation of the spatial listening zone.

[0119] A rotatable platform can be understood as a movable base or structure capable of rotating about one or more axes. In this context, it can serve as a mounting point for ultrasonic transducers, allowing them to be reoriented as needed. Angular degrees of freedom can be understood as the ability to rotate or pivot about a specific axis. This allows the platform to adjust the direction of the ultrasonic signal in a circular or arc-shaped manner. A directional ultrasonic acoustic signal can be understood as a highly focused beam of ultrasound propagating in a specific direction with minimal diffusion. The ability to adjust these signals enhances the accuracy and adaptability of spatial listening zone creation.

[0120] In various examples, directing the first and second directional ultrasonic acoustic signals toward the spatial listening zone can include dynamically adjusting the spatial listening zone based on the current user's location or user feedback. This dynamic adjustment allows the electronic consumer device to adapt in real time to changes in the user's location or preferences. By continuously updating the spatial listening zone, the device can maintain a consistent and high-quality audio experience for the user even as they move within the environment. User feedback can be used to fine-tune the listening zone to suit personally choreographed or specific acoustic conditions. This adaptive approach can enhance the overall effectiveness of the spatial audio system, potentially improving user satisfaction and the applicability of the technology in a variety of real-world scenarios.

[0121] Dynamic spatial adjustment can be understood as real-time modification of the size, shape, or position of the spatial listening area. This may involve altering parameters of the ultrasonic signal to reposition or reshape the area that produces audible sound. The current user's position can refer to the user's instantaneous location within the device's operating environment, which can be determined through various sensing technologies. User feedback can be understood as input provided by the user, whether explicitly through control or implicitly through behavior, indicating their preferences for the audio experience. This feedback can be used to improve the characteristics of the spatial listening area.

[0122] In various examples, dynamically adjusting the spatial listening area can include using a camera-based tracking system to detect the user's head position. The method can also include adjusting the listening position around the detected position. By employing a camera-based tracking system, electronic consumer devices can continuously and accurately monitor the user's head position. This real-time tracking allows the device to maintain an accurate spatial listening area even when the user moves or changes orientation. Adjusting the listening position around the detected position can create a buffer that takes into account small movements of the user's head, potentially resulting in a more stable and comfortable listening experience. This approach reduces the need for frequent fine-tuning and provides a more seamless audio experience as the user naturally changes position.

[0123] Camera-based tracking systems can be understood as visual sensing technology that uses one or more cameras to detect and monitor the position of objects or individuals in space. In this context, it may be specifically tuned to identify and track a user's head. The user's head position can be considered a key reference point for optimizing the spatial listening area, as it typically corresponds to the position of the user's ears. Adjusting the surrounding listening position can be understood as modifying the spatial characteristics of the ultrasound signal to create an effective listening area that not only covers the precisely detected location but also a small area surrounding it. This can provide some flexibility in the system's response to user movement.

[0124] In various examples, dynamically and spatially adjusting the spatial listening area may include providing at least one directional microphone mounted on a rotatable platform and aligned with a first and a second ultrasonic transducer. The system can utilize the rotatable platform to perform a two-dimensional acoustic search mode, changing the angular position in both the horizontal and vertical planes. During this search mode, the system can monitor microphone signals from at least one directional microphone, where the microphone signals may correspond to sound from a user. Based on the monitored microphone signals, the system can identify the angular position corresponding to the user's location. The directions of the first and second directional ultrasonic acoustic signals emitted by the ultrasonic transducers can then be adjusted to align with the identified angular position. This approach enables precise user localization within the environment, potentially improving the accuracy of spatial listening area placement. The two-dimensional search mode allows for a comprehensive scan of the surrounding space, increasing the likelihood of correctly identifying the user's location even in complex acoustic environments.

[0125] A directional microphone can be understood as an acoustic sensor designed to be more sensitive to sound from a specific direction while attenuating sound from other directions. In this context, it can be used to locate the source of user-generated sounds.

[0126] Two-dimensional acoustic search can be understood as a systematic approach to scanning the surrounding space by rotating a platform in both horizontal and vertical planes. This can allow for a comprehensive survey of the acoustic environment to locate the user.

[0127] An angular position can be understood as the specific orientation of a rotatable platform, defined by its rotation in both the horizontal and vertical planes. The identified angular position can represent the direction from which the user's voice is received most clearly.

[0128] In various examples, dynamically and spatially adjusting the spatial listening area may include providing at least one directional microphone mounted on a rotatable platform and aligned with a first and a second ultrasonic transducer. The method can utilize the rotatable platform to perform a two-dimensional acoustic search mode, changing the angular position in both horizontal and vertical planes. During this search mode, the method can monitor microphone signals from the at least one directional microphone, where the microphone signals correspond to sound from a user. An optimal angular position corresponding to the user's position can be identified based on the monitored microphone signals. The method can then adjust the direction of the first and second directional ultrasonic acoustic signals emitted by the ultrasonic transducers to align with the identified optimal angular position. This method enables precise localization of the user's position in three-dimensional space, potentially improving the accuracy of the spatial listening area. The two-dimensional search mode allows for comprehensive environmental coverage, increasing the likelihood of accurately detecting the user's position. By aligning the ultrasonic transducers with the identified optimal position, the method can improve the efficiency of spatial audio delivery and potentially reduce unwanted sound spillover into the surrounding area.

[0129] A directional microphone can be understood as an audio input device designed to be more sensitive to sound from a specific direction while attenuating sound from other directions. In this context, it can be used to detect and locate a user's voice or other sounds. A two-dimensional acoustic search pattern can be understood as a systematic scanning movement of a rotating platform, covering both horizontal and vertical angles to thoroughly investigate the acoustic environment. The optimal angular position can be understood as the orientation of the rotating platform, which results in the strongest or clearest reception of the user's voice and may indicate the most direct path between the user and the microphone.

[0130] In various examples, at least one directional microphone may include at least a first directional microphone and a second directional microphone. The method may involve monitoring a first audio signal from the first directional microphone and a second audio signal from the second directional microphone, where these signals correspond to a voice from a user. The method may compare the first audio signal with the second audio signal to determine a difference in signal strength. Based on this difference, an optimal angular position can be identified. By utilizing two directional microphones, this method can achieve more precise positioning of the user via triangulation. The comparison of signal strength between the two microphones can provide additional spatial information, potentially improving the accuracy of user position detection. This dual-microphone approach in a horizontal and / or vertical plane can also help distinguish the user's voice from background noise or reflections, potentially enhancing the robustness of the spatial listening zone adjustment process. For example, the dual-microphone method can be applied to a single plane (e.g., in a horizontal or vertical plane), or to two planes (i.e., in both a horizontal and a vertical plane) using four microphones, or to three directional microphones forming a triangle, such that the directional surfaces of all microphones intersect, resulting in a spatial region covered by all three directional surfaces. The directional surfaces are artificial surfaces formed in 3D polar coordinates: h-angle, v-angle, and distance, where h-angle is the angular offset of the projection of an orthogonal line drawn from the microphone diaphragm onto the horizontal plane, with a positive angular offset corresponding to a counter-clockwise angular direction considered from a top view relative to the microphone mounting position; v-angle is the angular offset of the projection of an orthogonal line drawn from the microphone diaphragm onto the vertical plane, with a positive angular offset corresponding to a counter-clockwise angular direction considered from a lateral right-angle view relative to the top view; the radius coordinates show the measured intensity of the received sound from the reference signal at the angular positions h-angle and v-angle.

[0131] In this context, signal strength can be understood as the intensity or amplitude of the audio signal received by each microphone. The difference in signal strength between two microphones may indicate the relative direction of the sound sources. Triangulation can be understood as a method of determining the location of a point by measuring the angle between a known point at either end of a fixed baseline and that of a given point. In this audio context, two microphones are known points, and the difference in their signals helps determine the angle relative to the user's position.

[0132] In various examples, dynamically and spatially adjusting the spatial listening area may include providing an infrared laser coupled to a first and a second ultrasonic transducer on a rotatable platform. The infrared laser can be activated to emit an infrared laser beam aligned with the direction of the average angular maximum intensity of the first and second directional ultrasonic acoustic signals emitted by the first and second ultrasonic transducers in both horizontal and vertical planes. In other words, the laser beam should be directed toward the direction of maximum intensity of the resulting audible signal, which typically corresponds to the average angular offset of the first and second ultrasonic transducers (in both planes: horizontal and vertical). At least two cameras in the electronic consumer device are positioned at a distance from each other to capture images of the user and the location of a point on the infrared beam. The captured images can be analyzed to determine the actual position of the directional ultrasonic acoustic signal relative to the user's position. This actual position can be compared to the user's position. Based on this comparison, the spatial listening area can be adjusted. This method allows for precise visual feedback on the alignment of the ultrasonic signal with the user's position. Using an infrared laser provides a non-invasive method for visualizing the direction of the ultrasonic signal. A dual-camera setup allows for depth perception and more accurate spatial analysis, potentially improving the precision of listening zone adjustment.

[0133] An infrared laser can be understood as a device that emits a narrow beam of infrared light, invisible to the human eye but detectable by certain cameras (such as infrared cameras). In this context, it can serve as a visual indicator of the direction of ultrasonic signals. The point of the infrared beam can refer to the point where the infrared laser beam intersects a surface, providing a visible reference point for the direction of the ultrasonic signal. Analyzing captured images may involve using computer vision techniques to extract relevant spatial information from the visual data collected by the camera. The actual location of the directional ultrasonic acoustic signal can be understood as the real-world location and orientation of the ultrasonic beam, as indicated by the infrared laser point.

[0134] In various examples, dynamically adjusting the spatial listening zone may include providing an infrared laser coupled to a first and a second ultrasonic transducer on a rotatable platform. The infrared laser can be activated to emit an infrared beam aligned with the directions of the first and second directional ultrasonic acoustic signals. At least two cameras in the electronic consumer device are positioned at a distance from each other to capture images of the user and the position of a point on the infrared beam. The captured images can be analyzed to determine the actual position of the directional ultrasonic acoustic signal relative to the user's position. This actual position can be compared to the user's position, and the spatial listening zone can be adjusted based on this comparison. This method allows for precise visual feedback on the alignment of the ultrasonic signal with the user's position. The use of an infrared laser can provide invisible guidance for the ultrasonic signal, potentially improving the accuracy of spatial listening zone adjustment without visually disturbing the user.

[0135] An infrared laser can be understood as a device that emits a narrow beam of infrared light, invisible to the human eye but detectable by certain cameras. In this context, it can serve as a visual indicator of the average angular maximum intensity direction of ultrasonic signals emitted by two transducers. The point of intersection of the infrared beam can be understood as the point where the infrared laser beam intersects a surface, providing a visible reference point for the direction of the ultrasonic signal in the captured image. There are two ultrasonic signals emitted by two different transducers, whose angular spatial orientations may differ slightly (to produce the intersection of the two ultrasonic beams).

[0136] If we define the term "angular orientation of the ultrasonic loudspeaker (rather than the transducer)," we can define the laser beam's positioning in another way. Since an ultrasonic loudspeaker is a unit comprising two ultrasonic transducers, the angular position of the laser beam would correspond to the angular orientation of the ultrasonic loudspeaker (i.e., a line orthogonal to the plane surface of the ultrasonic loudspeaker). Mathematically, the second definition is less precise. On the other hand, it is easier for non-experts to understand because it uses simpler terminology.

[0137] In various examples, dynamically and spatially adjusting the spatial listening zone may include receiving location information from a portable user acoustic device operating at the user's location and communicatively coupled to an electronic consumer device. This location information may represent the relative position of the portable user device with respect to the electronic consumer device. The positions of a first ultrasonic transducer and a second ultrasonic transducer may be adjusted based on the received location information to spatially adjust the spatial listening zone. This approach can potentially improve the accuracy of spatial listening zone adjustment by utilizing accurate positioning data provided by the user's personal device. Using a portable user device allows for continuous tracking of the user's position, enabling real-time adjustment of the spatial listening zone as the user moves.

[0138] In some examples, location information may include signals from at least one microphone of a portable user device. The method may include transmitting test signals via a first ultrasonic transducer and a second ultrasonic transducer. Audio feedback signals captured by at least one microphone of the portable user device in response to the test signals may be received via the portable user device using an existing wireless connection (e.g., Bluetooth or WiFi). This audio feedback signal may be analyzed to determine characteristics of the received test signals, such as intensity. Based on this analysis, the positions of the first and second ultrasonic transducers may be adjusted. An angle correction control signal may be transmitted via the portable user device to adjust the direction of the first and second directional ultrasonic acoustic signals to optimize the sound intensity at the location of the portable user device. This iterative process of test signal transmission, feedback analysis, and position adjustment allows for fine-tuning of the spatial listening area, potentially leading to improved audio quality and more precise targeting of the ultrasonic signals.

[0139] Portable user acoustic devices can be understood as personal electronic devices capable of audio processing and communication, such as smartphones or wearable devices. Location information in this context can refer to data indicating the position and orientation of the portable device relative to the electronic consumer device. Test signals can be understood as specific audio patterns transmitted to evaluate the acoustic characteristics of the environment and the effectiveness of the current ultrasonic transducer settings. Angle correction control signals can be understood as instructions sent from the portable device to the electronic consumer device to adjust the direction of the ultrasonic signals based on analysis of the received test signals.

[0140] In various examples, the method may include: providing at least one loudspeaker operating within an audible range, as part of a consumer audio device, for directly emitting sound within the audible frequency range based on an input audio signal. The method may also include: selectively switching between a directional ultrasonic mode utilizing a first and a second ultrasonic transducer and a conventional wide-diffusion mode utilizing at least one electromagnetic loudspeaker, based on the input audio signal. This dual-mode approach can allow the electronic consumer device to adjust its audio output strategy based on the characteristics of the input audio signal or user preferences. The directional ultrasonic mode can be used to create a focused, individual listening area, while the conventional wide-diffusion mode can be used for a wider sound distribution. By selectively switching between these modes, the device can optimize its audio output for different scenarios, potentially enhancing versatility and user experience. The ability to use a conventional loudspeaker also provides a rollback option for situations where directional audio is not needed or desired.

[0141] A loudspeaker operating within the audible range can be understood as a conventional electromagnetic transducer that converts electrical signals into sound waves within the range of human hearing, typically 20 Hz to 20 kHz. Directional ultrasonic mode can refer to the operation of using an ultrasonic transducer to create a focused, directional beam of audio through parametric audio effects. This mode can result in highly localized sound. Conventional wide-diffusion mode can be understood as a traditional method of sound reproduction where audio is broadcast from the loudspeaker over a wide area without the focusing directionality of an ultrasonic system. Selectively switching between modes may involve analyzing the input audio signal or considering user settings to determine the most suitable audio output method for the current situation.

[0142] In various examples, directing a first directional ultrasonic acoustic signal and a second directional ultrasonic acoustic signal toward a spatial listening area can include creating indirect beam paths for these signals. These indirect paths can form a spatial listening area at the user's location after reflection from at least one reflective surface. This approach can create individual listening areas located in spatial regions inaccessible to direct sound waves due to the presence of obstacles. By utilizing reflective surfaces, this method can extend the reach of ultrasonic signals to areas that would otherwise be obstructed. This technique can allow for more flexible placement of electronic consumer devices, as it may not require direct line-of-sight contact with the user. The creation of indirect beam paths enables the formation of spatial listening areas in complex environments with various obstacles, potentially increasing the versatility and applicability of the system in real-world environments. By utilizing an indirect path of at least one of the ultrasonic sound signals, the listening area can acquire a more defined shape. Specifically, the area can become narrower or smaller and be clearly defined in all spatial directions. For example, on a horizontal plane surrounding the listener, the area may be spatially defined or restricted in all directions. This indirect path technique can allow for precise control over the size and shape of the listening area, potentially enhancing focus and privacy in the audio experience. The ability to create listening zones in areas where direct line of sight is obstructed increases the system’s flexibility and applicability in complex environments with various obstacles or architectural features.

[0143] Indirect beam paths can be understood as the routes taken by ultrasonic signals, involving one or more reflections on surfaces before reaching a predetermined target area. These paths allow signals to bypass obstacles or reach areas not directly in the transducer's line of sight. In this context, a reflecting surface can refer to any object or structure in the environment capable of reflecting ultrasonic waves, such as walls, ceilings, or furniture. Spatial areas that sound waves cannot directly reach can be understood as locations that cannot be reached via a straight path from the ultrasonic transducer due to interference obstacles. This can include areas around corners or behind large objects. The ability to create listening zones in such areas can significantly enhance the system's flexibility and practicality in a variety of real-world scenarios.

[0144] In various examples, the method may include providing a stereo signal to the user based on four ultrasonic transducers mounted on a rotatable platform. The first and second ultrasonic transducers may correspond to the left audio channel, while the third and fourth ultrasonic transducers may correspond to the right audio channel. The third and fourth ultrasonic transducers may operate in a frequency range at least 20 kHz different from the frequency range of the first and second transducers. This configuration allows for the creation of a more immersive stereo field within the spatial listening area. By using separate transducer pairs for the left and right channels, the system can achieve improved stereo separation and spatial audio effects. The significant frequency difference between the two transducer pairs may help minimize interference between the left and right channels, potentially resulting in clearer stereo imaging. This approach can enhance the overall audio experience by providing more precise spatial cues and a wider sound field within a constrained listening area.

[0145] Stereo signals can be understood as audio formats that use two separate channels (left and right) to create a sense of direction and space in sound reproduction. In this context, it may involve using two pairs of ultrasonic transducers to produce this effect within a spatial listening area. A rotatable platform can be understood as a movable base for mounting the ultrasonic transducers, thus allowing for orientation adjustments to the entire array. A frequency range difference of at least 20 kHz between the transducer pairs can be understood as a significant separation of the ultrasonic frequencies used for modulation. This separation can help prevent crosstalk or interference between the left and right channels, potentially leading to improved stereo separation and clarity in the demodulated audible sound.

[0146] In various examples, electronic consumer devices can include portable or stationary entertainment audio devices. This can encompass a wide range of devices designed specifically for audio playback and entertainment purposes. In the case of portable devices, this could include smartphones, tablets, portable speakers, or dedicated mobile audio players. These portable devices can use directional audio capabilities to create personal listening zones in various environments, such as public transportation, shared offices, or outdoor spaces. The ability to generate localized audio fields can enhance privacy and minimize disturbance to others nearby. For stationary entertainment audio devices, this could include home theater systems, smart speakers, desktop computer audio equipment, or high-end audio equipment. In a home environment, directional audio technology can be used to create individual listening zones for different family members in a shared space, allowing each person to enjoy their preferred audio content without interference. In the context of a home theater, this system can be used to optimize surround sound effects, directing specific audio channels to precise locations around the viewer.

[0147] Portable entertainment audio devices can be understood as any mobile electronic device capable of playing back audio, easily carried and used in various locations. These devices are typically battery-powered and designed for use anytime, anywhere. Fixed entertainment audio devices can refer to audio equipment that is typically located in a fixed position and not intended for frequent movement. These devices are usually larger, may require a constant power supply, and are designed for optimal performance in a specific environment. Entertainment in this context can encompass a wide range of audio content, including music, podcasts, audiobooks, movies, TV shows, and games. Applying directional audio technology to these devices can enhance their functionality by providing better control over the audio environment and improving the overall listening experience.

[0148] In various examples, the first and second ultrasonic transducers can form a parametric loudspeaker array. The audible sound signal generated by this array can be in the frequency range below 20 kHz, corresponding to the typical range of human hearing. Simultaneously, the first and second directional ultrasonic acoustic signals emitted by the transducers can be in the frequency range above 40 kHz. This configuration can use parametric acoustic principles to create highly directional audio. By using ultrasonic frequencies above 40 kHz, the system can generate a narrow, focused sound beam that can travel long distances with minimal spread. The interaction of these ultrasonic waves in the air can lead to demodulation of the signal, thereby producing audible sound within the target listening area. This approach allows for precise control of audio delivery, potentially reducing sound spillover and enhancing privacy. Using frequencies far above the audible range for the carrier signal also minimizes potential interference with other audio systems operating in the environment.

[0149] Parametric loudspeaker arrays can be understood as systems that use ultrasound waves to generate audible sound through nonlinear interactions in the air. This technology utilizes the parametric effect, where the interaction of two high-frequency waves produces lower-frequency components that fall within the audible range. The frequency range below 20 kHz can be understood as the range of sound waves normally audible to humans, encompassing all normal speech and music frequencies. The frequency range above 40 kHz can be understood as part of the ultrasonic spectrum, far beyond human hearing capabilities. These high frequencies can be used as carrier waves to transmit audio information, which is then demodulated in the air to produce audible sound. This technology allows for the creation of highly directional sound beams that can be precisely targeted at specific listening areas.

[0150] This configuration generates a first subharmonic within the audible frequency range using the nonlinear interaction of a first directional ultrasonic acoustic signal and a second directional ultrasonic acoustic signal in the air within a spatial listening zone. This subharmonic can be obtained as the difference frequency harmonic of two ultrasonic waves emitted from the first and second ultrasonic transducers and can substantially correspond to the desired audible sound signal. The nonlinear interaction and the generation of the first subharmonic can occur substantially only within the spatial listening zone, which can correspond to the intersection of the first and second directional ultrasonic acoustic signals. This method enables the creation of highly localized audible sound without conventional loudspeakers. By using ultrasonic frequencies above 40 kHz, the system minimizes interference with other electronic devices and reduces the risk of hearing damage from prolonged exposure to high-frequency sounds. Generating audible sound through nonlinear interaction can result in a highly directional audio beam, potentially enhancing privacy and reducing sound pollution in the surrounding area.

[0151] A parametric loudspeaker array can be understood as a system that uses ultrasound to generate audible sound through nonlinear acoustic effects in the air. The audible frequency range typically refers to sound between 20 Hz and 20 kHz, which is the range of human hearing. Ultrasonic frequencies above 40 kHz are far beyond human hearing. In such an array, the frequencies emitted by all the ultrasonic transducers do not interact with each other during demodulation, resulting in audible frequencies. For this reason, the difference between the two carrier operating frequencies of different ultrasonic transducer pairs must be no less than 20 kHz. An example of a correct distribution of carrier frequencies could be: 40 kHz for the first pair of transducers, 60 kHz for the second pair, 80 kHz for the third pair, and so on.

[0152] In this context, nonlinear interaction refers to the phenomenon where high-intensity ultrasonic waves interact in the air to produce low-frequency components within the audible range. Subharmonics can be understood as frequency components below the fundamental frequency of the original ultrasonic signal. Difference harmonics specifically refer to the frequency difference between two ultrasonic signals, which in this case produces the desired audible sound. The crossover region can be understood as the area where the beams from two ultrasonic transducers overlap, thus creating the necessary conditions for the occurrence of nonlinear acoustic effects.

[0153] Directional ultrasonic acoustic signals can also be called directional ultrasonic beams. An ultrasonic beam corresponds to a highly focused acoustic signal operating within the ultrasonic frequency range, typically above 20 kHz, which exceeds the upper limit of human hearing. This beam exhibits exceptional directionality, meaning it propagates through the air with minimal spatial spread compared to audible sound waves. The beam is generated using a specialized ultrasonic transducer that emits high-frequency sound waves with a narrow radiation mode. This narrow mode is achieved through a combination of transducer design and the physical properties of ultrasound, whose wavelengths are much shorter than those of audible sound. Directional ultrasonic beams maintain coherence over longer distances than conventional audio signals. This property allows acoustic energy to be precisely targeted to specific areas in space. In various examples, these beams are often used in pairs or arrays. When two ultrasonic beams with slightly different frequencies intersect in the air, they interact non-linearly, producing a difference in tone within the audible range. This phenomenon, known as parametric audio or acoustic heterophony, effectively creates a virtual sound source within the beam path. Ultrasonic beams exhibit strong directional properties, meaning that sound energy is concentrated along a predetermined path or within a predetermined narrow angle, rather than spreading in all directions.

[0154] Acoustic search patterns can refer to a systematic method or sequence of movements used to locate and optimize the orientation of a sound source or the position of a listener relative to a directional audio system. In the context of directional ultrasonic beams, acoustic search patterns can include one or more of the following: a predefined series of angular adjustments in both the horizontal (azimuth) and vertical (elevation) planes; control of the movement of the ultrasonic transducer or the platform to which it is mounted; a process of scanning the surrounding space to detect the optimal sound reception or transmission point; continuous or discrete sampling of acoustic feedback at various locations during the search; and an algorithm for analyzing the collected acoustic data to determine the optimal orientation of the ultrasonic beam. Acoustic search patterns can be implemented, for example, using servo motors to control the orientation of the ultrasonic transducer on a movable mounting platform, or electronic beamforming techniques in a transducer array. The goal of acoustic search patterns is to efficiently locate the optimal orientation of the ultrasonic beam to produce the desired audible effect in the target listening area while minimizing sound spillover into other areas. This process can be automated to adapt to changes in listener position or environmental conditions, or manually controlled by the user to fine-tune the audio experience.

[0155] Identifying the optimal angular position based on the difference in signal strength between a first audio signal and a second audio signal may include identifying the angular error between the current orientation of the ultrasonic transducer and the desired target position. This process may include one or more of the following steps: Calculate the difference in signal strength or phase between two or more strategically positioned directional microphones on the device. Interpret this difference as an angular deviation from the optimal position. Determine the magnitude and direction of the angular error on both the azimuth (horizontal) and elevation (vertical) planes. In various examples, this may be performed using one or more of the following steps: Compare the amplitude of the received signals from the microphones (e.g., left to right for azimuth, up to down for elevation). Analyze the phase difference between the microphone signals to estimate the direction of sound arrival. Use a differential control system where the error signal is proportional to the difference in microphone outputs. The angular error can be expressed as a vector with two components. Azimuth error ( ): Angular deviation on the horizontal plane. Elevation angle error ( ): Angular deviation in the vertical plane. The goal is to minimize these two error components, bringing them as close to zero as possible. This is typically achieved through an iterative process of adjustment and measurement, usually implemented as a closed-loop control system. Optimal angular position is achieved when one or more of the following criteria are met: Minimization of the difference in signal strength between paired microphones; the calculated angular error is below a predefined threshold; and the quality of the audio signal in the target listening area is maximized.

[0156] Infrared lasers used to adjust the spatial listening area in directional audio systems can be low-power solid-state devices operating in the near-infrared to short-wavelength infrared spectrum. Typical wavelength ranges can be from 780 nm to 1550 nm, with common choices being 808 nm, 980 nm, or 1064 nm, depending on the specific requirements of the application. These wavelengths are chosen because they are invisible to the human eye and have relatively low absorption by atmospheric water vapor. The laser can operate in various modes, including continuous wave (CW), quasi-continuous wave (QCW), or pulsed mode. CW operation provides constant output and is suitable for continuous tracking applications. QCW mode features periodic switching cycles, reducing average power consumption while maintaining functionality. Pulsed operation is typically in the nanosecond to microsecond range and can be used for time-of-flight measurements or to increase peak power while maintaining low average power for eye safety. Output power can be in the range of 1 to 5 mW, ensuring eye safety even in the event of accidental exposure. The beam can be collimated to maintain a narrow profile over the operating distance, with a divergence angle typically less than 1 mrad. Such a beam profile allows for precise collinear alignment with the ultrasonic beam path. Laser diodes can be coupled with beamforming optics to optimize output characteristics.

[0157] The system can employ at least two infrared-sensitive cameras, positioned separately at known distances, forming a stereo configuration. This setup enables precise 3D positioning using triangulation techniques. The cameras can be equipped with sensors sensitive to the infrared spectrum, allowing them to detect both the infrared laser point and the user's face, even in low-light conditions. Using this dual-camera arrangement, the system can calculate the 3D coordinates of the infrared laser point by analyzing its position in images from both cameras, assuming the distance between the two cameras and / or their positions relative to the laser beam emitter and / or an ultrasonic speaker and / or another reference point in the fixed audio system are known. Simultaneously, for example, a facial recognition algorithm can process the stereo images to determine the 3D position of the user's face or head. The system can then calculate the difference between these two sets of coordinates to determine the accuracy of the current beam alignment and calculate necessary adjustments to the ultrasonic beam direction. This image processing and coordinate calculation can be performed in real time, potentially allowing for dynamic tracking and adjustment of the ultrasonic beam as the user moves. Initial calibration may be required to establish the spatial relationships between the cameras, the infrared laser, and the ultrasonic transducer. By utilizing this dual-camera approach, the system can provide the necessary data for precise 3D tracking and alignment, potentially ensuring that the ultrasonic beam is accurately guided to the optimal listening position. This method offers a robust solution to maintain the effectiveness of the individual listening zone as the user's position changes.

[0158] Sending control signals to adjust the spatial listening area can be performed using one of the standard hardware and software devices and methods commonly used in audio and control systems. The process can involve a combination of digital and analog components, as well as software algorithms, to ensure precise and sensitive adjustment of the speaker orientation. The hardware used to send these signals can include microcontrollers, digital signal processors (DSPs), or field-programmable gate arrays (FPGAs). These devices can be selected based on their processing power, real-time capabilities, and integration potential with other system components. In some implementations, a dedicated system-on-a-chip (SoC) solution may be used, combining processing and communication functions in a single package. The communication protocol used to transmit angle correction control signals can vary depending on the system architecture. Common methods can include I2C, SPI, or UART for short-range onboard communication. For wireless implementations, protocols such as Bluetooth Low Energy (BLE), Wi-Fi, or proprietary RF solutions may be utilized, allowing for flexible positioning of the control unit. Software methods for generating these control signals can involve closed-loop feedback systems, employing algorithms such as PID (proportional-integral-derivative) control or more advanced techniques such as adaptive or predictive control. These algorithms can process inputs from various sensors, including accelerometers, gyroscopes, or optical encoders, to determine the current angular position and calculate necessary corrections. The actual control signal can be generated as a pulse-width modulation (PWM) signal, analog voltage, or digital data stream, depending on the requirements of the actuator (such as a servo motor) responsible for adjusting the speaker's orientation. In more complex systems, error correction and signal verification techniques can be implemented to ensure the reliability and accuracy of the control process.

[0159] Loudspeakers operating within the audible range can refer to any acoustic transducer designed to produce sound waves within the spectrum of human hearing, typically spanning from 20 Hz to 20 kHz. This broad category encompasses a variety of loudspeaker technologies, each utilizing different principles to produce audible sound. Conventional electrodynamic loudspeakers are perhaps the most common, using voice coils and permanent magnets to convert electrical signals into mechanical movement of a diaphragm. However, other types also offer unique characteristics and advantages.

[0160] For example, electrostatic loudspeakers can employ a thin, charged diaphragm suspended between two conductive plates, and are often praised for their clarity and low distortion. Planar magnetic loudspeakers may use a planar diaphragm with embedded conductors placed in a magnetic field, thus combining the advantages of both electrostatic and dynamic designs. Some innovative methods can integrate audio output directly into display technologies, such as using the surface of an LCD or OLED screen as a planar sound wave emitter.

[0161] The third and fourth ultrasonic transducers operate in a frequency range that differs from the frequency ranges of the first and second transducers by at least 20 kHz. This is to avoid cross-channel interactions, such as an ultrasonic wave emitted by one of the left transducers interacting with an ultrasonic wave emitted by either of the right transducers, producing an undesirable audible sound wave.

[0162] This circuit transforms the audio digital signal into two high-frequency signals (by shifting its spectrum), which are then emitted by the ultrasonic transducer and interact with each other in the air. Utilizing its nonlinear properties for this frequency range, it results in an audio signal as a difference frequency subharmonic.

[0163] The disclosed technology has been described for creating individual spatial listening zones using ultrasonic transducers in electronic consumer devices such as televisions, portable speakers, and other consumer audio playback devices. Generally, other implementations may include, but are not limited to, smartphones and tablets, wearable devices, virtual and augmented reality systems, teleconferencing and remote collaboration tools, automotive audio systems, and assistive listening devices.

[0164] Although the disclosed technology has been described with respect to certain preferred embodiments, equivalents and modifications will occur to those skilled in the art upon reading and understanding this specification. This disclosure includes all such equivalents and modifications and is limited only by the scope of the appended claims.

[0165] Similar techniques can be readily applied to other kinds and types of audio systems, such as buildings, or any kind of outdoor or indoor sound systems, which may include, for example, the surface of a solid material for reflecting ultrasonic beams into areas that would otherwise be difficult or impossible to reach to produce sound.

Claims

1. A method for creating a personal spatial listening zone for a user of an electronic consumer device, the method comprising, at the electronic consumer device: - Receive an input audio signal and output a desired audible sound signal to the user based on the input audio signal; - Process the input audio signal to generate a first ultrasonic electro-audio signal and a second ultrasonic electro-audio signal, wherein the processing includes modulating one of the first ultrasonic electro-audio signal and the second ultrasonic electro-audio signal as a carrier signal using a signal obtained from the input audio signal; - The first ultrasonic electro-audio signal is provided to the first ultrasonic transducer, and the second ultrasonic electro-audio signal is provided to the second ultrasonic transducer, wherein the first ultrasonic transducer and the second ultrasonic transducer are included in the electronic consumer device. - The first ultrasonic electro-audio signal and the second ultrasonic electro-audio signal are converted into a first directional ultrasonic acoustic signal and a second directional ultrasonic acoustic signal respectively through the first ultrasonic transducer and the second ultrasonic transducer; - Directing the first directional ultrasonic acoustic signal and the second directional ultrasonic acoustic signal to a spatial listening area, the spatial listening area being a restricted area within the space where the user is located; and - An audible sound signal for the user is generated within the spatial listening area, wherein the audible sound signal substantially corresponds to the desired audible sound signal based on the input audio signal, and wherein the audible sound signal is generated by the nonlinear interaction of the first directional ultrasonic acoustic signal and the second directional ultrasonic acoustic signal in the air within the spatial listening area.

2. The method of claim 1, further comprising: - A first ultrasonic transducer and a second ultrasonic transducer are provided mounted on a rotatable platform of the electronic consumer device, the rotatable platform being configured to adjust the direction of the first directional ultrasonic acoustic signal and the second directional ultrasonic acoustic signal emitted by the first ultrasonic transducer and the second ultrasonic transducer in at least one angular degree of freedom.

3. The method of any one of claims 1 to 2, wherein guiding the first directional ultrasonic acoustic signal and the second directional ultrasonic acoustic signal toward the spatial listening area comprises: - Dynamically adjust the spatial listening area based on the current user's location or user feedback.

4. The method of claim 3, wherein dynamically adjusting the spatial listening area comprises: - Use a camera-based tracking system to detect the position of the user's head; as well as - Adjust the listening position based on the detected location.

5. The method of claim 3, wherein dynamically adjusting the spatial listening area comprises: - Provide at least one directional microphone mounted on the rotatable platform and aligned with the first ultrasonic transducer and the second ultrasonic transducer; - Utilize the rotatable platform to perform a two-dimensional acoustic search mode, changing the angular position on the horizontal and vertical planes; - Monitor microphone signals from the at least one directional microphone during the acoustic search mode, wherein the microphone signals correspond to the voice from the user; - Based on the monitored microphone signals, identify the optimal angular position corresponding to the user's position; and - Adjust the directions of the first directional ultrasonic acoustic signal and the second directional ultrasonic acoustic signal emitted by the ultrasonic transducer to align with the identified optimal angular position.

6. The method of claim 5, wherein the at least one directional microphone comprises at least a first directional microphone and a second directional microphone, the method further comprising: - Monitor a first audio signal from the first directional microphone and a second audio signal from the second directional microphone, wherein the first audio signal and the second audio signal correspond to the voice from the user; - Compare the first audio signal with the second audio signal to determine the difference in signal strength; as well as - Based on the difference, identify the optimal angle position.

7. The method of claim 3, wherein dynamically adjusting the spatial listening area comprises: - Provide an infrared laser coupled to the first ultrasonic transducer and the second ultrasonic transducer on the rotatable platform; - Activate the infrared laser to emit an infrared beam, the infrared beam being aligned with the direction of the first directional ultrasonic acoustic signal and the second directional ultrasonic acoustic signal emitted by the first ultrasonic transducer and the second ultrasonic transducer; - The user's image and the location of the point of the infrared beam are captured by at least two cameras positioned at a certain distance from each other in the electronic consumer device; - Analyze the captured images to determine the actual location of the directional ultrasonic acoustic signal relative to the user's position; - Compare the actual location with the user's location; - Based on the comparison, adjust the spatial listening area.

8. The method of claim 3, wherein dynamically adjusting the spatial listening area comprises: - Receive location information from a portable user acoustic device operated by the user at the user's location and communicatively coupled to the electronic consumer device, the location information indicating the relative position of the portable user device relative to the electronic consumer device; - Adjust the angular positions of the first and second ultrasonic transducers based on the received position information to spatially adjust the spatial listening area.

9. The method of claim 8, wherein the location information includes an audio feedback signal from at least one microphone of the portable user device, and wherein spatially adjusting the spatial listening area includes: - The test signal is transmitted through the first ultrasonic transducer and the second ultrasonic transducer; - Receive the audio feedback signal captured by the at least one microphone in response to the test signal via the portable user device; - Analyze the audio feedback signal to determine the characteristics of the received test signal; - A determined angle correction control signal is used to adjust the direction of the first directional ultrasonic acoustic signal and the second directional ultrasonic acoustic signal emitted through the first ultrasonic transducer and the second ultrasonic transducer, in order to optimize the characteristics at the location of the portable user device; and - Adjust the angular positions of the first ultrasonic transducer and the second ultrasonic transducer based on the angle correction control signal.

10. The method of any one of the preceding claims, further comprising: - Provides at least one speaker operating within the audible range as part of a consumer audio device for directly emitting sound within the audible frequency range based on the input audio signal, and - Based on the input audio signal, selectively switch between a directional ultrasound mode utilizing the first and second ultrasonic transducers and a conventional wide-diffusion mode utilizing the at least one conventional loudspeaker to directly emit sound in the audible frequency range.

11. The method of any one of the preceding claims, wherein directing the first directional ultrasonic acoustic signal and the second directional ultrasonic acoustic signal toward the spatial listening area comprises: - Create indirect beam paths for the first directional ultrasonic acoustic signal and the second directional ultrasonic acoustic signal to form the spatial listening area at the user's location after reflection from at least one reflective surface, thereby creating the individual listening area located in a spatial region that the sound waves cannot directly reach due to the presence of obstacles.

12. The method of any one of the preceding claims, further comprising: - A stereo signal is provided to the user based on four ultrasonic transducers mounted on the rotatable platform, wherein the first and second ultrasonic transducers correspond to the left audio channel, and the third and fourth ultrasonic transducers correspond to the right audio channel, wherein the third and fourth ultrasonic transducers operate in a frequency range that differs from the frequency range of the first and second transducers by at least 20 kHz.

13. The method of any one of the preceding claims, wherein the electronic consumer device includes a portable or stationary entertainment audio device.

14. The method of any one of the preceding claims, wherein the first ultrasonic transducer and the second ultrasonic transducer form a parametric loudspeaker array, wherein the audible sound signal is in a frequency range below 20 kHz, and the first directional ultrasonic acoustic signal and the second directional ultrasonic acoustic signal are in a frequency range above 40 kHz.

15. The method of any one of the preceding claims, wherein the nonlinear interaction of the first directional ultrasonic acoustic signal and the second directional ultrasonic acoustic signal in the air within the spatial listening zone generates a first subharmonic in the audible frequency range, the first subharmonic being obtained as a difference frequency harmonic of two ultrasonic waves emitted by the first ultrasonic transducer and the second ultrasonic transducer, substantially corresponding to the desired audible sound signal, the nonlinear interaction and the generation of the first harmonic occurring substantially only within the spatial listening zone, the spatial listening zone corresponding to the intersection of the first directional ultrasonic acoustic signal and the second directional ultrasonic acoustic signal emitted by the first ultrasonic transducer and the second ultrasonic transducer.

16. An electronic consumer device configured to create a personal listening space for a user, comprising: - First ultrasonic transducer; - Second ultrasonic transducer; - A conversion circuit, operatively coupled to the first ultrasonic transducer and the second ultrasonic transducer, the conversion circuit being configured to: - Receive input audio signals; - Process the input audio signal to generate a first ultrasonic electro-audio signal and a second ultrasonic electro-audio signal, wherein the processing includes modulating one of the first ultrasonic electro-audio signal and the second ultrasonic electro-audio signal as a carrier signal using a signal obtained from the input audio signal; - Provide the first ultrasonic electro-audio signal to the first ultrasonic transducer, and provide the second ultrasonic electro-audio signal to the second ultrasonic transducer; The first ultrasonic transducer and the second ultrasonic transducer are respectively configured to convert the first ultrasonic electro-audio signal and the second ultrasonic electro-audio signal into a first directional ultrasonic acoustic signal and a second directional ultrasonic acoustic signal. as well as - Guide the first directional ultrasonic acoustic signal and the second directional ultrasonic acoustic signal to a spatial listening area, which is a restricted area in the space where the user is located; The first and second directional ultrasonic acoustic signals interact nonlinearly with the air within the spatial listening area, resulting in the demodulation of an audible sound signal for the user, which substantially corresponds to a desired audible sound signal based on the input audio signal, and which is substantially confined to the spatial listening area and substantially inaudible outside the spatial listening area.

Images