Electronic device, method and computer program

Abstract

An electronic device (12) comprising circuitry configured to reduce an imbalance in a spatial sound field by generating a virtual compensation source (27) based on the position of a virtual sound source (26) and a frequency of the imbalance related to the virtual sound source.

Description

[0001] ELECTRONIC DEVICE, METHOD AND COMPUTER PROGRAM

[0002] TECHNICAL FIELD

[0003] The present disclosure generally pertains to methods, devices and systems for the generation of spatial sound fields.

[0004] TECHNICAL BACKGROUND

[0005] Spatial audio and sound field reproduction technologies are at the forefront of modem audio engineering, aiming to create immersive and realistic auditory experiences. These technologies are pivotal in various applications, including home theater systems, virtual reality (VR), augmented reality (AR), and professional audio production.

[0006] Spatial audio refers to the technique of processing and reproducing sound in a way that mimics how we naturally perceive it in a three-dimensional space. As an enhancement to traditional stereo systems that deliver sound from two channels (left and right), more advanced spatial audio systems utilize multiple channels and sophisticated processing algorithms to position sound sources in a three-dimensional environment. This creates the illusion that sound is coming from specific directions and distances, enhancing the listener's sense of presence and immersion.

[0007] One of the most common implementations of spatial audio is surround sound systems. Standard configurations like 5.1 and 7.1 systems consist of five or seven loudspeakers arranged around the listener and one or two subwoofers for low-frequency effects. These systems are designed to envelop the listener in sound, creating a more engaging audio experience.

[0008] To overcome the limitations of traditional surround sound systems, advanced methods like Wavefield Synthesis (WFS) have been developed. WFS is based on the Huygens principle, which states that every point on a wavefront can be considered a source of secondary spherical waves. By using an array of closely spaced loudspeakers, WFS can recreate the wavefronts of a sound field, making it appear as though the sound is originating from a specific point in space. This method can create a more extensive and consistent sound field, allowing listeners to experience the spatial effects from a wider range of positions.

[0009] Spatial audio technologies have found applications in various fields. In home entertainment, they enhance the viewing experience by providing realistic soundscapes for movies and games. In VR and AR, spatial audio is crucial for creating believable virtual environments, aiding in user immersion and interaction. Although there exist techniques for providing spatial audio, there exists a need to improve on the currently available techniques.

[0010] SUMMARY

[0011] According to a first aspect the disclosure provides an electronic device comprising circuitry configured to reduce an imbalance in a spatial sound field by generating a virtual compensation source based on the position of a virtual sound source and a frequency of the imbalance related to the virtual sound source.

[0012] According to a further aspect the disclosure provides a method for reducing an imbalance in a spatial sound field, the method comprising: generating a virtual compensation source based on a position of a virtual sound source; identifying a frequency of the imbalance related to the virtual sound source; and adjusting the virtual compensation source based on the identified frequency to reduce the imbalance in the spatial sound field.

[0013] According to a further aspect the disclosure provides a computer program comprising computer executable instructions that, when executed by a processor, cause the processor to perform a method for reducing an imbalance in a spatial sound field, the method comprising: generating a virtual compensation source based on a position of a virtual sound source; identifying a frequency of the imbalance related to the virtual sound source; and adjusting the virtual compensation source based on the identified frequency to reduce the imbalance in the spatial sound field.

[0014] BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Embodiments are explained by way of example with respect to the accompanying drawings, in which:

[0016] Fig. 1 illustrates a block diagram of an exemplary spatial audio system designed for vehicles;

[0017] Fig. 2 illustrates a spatial audio system that is integrated within a vehicle, the spatial audio system rendering an omnidirectional sound source;

[0018] Fig. 3 shows the spatial audio system of Fig. 2, the spatial audio system rendering a virtual compensation source 27 for mitigating perception imbalances related to the virtual sound source 26;

[0019] Fig. 4 represents the frequency response of a bandpass filter centered at 600 Hz with a Q factor of 8; Fig. 5 shows a flow chart of a process for generating a virtual compensation source; and

[0020] Fig. 6 is a diagram of assistance in explaining an example of installation positions of an outsidevehicle information detecting section and an imaging section.

[0021] DETAILED DESCRIPTION OF EMBODIMENTS

[0022] Before a detailed description of the embodiments under reference of Fig. 1 is given, general explanations are made.

[0023] The embodiments disclose an electronic device comprising circuitry configured to reduce an imbalance in a spatial sound field by generating a virtual compensation source based on the position of a virtual sound source and a frequency of the imbalance related to the virtual sound source.

[0024] An electronic device may be any device that utilizes electronic or digital technology for its operation. The electronic device may be designed to record, manipulate, and reproduce sound. The electronic device may be part of an audio system can include components like microphones, mixers, amplifiers, and speakers, as well as digital signal processors for handling tasks like equalization, compression, and spatial sound processing. Audio systems can be found in a variety of environments, from home entertainment setups and car audio systems to professional recording studios and concert venues. The electronic device could be an audio system specifically designed to manage and enhance the spatial sound field within a given space.

[0025] Circuitry may include a processor, a memory (RAM, ROM or the like), a storage, input means (mouse, keyboard, camera, etc.), output means (display (e.g. liquid crystal, (organic) light emitting diode, etc.), loudspeakers, etc., a (wireless) interface, etc., as it is generally known for electronic devices (computers, smartphones, etc.). Moreover, it may include sensors for sensing still image or video image data (image sensor, camera sensor, video sensor, etc.), for sensing a fingerprint, for sensing environmental parameters (e.g. radar, humidity, light, temperature), etc. Circuity may further comprise any electronic components that allow for the execution of specific functions within an electronic device. This can include passive components like resistors and capacitors, active components like transistors and integrated circuits, and the interconnecting wires or traces that form electrical paths between them.

[0026] A spatial sound field may be any distribution of sound in a three-dimensional space. It may take into account factors like the location of sound sources, the direction of sound propagation, and the position of the listener. In other words, the spatial sound field relates to how sound exists and is experienced within a given space. A virtual sound source refers to a simulated or 'virtual' source of sound in a three-dimensional audio environment. Unlike a physical sound source, such as a speaker or a musical instrument, a virtual sound source does not necessarily exist physically but is created digitally within the audio system. The sound from a virtual source is often produced through audio processing techniques and played through physical speakers, but it is made to appear as though it is coming from a specific location in the virtual or physical space. This can enhance the immersive quality of the audio experience, making it seem as though sounds are coming from different directions, distances, or even moving around in the space.

[0027] An imbalance related to virtual sound source may refer to any kind of unevenness or disproportion in the spatial sound field that's associated with the virtual sound source. This could be due to factors like unequal sound levels at different frequencies, unequal sound distribution in different directions, or phase differences between different sound waves.

[0028] Reducing an imbalance in a spatial sound field may for example comprise compensating or partly compensating for the imbalance in the spatial sound field, or mitigating the imbalance in the spatial sound field to some extent.

[0029] The position of a virtual sound source refers to the location within a spatial sound field from which a virtual sound source is perceived to originate.

[0030] A virtual compensation source may be a simulated or 'virtual' source of sound that is used to counteract or compensate for some form of audio imbalance. The virtual compensation source may for example be digitally produced and rendered by a loud speaker system, in particular a 3D sound Tenderer.

[0031] In another aspect of the embodiments, the virtual compensation source is configured to be active in a narrow frequency band around the frequency of the imbalance. In this way, the virtual compensation source will only produce sound in a specific frequency range that corresponds to the imbalance in the spatial sound field. This allows for targeted correction of the imbalance without affecting other frequencies.

[0032] In a further aspect of the embodiments, the circuitry is configured to apply a bandpass filter to the virtual compensation source to ensure that the virtual compensation source is only active in the narrow frequency band around the frequency of the imbalance. By applying this filter, the virtual compensation source will only produce sound within a specific frequency range that corresponds to the imbalance in the spatial sound field. This helps to focus the corrective effect of the virtual compensation source on the specific frequencies that need adjustment. Another aspect of the embodiments involves the circuitry being configured to determine the position of the compensation source based on the position of the virtual sound source. By analyzing the position of the virtual sound source, the circuitry can calculate and determine the appropriate position for the virtual compensation source. This allows for precise placement of the virtual compensation source to effectively counteract the imbalance in the spatial sound field.

[0033] In yet another aspect of the embodiments, the circuitry is configured to determine the position of the compensation source further based on the frequency of the imbalance. By considering the frequency of the imbalance, the circuitry can optimize the placement of the virtual compensation source to effectively address the specific frequency-related imbalance in the spatial sound field. This ensures that the corrective measures are targeted and tailored to the specific frequency range that needs adjustment.

[0034] An additional aspect of the embodiments includes the circuitry being configured to position the virtual compensation source in a manner that creates destructive interference with the sound field of the virtual sound source, thereby mitigating the perceived imbalance in the spatial sound field. By creating destructive interference, the circuitry may aim to mitigate or reduce the perceived imbalance in the spatial sound field. This can help create a more balanced and immersive audio experience for the listener.

[0035] In a further aspect of the embodiments, the position of the virtual compensation source is determined to be a lambda half distance from the virtual sound source in the direction of the listener. For example, the positioning may be based on the concept of a lambda half distance, which refers to the distance that corresponds to half the wavelength of the sound at the specific frequency. This distance is chosen to create a phase shift of 180 degrees, which helps in achieving spatial cancellation and effectively mitigating the imbalance in the spatial sound field.

[0036] In another aspect of the embodiments, the circuitry is configured to determine the position of the compensation source based on the position of a listener. By analyzing the position of the listener within the spatial sound field, the circuitry can calculate and determine the appropriate position for the virtual compensation source. This may allow for precise placement of the virtual compensation source based on the listener's position, ensuring an optimal audio experience tailored to the listener's location within the space.

[0037] Another aspect of the embodiments involves the circuitry being configured to render the virtual compensation source based on the frequency of the imbalance, and the position of the virtual sound source. The frequency of the imbalance refers to the specific frequency at which the imbalance occurs in the spatial sound field. The circuitry takes this frequency into account when rendering the virtual compensation source, ensuring that it is active and produces sound specifically in the range of frequencies that need adjustment. This targeted approach allows for precise correction of the imbalance without affecting other frequencies.

[0038] In a further aspect of the embodiments, the circuitry is configured to determine a gain for the virtual compensation source, and to render the virtual compensation source based on the determined gain. The gain may refer to the amplification or attenuation of the sound produced by the virtual compensation source. It represents the level or intensity at which the virtual compensation source is rendered. The circuitry may analyze various factors, such as the characteristics of the spatial sound field, the frequency of the imbalance, and the position of the virtual sound source, to determine the appropriate gain for the virtual compensation source.

[0039] Once the gain is determined, the circuitry may render the virtual compensation source with this specific gain value. This means that the audio system will produce the sound from the virtual compensation source at the determined level, which is optimized to effectively counteract the imbalance in the spatial sound field. By adjusting the gain, the circuitry can ensure that the virtual compensation source contributes to balancing the sound field and creating a more immersive and realistic auditory experience.

[0040] In yet another aspect of the embodiments, the gain of the virtual compensation source is determined based on the position of the virtual compensation source. The position of the virtual compensation source refers to its specific location within the spatial sound field. This aspect may involve using this position to determine the appropriate gain for the virtual compensation source.

[0041] An additional aspect of the embodiments includes the circuitry being configured to use a 3D audio Tenderer to generate the virtual compensation source. A 3D audio Tenderer may be a component or software algorithm that is capable of processing audio signals in a way that creates a three-dimensional sound field. It can simulate the perception of sound coming from various directions and distances, enhancing the immersive experience for the listener. The 3D audio Tenderer may take into account factors such as the position of the virtual sound source and other parameters to create a sound output that effectively counteracts the imbalance in the spatial sound field. By using a 3D audio Tenderer, the circuitry may create a virtual compensation source that is precisely positioned and tailored to address the specific imbalances in the sound field. The 3D audio Tenderer may process the audio signals and applies spatial effects, such as positioning, phase manipulation, and amplitude adjustments, to generate the virtual compensation source. The virtual compensation source produced by the 3D audio Tenderer may then rendered through the audio system, ensuring that it effectively compensates for the imbalance and enhances the overall audio experience.

[0042] In a further aspect of the embodiments, the virtual compensation source is a monopole. A monopole, in the context of audio, refers to a sound source that radiates sound equally in all directions. It emits sound waves uniformly in a 360-degree pattern, with no preference for any specific direction. In this aspect, the virtual compensation source is designed as a monopole. This means that the sound produced by the virtual compensation source is evenly distributed in all directions within the spatial sound field. By utilizing a monopole as the virtual compensation source, the circuitry can ensure that the corrective sound waves are uniformly propagated throughout the space. The monopole emits sound waves equally in all directions, allowing for a uniform distribution of the corrective sound within the spatial sound field to address any imbalances.

[0043] According to an aspect of the embodiments the circuitry is configured to optimize the spatial sound field of the interior acoustics of the vehicle. By optimizing the spatial sound field, the circuitry aims to enhance the audio experience for the occupants of the vehicle. This can involve creating a more immersive and realistic sound environment, ensuring a balanced sound distribution throughout the cabin, and minimizing any audio imbalances or sound field distortions.

[0044] In a further aspect of the embodiments, the electronic device is part of a public address system. A public address system is a system used to amplify and distribute sound over a large area or venue, such as a stadium, conference center, or public space. It typically includes components such as microphones, amplifiers, and loudspeakers, which work together to deliver sound to a large audience. In this aspect, the electronic device described in the embodiments is specifically designed to be integrated into a PA system. It functions as a component of the overall system, contributing to the amplification, processing, and distribution of sound. The electronic device may include circuitry and software that are specifically tailored for the requirements of a PA system. This could include features such as advanced audio processing algorithms, spatial sound processing capabilities, and adaptive control mechanisms. By being part of a PA system, the electronic device can help optimize the sound distribution, enhance the clarity and intelligibility of the audio, and ensure that the sound reaches the intended audience effectively. It may also provide functionalities for managing and controlling the PA system, such as adjusting volume levels, configuring speaker placement, and integrating with other audio sources. In another aspect of the embodiments, the circuitry is configured to track the position of the listener and dynamically adjust the position of the virtual compensation source based on changes in the listener's position. The circuitry may for example be configured to monitor and track the movement or changes in the position of the listener within the spatial sound field. This can be done using various sensors or tracking mechanisms, such as cameras, infrared sensors, or motion detectors. Based on the tracked position of the listener, the circuitry can dynamically adjust the position of the virtual compensation source. This means that as the listener moves or changes position, the virtual compensation source is repositioned accordingly to maintain an optimal audio experience. By dynamically adjusting the position of the virtual compensation source based on the listener's movement, the circuitry ensures that the corrective sound is effectively delivered to the listener regardless of their location within the spatial sound field. This helps to maintain a balanced and immersive audio experience as the listener moves around or changes position.

[0045] In yet another aspect of the embodiments, the circuitry is configured to dynamically adjust the position of the virtual compensation source based on the movement of a sound source in a live performance. In a live performance, such as a concert or theater production, sound sources like performers or instruments often move around the stage. This aspect of the embodiments focuses on the ability of the circuitry to track the movement of these sound sources and adjust the position of the virtual compensation source accordingly. By monitoring the movement of the sound source, the circuitry can dynamically update the position of the virtual compensation source to maintain an optimal audio experience. By dynamically adjusting the position of the virtual compensation source based on the movement of the sound source, the circuitry can provide a more balanced and immersive audio experience for the audience during live performances or for an actor.

[0046] An additional aspect of the embodiments includes the circuitry being configured to mitigate potential feedback loops in a public address system. Feedback occurs in a PA system when sound from the loudspeakers is picked up by the microphones, amplified, and then played back through the loudspeakers again, creating a looping effect. This can result in a high-pitched squealing or howling sound, which can be disruptive and unpleasant. In this aspect, the circuitry is specifically designed to prevent feedback loops in the PA system. It utilizes various techniques and algorithms to analyze the audio signals and identify frequencies that could potentially cause feedback. Once potential feedback frequencies are identified, the circuitry takes corrective measures to mitigate the feedback. This can involve dynamically adjusting the gain, phase, or position of the virtual compensation source or other audio processing techniques to prevent the looping effect from occurring. By actively monitoring and mitigating potential feedback, the circuitry helps maintain a clean and clear audio output in the PA system. This ensures that the sound is amplified and distributed without any disruptive feedback noise, providing a better experience for both the audience and the performers.

[0047] Another aspect of the embodiments is a method for compensating for an imbalance in a spatial sound field, which involves generating a virtual compensation source based on a position of a virtual sound source, identifying a frequency of the imbalance related to the virtual sound source, and adjusting the virtual compensation source based on the identified frequency to compensate for the imbalance in the spatial sound field.

[0048] In yet another aspect of the embodiments, there is a computer program comprising computer executable instructions that, when executed by a processor, cause the processor to perform the method of compensating for an imbalance in a spatial sound field.

[0049] In another aspect of the embodiments, a computer-readable medium is provided comprising computer executable instructions that, when executed by a processor, cause the processor to perform the method of compensating for an imbalance in a spatial sound field, thereby improving the audio experience for users.

[0050] Spatial audio in automotive applications

[0051] Spatial audio is increasingly being applied in cars to enhance the in-car audio experience. Car and audio system manufacturers are leveraging advanced spatial audio technologies to create a more immersive and enjoyable listening environment within the vehicle.

[0052] Spatial audio systems in cars aim to provide a surround sound experience that envelops the passengers, making them feel as though the music or audio is coming from all around rather than just from specific speakers. This is achieved through the careful placement of multiple speakers throughout the cabin and the use of advanced audio processing algorithms.

[0053] Car audio systems may for example use multi-speaker arrays (multiple strategically placed speakers) to create a surround sound experience. These systems are designed specifically for the car's interior to ensure optimal sound distribution.

[0054] Fig. 1 illustrates a block diagram of an exemplary spatial audio system designed for vehicles. The system achieves an immersive surround sound experience for passengers within the car cabin. An audio source 11 provides the initial audio signal. This source can include various inputs such as radio, Bluetooth, or a media player. The audio signal from the audio source 11 is transmitted to the audio processing unit 12. Audio processing unit 12 employs advanced audio processing algorithms (e.g. DSP algorithms) to convert the incoming audio signal into surround sound signals. The audio processing unit 12 may also optimize the audio for the car's interior acoustics.

[0055] The audio processing unit 12 further manages the distribution of the surround sound signals to the appropriate speakers within the vehicle, ensuring that each speaker receives the correct signal to maintain the surround sound effect.

[0056] From the audio processing unit 12 the surround sound signals are sent to amplifiers 14. These amplifiers 14 enhance the power of the surround sound signals to a level suitable for driving the speakers.

[0057] The amplified audio signals are then distributed to multiple strategically placed speakers throughout the vehicle cabin, including: a Front Left Speaker SP1-FL, a Front Right Speaker SP- FR, a Center Speaker SP-C, a Rear Left Speaker SP-RL, a Rear Right Speaker SP-RR, and a Subwoofer SP-SW, which handles low-frequency sounds for a richer audio experience.

[0058] The system may also include a microphone 15 to further enhance the audio experience. The microphone 15 can be used for noise cancellation or to dynamically adjust the audio based on real-time cabin acoustics.

[0059] The spatial audio system described above provides a surround sound experience by using a combination of strategically placed speakers and sophisticated audio processing techniques. This configuration ensures that passengers perceive the sound as coming from all directions, creating a more immersive and enjoyable auditory environment.

[0060] While the described spatial audio system illustrates a 5.1 surround sound configuration, this is only one example of how spatial audio can be implemented in vehicles. Other implementations may use different speaker configurations and technologies to achieve a similar immersive audio experience. For instance, a 7.1 surround sound system can include additional rear speakers, providing even more detailed sound localization and depth. Alternatively, some systems might incorporate height speakers to create a 3D audio effect, enhancing the sense of audio immersion by simulating sounds coming from above. Advanced implementations may also use fewer speakers in conjunction with sophisticated audio processing algorithms, such as wave field synthesis or object-based audio rendering, to create a virtual surround sound environment. In the embodiments described below, Digital Signal Processing (DSP) algorithms process audio signals to create virtual sound sources and optimize the listening experience. They can adapt to the acoustic environments and adjust sound output in real-time based on cabin acoustics and listener preferences. For example, the embodiments described below adjust the sound output to compensate for reflections, absorption, and other acoustic factors, ensuring a consistent audio quality regardless of seating position.

[0061] 3D audio rendering for audio optimization

[0062] When reproducing a sound field inside of a closed space (e.g. a car cabin) an audio system will always excite the room. This excitation will result in a superposition of the modal sound field created by the room and the intended sound field intended by the actual audio system. This may lead to unpleasant dips and peaks in the resulting sound field, that may annoy the listener.

[0063] An uneven amplitude frequency response can be made smoother by classical equalizing, but this method cannot alter the spatial distribution of the sound field disturbance by the room (for instance the difference in sound pressure at the different ears of the listener).

[0064] In the embodiments described below, the above problem of uneven sound distribution in a closed space, such as a car cabin, is resolved by using a 3D audio Tenderer to compensate for or reduce audio imbalances. The 3D audio Tenderer, which may include a monopole synthesis or another spatial Tenderer that incorporates the phase of the signal, is designed to create a balanced, immersive auditory experience for the listener.

[0065] Referring to Fig. 2, a spatial audio system is integrated within a vehicle, the spatial audio system rendering an omnidirectional sound source. A chassis 21 of the vehicle provides the main structural framework of the vehicle. A driver's seat 22 is located at the front left side of the vehicle, this seat accommodates the driver 24. A passenger's seat 23 is positioned at the front right side of the vehicle, this seat accommodates the front passenger. A dashboard 24 is provided as control panel located in front of the driver and passenger, containing various instruments and controls. The vehicle is equipped with a spatial audio system such as described in Fig. 1. The spatial audio system includes a multi-speaker array strategically placed throughout the vehicle interior to achieve optimal sound distribution. The speakers are positioned to ensure that audio output creates an immersive surround sound experience. Advanced audio processing algorithms, in particular a 3D audio Tenderer, are employed to manipulate the audio signals, making the passengers perceive the sound as coming from various directions rather than from specific speaker locations. The 3D audio Tenderer (e.g. implemented by the audio processing unit 12 shown in Fig. 1) renders an intended sound source 26. It assumed here that this intended sound source 26 is of the omnidirectional type (a monopole). The 3D audio Tenderer may for example simulate the presence of an omnidirectional sound source by adjusting the output levels and phase of each speaker, creating a sound field that appears to radiate uniformly from a central point. Alternatively, the 3D audio Tenderer may use sound field synthesis techniques to render an omnidirectional sound source.

[0066] An omnidirectional sound source, or a monopole, radiates sound equally in all directions. In a free-field environment, this would mean that the sound pressure levels are the same at any point equidistant from the source. This would result in a balanced perception of sound source 26 by the left and right ear of driver 25 in vehicle 21. In the closed space of the car cabin, however, the above described problem of uneven sound generates audio imbalances.

[0067] It is assumed here that the sound field excites the interior of a car and creates an imbalance at a given frequency f. It is further assumed that this imbalance results in a higher volume perceived at the driver’s left ear than the right ear due to modal excitation of the space. This problem is addressed by instructing the 3D audio Tenderer to place a second virtual sound source (also a monopole).

[0068] Fig. 3 shows the spatial audio system of Fig. 2, the spatial audio system rendering a virtual compensation sound source 27 for mitigating perception imbalances related to the virtual sound source 26. Like the intended virtual sound source 26, the virtual compensation source 27 is of the omnidirectional type. The virtual source 27 is positioned between the desired source 26 and the listener 25, using half of the wavelength lambda at the given frequency.

[0069] The virtual compensation source 27 is configured to be only active in the near frequency band of the problematic frequency f. A narrow band pass filter is used to achieve the response.

[0070] The gain is adjusted so that the level follows the monopole gain law. This results in that the frequency is entirely cancelled out. The distance of lambda-half functions as a phase shift of 180 degrees, creating the spatial cancellation. Since this effect is altering the complete sound field in the given space, it can equalize the pressure difference between the ears, which is not possible using a classical equalizer.

[0071] In the following, an implementation example for a narrow band pass filter is provided.

[0072] A bandpass filter allows frequencies within a certain range (the passband) to pass through while attenuating frequencies outside this range. The center frequency is the midpoint of this range. A bandpass filter may for example be implemented according to the Butterworth design. A Butterworth bandpass filter is a type of filter that allows signals within a certain frequency range to pass through while attenuating frequencies outside this range.

[0073] The Butterworth filter may for example be described by a center frequency and a Q Factor. The center frequency is where the filter has maximum gain. The filter is designed to have the maximum gain (or least attenuation) at this frequency. Essentially, it is the frequency at which the filter is most "transparent," letting signals pass through with minimal resistance.

[0074] The Q factor indicates the selectivity of the filter. A higher Q factor means a narrower passband, providing a more selective frequency range around the center frequency. A higher Q indicates a narrower passband, meaning the filter will only allow a smaller range of frequencies around the center frequency to pass through. Conversely, a lower Q indicates a wider passband, allowing a broader range of frequencies to pass through.

[0075] A Butterworth filter has a flat response in the passband. This means that within the frequency range it is designed to pass, there are no ripples or variations in the gain. Still further, the Butterworth filter has a smooth roll-off. That is, the Butterworth design ensures a smooth transition from the passband to the stopband, meaning that the attenuation increases gradually rather than abruptly as the frequency moves away from the center frequency.

[0076] Fig. 4 represents the frequency response of a bandpass filter centered at 600 Hz with a Q factor of 8. The plot uses a logarithmic frequency scale and labels the frequency axis in kilohertz (kHz). The plot illustrates how the bandpass filter behaves across different frequencies, with a focus on its selective frequency passing capability around 600 Hz and its attenuation outside this range.

[0077] The horizontal axis represents frequency in kilohertz (kHz). The scale is logarithmic, ranging from 0.01 kHz to 20 kHz, which covers the audible frequency spectrum. The dashed line 41 marks the center frequency of 600 Hz. It visually highlights the frequency at which the filter is designed to have maximum gain.

[0078] The vertical axis represents gain in decibels (dB), ranging from -15 dB to 5 dB. This limited range focuses on the filter's behavior near the passband, omitting the deeper attenuation levels. The gain is shown in decibels (dB), a logarithmic measure of the amplitude ratio. Positive dB values indicate amplification, while negative values indicate attenuation.

[0079] The solid curve 42 represents the gain, i.e. the magnitude of the filter's frequency response in decibels. It shows how the filter attenuates or amplifies signals at different frequencies. The gain is highest at the center frequency of 600 Hz, where the filter is designed to pass frequencies with minimal attenuation. As the frequency moves away from 600 Hz, the gain decreases, showing how the filter attenuates frequencies outside the passband.

[0080] It is noted here that the Butterworth design is only one possible method of realizing a bandpass filter. While it is well-known for its maximally flat frequency response in the passband and smooth roll-off characteristics, there are several alternative designs that can be used as an alternative. For instance, the Chebyshev filter offers a steeper roll-off than the Butterworth filter but introduces ripples in the passband. The Elliptic filter, also known as the Cauer filter, provides the steepest roll-off for a given order but with ripples in both the passband and the stopband. Another option is the Bessel filter, which is designed to have a maximally flat phase response, making it ideal for applications requiring minimal signal distortion in the time domain.

[0081] Fig. 5 shows a flow chart of a process for generating a virtual compensation source. The process steps 51-56 describe the steps involved in determining the position and gain of the virtual compensation source and rendering it to address the imbalance related to the virtual sound source.

[0082] At 51, a position Poof a virtual sound source is obtained. This position represents the location from which the intended sound is supposed to originate. At 52, a position PL of the listener is determined. This is the l ocation of the li stener (in parti cular the head of the li stener) within the vehicle cabin. At 53, a frequency f of an imbalance related to the virtual sound source is determined. This frequency represents the specific frequency at which the imbalance occurs. At 54, it is determined a position Pcof a virtual compensation source based on the Poposition of virtual sound source, based on the position PL of the listener and based on the frequency f. By positioning the virtual compensation source between the virtual sound source and the listener, it helps mitigate the perception imbalances caused by the virtual sound source. At 55, a gain g of the virtual compensation source is determined. By adjusting the gain, it is possible to cancel out the imbalance at the specific frequency / At 56, the virtual compensation source is rendered based on the determined frequency / the determined position Pcand the determined gain g. This means that the audio system will produce the sound from the virtual compensation source in such a way that it compensates for or reduces the imbalance caused by the virtual sound source.

[0083] The position of the virtual sound source (see 51 in Fig. 5 above) can be determined in different ways, depending on the specific setup of the audio system. For example, the virtual sound source may be associated with the position of a virtual speaker within the audio system. Virtual speakers are simulated audio sources that can be placed in different locations within the virtual environment. The position of the virtual sound source can be obtained by determining the position of the virtual speaker within this virtual environment. For example, in channel-based audio formats, metadata can be used to assign specific channels to different sound objects within the audio scene. By analyzing the channel assignment metadata, the position of the sound object can be inferred. For example, if a sound object is assigned to channels corresponding to the front left speaker, its position can be determined as being located in the front left direction.

[0084] Alternatively, the virtual sound source may be associated with the position of a virtual sound object. A virtual sound object represents a specific audio element or entity within the audio scene. The position of the virtual sound source can be obtained by determining the position of the virtual sound object within the audio system. The position of a sound object can for example be obtained from metadata associated with the sound object. Metadata refers to additional information about the sound object that is stored alongside the audio data. Some audio formats, such as Ambisonics or Object-Based Audio formats, include spatial metadata that explicitly define the position of the sound object. This metadata may include coordinates (e.g., Cartesian or spherical coordinates) that specify the location of the sound object within the virtual or physical space. By extracting this spatial metadata, the position of the sound object can be obtained. Sound object metadata can also include positional tags or labels that provide information about the position of the sound object. These tags can be descriptive labels (e.g., "front center," "rear right") or more specific positional information (e.g., "x = 3, y = 2, z = -1"). By extracting and interpreting these positional tags, the position of the sound object can be derived. The availability and format of metadata related to sound objects can vary depending on the audio system and the specific audio format being used. Therefore, the method for obtaining the position of a sound object from its metadata may vary accordingly.

[0085] In both cases, the position of the virtual sound source is typically represented using coordinates in a virtual space or a physical space, depending on the audio system setup. This position information can be used for accurately rendering the sound from the original source and ensuring its intended spatial distribution within the closed space, such as a car cabin.

[0086] There are also many possibilities for the skilled person to determine the location of the listener (see 52 in Fig. 5 above), specifically the position of the head. For example, when the listener is seated in the front seat of a car, automotive sensors may allow to determine this position. Modem car seats often include position sensors that can detect the position and adjustments made by the driver or front passenger. These sensors may measure seat height, recline angle, and fore / aft position. By using this information, the approximate position of the head can be estimated. In addition or alternatively, seatbelt sensors as commonly found in cars can provide information about whether the occupant is buckled up or not. These sensors can be used to infer the presence and approximate position of the head, as the head is typically in close proximity to the seatbelt. Cars may also be equipped with occupancy sensors that detect the presence and position of occupants in the seats. These sensors can use technologies such as weight sensors or pressure sensors to estimate the position of the head. Infrared sensors or cameras can be used to track the position of the head using infrared light. By emitting and detecting infrared signals, these sensors can create a 3D map of the cabin and accurately determine the position of the head. Depth cameras or time-of-flight cameras utilize advanced imaging techniques to calculate the depth and distance of objects within their field of view. These cameras can also be used to detect the position of the head within the car cabin. Some cars may employ gesture recognition systems that use cameras or sensors to detect and interpret the movements and positions of the driver's hands and head. By analyzing these gestures, the position of the head can be determined. The specific implementation can vary, but the goal is to estimate the position of the head for optimal audio rendering and sound distribution within the car cabin according to the processes described above.

[0087] The frequency (see 53 in Fig. 5 above) of an imbalance related to the virtual sound source can also be determined in several ways. The imbalance described above can be for example be tested with exciting the space with a sinoide test tone, which sweeps through the audible frequency spectrum. The sinusoidal test tone is emitted from the position of the virtual sound source. It is played through the audio system, covering a predetermined range of frequencies. At one or more certain frequencies, the listener may perceive a different sound pressure at each ear. Such frequencies can be observed by using a dummy head with two microphones (each located at the respective ear of the listener) or by a human experiencing the test signal. The dummy head is designed to mimic the human head and ears, allowing for accurate measurement of the sound pressure at each ear. By capturing the audio signals from the microphones, it is possible to analyze the recorded data and identify the specific frequency at which the imbalance occurs. Alternatively, the imbalance can be observed through subjective perception by a human listener experiencing the test signal. The listener can provide feedback or indicate when they perceive a difference in sound pressure between their ears at a particular frequency. By conducting such tests and analyzing the recorded data or subjective feedback, it becomes possible to determine the frequency at which the imbalance related to the virtual sound source occurs. This information can be used for subsequent steps, such as positioning the virtual compensation source and determining its gain, to address and compensate for the imbalance.

[0088] By moving the virtual sound source in space and recording the measurements using the dummy head or subjective feedback from human listeners, it allows for a more comprehensive understanding of how the imbalance varies with different positions. This information can then be used to optimize the positioning and compensation techniques for the virtual sound source in subsequent steps. By calibrating the system for various positions, it ensures that the virtual compensation source is accurately positioned and adjusted to address imbalances at different locations within the spatial sound field. This can help create a more balanced and immersive audio experience for listeners, regardless of the position of the virtual sound source. The system may include software or hardware that directly interfaces with the measurement devices and it may use built-in data logging capabilities. This may allow the system to automatically record and store the data during the calibration phase. The data can be saved in a specified format or database for later retrieval and analysis by the audio rendering system.

[0089] The position of the virtual compensation source can be determined based on the position of virtual sound source, based on the position of the listener and based on the frequency of the imbalance. In this way, the audio system calculates a suitable position for the virtual compensation source, which should be strategically placed within the listening environment. The goal is to position the virtual compensation source in such a way that it can effectively counteract the imbalance caused by the virtual sound source, providing a more balanced and immersive auditory experience for the listener. For example, a lambda half placement of an optimization monopole may be applied such as described with regard to Fig. 3 above. This means that the virtual compensation source is positioned at a distance of lambda half, which corresponds to half of the wavelength at the frequency of the imbalance. This placement introduces a phase shift of 180 degrees, creating spatial cancellation and effectively counteracting the imbalance. Once the frequency of the imbalance is known, the wavelength of the sound at that frequency can be calculated. In air, the speed of sound is approximately 343 meters per second. The wavelength (lambda) can be calculated using the formula: lambda = speed of sound / frequency. The lambda-half distance is calculated by dividing the wavelength by two. This distance corresponds to a phase shift of 180 degrees, which is used to create spatial cancellation and counteract the sound imbalance. The virtual compensation source can then be positioned at the lambda-half distance from the virtual sound source, in the direction of the listener. This strategic placement ensures that the virtual compensation source effectively counteracts the imbalance caused by the virtual sound source , resulting in a more balanced sound field.

[0090] It should however be noted that the lambda-half placement for the virtual compensation source is only an example to create spatial cancellation and counteract the sound imbalance. There are other distances that can also result in cancellation (e.g. a 180-degree phase shift). A phase shift of 180 degrees can also occur at other distances depending on the characteristics of the sound wave. While the lambda-half placement is one effective way to introduce a phase shift of 180 degrees and create spatial cancellation, there can certainly be other distances that can achieve the same effect depending on the specific circumstances.

[0091] The virtual compensation source is rendered (56 in Fig. 5) based on the frequency of the imbalance, the position of the virtual compensation source and based on a gain factor. The virtual compensation source does replicate the sound signal of the virtual sound source . A narrow bandpass filter is then applied to this signal to ensure that the virtual compensation source is only active in a narrow frequency band around the problematic frequency (see Fig. 4 and corresponding description). This means that the 3D rendering engine of the audio system will produce the sound from the virtual compensation source in such a way that it mitigates the imbalance caused by the virtual sound source be suppressing the problematic frequencies.

[0092] The appropriate gain factor for the virtual compensation source (55 in Fig. 5) can be determined according to the principles well-known to the skilled person. For example, the gain of the virtual compensation source may be chosen based on the loudness of the virtual sound source compensating for any loudness loss effected by the lambda-half placement. The gain can be adjusted such that when the original and the corrective sound signals combine, they effectively cancel each other out at the problematic frequency, leading to a cancellation of the sound imbalance. This corresponds to a form of active noise control or destructive interference, where two sound waves of the same frequency and amplitude, but out of phase by 180 degrees (provided by the lambda-half placement), combine to effectively cancel each other out. So the gain of the virtual compensation source is set in a way that it matches the amplitude of the problematic frequency of the virtual sound source at the location of the listener.

[0093] According to an example, the loss in loudness due to the lambda-half displacement between the virtual compensation source and the virtual sound source is determined based on the principles of sound propagation and the inverse square law (monopole gain law). In an ideal free-field environment (an open space with no reflections), sound pressure level decreases as you move further from the sound source. This is due to the sound energy spreading out over an increasingly larger area as it propagates away from the source. This principle is encapsulated in the inverse square law, which states that the intensity (or power per unit area) of a sound wave decreases with the square of the distance from the source. So, if the system places the virtual compensation source at a lambda-half distance from the virtual sound source in the direction of the listener, there will be some reduction in loudness appropriate due to this decreased distance. To calculate this loss, the system can use the inverse square law equation: L2 = LI + 20 Iogl0(dl / d2), where: LI is the sound level at the virtual sound source , L2 is the sound level at the virtual compensation source, dl is the distance from the virtual sound source to the listener and d2 is the distance from the virtual compensation source (which is at a lambda-half distance) to the listener. Based on this principle the sound level at the virtual compensation source (L2) that's needed to compensate for the lambda-half displacement can be determined. This results in the gain needed for the virtual compensation source to effectively cancel out the imbalance at the problematic frequency. This is a possible way to determine the gain. According to other examples, the calculation could take into account various other factors, including the specific acoustics of the car cabin and any sound reflections or absorptions that might occur.

[0094] Improving driving experience through modal structure alteration

[0095] The system described in the embodiments can also be used to alter the modal structure of an external excitation of the system. For example, inside of a driving car the modal structure of the sound field can be altered, and the driving experience being improved.

[0096] The modal structure of a sound field is essentially the pattern of how sound waves interact with the environment, particularly in enclosed spaces like a car cabin. These interactions can result in certain frequencies being amplified (resonant frequencies or modes) or diminished (anti-resonant frequencies or nulls), based on factors like the dimensions of the space, the materials used, and the positions of sound sources and listeners.

[0097] External excitations, such as road noise, engine noise, or wind noise while driving can excite these modes in a car cabin, leading to an uneven and potentially unpleasant sound environment inside the car.

[0098] The system described above can be used to alter this modal structure and counteract these undesirable effects. By using a 3D audio Tenderer and virtual compensation sources, as described in the previous embodiments, the system can effectively manipulate the sound field inside the car. It can position virtual sound sources and control their output in such a way to interfere destructively with the undesirable frequencies, thus cancelling them out. In other words, the system can be used to introduce additional sounds to the sound field that effectively 'fill in' the nulls and 'smooth out' the resonant peaks in the modal structure. This results in a more even, balanced sound environment inside the car, improving the driving experience. The sound environment inside the car becomes less dependent on the external excitations and more influenced by the controlled output of the audio system.

[0099] This can be used to create a more comfortable driving experience, or to enhance the perception of the car's audio system, making music or other audio content sound better inside the car.

[0100] Enhancing Public Address (PA) systems through room mode manipulation

[0101] Although in the embodiments described above the audio system is described in the context of automotive applications, the applications are not limited to this use case. Another application would for example be a public address (PA) audio amplification system. Here also the excited room modes lead to sound colorations and possible feedback problems from the microphones used. Due to the acoustic of the venue, modal problems may occur, and some frequencies can cause a feedback loop (leading to an excessive level at those frequencies and matching path lengths). A state-of-the-art system will use an equalizer to tame those excessive frequencies.

[0102] A public address (PA) system is used to amplify and distribute sound across a large space, such as a concert hall, stadium, or conference center. These systems typically consist of microphones, amplifiers, and loudspeakers. When sound is amplified in such large spaces, it can excite the room modes, or resonant frequencies, of the space. These resonances can cause certain frequencies to be amplified more than others, leading to sound colorations or distortions. For example, some notes might sound louder and more resonant, while others might be harder to hear. This uneven amplification can negatively impact the clarity and quality of the sound.

[0103] Additionally, the excited room modes can also lead to feedback problems. Feedback occurs when sound from the loudspeakers is picked up by the microphones, amplified, and played back through the loudspeakers, creating a looping effect. This can result in a high-pitched squealing or howling sound, which can be very unpleasant and disruptive.

[0104] By using a system as described in the embodiments above, these issues can be addressed. Virtual compensation sources can be used to manipulate the sound field in the space, reducing resonances and balancing the sound distribution. This leads to a more even, clear sound, reducing colorations and potential feedback issues. In this way, feed-back-loops can be tamed very effectively, because the feed-back is removed in a spatial manner. The described system can dynamically adapt to changes in the acoustical environment and even to moving sound sources. For example, the above-described system and processes can be utilized to dynamically follow the changed acoustical path length (which leads to the feed-back) and thus dynamically avoid problematic conditions. For example, the location of the virtual compensation source could be linked to a moving object in space (e.g., the lead vocalist on the stage). Thus the feedback of the complete PA system could be avoided in dynamically changing environments.

[0105] In a live sound situation, such as a concert, the acoustical path length can change dynamically. This is the distance that sound travels from a source (e.g., a vocalist or instrument on stage) to a microphone. As performers move around the stage, these path lengths change, which can impact the sound picked up by the microphones and potentially lead to feedback. However, by dynamically adjusting the location of the virtual compensation source (the virtual compensation source) to follow these changes, the system can continue to effectively counteract any potential feedback. For instance, if the lead vocalist moves around the stage, the virtual compensation source could be adjusted to 'follow' them, maintaining the correct phase relationship to prevent feedback.

[0106] This dynamic adjustment of the virtual compensation source allows the system to effectively 'track' moving sound sources and continue to provide effective feedback prevention. This would be especially valuable in live sound situations, preventing feedback issues and ensuring a clearer, better sound for both the performers and the audience.

[0107] In this way, the principles of 3D audio rendering and active noise control can be applied to improve the performance of public address systems, enhancing the listening experience for audiences in large spaces. The system can be used for avoiding feed-back-loops for public address sound reinforcement system. It could be utilized in a static or dynamic fashion (e.g., following the motion of a lead singer in a live performance).

[0108] Rendering system

[0109] The described principles can be applied to create an immersive and balanced audio experience within a closed room such as a vehicle. A 3D audio Tenderer and multiple speakers are used to create a virtual sound source, which is then fine-tuned by a corrective virtual sound source to mitigate any imbalances. The system can adjust dynamically to changes in the listener's position or external noise, ensuring a consistent and high-quality audio experience regardless of the driving conditions. In large spaces like concert halls or stadiums, the principles of 3D audio rendering and active noise control can be used to enhance the performance of public address systems. By manipulating the sound field and introducing corrective virtual sound sources, the system can reduce resonances, balance sound distribution, and minimize feedback issues. This results in a clearer, more balanced sound for the audience.

[0110] In live performance situations, the system can dynamically adjust the location of the virtual compensation source to 'track' moving performers, maintaining the correct phase relationship to prevent feedback. This ensures a clearer, better sound for both the performers and the audience.

[0111] In each case, the system utilizes a combination of 3D audio rendering, active noise control, and dynamic adaptation to create a balanced and immersive audio experience. The specific implementation may vary based on factors like the number of speakers available and the specific acoustic properties of the environment.

[0112] As the frequency of the sound increases, the complexity and precision required for the audio system to effectively control and correct the sound also increase. This is because higher frequencies have shorter wavelengths and are more directional, making their manipulation more challenging.

[0113] However, with the right rendering system capabilities and an appropriate number of speakers, the system can effectively handle these higher frequencies. More speakers allow for finer control over these frequencies, enhancing the effectiveness of the system.

[0114] Implementation of a 3D audio Tenderer in an automotive environment

[0115] The technology according to an embodiment of the present disclosure is applicable to various products. For example, the technology according to an embodiment of the present disclosure may be implemented as a device included in a mobile body that is any of kinds of automobiles, electric vehicles, hybrid electric vehicles, motorcycles, bicycles, personal mobility vehicles, airplanes, drones, ships, robots, construction machinery, agricultural machinery (tractors), and the like. In the following it is described an implementation of a 3D audio Tenderer in an automotive environment.

[0116] FIG. 6 is a block diagram depicting an example of schematic configuration of a vehicle control system 7000 as an example of a mobile body control system to which the technology according to an embodiment of the present disclosure can be applied.

[0117] The vehicle control system 7000 shown in Fig. 6 is a comprehensive system that encompasses several electronic control units connected via a communication network 7010. This system manages various aspects of the vehicle's operation, including driving, body system, battery control, outside-vehicle information detection, in-vehicle information detection, and integrated control.

[0118] A microcomputer 7610, within an integrated control unit 7600 of the vehicle control system 7000, is responsible for executing arithmetic processing according to various programs. In the context of the audio correction technology described in the embodiments above, the microcomputer 7610 is responsible for executing DSP related algorithms such as algorithms to perform 3D audio rendering, to determine the imbalance in audio distribution, to calculate the position of the corrective virtual sound source, and to adjust the gain of this source.

[0119] The sound / image output section 7670 is part of the functional configuration of the integrated control unit 7600. It transmits output signals of sound or images to the appropriate devices for audibly or visually presenting information to an occupant of the vehicle or the outside of the vehicle. In the context of the audio correction technology, the sound / image output section 7670 handles the output of audio through the speakers, manipulated as per the calculations from the microcomputer 7610.

[0120] The audio speaker 7710 is one of the output devices connected to the sound / image output section 7670. It is responsible for converting the audio signal into sound that the vehicle occupants can hear. In the context of the audio correction technology, the audio speaker 7710 (or speakers, if the system utilizes multiple speakers for spatial audio rendering) outputs the sound that has been processed by the 3D audio Tenderer and the corrective virtual sound source to create a balanced, immersive audio experience.

[0121] In summary, in the context of the described audio correction technology, the vehicle control system 7000 provides an integrated system for analyzing, processing, and outputting audio in a way that compensates for or reduces imbalances and enhances the overall auditory experience within the vehicle.

[0122] The vehicle control system 7000 includes a plurality of electronic control units connected to each other via a communication network 7010. In the example depicted in FIG. 6, the vehicle control system 7000 includes a driving system control unit 7100, a body system control unit 7200, a battery control unit 7300, an outside-vehicle information detecting unit 7400, an in-vehicle information detecting unit 7500, and an integrated control unit 7600. The communication network 7010 connecting the plurality of control units to each other may, for example, be a vehicle-mounted communication network compliant with an arbitrary standard such as controller area network (CAN), local interconnect network (LEST), local area network (LAN), FlexRay (registered trademark), or the like.

[0123] Each of the control units includes: a microcomputer that performs arithmetic processing according to various kinds of programs; a storage section that stores the programs executed by the microcomputer, parameters used for various kinds of operations, or the like; and a driving circuit that drives various kinds of control target devices. Each of the control units further includes: a network interface (I / F) for performing communication with other control units via the communication network 7010; and a communication I / F for performing communication with a device, a sensor, or the like within and without the vehicle by wire communication or radio communication. A functional configuration of the integrated control unit 7600 illustrated in FIG. 6 includes a microcomputer 7610, a general -purpose communication I / F 7620, a dedicated communication I / F 7630, a positioning section 7640, a beacon receiving section 7650, an in- vehicle device I / F 7660, a sound / image output section 7670, a vehicle-mounted network I / F 7680, and a storage section 7690. The other control units similarly include a microcomputer, a communication I / F, a storage section, and the like.

[0124] The driving system control unit 7100 controls the operation of devices related to the driving system of the vehicle in accordance with various kinds of programs. For example, the driving system control unit 7100 functions as a control device for a driving force generating device for generating the driving force of the vehicle, such as an internal combustion engine, a driving motor, or the like, a driving force transmitting mechanism for transmitting the driving force to wheels, a steering mechanism for adjusting the steering angle of the vehicle, a braking device for generating the braking force of the vehicle, and the like. The driving system control unit 7100 may have a function as a control device of an antilock brake system (ABS), electronic stability control (ESC), or the like.

[0125] The driving system control unit 7100 is connected with a vehicle state detecting section 7110. The vehicle state detecting section 7110, for example, includes at least one of a gyro sensor that detects the angular velocity of axial rotational movement of a vehicle body, an acceleration sensor that detects the acceleration of the vehicle, and sensors for detecting an amount of operation of an accelerator pedal, an amount of operation of a brake pedal, the steering angle of a steering wheel, an engine speed or the rotational speed of wheels, and the like. The driving system control unit 7100 performs arithmetic processing using a signal input from the vehicle state detecting section 7110, and controls the internal combustion engine, the driving motor, an electric power steering device, the brake device, and the like. The body system control unit 7200 controls the operation of various kinds of devices provided to the vehicle body in accordance with various kinds of programs. For example, the body system control unit 7200 functions as a control device for a keyless entry system, a smart key system, a power window device, or various kinds of lamps such as a headlamp, a backup lamp, a brake lamp, a turn signal, a fog lamp, or the like. In this case, radio waves transmitted from a mobile device as an alternative to a key or signals of various kinds of switches can be input to the body system control unit 7200. The body system control unit 7200 receives these input radio waves or signals, and controls a door lock device, the power window device, the lamps, or the like of the vehicle.

[0126] The battery control unit 7300 controls a secondary battery 7310, which is a power supply source for the driving motor, in accordance with various kinds of programs. For example, the battery control unit 7300 is supplied with information about a battery temperature, a battery output voltage, an amount of charge remaining in the battery, or the like from a battery device including the secondary battery 7310. The battery control unit 7300 performs arithmetic processing using these signals, and performs control for regulating the temperature of the secondary battery 7310 or controls a cooling device provided to the battery device or the like.

[0127] The outside-vehicle information detecting unit 7400 detects information about the outside of the vehicle including the vehicle control system 7000. For example, the outside-vehicle information detecting unit 7400 is connected with at least one of an imaging section 7410 and an outsidevehicle information detecting section 7420. The imaging section 7410 includes at least one of a time-of-flight (ToF) camera, a stereo camera, a monocular camera, an infrared camera, and other cameras. The outside-vehicle information detecting section 7420, for example, includes at least one of an environmental sensor for detecting current atmospheric conditions or weather conditions and a peripheral information detecting sensor for detecting another vehicle, an obstacle, a pedestrian, or the like on the periphery of the vehicle including the vehicle control system 7000.

[0128] The environmental sensor, for example, may be at least one of a rain drop sensor detecting rain, a fog sensor detecting a fog, a sunshine sensor detecting a degree of sunshine, and a snow sensor detecting a snowfall. The peripheral information detecting sensor may be at least one of an ultrasonic sensor, a radar device, and a LIDAR device (Light detection and Ranging device, or Laser imaging detection and ranging device). Each of the imaging section 7410 and the outsidevehicle information detecting section 7420 may be provided as an independent sensor or device, or may be provided as a device in which a plurality of sensors or devices are integrated. The outside-vehicle information detecting unit 7400 makes the imaging section 7410 image an image of the outside of the vehicle, and receives imaged image data. In addition, the outsidevehicle information detecting unit 7400 receives detection information from the outside-vehicle information detecting section 7420 connected to the outside-vehicle information detecting unit 7400. In a case where the outside-vehicle information detecting section 7420 is an ultrasonic sensor, a radar device, or a LIDAR device, the outside-vehicle information detecting unit 7400 transmits an ultrasonic wave, an electromagnetic wave, or the like, and receives information of a received reflected wave. On the basis of the received information, the outside-vehicle information detecting unit 7400 may perform processing of detecting an object such as a human, a vehicle, an obstacle, a sign, a character on a road surface, or the like, or processing of detecting a distance thereto. The outside-vehicle information detecting unit 7400 may perform environment recognition processing of recognizing a rainfall, a fog, road surface conditions, or the like on the basis of the received information. The outside-vehicle information detecting unit 7400 may calculate a distance to an object outside the vehicle on the basis of the received information.

[0129] In addition, on the basis of the received image data, the outside-vehicle information detecting unit 7400 may perform image recognition processing of recognizing a human, a vehicle, an obstacle, a sign, a character on a road surface, or the like, or processing of detecting a distance thereto. The outside-vehicle information detecting unit 7400 may subject the received image data to processing such as distortion correction, alignment, or the like, and combine the image data imaged by a plurality of different imaging sections 7410 to generate a bird’s-eye image or a panoramic image. The outside-vehicle information detecting unit 7400 may perform viewpoint conversion processing using the image data imaged by the imaging section 7410 including the different imaging parts.

[0130] The in-vehicle information detecting unit 7500 detects information about the inside of the vehicle. The in-vehicle information detecting unit 7500 is, for example, connected with a driver state detecting section 7510 that detects the state of a driver. The driver state detecting section 7510 may include a camera that images the driver, a biosensor that detects biological information of the driver, a microphone that collects sound within the interior of the vehicle, or the like. The biosensor is, for example, disposed in a seat surface, the steering wheel, or the like, and detects biological information of an occupant sitting in a seat or the driver holding the steering wheel. On the basis of detection information input from the driver state detecting section 7510, the in- vehicle information detecting unit 7500 may calculate a degree of fatigue of the driver or a degree of concentration of the driver, or may determine whether the driver is dozing. The in- vehicle information detecting unit 7500 may subject an audio signal obtained by the collection of the sound to processing such as noise canceling processing or the like.

[0131] The integrated control unit 7600 controls general operation within the vehicle control system 7000 in accordance with various kinds of programs. The integrated control unit 7600 is connected with an input section 7800. The input section 7800 is implemented by a device capable of input operation by an occupant, such, for example, as a touch panel, a button, a microphone, a switch, a lever, or the like. The integrated control unit 7600 may be supplied with data obtained by voice recognition of voice input through the microphone. The input section 7800 may, for example, be a remote control device using infrared rays or other radio waves, or an external connecting device such as a mobile telephone, a personal digital assistant (PDA), or the like that supports operation of the vehicle control system 7000. The input section 7800 may be, for example, a camera. In that case, an occupant can input information by gesture. Alternatively, data may be input which is obtained by detecting the movement of a wearable device that an occupant wears. Further, the input section 7800 may, for example, include an input control circuit or the like that generates an input signal on the basis of information input by an occupant or the like using the above-described input section 7800, and which outputs the generated input signal to the integrated control unit 7600. An occupant or the like inputs various kinds of data or gives an instruction for processing operation to the vehicle control system 7000 by operating the input section 7800.

[0132] The storage section 7690 may include a read only memory (ROM) that stores various kinds of programs executed by the microcomputer and a random access memory (RAM) that stores various kinds of parameters, operation results, sensor values, or the like. In addition, the storage section 7690 may be implemented by a magnetic storage device such as a hard disc drive (HDD) or the like, a semiconductor storage device, an optical storage device, a magneto-optical storage device, or the like.

[0133] The general-purpose communication I / F 7620 is a communication I / F used widely, which communication I / F mediates communication with various apparatuses present in an external environment 7750. The general-purpose communication I / F 7620 may implement a cellular communication protocol such as global system for mobile communications (GSM (registered trademark)), worldwide interoperability for microwave access (WiMAX (registered trademark)), long term evolution (LTE (registered trademark)), LTE-advanced (LTE-A), or the like, or another wireless communication protocol such as wireless LAN (referred to also as wireless fidelity (Wi-Fi (registered trademark)), Bluetooth (registered trademark), or the like. The general-purpose communication I / F 7620 may, for example, connect to an apparatus (for example, an application server or a control server) present on an external network (for example, the Internet, a cloud network, or a company-specific network) via a base station or an access point. In addition, the general-purpose communication I / F 7620 may connect to a terminal present in the vicinity of the vehicle (which terminal is, for example, a terminal of the driver, a pedestrian, or a store, or a machine type communication (MTC) terminal) using a peer to peer (P2P) technology, for example.

[0134] The dedicated communication I / F 7630 is a communication I / F that supports a communication protocol developed for use in vehicles. The dedicated communication I / F 7630 may implement a standard protocol such, for example, as wireless access in vehicle environment (WAVE), which is a combination of institute of electrical and electronic engineers (IEEE) 802.1 Ip as a lower layer and IEEE 1609 as a higher layer, dedicated short range communications (DSRC), or a cellular communication protocol. The dedicated communication I / F 7630 typically carries out V2X communication as a concept including one or more of communication between a vehicle and a vehicle (Vehicle to Vehicle), communication between a road and a vehicle (Vehicle to Infrastructure), communication between a vehicle and a home (Vehicle to Home), and communication between a pedestrian and a vehicle (Vehicle to Pedestrian).

[0135] The positioning section 7640, for example, performs positioning by receiving a global navigation satellite system (GNSS) signal from a GNSS satellite (for example, a GPS signal from a global positioning system (GPS) satellite), and generates positional information including the latitude, longitude, and altitude of the vehicle. Incidentally, the positioning section 7640 may identify a current position by exchanging signals with a wireless access point, or may obtain the positional information from a terminal such as a mobile telephone, a personal handyphone system (PHS), or a smart phone that has a positioning function.

[0136] The beacon receiving section 7650, for example, receives a radio wave or an electromagnetic wave transmitted from a radio station installed on a road or the like, and thereby obtains information about the current position, congestion, a closed road, a necessary time, or the like. Incidentally, the function of the beacon receiving section 7650 may be included in the dedicated communication I / F 7630 described above.

[0137] The in-vehicle device I / F 7660 is a communication interface that mediates connection between the microcomputer 7610 and various in-vehicle devices 7760 present within the vehicle. The in- vehicle device I / F 7660 may establish wireless connection using a wireless communication protocol such as wireless LAN, Bluetooth (registered trademark), near field communication (NFC), or wireless universal serial bus (WUSB). In addition, the in-vehicle device I / F 7660 may establish wired connection by universal serial bus (USB), high-definition multimedia interface (HDMI (registered trademark)), mobile high-definition link (MHL), or the like via a connection terminal (and a cable if necessary) not depicted in the figures. The in-vehicle devices 7760 may, for example, include at least one of a mobile device and a wearable device possessed by an occupant and an information device carried into or attached to the vehicle. The in-vehicle devices 7760 may also include a navigation device that searches for a path to an arbitrary destination. The in-vehicle device I / F 7660 exchanges control signals or data signals with these in-vehicle devices 7760.

[0138] The vehicle-mounted network I / F 7680 is an interface that mediates communication between the microcomputer 7610 and the communication network 7010. The vehicle-mounted network I / F 7680 transmits and receives signals or the like in conformity with a predetermined protocol supported by the communication network 7010.

[0139] The microcomputer 7610 of the integrated control unit 7600 controls the vehicle control system 7000 in accordance with various kinds of programs on the basis of information obtained via at least one of the general-purpose communication I / F 7620, the dedicated communication I / F 7630, the positioning section 7640, the beacon receiving section 7650, the in-vehicle device I / F 7660, and the vehicle-mounted network I / F 7680. For example, the microcomputer 7610 may calculate a control target value for the driving force generating device, the steering mechanism, or the braking device on the basis of the obtained information about the inside and outside of the vehicle, and output a control command to the driving system control unit 7100. For example, the microcomputer 7610 may perform cooperative control intended to implement functions of an advanced driver assistance system (ADAS) which functions include collision avoidance or shock mitigation for the vehicle, following driving based on a following distance, vehicle speed maintaining driving, a warning of collision of the vehicle, a warning of deviation of the vehicle from a lane, or the like. In addition, the microcomputer 7610 may perform cooperative control intended for automatic driving, which makes the vehicle to travel autonomously without depending on the operation of the driver, or the like, by controlling the driving force generating device, the steering mechanism, the braking device, or the like on the basis of the obtained information about the surroundings of the vehicle.

[0140] The microcomputer 7610 may generate three-dimensional distance information between the vehicle and an object such as a surrounding structure, a person, or the like, and generate local map information including information about the surroundings of the current position of the vehicle, on the basis of information obtained via at least one of the general-purpose communication I / F 7620, the dedicated communication I / F 7630, the positioning section 7640, the beacon receiving section 7650, the in-vehicle device I / F 7660, and the vehicle-mounted network I / F 7680. In addition, the microcomputer 7610 may predict danger such as collision of the vehicle, approaching of a pedestrian or the like, an entry to a closed road, or the like on the basis of the obtained information, and generate a warning signal. The warning signal may, for example, be a signal for producing a warning sound or lighting a warning lamp.

[0141] The sound / image output section 7670 transmits an output signal of at least one of a sound and an image to an output device capable of visually or auditorily notifying information to an occupant of the vehicle or the outside of the vehicle. In the example of FIG. 6, an audio speaker 7710, a display section 7720, and an instrument panel 7730 are illustrated as the output device. The display section 7720 may, for example, include at least one of an on-board display and a head-up display. The display section 7720 may have an augmented reality (AR) display function. The output device may be other than these devices, and may be another device such as headphones, a wearable device such as an eyeglass type display worn by an occupant or the like, a projector, a lamp, or the like. In a case where the output device is a display device, the display device visually displays results obtained by various kinds of processing performed by the microcomputer 7610 or information received from another control unit in various forms such as text, an image, a table, a graph, or the like. In addition, in a case where the output device is an audio output device, the audio output device converts an audio signal constituted of reproduced audio data or sound data or the like into an analog signal, and auditorily outputs the analog signal.

[0142] Incidentally, at least two control units connected to each other via the communication network 7010 in the example depicted in FIG. 6 may be integrated into one control unit. Alternatively, each individual control unit may include a plurality of control units. Further, the vehicle control system 7000 may include another control unit not depicted in the figures. In addition, part or the whole of the functions performed by one of the control units in the above description may be assigned to another control unit. That is, predetermined arithmetic processing may be performed by any of the control units as long as information is transmitted and received via the communication network 7010. Similarly, a sensor or a device connected to one of the control units may be connected to another control unit, and a plurality of control units may mutually transmit and receive detection information via the communication network 7010. Incidentally, a computer program for realizing the functions of the spatial audio system according to the present embodiment described with reference to Fig. 1 can be implemented in one of the control units or the like. In addition, a computer readable recording medium storing such a computer program can also be provided. The recording medium is, for example, a magnetic disk, an optical disk, a magneto-optical disk, a flash memory, or the like. In addition, the above-described computer program may be distributed via a network, for example, without the recording medium being used.

[0143] In the vehicle control system 7000 described above, the spatial audio system according to the present embodiment described with reference to Fig. 1 can be applied to the integrated control unit 7600 in the application example depicted in FIG. 6. For example, the audio processing unit 12 of the spatial audio system may correspond to the microcomputer 7610. For example, the process described in Fig. 5 can be performed by microcomputer 7610.

[0144] In addition, at least part of the constituent elements of the spatial audio system described with reference to Fig. 1 may be implemented in a module (for example, an integrated circuit module formed with a single die) for the integrated control unit 7600 depicted in Fig. 6. Alternatively, the spatial audio system described with reference to Fig. 1 may be implemented by a plurality of control units of the vehicle control system 7000 depicted in FIG. 6.

[0145] Note that the present technology can also be configured as described below.

[0146] [1] An electronic device (12) comprising circuitry configured to reduce an imbalance in a spatial sound field by generating a virtual compensation source (27) based on the position of a virtual sound source (26) and a frequency of the imbalance related to the virtual sound source.

[0147] [2] The electronic device according to [1], wherein the virtual compensation source (27) is configured to be active in a narrow frequency band around the frequency of the imbalance.

[0148] [3] The electronic device according to [1] or [2], wherein the circuitry is configured to apply a bandpass filter to the virtual compensation source (27) to ensure that the virtual compensation source is only active in the narrow frequency band around the frequency of the imbalance.

[0149] [4] The electronic device according to any one of [1] to [3], wherein the circuitry is configured to determine the position of the compensation source (27) based on the position of the virtual sound source (26).

[0150] [5] The electronic device according to [4], wherein the circuitry is configured to determine the position of the compensation source (27) further based on the frequency of the imbalance. [6] The electronic device according to any one of [1] to [5], wherein the circuitry is configured to position the virtual compensation source (27) in a manner that creates destructive interference with the sound field of the virtual sound source (26), thereby mitigating the perceived imbalance in the spatial sound field.

[0151] [7] The electronic device of [6], wherein the position of the virtual compensation source (27) is determined to be a lambda half distance from the virtual sound source (26) in the direction of the listener.

[0152] [8] The electronic device according to any one of [1] to [7], wherein the circuitry is configured to determine the position of the compensation source (27) based on the position of a listener.

[0153] [9] The electronic device according to any one of [1] to [8], wherein the circuitry is configured to render the virtual compensation source (27) based on the frequency of the imbalance, and the position of the virtual sound source (26).

[0154]

[0010] The electronic device according to any one of [1] to [9], wherein the circuitry is configured to determine a gain for the virtual compensation source (27), and to render the virtual compensation source based on the determined gain.

[0155]

[0011] The electronic device of

[0010] , wherein the gain of the virtual compensation source (27) is determined based on the position of the virtual compensation source.

[0156]

[0012] The electronic device according to any one of [1] to

[0011] , wherein the circuitry is configured to use a 3D audio Tenderer to generate the virtual compensation source (27).

[0157]

[0013] The electronic device of any one of [1] to

[0012] , wherein the virtual compensation source (27) is a monopole.

[0158]

[0014] The electronic device according to any one of

[0001] to

[0013] , wherein the circuitry is further configured to track the position of the listener and dynamically adjust the position of the virtual compensation source (27) based on changes in the listener's position.

[0159]

[0015] The electronic device according to

[0014] , wherein the circuitry is configured to optimize the spatial sound field of the interior acoustics of the vehicle.

[0160]

[0016] The electronic device according to any one of [1] to

[0015] , wherein the electronic device is part of a public address system.

[0161]

[0017] The electronic device according to any one of [1] to

[0016] , wherein the circuitry is configured to dynamically adjust the position of the virtual compensation source (27) based on the movement of a sound source in a live performance.

[0162]

[0018] The electronic device according to

[0016] or

[0017] , wherein the circuitry is configured to mitigate potential feedback in a public address system.

[0163]

[0019] A method for compensating for an imbalance in a spatial sound field, the method comprising: generating a virtual compensation source (27) based on a position of a virtual sound source (26); identifying a frequency of the imbalance related to the virtual sound source (26); and adjusting the virtual compensation source (27) based on the identified frequency to reduce the imbalance in the spatial sound field.

[0164]

[0020] A computer program comprising computer executable instructions that, when executed by a processor, cause the processor to perform the method of

[0019] ,

[0165] List of reference signs:

[0166] 11 Audio source

[0167] 12 Audio processing unit

[0168] 14 Amplifiers

[0169] 15 Microphone

[0170] 21 Vehicle chassis

[0171] 22 Driver's seat

[0172] 23 Passenger's seat

[0173] 24 Dashboard

[0174] 25 Driver

[0175] 26 Intended sound source

[0176] 27 Virtual compensation source

[0177] 41 Dashed line marking center frequency

[0178] 42 Solid curve representing the gain

[0179] 51 Step to obtain posi ti on of virtual sound source

[0180] 52 Step to determine position of listener

[0181] 53 Step to determine frequency of imbalance

[0182] 54 Step to determine position of virtual compensation source

[0183] 55 Step to determine gain of virtual compensation source

[0184] 56 Step to render virtual compensation source

[0185] 7000 Vehicle control system

[0186] 7010 Communication network

[0187] 7100 Driving system control unit

[0188] 7200 Body system control unit

[0189] 7300 Battery control unit

[0190] 7400 Outside-vehicle information detecting unit

[0191] 7500 In-vehicle information detecting unit

[0192] 7600 Integrated control unit

[0193] 7610 Mi crocomputer

[0194] 7620 General -purpose communication IZF 7630 Dedicated communication I / F

[0195] 7640 Positioning section

[0196] 7650 Beacon receiving section

[0197] 7660 In-vehicle device I / F

[0198] 7670 Sound / image output section

[0199] 7680 Vehicle-mounted network I / F

[0200] 7690 Storage section

[0201] 7710 Audio speaker

[0202] 7720 Display section

[0203] 7730 Instrument panel

[0204] 7750 External environment

[0205] 7760 In-vehicle devices

[0206] 7800 Input section

Claims

CLAIMS1. An electronic device (12) comprising circuitry configured to reduce an imbalance in a spatial sound field by generating a virtual compensation source (27) based on the position of a virtual sound source (26) and a frequency of the imbalance related to the virtual sound source.

2. The electronic device according to claim 1, wherein the virtual compensation source (27) is configured to be active in a narrow frequency band around the frequency of the imbalance.

3. The electronic device according to claim 1, wherein the circuitry is configured to apply a bandpass filter to the virtual compensation source (27) to ensure that the virtual compensation source is only active in the narrow frequency band around the frequency of the imbalance.

4. The electronic device according to claim 1, wherein the circuitry is configured to determine the position of the compensation source (27) based on the position of the virtual sound source (26).

5. The electronic device according to claim 4, wherein the circuitry is configured to determine the position of the compensation source (27) further based on the frequency of the imbalance.

6. The electronic device according to claim 1, wherein the circuitry is configured to position the virtual compensation source (27) in a manner that creates destructive interference with the sound field of the virtual sound source (26), thereby mitigating the perceived imbalance in the spatial sound field.

7. The electronic device of claim 6, wherein the position of the virtual compensation source (27) is determined to be a lambda half distance from the virtual sound source (26) in the direction of the listener.

8. The electronic device according to claim 1, wherein the circuitry is configured to determine the position of the compensation source (27) based on the position of a listener.

9. The electronic device according to claim 1, wherein the circuitry is configured to render the virtual compensation source (27) based on the frequency of the imbalance, and the position of the virtual sound source (26).

10. The electronic device according to claim 1, wherein the circuitry is configured to determine a gain for the virtual compensation source (27), and to render the virtual compensation source based on the determined gain.

11. The electronic device of claim 10, wherein the gain of the virtual compensation source (27) is determined based on the position of the virtual compensation source.

12. The electronic device according to claim 1, wherein the circuitry is configured to use a 3D audio Tenderer to generate the virtual compensation source (27).

13. The electronic device of claim 1, wherein the virtual compensation source (27) is a monopole.

14. The electronic device according to claim 1, wherein the circuitry is further configured to track the position of the listener and dynamically adjust the position of the virtual compensation source (27) based on changes in the listener's position.

15. The electronic device according to claim 14, wherein the circuitry is configured to optimize the spatial sound field of the interior acoustics of the vehicle.

16. The electronic device according to claim 1, wherein the electronic device is part of a public address system.

17. The electronic device according to claim 1, wherein the circuitry is configured to dynamically adjust the position of the virtual compensation source (27) based on the movement of a sound source in a live performance.

18. The electronic device according to claim 16, wherein the circuitry is configured to mitigate potential feedback in a public address system.

19. A method for compensating for an imbalance in a spatial sound field, the method comprising: generating a virtual compensation source (27) based on a position of a virtual sound source (26); identifying a frequency of the imbalance related to the virtual sound source (26); and adjusting the virtual compensation source (27) based on the identified frequency to reduce the imbalance in the spatial sound field.

20. A computer program comprising computer executable instructions that, when executed by a processor, cause the processor to perform the method of claim 19.

Images