Sound reproduction system architectures

Abstract

Generating audio signals for an array of loudspeakers. The method comprises: receiving a first plurality of input audio signals to be reproduced, by the array, at a respective plurality of control points in an acoustic environment. This is followed by receiving a first set of filters and generating a respective first output audio signal for each of the loudspeakers in the array by applying the first set of filters to the first plurality of input audio signals. Subsequent to receiving the first plurality of input audio signals, repeating the receiving to receive a second plurality of input audio signals; and repeating the generating to generate a respective second output audio signal for each of the loudspeakers in the array by applying the first set of filters to the second plurality of input audio signals. An embodiment involves applying filters (possibly based on head related transfer functions) to the audio, where the filters are determined based on a position estimate determined via an image sensor.

Description

Field

[0001] The present disclosure relates generally to a method of generating audio signals for an array of loudspeakers and a corresponding apparatus and computer program. Background

[0002] Loudspeaker arrays may be used to reproduce a plurality of different audio signals at a plurality of control points, such as the ears of one or more listeners. The audio signals that are applied to the loudspeaker array are generated using filters, which may be designed so as to avoid cross-talk or synthesise a desired array radiation pattern. However, the determination of the weights of these filters may be computationally expensive. Summary

[0003] Aspects of the present disclosure are defined in the accompanying independent claims. Overview of disclosure

[0004] There is disclosed a computer-implemented method of generating audio signals for an array of loudspeakers.

[0005] The method is performed by a sound reproduction system.

[0006] The method comprises receiving a first plurality of input audio signals to be reproduced, by the array, at a respective plurality of control points in an acoustic environment.

[0007] Optionally, each of the plurality of input audio signals is different.

[0008] Optionally, at least one of the plurality of input audio signals is different from at least one other one of the plurality of input audio signals.

[0009] The method comprises receiving a first set of filters.

[0010] The method comprises generating a respective first output audio signal for each of the loudspeakers in the array by applying the first set of filters to the first plurality of input audio signals .

[0011] Optionally, the method comprises, subsequent to receiving the first plurality of input audio signals, repeating the receiving to receive a second plurality of input audio signals.

[0012] Optionally, the method comprises repeating the generating to generate a respective second output audio signal for each of the loudspeakers in the array by applying the first set of filters to the second plurality of input audio signals.

[0013] Optionally, the first set of filters is received from a non-real-time process .

[0014] Optionally, the non-real-time process is performed by one or more first processors that are not capable of guaranteeing a response within a specified time constraint.

[0015] Optionally, the receiving of the first plurality of input signals, the repeating thereof, the generating and the repeating thereof are performed by one or more real-time processes.

[0016] Optionally, the one or more real-time processes are performed by one or more second processors that are capable of guaranteeing a response within a specified time constraint.

[0017] Optionally, the non-real-time process is performed by one or more first processors having respective first clock rates, and the one or more real-time processes are performed by one or more second processors having respective second clock rates. Optionally, the one or more first clock rates are each different from the one or more second clock rates. Optionally, the one or more first clock rates are higher than the one or more second clock rates.

[0018] Optionally, the one or more real-time processes guarantee a response within a respective specified time constraint, and the non-real-time process does not guarantee a response within a specified time constraint.

[0019] Optionally, the one or more real-time processes have respective first priorities, the non-real-time process has a second priority, and the first priorities are higher than the second priority.

[0020] Optionally, the one or more real-time processes are performed, on average, at respective first frequencies and the non-real-time process is performed, on average, at a second frequency. Optionally, the first frequencies are higher than the second frequency.

[0021] Optionally, the first frequencies are based on a frame size of the first plurality of input audio signals.

[0022] Optionally, the first frequencies are based on a sampling frequency of the first plurality of input audio signals.

[0023] Optionally, the method further comprises, prior to receiving the first set of filters: receiving a first estimate of a position of each of the plurality of control points; and determining, based on the first estimate, the first set of filters .

[0024] Optionally, determining the first set of filters comprises determining one or more audio processing parameters for each of the plurality of control points. Optionally, the first estimate of the position of a given one of the plurality of control points is between a third position and a fourth position. Optionally, determining the one or more audio given control point comprises: retrieving one or more first parameters for the third position audio processing parameters for th processing parameters for the stored audio processing and one or more second stored ie fourth position; and determining the one or more audio processing parameters for the given control point based on the retrieved one or more first stored audio processing parameters and the retrieved one or more second stored audio processing parameters.

[0025] Optionally, the one or more audio processing parameters are filter weights of the first set of filters.

[0026] Optionally, the method further comprises, prior to receiving the first estimate: determining the first estimate.

[0027] Optionally, the first estimate is determined based on a signal captured by a sensor. Optionally, the sensor is an image sensor .

[0028] Optionally, the first estimate is determined based on a plurality of signals captured by a corresponding plurality of image sensors .

[0029] Optionally, the image sensor, or each of the plurality of image sensors, is a visible light sensor (i.e., a conventional, or non-infrared sensor), an infrared sensor, an ultrasonic sensor, a radar sensor, a time of flight (ToF) sensor, an extremely high frequency (EHF) sensor (or 'mmWave sensor' ), or a LiDAR sensor.

[0030] Optionally, the non-real-time process is performed at a frequency based on a sampling frequency of the sensor.

[0031] Optionally, the method further comprises: subsequent to receiving the second plurality of input audio signals, repeating the receiving to receive a third plurality of input audio signals; receiving a second set of filters; and generating a respective third output audio signal for each of the loudspeakers in the array by applying the second set of filters to the third plurality of input audio signals.

[0032] Optionally, the method further comprises, prior to receiving the second set of filters: receiving a second estimate of a position of each of the plurality of control points, determining, based on the second estimate, the second set of filters .

[0033] Optionally, the method further comprises, prior to receiving the second estimate: determining the second estimate.

[0034] Optionally, the method further comprises: outputting, to the array, the respective first output audio signals and respective second output audio signals for each of the loudspeakers in the array.

[0035] Optionally, the outputting is performed at an audio device.

[0036] Optionally, the receiving of the first plurality of input signals, the repeating thereof, the generating and the repeating thereof are performed at an audio device.

[0037] Optionally, the determining of the first set of filters is performed at the audio device.

[0038] Optionally, the determining of the first estimate is performed at the audio device.

[0039] Optionally, the determining of the first estimate is performed at the sensor.

[0040] Optionally, the first estimate is received from a host device .

[0041] Optionally, the first set of filters is received from a host device .

[0042] Optionally, the receiving of the first plurality of input signals, the repeating thereof, the generating, and the repeating thereof are performed at a host device which is communicatively coupled to an audio device.

[0043] Optionally, the determining of the first set of filters is performed at the host device.

[0044] Optionally, the determining of the first estimate is performed at the host device.

[0045] Optionally, the host device and the audio device are each housed in a separate housing.

[0046] Optionally, the first plurality of input audio signals and second plurality of input audio signals are received from a source device without passing through the host device.

[0047] Optionally, the host device and the audio device are communicatively coupled by an external data communication interface. Optionally, the external data communication interface is a Universal Serial Bus, USB, interface.

[0048] Optionally, an external data communication interface is a data communication interface for connecting a computer to external peripheral devices, rather than to connect components within a computer .

[0049] Optionally, the external data communication interface is a wired communication interface.

[0050] Optionally, the external data communication interface is a wireless communication interface.

[0051] Optionally, the array is housed in a (or the) housing of the audio device.

[0052] Optionally, the sensor is housed in a (or the) housing of the audio device.

[0053] There is disclosed an apparatus configured to perform any of the methods described herein.

[0054] There is disclosed a computer program comprising instructions which, when executed by one or more processors of a computing system, cause the computing system to perform any of the methods described herein.

[0055] There is disclosed a computer-readable medium comprising instructions which, when executed by one or more processors of a computing system, cause the computing system to perform any of the methods described herein.

[0056] There is disclosed a data carrier signal comprising instructions which, when executed by one or more processors of a computing system, cause the computing system to perform any of the methods described herein. Brief description of the drawings

[0057] Examples of the present disclosure will now be explained with reference to the accompanying drawings in which: Fig. 1 is a high-level overview of processes involved in providing user-tracked adaptive audio. Fig. 2 is a schematic of a basic, reference architecture for implementing the processes of Fig. 1. Fig. 3A is a schematic of a first architecture for implementing the methods described herein. Fig. 3B is a schematic of an example of the first architecture. Fig. 4A is a schematic of a second architecture for implementing the methods described herein. Fig. 4B is a schematic of an example of the second architecture. Fig. 5A is a schematic of a third architecture for implementing the methods described herein. Fig. 5B is a schematic of an example of the third architecture. Fig. 6A is a schematic of a fourth architecture for implementing the methods described herein. Fig. 6B is a schematic of an example of the fourth architecture. Fig. 7A is a schematic of a fifth architecture for implementing the methods described herein. Fig. 7B is a schematic of an example of the fifth architecture. Fig. 8A is a schematic of a sixth architecture for implementing the methods described herein. Fig. 8B is a schematic of an example of the sixth architecture. Figs. 9A, 9B, and 9C are respective parts of a flowchart of a method which may be performed using any of the first, second, third, fourth, fifth or sixth architectures. Fig. 10 is an exemplary apparatus for implementing any of the methods described herein.

[0058] Throughout the description and the drawings, like reference numerals refer to like parts. Detailed description Overview

[0059] In general terms, the present disclosure relates to a computer-implemented method of generating audio signals for an array of loudspeakers. The method may involve receiving a first set of audio signals to be reproduced, by the array, at various control points in an acoustic environment (e.g., a room); receiving a set of filters; and generating output signals for each of the loudspeakers in the array by applying the set of filters to the input audio signals. The method is repeated for further sets of input audio signals without necessarily applying a new, updated set of filters. The sets of filters can thus be updated less frequently than new sets of input signals are received, allowing for reduced computational load and for the proportion of computations that are subject to a requirement to be performed in real-time to be reduced. Principles of operation

[0060] The approach of the present disclosure is suitable for use in audio reproduction systems with loudspeakers. More specifically, the approach of the present disclosure is suitable for use in a sound reproduction device (e.g., a soundbar or a laptop) that is connected to a user-detection-and-tracking system. In the approach of the present disclosure, the processing for the sound reproduction device and the processing for the user-detection-and-tracking system may be implemented in processes with different priority levels, or in different hardware processors, in order to obtain a lower cost implementation, an implementation with reduced audio latency, and / or an implementation which allows operation when not connected to a main host computer.

[0061] Fig. 1 shows some processes which may be involved in generating audio signals for an array of loudspeakers, such as those in user-tracking-based audio systems (i.e., the processes involved in user-adaptive audio reproduction). The main processes involved in such a system may be: user tracking (or 'user detection and tracking', or 'detection and tracking', or 'tracking') S121, calculation S122 of user-position-dependent audio parameters (or 'audio processing parameter calculation', or 'filter calculation', or 'filter computation'), and computation S123 of speaker output signals by applying time-varying signal processing to obtain speaker outputs (or 'filtering').

[0062] User tracking S121 may be implemented on a user tracking processor, and may involve generating user position data 111. User tracking may find the position of one or more users with respect to one or more speakers of a sound reproduction device. This may be achieved using user detection and tracking software in the form of instructions for a dedicated user detection and tracking processor. The user detection and tracking processor may be connected to any kind of user position sensor (e.g., camera, radar, infrared, wearable) that provides information to be processed by the (user) detection and tracking processor so that it can output user position data 111. User tracking S121 is generally a process that need not be implemented in real-time. This is because, even if the user's position is not tracked in real-time, signals may still be output from the sound device at an expected rate - they may simply not be fully adapted to the user's current position.

[0063] Calculation S122 of audio signal processing parameters (e.g., user-position-dependent audio parameters) may be implemented on an audio processing and parameter calculation / extraction processor, which may use the user position data 111 and may generate audio signal processing parameters 112. This process may happen at a predetermined frequency, or every time a new user position is detected. User position data 111 is used to calculate audio processing parameters 112 (e.g., filter weights / coefficients and / or digital signal processing, DSP, values) that correspond to the latest user position 111. The parameters may be calculated in the same processor associated with the user tracking, or in a processor for audio digital signal processing (DSP), as described below. Alternatively or additionally, the audio processing parameters 112 can be extracted from a lookup table stored in memory and interpolated in between user positions. Calculation S122 of audio signal processing parameters is generally a process that need not be implemented in real-time. This is because, even if the audio signal processing parameters are not calculated in real-time, signals may still be output from the sound device at the expected rate - they may simply not be fully adapted to the user's current position.

[0064] Computation S123 of speaker output signals 113 (i.e., filtering) may be implemented on an audio DSP processor. This calculation may involve filtering of input signals 114 with digital signal processing effects (e.g., applying finite impulse response -FIR - filters, infinite impulse response - HR - filters, or gain and delay elements) to obtain a set of speaker output signals 113 that can provide a particular audio effect. In real-time audio applications, computation S123 of speaker output signals is generally a process that needs to be implemented in real-time, as it requires continuous filtering of input audio signals (i.e., input audio samples) 114 to achieve good quality and smooth audio output 113.

[0065] Some implementations for generating audio signals for an array of loudspeakers may combine the user tracking, audio parameter calculating and audio processing in the same physical processing unit. Fig. 2 shows a schematic of such an implementation as an example. In Fig. 2, a device 201 comprises a processor 202, a sensor 203, and an array of loudspeakers 216. The processor 202 is configured to perform processes S121, S122, and S123. The sensor 203 is coupled to the processor 202 in order, for example, to provide data for the user tracking process S121. The processor 202 is configured to output the audio signals 113 to the array of loudspeakers 216.

[0066] The approach of the present disclosure allows for some issues associated with implementing user tracking audio systems on low-cost processing platforms to be mitigated.

[0067] User-tracking-based audio systems are typically used for reproduction of spatial audio through loudspeakers or for the creation of listening zones tailored to the user's position. As the user positions vary, the position of the users may be determined and also parameters of the audio reproduction system are calculated and adapted to the user's positions at different points in time (i.e., instantaneous positions). However, as a result, user-tracking-based audio systems may have to rely on a large amount of computational resources to perform the main processes which may be involved in such a system, e.g., the processes described above in relation to Fig. 1.

[0068] Said processes may be performed on a single system on chip (SoC) for implementation simplicity; however, this can result in significant costs for producing a user-tracking-based audio system, as the system may then require a computationally expensive multipurpose processor, making platforms with the required combined user tracking and audio processing complex and expensive.

[0069] In some implementations, audio processing, audio parameter calculation, and user tracking may be split into various processing units. For example, the split may be between a microcontroller, video processing unit or electromagnetic tracking device (e.g., providing user tracking and / or audio parameter calculation), and an audio digital signal processor (e.g., providing audio processing).

[0070] Thus, the approach of the present disclosure may provide a low-cost implementation of a user-tracking-based audio system. The approach of the present disclosure may also have advantages from an audio point of view, as digital signal processors allow for reduced audio latency compared to other multipurpose processors used in the single SoC implementations referred to above.

[0071] The approach of the present disclosure provides novel ways (e.g., architectures, methods, apparatus) for implementing the processes involved in a user-tracking-based audio system.

[0072] The approach of the present disclosure is for a usertracking-based audio system (i.e., a digital audio reproduction system that adapts the audio reproduction with respect to the position of one or more users). Several processes may be involved in obtaining such reproduction / output, each process running at different levels of priority in one or more software processors: real-time and non-real-time.

[0073] Real-time processes are those which have a fixed deadline by which the result must be generated and, therefore, real-time processes are those which have a higher, or maximum, priority from a software hierarchy point of view. Real-time processes may have to run continuously and may not be interruptible by other software processes. For example, the (digital) filtering S123 of audio samples 114 to obtain processed speaker output signals 113 may need to occur in real-time and with maximum priority; otherwise, there could be digital samples 114 that are not correctly output, causing audio dropouts coming from the audio device.

[0074] Non-real-time processes do not have a fixed deadline by which the result must be generated and, therefore, are those which do not have to be run with maximum priority and may instead be run with lower priority. A delay in generating the results of non-real-time processes does not generally cause a drastic negative effect on the audio user experience, or indeed sometimes does not even cause a noticeable effect. Accordingly, these processes have a lower position in the software hierarchy and may be interruptible by other processes that have a higher priority. The calculation and tracking S121 of the user position and the calculation S122 of audio processing parameters given a user position input are both processes that, in some implementations, do not have to be run in real-time, and their performance can wait a few milliseconds to happen if interrupted by other processes, without the user perceiving any degradation in the audio.

[0075] As a result of the different priority of the various processes involved in generating audio signals for an array of loudspeakers, said processes may be implemented in different specialised processors. User tracking processes S121 may be implemented in a (micro-) processor or specialised hardware with accelerators. Audio parameter calculation S122 may be implemented in high-level (micro-) processors that can provide instruction sets, or execute libraries for special mathematical functions, that may not be available, or may be costly, on digital signal processors used for audio processing. Audio filtering and computation S123 of speaker signals 113 may be implemented in dedicated audio digital signal processors that can perform deterministic filtering to produce an output without being interrupted by other processes.

[0076] In some implementations of a user-tracking-based audio system or user-adaptive audio system, a combination of various types of processing hardware may therefore be used: vision processors, microprocessors, and audio digital signal processors. Some implementations combine the three kinds of software processes S121, S122, S123 described above within a single, high computing power, microprocessor and run those processes at a same priority. However, as explained above, splitting the different processes onto different hardware types (e.g., a different processor for each of S121, S122 and S123), or at least running those processes at different priorities, may yield a more efficient and cost-optimised implementation, and / or a more versatile audio device with reduced latency.

[0077] This may be particularly advantageous when, for example, the user tracking based audio system is an audio device accessory for a personal computer (PC) , as some of the processes can then be performed on the host PC to simplify the implementation of the sound device . Architectures

[0078] Several architectures 300, 400, 500, 600, 700, or 800 which can be used for performing processes S121, S122 and S123, or a method 900 of generating audio signals for an array of loudspeakers 316, are now described.

[0079] A first architecture 300 is shown in Fig. 3A. In this architecture, an audio device (or 'sound device', or 'sound reproduction device' , or 'soundbar' ) 301 performs processes S121, S122 and S123 to output output audio signals 113. The audio device 301 may be communicatively coupled with a host device (or 'host personal computer (PC)', or 'PC') 302, from which it may receive input audio signals 114.

[0080] Architecture 300 may be suitable for a standalone audio device, i.e., the audio device 301 performs user tracking S121, calculation S122 of audio parameters, and computation S123 of speaker output signals.

[0081] In this architecture, the microprocessor or system performing the user tracking may be embedded in the audio device 301.

[0082] A single processor may perform all of processes S121 S122 and S123. In this case, the single processor may need to be relatively powerful. However, each of those processes may be run with different priority levels, and / or may be run at different frequencies. In this way, process S123 can be performed in real-time even if the single processor is not particularly powerful. As a result, the processing power required to be provided by the single processor may be reduced.

[0083] An example of a possible implementation of the first architecture 300 is shown in Fig. 3B. The audio device 301 may have a user tracking processor 311 for user tracking S121, an audio parameter processor 312 for calculation S122 of audio parameters, a control processor 315 for controlling behaviour of the audio device (e.g., functionality that is not related to providing user-adaptive audio) , and an audio DSP processor 313 for computation S123 of speaker output signals. The host device 302 may have a content storage 314 which may provide input audio signals. The audio device 301 may also comprise user tracking sensors 321 for estimating a position of a control point, a user interface (UI) control processor 322 for rendering a user interface of the audio device 301, and / or amplifiers 323 to amplify outputs of DSP processor 313.

[0084] In this architecture, a set of different processors may be used to perform the different processes, as shown in Fig. 3B. A microprocessor or video processor unit 311 may be used to obtain user position estimates from the tracking sensor feed. The calculation of audio processing parameters may be performed in a microprocessor 312. This can be the same physical processor as is used for the user tracking processing, or alternatively, audio parameter calculation can be implemented in a different signal processor. The calculation of audio processing parameters can be performed in real-time according to the user position or the parameters can be retrieved from a lookup table stored in the device's memory. The processing of the inputs to obtain audio speaker signals may be implemented in an audio digital signal processor or other real-time processor 313.

[0085] In some implementations, the user tracking sensor 321 and user tracking processor 311 may be integrated. The integrated user tracking sensor and user tracking processor may thus output user position data 111, avoiding the need for any other processors of the audio device 301 to perform the user tracking S121.

[0086] A second architecture 400 is shown in Fig. 4A. In this architecture, the host device 302 performs processes S121 and S122, and the audio device 301 performs process S123.

[0087] From a system and product design perspective, it may be useful to keep flexibility so that a software package can work with as many operating systems (OSs) as possible. This can be difficult to achieve if the audio DSP for the computation of the speaker signals is performed inside the host device, as it may be timeconsuming and resource-intensive to implement and maintain audio processing software with real-time priority in different processors with different operating systems. Therefore, broader host OS support is facilitated by the above splitting of processes in architecture 400.

[0088] Furthermore, from an audio processing perspective and from a product design point of view, performing audio DSP for the computation of the speaker signals may require addressing the following two issues. The first issue is that performing audio DSP in the host computer may add extra processing latency, as external audio input signals need to be received into the host computer first. This may add extra delay that may be unacceptable for some applications. The second issue is that in this case the audio device needs to be connected to the computer and the computer needs to be switched on.

[0089] To mitigate these two issues, this second architecture allows performance of audio DSP for the computation of the speaker signals in a processor embedded in the audio device. The processor may then ensure reduced latency of external audio inputs and can also have stored in memory basic audio parameters (e.g., providing a fixed beamforming pattern or a non-adaptive output) that allow for the system to function even if not connected to a host computer or if the host computer is switched off.

[0090] Generally, the most resource expensive processing is the user tracking, which is usually a non-real-time process. In this case, the video processing can be offloaded to the host computer without compromising the real-time requirements of the audio signal processing. The audio parameter calculation (or parameter extraction from a lookup table) can be performed in the host device and transferred to the audio device, or can happen inside the audio device, on an embedded microprocessor, or by using pre-calculated parameters from the memory of the device in the DSP for preparing the loudspeaker signals. In this case, the calculation of the audio processing parameters may happen in the host device.

[0091] An example of a possible implementation of architecture 400 is shown in Fig. 4B. Architecture 400 may be suitable for a hostbased implementation in which the host device 302 performs user tracking S121 and calculation S122 of audio parameters, and the audio device 301 performs computation S123 of speaker output signals. In this example, the host device 302 comprises a user tracking processor 311 for user tracking S121 and an audio parameter processor 312 for calculation S122 of audio parameters, and the audio device 301 comprises an audio DSP processor 313 for preparation S123 of speaker output signals.

[0092] A third architecture 500 is shown in Fig. 5A. In this architecture, the host device 302 performs process S121, and the audio device 301 performs processes S122 and S123.

[0093] The calculation of the audio processing parameters S122 can be performed in a same process as the filtering S123. Alternatively, the calculation of the audio processing parameters S122 may, for example, be performed in an interrupt routine or a (real-time operating system) task having a lower priority than the task running the filtering S123.

[0094] The audio processing parameters may be calculated for each newly informed user position, or alternatively can be retrieved from audio processing parameters stored in a lookup table in memory.

[0095] An example of a possible implementation of architecture 500 is shown in Fig. 5B. Architecture 500 may be suitable for a hostbased implementation in which the host device 302 performs user tracking S121, and the audio device 301 performs calculation S122 of audio parameters and computation S123 of speaker output signals (i.e., with an external calculation S122). In this example, the host device 302 comprises a user tracking processor 311 for user tracking S121, and the audio device 301 comprises an audio parameter processor 312 for calculation S122 of audio parameters and an audio DSP processor 313 for computation S123 of speaker output signals.

[0096] A fourth architecture 600 is shown in Fig. 6A. In this architecture, the host device 302 performs processes S121, S122 and S123 .

[0097] Some audio systems are designed to be used as peripherals or accessories for computers. In this case, if the system is generally going to be connected to a PC, it is possible to offload the required user tracking, audio parameter calculation and audio processing to the host PC, thus yielding a simpler implementation on the audio hardware side. For example, in the fourth architecture, the tracking, calculation and audio processing are offloaded to the host computer. This may be desirable, for example, to reduce the bill of material and parts cost of a user-tracking capable audio device .

[0098] An example of a possible implementation of architecture 600 is shown in Fig. 6B. Architecture 600 may be suitable for a fully host-based implementation in which the host performs user tracking S121, calculation S122 of audio parameters, and computation S123 of speaker output signals. In this example, the host device 302 comprises a user tracking processor 311 for user tracking S121, an audio parameter processor 312 for calculation S122 of audio parameters, and an audio DSP processor 313 for computation S123 of speaker output signals.

[0099] A fifth architecture 700 is shown in Fig. 7A. In this architecture, the host device 302 performs processes S122 and S123, and the audio device 301 performs process S121.

[0100] This architecture is similar to the fourth architecture 600. However, in the fifth architecture 700, process S121 is performed by the audio device 301 rather than the host device 302. Such an implementation may be suitable when the user tracking processor 311 performing process S121 is integrated with the user tracking sensor 321 in the audio device 301.

[0101] An example of a possible implementation of architecture 700 is shown in Fig. 7B. Architecture 700 may be suitable for a mixed implementation in which the host performs calculation S122 of audio parameters and computation S123 of speaker output signals, and the audio device performs user tracking S121. In this example, the host device 302 comprises an audio parameter processor 312 for calculation S122 of audio parameters, and an audio DSP processor 313 for computation S123 of speaker output signals, and the audio device 301 comprises a user tracking processor 311 for user tracking S121.

[0102] A sixth architecture 800 is shown in Fig. 8A. In this architecture, the host device 302 performs process S122, and the audio device 301 performs processes S121 and S123.

[0103] This architecture is similar to the second architecture 400. However, in the sixth architecture 800, process S121 is performed by the audio device 301 rather than the host device 302. Such an implementation may be suitable when the user tracking processor 311 performing process S121 is integrated with the user tracking sensor 321 in the audio device 301.

[0104] An example of a possible implementation of architecture 800 is shown in Fig. 8B. Architecture 800 may be suitable for a mixed implementation in which the host device 302 performs calculation S122 of audio parameters, and the audio device 301 performs user tracking S121 and computation S123 of speaker output signals. In this example, the host device 302 comprises an audio parameter processor 312 for calculation S122 of audio parameters, and the audio device 301 comprises a user tracking processor 311 for user tracking S121 and an audio DSP processor 313 for preparation S123 of speaker output signals.

[0105] In any of the first, second, third, fourth, fifth or sixth architectures 300, 400, 500, 600, 700 and 800, the host device 302 and the audio device 301 may be communicatively coupled by an external data communication interface (e.g., a Universal Serial Bus, USB, interface). An external data communication interface is a data communication interface for connecting a computer to external peripheral devices, rather than to connect components within a computer, and may be a wired or wireless communication interface. Method of generating audio signals

[0106] A computer-implemented method 900 of generating audio signals 413 for an array of loudspeakers 316 may comprise steps S901 to S917, shown in Figs. 9A, 9B and 9C. The method 900 may be performed by a sound reproduction system. More specifically, the method may be implemented on any of architectures 300, 400, 500, 600, 700 and 800 described above, in a manner which will be explained below with reference to the steps of method 900 and each of architectures 300, 400, 500, 600, 700 and 800.

[0107] At step S901, a first plurality of input audio signals (or 'input signals' ) d 414A may be received to be reproduced by the array 316 at a respective plurality of control points in an acoustic environment. The first plurality of input audio signals 414A may be received from a content storage 314.

[0108] The plurality of control points may be positioned at the positions of the users. The position of a particular user may be a position of a centre of a head of the particular user. The plurality of control points may be positioned at ears of the users.

[0109] Each of the plurality of input audio signals may be different, or at least one of the plurality of input audio signals may be different from at least one other one of the plurality of input audio signals. For example, to reproduce the same left and right ear signals in the ears of three listeners, the plurality of input audio signals may comprise first, second and third signals to be reproduced at the left ear of respective first, second and third listeners, and fourth, fifth and sixth signals to be reproduced at the left ear of the respective first, second and third listeners. The first, second and third signals may then be the same, and the fourth, fifth and sixth signals may also be the same, but the first / second / third signals may be different from the fourth / fifth / sixth signals. When only some of the plurality of input audio signals are different from each other, the receiving of step S901 may comprise receiving (e.g., from content storage 314) a plurality of unique input audio signals (in this case, the first / second / third signal, corresponding to the left ear signal, and the fourth / fifth / sixth signal, corresponding to the right ear signal), and forming the plurality of input audio signals - which may include duplicates - from the plurality of unique input audio signals, ready for filtering in step S906.

[0110] At step S902, a first estimate 411A of a position of each of the plurality of control points may be determined.

[0111] The first estimate 411A may be determined based on a signal captured by a sensor 321 (e.g., an image sensor). Alternatively, the first estimate 411A may be determined based on a plurality of signals captured by a corresponding plurality of image sensors 321. The image sensor, or each of the plurality of image sensors, may be a visible light sensor (i.e., a conventional, or non-infrared sensor) , an infrared sensor, an ultrasonic sensor, a radar sensor, an extremely high frequency (EHF) sensor (or 'mmWave sensor' ) , or a LiDAR sensor.

[0112] At step S903, the first estimate 411A may be received. The first estimate 411A may be received from another device or component of a device, or may be received locally when steps S902 and S903 are performed by a same processor. The term 'receiving' does not, therefore, necessarily imply receipt from another device, or even another processor.

[0113] At step S904, a first set of filters H 415A may be determined based on the first estimate 411A. The set of filters H may be determined such that, when the output audio signals are output to the array of loudspeakers 316, substantially only the respective one of the plurality of input audio signals is reproduced at each of the plurality of control points. An approach for determining the set of filters is briefly described in the next section.

[0114] The determining of the set of filters may be performed using previously computed data. Thus, to determine the first set of filters 415A, one or more stored audio processing parameters for each of the control points may be retrieved. The audio processing parameters may be weights of the set of filters 415A, or may be parameters used to compute those weights.

[0115] The positions of the control points for which audio processing parameters have been previously computed may not precisely match the positions of the control points for which a set of filters is needed. Thus, to determine the first set of filters 415A, one or more audio processing parameters for each of the plurality of control points may be determined. The first estimate 411A of the position of a given one of the plurality of control points may be between a third position and a fourth position. Determining the one or more audio processing parameters for the given control point may comprise retrieving one or more first stored audio processing parameters for the third position and one or more second stored audio processing parameters for the fourth position. Determining the one or more audio processing parameters for the given control point may then be based on the retrieved one or more first stored audio processing parameters and the retrieved one or more second stored audio processing parameters. For example, the one or more audio processing parameters for the given control point may be determined as an average, or a weighted average, of the retrieved one or more first stored audio processing parameters and the retrieved one or more second stored audio processing parameters.

[0116] At step S905, the first set of filters 415A may be received.

[0117] At step S906, a respective first output audio signal 413A for each of the loudspeakers in the array 316 may be generated (or 'determined') by applying the first set of filters 415A to the first plurality of input audio signals 414A.

[0118] The output audio signal 413A for a particular loudspeaker in the array of loudspeakers may be based on each of the plurality of input audio signals 414A.

[0119] The first set of filters 415A may be applied in the frequency domain. In this case, a transform, such as a fast Fourier transform (FFT), is applied to the plurality of input audio signals 414A, the filters are applied, and an inverse transform is then applied to obtain the output audio signals 413A. The first set of filters 415A may alternatively be applied in the time domain.

[0120] At step S907, the respective first output audio signals 413A for each of the loudspeakers in the array 316 may be output to the array.

[0121] Some or all of steps S901 to S907 may optionally be repeated once, twice, or more times with another plurality of input signals 414 .

[0122] In particular, in a first repetition of some or all of steps S901 to S907, the set of filters 415 may not be updated. Thus, at step S908, subsequent to receiving the first plurality of input audio signals 414A, the receiving S901 may be repeated to receive a second plurality of input audio signals 414B. The receiving may be repeated in the same manner as in step S901, but at a later time.

[0123] At step S909, the generating S906 may be repeated to generate a respective second output audio signal 413B for each of the loudspeakers in the array 316 by applying the first set of filters 415A to the second plurality of input audio signals 414B. The generating may be repeated in the same manner as in step S906.

[0124] At step S910, the respective second output audio signals 413B for each of the loudspeakers in the array 316 may be output to the array. The outputting may be repeated in the same manner as in step S907.

[0125] In a second repetition of the set of filters may be updated receiving the second plurality of receiving S901 may be repeated to audio signals 414C. The receiving as in steps S901 and S908, but at some or all of steps S901 to S907, . Thus, at step S911, subsequent to input audio signals 414B, the receive a third plurality of input may be repeated in the same manner a later time.

[0126] At step S912, subsequent to determining the first estimate 411A of the position of each of the plurality of control points, a second estimate 411B of the position of each of the plurality of control points may be determined. The determining may be performed in the same manner as in step S902, but at, and / or using sensor data obtained at, a later time than in step S902.

[0127] At step S913, the second estimate 411B of the position of each of the plurality of control points may be received. The receiving may be performed in the same manner as in step S903, but at a later time than in step S903.

[0128] At step S914, subsequent to determining the first set of filters 415A, a second set of filters 415B may be determined based on the second estimate 411B. The determining may be performed in the same manner as in step S904, but using an estimate 411B of the position of each of the plurality of control points obtained at, and / or determined using sensor data obtained at, a later time than in step S904.

[0129] At step S915, the second set of filters 415B may be received. The receiving may be performed in the same manner as in step S905, but at a later time than in step S905.

[0130] At step S916, subsequent to generating the second output audio signals 413B, a respective third output audio signal 413C for each of the loudspeakers in the array 316 may be generated by applying the second set of filters 415B to the third plurality of input audio signals 414C. The generating may be performed in the same manner as in steps S906 and S909, but at a later time.

[0131] At step S917, subsequent to outputting the respective second output audio signals 413B, the respective third output audio signals 413C for each of the loudspeakers in the array 316 may be output to the array. The outputting may be performed in the same manner as in steps S907 and S910, but at a later time.

[0132] The steps of method 900 are described in a particular order. However, those skilled in the art will recognise that that order is not the only possible order in which steps S901 to S917 may be performed.

[0133] Furthermore, as would be understood by a skilled person, steps S901 to S907 can be performed with respect to successively received frames of a plurality of input audio signals. Accordingly, steps S901 to S907 need not all be completed before they begin to be repeated. For example, in some implementations, step S901 is performed a second time (e.g., as step S908 or 8911) before step 8907 has been performed a first time.

[0134] Some of steps S901 to S917 may not need to be performed in real-time. Thus, the first set of filters 415A may be received S905 from a non-real-time process (e.g., a process performing one or more of steps S902 to S905). A non-real-time process may not guarantee a response within a specified time constraint. In some implementations, a non-real-time process may be performed by one or more first processors that are not capable of guaranteeing a response within a specified time constraint.

[0135] Some of steps S901 to S917 may, in contrast, need to be performed in real-time. Thus, the receiving S901 of the first plurality of input signals, the repeating S908 thereof, the generating S906 and / or the repeating S909 thereof may be performed by one or more real-time processes (e.g., a process involving one or more of steps S901, S906 to S909, and S910) . A real-time process may guarantee a response within a respective specified time constraint. In some implementations, a real-time process may be performed by one or more second processors that are capable of guaranteeing a response within a specified time constraint.

[0136] In some implementations, the non-real-time process is performed by one or more first processors having respective first clock rates, and the one or more real-time processes are performed by one or more second processors having respective second clock rates. The one or more first clock rates may each be different from the one or more second clock rates. In other words, the one or more real-time processes and the non-real-time process can be performed by (one or more) processors having different clock rates. In some implementations, the one or more first clock rates may be higher than the one or more second clock rates, e.g., the one or more first clock rates may be 2.5 GHz and the one or more second clock rates may be 800 MHz.

[0137] Steps that need to be performed in real-time may be performed using a same processor as steps that need not be performed in real-time. Thus, the one or more real-time processes may have respective first priorities, the non-real-time process may have a second priority, and, in that case, the first priorities may be higher than the second priority. Similarly, the one or more realtime processes may be performed, on average, at respective first frequencies and the non-real-time process may be performed, on average, at a second frequency, the first frequencies being higher than the second frequency. The first frequencies may be based on a frame size of the first plurality of input audio signals 414A and / or on a sampling frequency of the first plurality of input audio signals 414A. The non-real-time process may be performed at a frequency based on a sampling frequency of the sensor 321.

[0138] It should be noted that the steps of method 900 may be used to implement processes S121, S122 and S123. Specifically, step S902 may be an implementation of user tracking S121; steps S903 to S904 may be an implementation of calculation S122 of user-position-dependent audio parameters; and steps S901 and S906 may be an implementation of computation S123 of speaker output signals. Similarly, estimates 411 (e.g., 411A, 411B) may correspond to user position data 111; output audio signals 413 (e.g., 413A, 413B, 413C) may correspond to speaker output signals 113; input signals 414 (e.g., 414A, 414B, 414C) may correspond to input signals 114. Filter determination

[0139] An approach for determining the set of filters H 415A / 415B of steps S904 and S914 is now described. Alternative approaches for determining the set of filters H are described in US 2019 / 0090060 Al, US 2021 / 0385605 Al, and US 2023 / 0007424 Al, the entire disclosures of which - except for any definitions, disclaimers or inconsistencies with the present disclosure - are incorporated herein by reference.

[0140] Consider a system in which the spatial coordinates of the loudspeakers are y1; ...,yL, whereas the coordinates of the M control points are xlz The matrix S(to), hereafter referred to as plant matrix, whose element Smi(oi) is the electro-acoustical transfer function between the Z-th loudspeaker and the m-th control point, is expressed as a function of the angular frequency w. The reproduced sound pressure signals at the M control points, p(o>) = ^(0)),...^^(0))^, for a given frequency 0) are given by p(o)) = S(o))q(o>), where q(^) is a vector whose L elements are the loudspeaker signals. These are given by q(o)) = H(o>)d(o>), where d(to) is a vector whose M elements are the M signals intended to be delivered to the various control points. H(o>) is a complex-valued matrix that represents the effect of the signal processing apparatus, succinctly referred to herein as "filters". It should be clear though that each element of H(w) is not necessarily a single filter, but can be the result of a combination of filters, delays, and other signal processing blocks.

[0141] In what follows, the dependency of variables on the frequency at will be dropped to simplify the notation. We have therefore that p ™ Slid (1)

[0142] An approach to design the filters is to compute H as the (regularised) inverse or pseudo-inverse of matrix S, or of a model of matrix S, that is H = t A; (2) where matrix G is a model or estimate of the plant matrix S, A is a regularisation matrix (for example for Tikhonov regularisation) , [-]H is the complex-transposed (Hermitian) operator, j = V-1, and T is a modelling delay.

[0143] In more general terms, the set of filters may be based on a plurality of filter elements [e.g., G] comprising a respective filter element for each of the control points and loudspeakers.

[0144] Each one of the plurality of filter elements [e.g., G] may be a frequency-independent delay-gain element [e.g., Gml = e~j(i>T:Qxm,yi)gmi^ _ Each one of the plurality of filter elements [e.g., G] may comprise a delay term [e.g., e v<’>T(xm.yO] and / or a gain term [e.g., gm>i] that is based on the relative position [e.g., xm] of one of the control points and one of the loudspeakers [e.g., y( ] .

[0145] Each one of the plurality of filter elements [e.g., G] may comprise an approximation of a respective transfer function [e.g., S,ni(to)] between an audio signal applied to a respective one of the loudspeakers and an audio signal received at a respective one of the control points from the respective one of the loudspeakers.

[0146] The approximation may be based on a free-field acoustic propagation model and / or a point-source acoustic propagation model. The approximation may account for one or more of reflections, refraction, diffraction or scattering of sound in the acoustic environment. The approximation may alternatively or additionally account for scattering from a head of one or more listeners. The approximation may alternatively or additionally account for one or more of a frequency response of each of the loudspeakers or a directivity pattern of each of the loudspeakers.

[0147] The approximation may be based on one or more head-related transfer functions, HRTFs. The one or more HRTFs may be measured HRTFs. The one or more HRTFs may be simulated HRTFs. The one or more HRTFs may be determined using a boundary element model of a head.

[0148] A filter element may be a weight of a filter. A plurality of filter elements may be any set of filter weights. A filter element may be any component of a weight of a filter. A plurality of filter elements may be a plurality of components of respective weights of a filter .

[0149] Generating the respective output audio signal for each of the loudspeakers in the array may comprise: generating a respective intermediate audio signal for each of the control points [e.g., m] by applying the or a first subset of filters [e.g., [GG^1 ] to the input audio signals [e.g., d] ; and generating the respective output audio signal for each of the loudspeakers by applying the or a second subset of filters [e.g., GH] to the intermediate audio signals.

[0150] The set of filters or the first subset of filters [e.g., [GGh]-1] may be determined based on an inverse of a matrix [e.g., [GGh]] containing the plurality of filter elements [e.g., G] .

[0151] The matrix [e.g., [GGH] ] containing the plurality of filter elements [e.g., G] may be regularised prior to being inverted [e.g., by regularisation matrix A].

[0152] The set of filters may be determined based on: in the frequency domain, a product of the or a matrix [e.g., Gh] containing the plurality of filter elements [e.g., G] and the inverse of the or a matrix [e.g., [GGH]] containing the plurality of filter elements [e.g., G] ; or an equivalent operation in the time domain. Implementation of method on disclosed architectures

[0153] As mentioned above, method 900 may be implemented on any of architectures 300, 400, 500, 600, 700 and 800. Examples of how method 900 may be performed on those architectures are set out below.

[0154] Different architectures may split various steps of method 900 between an audio device 301 and a host device 302. The host device and the audio device may each be housed in a separate housing. The array 316 may be housed in a housing of the audio device 301, i.e., the array 316 may be built-in. Alternatively, the array 316 may be housed outside the housing of the audio device 301. The sensor (or sensors) 321 may be housed in a housing of the audio device 301. Alternatively, the sensor (or sensors) 321 may be housed outside the housing of the audio device 301.

[0155] It will be understood that, because the steps of method 900 may be split between multiple devices, at each device, only some steps of method 900 need to be performed. The steps of method 900 that are performed at each of these devices are to be understood as forming methods in their own right. Thus, as but one example, a method may be performed from only steps S901, S905, S906 if, for example, steps S902 to S904 are performed at another device. Similarly, as the audio device 301 and the host device 302 may be manufactured (and sold) separately, the audio device 301 and the host device 302 should be understood as being disclosed both separately and in combination with each other.

[0156] When implementing the method 900 on the first architecture 300 (see Figs. 3A and 3B), the receiving S901 of the first plurality of input signals 414A, the repeating S908 thereof, the generating S906 and the repeating S909 thereof may be performed at the audio device 301. For example, steps S901 / S908 and / or S906 / S909 may be performed at audio DSP processor 313 of audio device 301.

[0157] The determining S904 of the first set of filters 415A may be performed at the audio device 301. For example, step S904 may be performed at the audio parameter processor 312 of audio device 301.

[0158] The determining S902 of the first estimate 411A may be performed, and the first estimate 411A may be received S903 at, the audio device 301. For example, step S902 may be performed at the user tracking processor 311 of audio device 301 and step S903 may be performed at the audio parameter processor 312 of audio device 301.

[0159] The first set of filters 415A may be received S905 by and from the audio device 301. For example, step S905 may be performed at the audio DSP processor 313 of audio device 301.

[0160] When implementing the method 900 on the second architecture 400 (see Figs. 4A and 4B), the receiving S901 of the first plurality of input signals 414A, the repeating S908 thereof, the generating S906 and the repeating S909 thereof may be performed at the audio device 301. For example, steps S901 / S908 and / or S906 / S909 may be performed at the audio DSP processor 313 of audio device 301.

[0161] The determining S904 of the first set of filters 415A may be performed at the host device 302. For example, S904 may be performed at the audio parameter processor 312 of host device 302.

[0162] The determining S902 of the first estimate 411A may be performed, and the first estimate may be received S903 at, the host device 302. For example, step S902 may be performed at the user tracking processor 311 of host device 302 and S903 may be performed at the audio parameter processor 312 of host device 302.

[0163] The first set of filters 415A may be received S905 by the audio device 301 from the host device 302. For example, step S905 may be performed at the audio DSP processor 313 of audio device 301.

[0164] When implementing the method 900 on the third architecture 500 (see Figs. 5A and 5B), the receiving S901 of the first plurality of input signals 414A, the repeating S908 thereof, the generating S906 and the repeating S909 thereof may be performed at the audio device 301. For example, steps S901 / S908 and / or S906 / S909 may be performed at the audio DSP processor 313 of audio device 301.

[0165] The determining S904 of the first set of filters 415A may be performed at the audio device 301. For example, S904 may be performed at the audio parameter processor 312 of audio device 301.

[0166] The determining S902 of the first estimate 411A may performed at the host device 302. The first estimate 411A may be received S903 by the audio device 301 from the host device 302. For example, step S902 may be performed at the user tracking processor 311 of host device 302 and step S903 may be performed at the audio parameter processor 312 of audio device 301.

[0167] The first set of filters 415A may be received S905 by and from the audio device 301. For example, step S905 may be performed at the audio DSP processor 313 of audio device 301.

[0168] When implementing the method 900 on the fourth architecture 600 (see Figs. 6A and 6B), the receiving S901 of the first plurality of input signals 414A, the repeating S908 thereof, the generating S906, and the repeating thereof S909 may be performed at the host device 302 which is communicatively coupled to the audio device 301. For example, steps S901 / S908 and / or S906 / S909 may be performed at the audio DSP processor 313 of host device 302.

[0169] The determining S904 of the first set of filters 415A may be performed at the host device 302. For example, step S904 may be performed at the audio parameter processor 312 of host device 302.

[0170] The determining S902 of the first estimate 411A may be performed, and the first estimate may be received S903 at, the host device 302. For example, S902 may be performed at the user tracking processor 311 of host device 302 and S903 may be performed at the audio parameter processor 312 of host device 302.

[0171] The first set of filters 415A may be received S905 by and from the host device 302. For example, step S905 may be performed at the audio DSP processor 313 of host device 302.

[0172] When implementing the method 900 on the fifth architecture 700 (see Figs. 7A and 7B) , the receiving S901 of the first plurality of input signals 414A, the repeating S908 thereof, the generating S906, and the repeating thereof S909 may be performed at the host device 302 which is communicatively coupled to the audio device 301. For example, steps S901 / S908 and / or S906 / S909 may be performed at the audio DSP processor 313 of host device 302.

[0173] The determining S904 of the first set of filters 415A may be performed at the host device 302. For example, step S904 may be performed at the audio parameter processor 312 of host device 302.

[0174] The determining S902 of the first estimate 411A may be performed at the audio device 301, and the first estimate may be received S903 at the host device 302 from the audio device 301. For example, S902 may be performed at the user tracking processor 311 of audio device 301 (which may be integrated with the user tracking sensor 321) and S903 may be performed at the audio parameter processor 312 of host device 302.

[0175] The first set of filters 415A may be received S905 by and from the host device 302. For example, step S905 may be performed at the audio DSP processor 313 of host device 302.

[0176] When implementing the method 900 on the sixth architecture 800 (see Figs. 8A and 8B), the receiving S901 of the first plurality of input signals 414A, the repeating S908 thereof, the generating S906 and the repeating S909 thereof may be performed at the audio device 301. For example, steps S901 / S908 and / or S906 / S909 may be performed at the audio DSP processor 313 of audio device 301.

[0177] The determining S904 of the first set of filters 415A may be performed at the host device 302. For example, S904 may be performed at the audio parameter processor 312 of host device 302.

[0178] The determining S902 of the first estimate 411A may be performed at the audio device 301, and the first estimate may be received S903 at the host device 302 from the audio device 301. For example, S902 may be performed at the user tracking processor 311 of audio device 301 (which may be integrated with the user tracking sensor 321) and S903 may be performed at the audio parameter processor 312 of host device 302.

[0179] The first set of filters 415A may be received S905 by the audio device 301 from the host device 302. For example, step S905 may be performed at the audio DSP processor 313 of audio device 301.

[0180] In any of the first to sixth architectures, the outputting S907 may be performed at the audio device 301. For example, the outputting may be to the array 316 of loudspeakers of audio device 301. Optionally, output audio signals 413 may be amplified at one or more amplifiers 323 of audio device 301 before S910 is performed.

[0181] In any of the first to sixth architectures, any of the first plurality of input audio signals 414A, the second plurality of input audio signals 414B, or the third plurality of input audio signals 414C may be received from a source device without passing through the host device 302. Input audio signals 414 may alternatively be stored in the content storage 314.

[0182] In any of the first to sixth architectures, when steps (e.g., steps S901 or S906) are repeated, they may be performed at the same device as the device at which they were first performed. More specifically, they may be performed at the same processor as the processor at which they were first performed. Computer implementation

[0183] A block diagram of an exemplary apparatus 1000 for implementing any of the methods described herein, or any portion thereof, such as method 900, is shown in Fig. 10. The exemplary apparatus 1000 may, for example, be used to implement any of device 201, audio device 301, or host device 302.

[0184] The apparatus 1000 comprises a processor 1010 (e.g., a digital signal processor, or a multipurpose processor) arranged to execute computer-readable instructions as may be provided to the apparatus 1000 via one or more of a memory 1020, a communication interface 1030, or an input interface 1050.

[0185] The memory 1020, for example a random-access memory (RAM), is arranged to be able to retrieve, store, and provide to the processor 1010, instructions and data that have been stored in the memory 1020. The communication interface 1030 is arranged to enable the processor 1010 to communicate with other devices (e.g., to receive filters) and / or to communicate with a communications network, such as the Internet. The communication interface 1030 may be the external data communication interface described above. The input interface 1050 is arranged to receive user inputs provided via an input device (not shown) such as a mouse, a keyboard, or a touchscreen. The processor 1010 may further be coupled to a display adapter 1040, which is in turn coupled to a display device (not shown). The processor 1010 may further be coupled to an audio interface 1060 which may be used to output audio signals to one or more audio devices, such as a loudspeaker array (or 'array of loudspeakers', or 'sound reproduction device') 216 / 316. The audio interface 1060 may comprise a digital-to-analog converter (DAC) (not shown), e.g., for use with audio devices with analog input(s).

[0186] The approaches described herein may be embodied on a computer-readable medium. Said computer-readable medium may be a non-transitory computer-readable medium. The computer-readable medium carries computer-readable instructions arranged for execution upon a processor so as to make the processor carry out any or all of the methods described herein.

[0187] The term "computer-readable medium" as used herein refers to any medium that stores data and / or instructions for causing a processor to operate in a specific manner. Such storage medium may comprise non-volatile media and / or volatile media. Non-volatile media may include, for example, optical or magnetic disks. Volatile media may include dynamic memory. Exemplary forms of storage medium include, a floppy disk, a flexible disk, a hard disk, a solid state drive, a magnetic tape, or any other magnetic data storage medium, a CD-ROM, any other optical data storage medium, any physical medium with one or more patterns of holes, a RAM, a PROM, an EPROM, a FLASH-EPROM, NVRAM, and any other memory chip or cartridge. Examples of the disclosure

[0188] There is provided a computer-implemented method 900 of generating audio signals 413 for an array 316 of loudspeakers, the method comprising: receiving S901 a first plurality of input audio signals 414A to be reproduced, by the array, at a respective plurality of control points in an acoustic environment; receiving S905 a first set of filters 415A; generating S906 a respective first output audio signal 413A for each of the loudspeakers in the array by applying the first set of filters to the first plurality of input audio signals; subsequent to receiving the first plurality of input audio signals, repeating S908 the receiving to receive a second plurality of input audio signals 414B; and repeating S909 the generating to generate a respective second output audio signal 413B for each of the loudspeakers in the array by applying the first set of filters to the second plurality of input audio signals.

[0189] There is provided an apparatus configured to perform the method 900, or a computer program comprising instructions which, when executed by one or more processors of a computing system, cause the computing system to perform the method 900, or a computer-readable medium comprising instructions which, when executed by one or more processors of a computing system, cause the computing system to perform the method 900, or a data carrier signal comprising instructions which, when executed by one or more processors of a computing system, cause the computing system to perform the method 900. Interpretation

[0190] Section titles are provided above to ease understanding of the disclosure, and are not to be construed as limiting the scope of the disclosure.

[0191] It will be appreciated that, although various approaches above may be implicitly or explicitly described as 'optimal', engineering involves trade-offs and so an approach which is optimal from one perspective may not be optimal from another. Furthermore, approaches which are slightly sub-optimal may nevertheless be useful. As a result, both optimal and sub-optimal solutions should be considered as being within the scope of the present disclosure.

[0192] Those skilled in the art will recognise that a wide variety of modifications, alterations, and combinations can be made with respect to the examples described herein without departing from the scope of the disclosed concepts. Those skilled in the art will thus recognise that the scope of the invention is not limited by the examples described herein, but is instead defined by the appended claims .

Claims

1. A computer-implemented method of generating audio signals for an array of loudspeakers, the method comprising:receiving a first plurality of input audio signals to be reproduced, by the array, at a respective plurality of control points in an acoustic environment;receiving a first set of filters;generating a respective first output audio signal for each of the loudspeakers in the array by applying the first set of filters to the first plurality of input audio signals;subsequent to receiving the first plurality of input audio signals, repeating the receiving to receive a second plurality of input audio signals; andrepeating the generating to generate a respective second output audio signal for each of the loudspeakers in the array by applying the first set of filters to the second plurality of input audio signals.

2. The method of claim 1, wherein the first set of filters isreceived from a non-real-time process.

3. The method of claim 2, wherein the non-real-time process isperformed by one or more first processors that are not capable ofguaranteeing a response within a specified time constraint.

4. The method of any preceding claim, wherein the receiving of the first plurality of input signals, the repeating thereof, the generating and the repeating thereof are performed by one or more real-time processes.

5. The method of any preceding claim, further comprising, prior to receiving the first set of filters:receiving a first estimate of a position of each of the plurality of control points; anddetermining, based on the first estimate, the first set offilters .

6. The method of claim 5, wherein determining the first set of filters comprises determining one or more audio processing parameters for each of the plurality of control points, wherein the first estimate of the position of a given one of the plurality of control points is between a third position and a fourth position, and wherein determining the one or more audio processing parameters for the given control point comprises:retrieving one or more first stored audio processing parameters for the third position and one or more second stored audio processing parameters for the fourth position; anddetermining the one or more audio processing parameters for the given control point based on the retrieved one or more first stored audio processing parameters and the retrieved one or more second stored audio processing parameters.

7. The method of any of claims 5 to 6, further comprising, priorto receiving the first estimate:determining the first estimate.

8. The method of any of claims 5 to 7, wherein the first estimate is determined based on a signal captured by a sensor, optionally wherein the sensor is an image sensor.

9. The method of claim 8, when dependent on claim 2, wherein thenon-real-time process is performed at a frequency based on asampling frequency of the sensor.

10. The method of any preceding claim, further comprising:subsequent to receiving the second plurality of input audio signals, repeating the receiving to receive a third plurality of input audio signals;receiving a second set of filters; andgenerating a respective third output audio signal for each ofthe loudspeakers in the array by applying the second set of filtersto the third plurality of input audio signals.

11. The method of any of claims 1 to 10, wherein the receiving of the first plurality of input signals, the repeating thereof, the generating and the repeating thereof are performed at an audio device .

12. The method of claim 11, when dependent on claim 5, wherein thedetermining of thefirstsetof filters is performed at the audiodevice .

13. The method of claim 12, when dependent on claim 7, wherein thedetermining of the first estimate is performed at the audio device.

14. The method of claim 12, wherein the first estimate is receivedfrom a host device.15 .The method of claim 11,wherein thefirst set of filters isreceived from a host device.

16. The method of claim 15, when dependent on claim 7, wherein the determining of the first set of filters and of the first estimate are performed at the host device.

17. The method of any of claims 1 to 10, wherein the receiving of the first plurality of input signals, the repeating thereof, the generating, and the repeating thereof are performed at a host device which is communicatively coupled to an audio device.

18. The method of claim 17, when dependent on claim 5, wherein the determining of the first set of filters is performed at the host device .

19. The method of claim 18, when dependent on claim 7, wherein thedetermining of the first estimate is performed at the host device.

20. The method of any of claims 17 to 19, wherein the host deviceand the audio device are each housed in a separate housing.

21. The method of any of claims 17 to 20, wherein the first plurality of input audio signals and second plurality of input audio signals are received from a source device without passing through the host device.

22. The method of any of claims 17 to 21, wherein the host deviceand the audio device are communicatively coupled by an external datacommunication interface, optionally wherein the external data communication interface is a Universal Serial Bus, USB, interface.

23. The method of any of claims 11 to 22, wherein the array ishoused in a or the housing of the audio device.

24. The method of any of claims 11 to 23, when dependent on claim 8, wherein the sensor is housed in a or the housing of the audio device .

25. An apparatus configured to perform the method of any preceding claim, ora computer program comprising instructions which, when executed by one or more processors of a computing system, cause the computing system to perform the method of any preceding claim, ora computer-readable medium comprising instructions which, when executed by one or more processors of a computing system, cause the computing system to perform the method of any preceding claim, ora data carrier signal comprising instructions which, when executed by one or more processors of a computing system, cause the computing system to perform the method of any preceding claim.42

Images