Modeling head-related impulse responses

Abstract

The invention relates to modeling head-related impulse responses, including methods for sound signal filtering. The method includes: generating a filter pair at a specific position specified by an elevation angle and an azimuth angle, the filter pair consisting of a right filter and a left filter; filtering the sound signal by using a right filter; and the sound signal is filtered by a left filter. Generating the filter pair comprises: i) obtaining at least a first set of elevation base function values at the elevation angle; ii) obtaining at least a first set of azimuth base function values at the azimuth angle; iii) generating a right filter with at least a first set of elevation basis function values, at least a first set of azimuth basis function values, right filter model parameters, iv) generating a left filter with at least a first set of elevation basis function values, at least a first set of azimuth basis function values, left filter model parameters.

Description

[0001] This application is a divisional application of PCT international application PCT / EP2020 / 079042, filed on October 15, 2020, entitled “Modeling Head-Related Impulse Response”, which has entered the Chinese national phase as patent application 202080072479.3. Technical Field

[0002] This disclosure relates to sound in the rendering space. Background Technology

[0003] We have two ears to capture sound waves traveling toward us. Figure 1 This diagram illustrates sound waves propagating towards a listener from a specified direction of arrival (DOA) defined by elevation and azimuth angles in spherical coordinates. Along their propagation path towards us, each sound wave interacts with our upper torso, head, outer ear, and the surrounding medium before reaching our left and right eardrums. This interaction results in temporal and spectral changes in the waveforms reaching the left and right eardrums, some of which are DOA-dependent. Our auditory system has learned to interpret these changes to infer various spatial characteristics of the sound waves themselves and the listener's perceived acoustic environment. This ability is called spatial hearing, which involves how we evaluate spatial cues embedded in the binaural signals (i.e., sound signals in the right and left ear canals) to infer the location of auditory events triggered by sound events (physical sound sources) and the acoustic characteristics caused by our physical environment (e.g., a small room, a tiled bathroom, an auditorium, a cave). This human ability (spatial hearing) can then be developed to create spatial soundscapes by reintroducing spatial cues into the binaural signals that lead to spatial perception of sound.

[0004] The main spatial cues include: 1) Angle-related cues: binaural cues (i.e., interaural level difference (ILD) and interaural time difference (ITD)) and monoaural (or spectral) cues; 2) Distance-related cues: intensity and direct-reverberation (D / R) energy ratio. Figure 2 Examples of ITD and spectral cues for sound waves propagating toward the listener are shown. These two figures illustrate the amplitude response of an HR filter pair obtained at 0 degrees elevation and 40 degrees azimuth (data from the CIPIC database: object ID 28. This database is publicly available and can be accessed via the URL www.ece.ucdavis.edu / cipic / spatial-sound / hrtf-data / ). Figure 1 and Figure 2In this paper, the convention of using the positive azimuth direction to the right is used, and this is also the convention used in the rest of the paper. However, some HR filter sets do use another convention (where the positive azimuth direction is to the left). The mathematical representation of the short-term DOA-related time and spectral changes (1-5 ms) of the waveform is the so-called head-related (HR) filter. The frequency domain (FD) representation of these filters is the so-called head-related transfer function (HRTF), and the time domain (TD) representation is the head-related impulse response (HRIR). Binaural rendering methods based on HR filters have been gradually established, in which spatial sound scenes are generated by directly utilizing HR filters at the desired location to filter the sound source signal. This method is particularly attractive for many emerging applications, such as virtual reality (VR), augmented reality (AR), mixed reality (MR), or extended reality (XR), as well as mobile communication systems, where headphones are often used.

[0005] Head-related (HR) filters are typically estimated based on measurements of the impulse response of a linear dynamic system that transforms the raw audio signal (input signal) into left-ear and right-ear signals (output signals) measurable within the ear canal of the listening object at a predefined set of elevation and azimuth angles on a sphere of constant radius from the listening object (e.g., an artificial head, mannequin, or human body). The estimated HR filters are typically set as FIR filters and can be used directly in this format. For efficient binaural rendering, HRTF pairs can be converted to interaural transfer functions (ITFs) or modified ITFs to prevent steep spectral peaks. Alternatively, HRTFs can be described using parametric representations. These parametric HRTFs are easily integrated with parametric multichannel audio encoders such as MPEG surround sound and Spatial Sound Object Coding (SAOC).

[0006] Rendering spatial sound signals to provide a convincing spatial perception of sound at any location in space requires pairs of HR filters for the corresponding locations, and therefore a set of HR filters for finely sampled locations on a 2D sphere. The minimum audible angle (MAA) characterizes the sensitivity of our auditory system as the angular displacement of a sound event. Regarding azimuth location, it is reported that for broadband noise bursts, the MAA is minimum (approximately 1 degree) in the forward and backward directions and much larger (approximately 10 degrees) for lateral sound sources. The MAA in the midplane increases with elevation. It is reported that for broadband noise bursts, the average MAA at elevation is as small as 4 degrees. Currently, several publicly available databases of densely sampled HR filters in space exist, such as the SADIE and CIPIC databases. However, particularly for samples at elevation, none of them fully meet the MAA requirements. Even though the SAIDE dataset for the Neumann KU100 and KEMAR mannequin heads contains more than 8,000 measurements, its sampling resolution at elevation angles between -15 and 15 degrees is only 15 degrees, while according to MAA studies, it requires 4 degrees. Inevitably, angular interpolation of the HR filter is needed to render the sound source at locations where the actual filter has not been measured. Figure 3 An example of sampling a mesh on a 2D sphere is shown, where each point indicates the location where the HR filter is measured.

[0007] Several different interpolation schemes have been developed for angular interpolation of HR filters. Typically, based on the angle interpolation on the sphere... Measurement results at the location are estimated Each HR filter pair Where r represents the right ear and l represents the left ear. Indicates the angle of elevation. This represents the azimuth angle. The task is to find the function... ,in It provides left and right filters that deliver sound rendering with good perceptual accuracy at unsampled angles. Once obtained Then you can be in the Generates left and right ear HR filters at any specified location. Note that the superscripts l or r are sometimes omitted for simplicity and to avoid confusion.

[0008] The two main methods for HRTF angle interpolation are as follows:

[0009] (1) Local Neighbor Method: The commonly used method is linear interpolation, in which the missing HRTF is inferred by weighting the contributions of the HRTFs measured at the nearest surrounding locations. The HRTFs can be preprocessed before interpolation, for example, by first converting the HRTFs measured at two or more nearest locations into minimum phase, and then applying linear interpolation.

[0010] (2) Methods of transformation: A more complex data-driven approach is to linearly transform the measured HRTF to another space defined by a set of basis functions, one set of which covers the elevation and azimuth dimensions and another set covers the frequency dimension. The basis functions can be obtained by eigenvalue decomposition of the covariance matrix of the measured HRTF [1,2]. In [3], complete and orthogonal spherical harmonic functions (SH) on a 2D sphere have been used to cover the elevation and azimuth dimensions, and complex exponential functions have been used to cover the frequency dimension. SH-based HRTF models have produced encouraging performance levels in terms of mean squared error (MSE) and perceived loudness stability [4]. Summary of the Invention

[0011] The ability to accurately and efficiently render spatially located sound sources is one of the key features of a spatial sound renderer based on HR filters. The spatial resolution of the HR filter set used in the renderer determines the spatial resolution of the rendered sound source. Using a coarsely sampled HR filter set on a 2D sphere, VR / AR / MR / XR users often report spatial discontinuities in moving sounds. These spatial discontinuities lead to audio-video synchronization errors, which significantly reduce immersion. Using a finely sampled HR filter set on a sphere is one solution. However, estimating the HR filter set based on input-output measurements on a fine grid that meets MAA requirements can be very time-consuming and tedious for both the subject and the experimenter. Alternatively, a more efficient approach is to infer spatially relevant information associated with missing HR filters given a sparsely sampled HR filter dataset.

[0012] The nearest-neighbor HR filter interpolation method assumes that the HR filter at each sampling location affects only a finite distance region. Then, the HR filter at unsampled locations is approximated as a weighted average of the HR filters at locations within a certain cutoff distance, or approximated based on a specified number of nearest points on a linear 2D grid. For example... ,in, At the unsampled location The estimated HR filter vector, and This method is simple and computationally inexpensive, leading to efficient implementations. However, the interpolation accuracy may not be sufficient to produce convincing spatial sound scenes. This is simply due to the fact that the variations between sample points are more complex than what a weighted average of filters can produce.

[0013] The method of change represents the HR filter as a linear combination of the basis function set (i.e. ,in, It is the first basis functions (The coefficients). Regardless of the basis functions, the coefficients are typically obtained by making the measured set of points... The least squares estimate is obtained by minimizing the sum of squared estimation errors, i.e. Given a set of basis functions, the coefficients are considered to be the "best" fit in the sense of solving a quadratic minima problem. In principle, there are no restrictions on the choice of basis functions. However, in practice, a practical choice can efficiently represent the basis function set of the HR filter set in terms of estimation accuracy, as well as in terms of the number and complexity of the basis functions.

[0014] Early work on modeling the HRTF amplitude response used principal component analysis (PC) as basis functions, where the PCs were obtained by eigenvalue decomposition of the covariance matrix of the HRTF amplitude response measured from 10 listeners at 265 source locations. Using only five PCs, the resulting model achieved nearly 90% of the variance in the original database. This model was efficient, representing the original dataset well without requiring interpolation of the HRTF at missing locations. Recently, a hybrid approach combining principal component analysis (PCA) with the nearest neighbor method was proposed, where model coefficients are approximated by partial derivatives. However, this hybrid approach only achieved results similar to nearest neighbor-based bilinear interpolation.

[0015] SH has been used to model the angular correlation of HRTF sets. The resulting model yielded encouraging performance levels in terms of mean squared error (MSE). However, unlike the basis functions in eigenvalue-based models (which are fixed PC vectors), SH basis functions are complex and costly to evaluate. Second-rate The order SH function is written as . It is a related Legendre polynomial, which is essentially... The second-order triangular polynomial. For the entire model, the highest value to be evaluated is... Rank SH.

[0016] To achieve high spatial resolution, the order of the SH representation should be as high as possible. The impact of the SH order on spatial aliasing has been investigated within the context of perceived spatial loudness stability, defined as how stable the loudness of a rendered sound scene is perceived across different head orientations. Subjective results indicate the need to prioritize higher orders (…). >10) SH HRTF indicates a technology used to facilitate high-quality, dynamic virtual sound scenes. This led to There are coefficients, among which... Corresponding to the number of frequency bins. Another study also modeled the frequency component of the HRTF using complex exponents, and the total number of coefficients is ,in, This refers to the truncated portion of the frequency representation. The results show that, in order to represent the HRTF across the entire frequency range up to 20kHz in terms of MSE, the order of SH needs to reach a level comparable to... The same size, and the truncated frequency portion is The number of coefficients is 38,440. Using such a high-order SH HRTF model to evaluate HRTF is practically impossible in real-time VR / AR / MR / XR systems.

[0017] This disclosure provides a process for generating HR filters that are sufficiently accurate and efficient for real-time VR / AR / MR / XR systems at any arbitrary location in space. In one embodiment, a variational approach is employed, wherein the spatial variation of the HR filter set is modeled using B-spline basis functions, and the filters are parameterized according to a time-domain FIR filter or some mapping in the frequency domain, where DFT is such a mapping. The resulting model is accurate in terms of MSE measurement and perception evaluation. It is efficient in that the total number of basis functions and computational cost required to evaluate the HR filters based on this model are far less than those required for models using spherical harmonic functions or other such complex basis functions.

[0018] Therefore, in one aspect, a method for filtering audio signals is provided. The method includes: generating a signal for filtering based on the elevation angle. and azimuth angle A filter pair at a specified location, the filter pair consisting of a right filter ( ) and left filter ( The method further includes: filtering the audio signal using the right filter and filtering the audio signal using the left filter. Generating the filter pair includes: i) obtaining at least a first elevation angle basis function value set at the elevation angle; ii) obtaining at least a first azimuth angle basis function value set at the azimuth angle; iii) generating the right filter using: a) at least the first elevation angle basis function value set, b) at least the first azimuth angle basis function value set, and c) right filter model parameters; and iv) generating the left filter using: a) at least the first elevation angle basis function value set, b) at least the first azimuth angle basis function value set, and c) left filter model parameters.

[0019] In another aspect, a filtering device for filtering audio signals is provided. The filtering device is adapted to perform a method comprising: generating an image from an elevation angle. and azimuth angle A filter pair at a specified location, the filter pair consisting of a right filter ( ) and left filter ( The method further includes: filtering the audio signal using the right filter and filtering the audio signal using the left filter. Generating the filter pair includes: i) obtaining at least a first elevation angle basis function value set at the elevation angle; ii) obtaining at least a first azimuth angle basis function value set at the azimuth angle; iii) generating the right filter using: a) at least the first elevation angle basis function value set, b) at least the first azimuth angle basis function value set, and c) right filter model parameters; and iv) generating the left filter using: a) at least the first elevation angle basis function value set, b) at least the first azimuth angle basis function value set, and c) left filter model parameters.

[0020] The main advantages of the proposed process include: a) it is more accurate than solutions based on bilinear PC, b) it is more efficient than solutions based on SH, c) model building does not require intensive sampling of the HR filter database, and d) the model occupies less memory space compared to the original HR filter database. These advantages make the proposed embodiment attractive for real-time VR / AR / MR / XR systems. Attached Figure Description

[0021] The accompanying drawings, which are incorporated herein and form part of the specification, illustrate various embodiments.

[0022] Figure 1 This illustrates sound waves propagating toward the listener from the specified direction of arrival (DOA) using elevation and azimuth angles in spherical coordinates.

[0023] Figure 2 An example of the ITD and spectral clues of a sound wave propagating toward the listener is shown.

[0024] Figure 3 An example of sampling a mesh on a 2D sphere is shown.

[0025] Figure 4 An HR filtering unit according to an embodiment is shown.

[0026] Figure 5 This is a flowchart illustrating one embodiment of HR filter modeling.

[0027] Figure 6 This is a flowchart illustrating the preprocessing steps for obtaining a zero-delay HR filter and an ITD, as described in the embodiments.

[0028] Figure 7A The delay estimates for the right ear HRTF (solid line) and left ear HRTF (dashed line) are shown on a horizontal plane with an elevation angle of 0 degrees and an azimuth angle from 0 degrees to 360 degrees.

[0029] Figure 7B The corresponding right ear HRTF (solid line) and left ear HRTF (dashed line) at a 90-degree azimuth angle are shown.

[0030] Figure 8 A block diagram depicts the modeling process according to an embodiment.

[0031] Figure 9 An example of B-spline basis functions is shown.

[0032] Figure 10 An example of a periodic basis function is shown.

[0033] Figure 11 The process according to an embodiment is shown.

[0034] Figure 12 An example of a periodic B-spline basis function is shown.

[0035] Figure 13A An example of B-spline basis functions is shown.

[0036] Figure 13B An example of standard B-spline basis functions is shown.

[0037] Figure 14A Another example of B-spline basis functions is shown.

[0038] Figure 14B The standard B-spline basis functions without smoothing conditions at 0 / 180 degrees of the knot-point are shown.

[0039] Figure 15A model representation of an HR filter dataset according to one embodiment is shown.

[0040] Figure 16 This is a block diagram of a system for generating zero-delay HR filter pairs and corresponding ITDs according to one embodiment.

[0041] Figure 17 An example is shown for positioning a given HR filter model representation. The process of generating zero-delay HR filter pairs.

[0042] Figure 18 An example is shown for positioning when given an ITD model representation. The process of generating ITD.

[0043] Figure 19 This is a flowchart illustrating a process according to an embodiment.

[0044] Figure 20 This is a flowchart illustrating a process according to an embodiment.

[0045] Figure 21 This is a block diagram of an HR filter device 2100 according to one embodiment. Detailed Implementation

[0046] Figure 4 An HR filtering unit 400 according to an embodiment is shown. The HR filtering unit 400 includes a rendering unit 402. The unit 400 further includes an HR filter generator 404 and an ITD generator 406 that generate HR filters and ITDs respectively at arbitrary elevation and azimuth angles requested in real time by the rendering unit 402. This requires efficient evaluation of left and right HR filter pairs based on HR filter models already loaded into the unit 400. It also requires efficient evaluation of ITDs based on ITD models already loaded into the unit. The HR filtering unit 400 will therefore have an interface 408 for loading HR filter models and ITD models from a database 410 of these models. The database of HR filter models is generated offline by estimating HR filter models from different HR filter databases.

[0047] 1. HR filter set modeling

[0048] As mentioned above, the HR filter is a mathematical representation of the angle-dependent spatial cues, including ITD, ILD, and spectral cues. ITD is defined as the time difference between the arrival times of the sound signal at both ears, such as... Figure 2As shown, we remove the frequency-independent time delay from the HR filters and retain it separately as the pure delay for each pair of HR filters. The remaining zero-delay HR filters contain the inter-ear phase difference (IPD), ILD, and spectral cues. The filters and ILD are modeled separately for azimuth and elevation angles.

[0049] The variation is performed using existing HR filter databases, most of which are publicly available. The HR filters in these databases are estimated based on acoustic measurements performed on different spatial sampling grids and are typically stored in different file formats, which is naturally advantageous for the lab providing the databases. Recently, a spatially oriented acoustic format (SOFA) has been developed for self-describing data with a consistent definition, unifying the representation of the different HR filter databases. Therefore, in one embodiment, the SOFA format is used, thus eliminating the additional effort required for exchanging data formats before modeling. More information about the SOFA format can be found at www.sofaconventions.org / mediawiki / index.php.

[0050] Figure 5 A flowchart illustrating an embodiment of HR filter modeling is provided, in which an HR filter set in SOFA format is loaded via the SOFA API. In the preprocessing unit, if frequency-independent delays are not provided in the raw database, this information is estimated for each HR filter. The HR filters are then split into zero-delay HR filters and ITDs. Finally, when modeling the unit, the zero-delay HR filters and ITDs are modeled as linear sums of continuous basis functions for elevation and azimuth angles, respectively.

[0051] The steps of preprocessing, HR filter model estimation, and ITD model estimation are described in more detail in the first three sections. A description of the entire model representation is then given.

[0052] 1.1 Preprocessing

[0053] The basic process for estimating the HR filter set based on measurement results includes the following steps:

[0054] (1) By placing at a specified elevation angle Azimuth And a known signal is emitted by a speaker located at a fixed distance from the object's head;

[0055] (2) Record the left and right ear signals of the subject using a microphone placed in or at the entrance of the subject's ear canal;

[0056] (3) Post-processing of the recorded raw data, primarily used to remove the response of the measurement system; and

[0057] (4) Using the known loudspeaker signal as the input signal and the preprocessed ear signal as the output signal, the HR filter as the impulse response of the linear dynamic system is estimated based on the preprocessed data.

[0058] There is typically a frequency-independent delay before the impulse response begins (onset). Some databases (e.g., the CIPIC database) provide onset information. However, most databases do not. As mentioned above, the HR filter set can be modeled as a combination of a minimum-phase system and a pure delay line. Delay estimation is required in this case. Given delay information, the ITD is simply calculated by subtracting the delay of the left-ear HR filter from the delay of the right-ear HR filter. Then, the delay is removed by windowing the HR filters, resulting in a zero-delay HR filter. Figure 6 The diagram shows a flowchart describing the preprocessing steps used to obtain the zero-delay HR filter and ITD.

[0059] In the time structure of HRIR, it is easy to observe a sudden increase in amplitude after the onset. Based on this temporal characteristic, one method for estimating the delay is to use an onset detection function that follows the energy envelope of the impulse response (IR). This onset detection function can be constructed as follows: ,in, yes A windowing function of sample length and This is the time step in samples between the two windows. To avoid blurring and for simplicity, the angle parameters and ear representations are omitted here. The length of the window can be chosen as the length of the portion covering 90% of the total energy of the HRIR. The above solution yields satisfactory results when strong shock transients exist in the HRIR. However, this is not always the case, therefore, by using the ratio of accumulated energy to total energy... Optimize the solution, wherein, It is the length of the HRIR. The accumulated energy is defined as... ,in, yes A window at a point. The total energy is Another optimization uses the derivative of the ratio and finds the index of the first sample when the derivative exceeds a certain threshold as the starting index (onset). The delay is measured in samples. It can be written as ,in It is a threshold. Generally, the threshold of the same-side HRTF is higher than that of the contralateral HRTF. Figure 7A and Figure 7B This example shows an estimate of HRTF latency using the Princeton HRTF dataset – object ID 27 (the database URL is www.princeton.edu / 3D3A / HRTFMeasurements.html). Figure 7A The curves in the figure show the delay estimates for the right ear HRTF (solid line) and left ear HRTF (dashed line) on a horizontal plane with an elevation angle of 0 degrees and an azimuth angle from 0 degrees to 360 degrees. The data hint shows the HRTF delay at a 90-degree azimuth angle. Figure 7B The diagram shows the right ear HRTF (solid line) and left ear HRTF (dashed line) at a 90-degree azimuth angle. An asterisk (*) highlights the detected onset.

[0060] Given a delay estimate, a zero-delay HR filter can be obtained by windowing the original HR filter. The most obvious position-dependent effect on the spectral content of the HR filter is traced to the outer ear or auricle, lasting approximately 0.3 milliseconds. The "shoulder bounce" effect appears later. The total length of the position-dependent IR typically does not exceed 1 millisecond. Therefore, a 1-millisecond rectangular window is long enough to preserve the main spectral-dependent cues. A longer window is not necessary if no additional position-dependent information is added.

[0061] 1.2 HR Filter Model Estimation

[0062] The HR filters for the right and left ears are modeled separately. The lengths are given below in two possible extended forms (elevation extended form and azimuth extended form). HR filter The general time-domain truncated (TD) FIR model, in which separate basis functions are used for elevation and azimuth.

[0063]

[0064] In the elevation angle extended form, there exists a single set of basis functions for the elevation angle dimension. And for azimuth dimension There are sets of basis functions, each set corresponding to an elevation angle index. , . It is the filter parameter vector The number of basis vectors in a dimensional vector space, and It is a length of Orthogonal basis vectors:

[0065] , , .

[0066] This is the set of model parameters that need to be estimated. Because the FIR model is truncated, Therefore, the HR filter model value equal .

[0067] The azimuth extension is a mirror image of the elevation extension, and has corresponding mirror terminology. From now on, we will show the properties of the elevation extension. These properties also hold true for the azimuth extension in a mirror sense, and those skilled in the art can derive those mirror properties based on these properties of the elevation extension.

[0068] The flexibility of the elevation angle extension form lies in its support for each elevation angle index. The azimuth basis functions are set by set. This comprehensive flexibility isn't always necessary, but using more than one set of azimuth basis functions is certainly a good idea. At elevation angles of + / -90 degrees directly above and below the listener, the HR filter is the same at different azimuth angles. This can be handled by using a single azimuth basis function equal to 1 for the elevation index p, which has basis functions contributing to elevation + / - 90 degrees. Other elevation indices can share a single, different set of azimuth basis functions, where the number of basis functions... Alternatively, several sets of carefully selected azimuth basis functions can be shared to capture the elevation-azimuth variation of the modeled filter set.

[0069] Below we will derive the properties of the general elevation angle extended form. However, those skilled in the art will understand that when the number of different sets of azimuth basis functions is less than... How to modify these properties.

[0070] To estimate model parameters Two things are needed.

[0071] (1) A minimization criterion needs to be specified, which is usually in the form of a measurement of the modeling error in the time domain, frequency domain, or a combination of both, and this criterion may even include a regularization term to reduce the tendency of overfitting the modeled data.

[0072] (2) Optimization methods for estimating the parameters that minimize the minimization criterion.

[0073] Figure 8 It describes the given and corresponding elevation and azimuth angles (i.e. The associated set of zero-delay HR filters The flowchart illustrates the modeling process. Given a list of elevation and azimuth angles, basis functions are constructed for the elevation and azimuth angles, respectively. Then, the least squares method is used to estimate the model parameters.

[0074] The model estimation process is described in more detail in section 1.2.1.

[0075] At this stage, the model specification is quite general because the two sets of basis functions have not yet been specified. and The key to obtaining accurate modeling and efficient evaluation of the HR filter model lies in the selection of these two sets of basis functions. After experimenting with different types of functions, we chose to use the basis functions we call periodic B-spline basis functions as the azimuth basis functions and the standard B-spline functions as the elevation basis functions. These chosen basis functions are explained in more detail in Section 1.2.2.

[0076] 1.2.1 Model Parameter Estimation

[0077] Given a set of basis functions and And in A set of zero-delay HR filters sampled from the right or left ear at different angle positions. The typical minimization criterion in the time domain is The sum of the norms of the modeling errors on the set of HR filters (right ear or left ear):

[0078]

[0079] in,

[0080] and

[0081] .

[0082] The number of parameters estimated is It should be far less than the number of available data samples. To avoid uncertain systems.

[0083] Because of the orthonormal basis vectors Since they are orthogonal vectors, the parameters... It can be done for each sample Solve independently. For each sample... The minimization criterion becomes:

[0084] .

[0085] It can be represented in matrix form as follows:

[0086] ,

[0087] in,

[0088] .

[0089] It is the linear least squares criterion. It is achieved by solving the standard equations. Obtain The minimized solution. However, directly minimizing the above cost function leads to an exact solution for the linear system. This solution is applicable to the data. The noise in the model is sensitive and can lead to overfitting. Then, Tikhonov regularization is applied, and the minimization criterion becomes:

[0090] .

[0091] in The size is The identity matrix and It has A zero-column vector with 0 elements.

[0092] This is also a linear least squares criterion. Similarly, it is achieved by solving the standard equation. Obtain The minimized solution, where can be The value is determined to make the matrix The condition number is less than 10 or some other value that leads to good model accuracy.

[0093] To achieve better numerical accuracy, by means of Singular Value Decomposition (SVD) Perform the actual solution:

[0094]

[0095] List and Spanning the same subspace. In orthogonal matrix The projection on is by Given and equal to This is obtained. The solution was subsequently obtained. This estimation is very efficient because it only requires a small-dimensional matrix. An SVD is then used to evaluate against The solution can be obtained in parallel. Utilizing... replace And utilize replace , which holds for as well.

[0096] Given the right-ear HR filter measurement results , we obtain the set of model parameters represented by , where each is a column vector of dimension . Similarly, given the left-ear HR filter measurement results , we obtain the set of model parameters represented by , where each is a column vector of dimension .

[0097] Specify the minimization criterion and in the time domain. It is easy to map them to the frequency domain as follows: Use the DFT transform or similar processing (e.g., interaural transfer function (ITF)) to map the time-domain vectors and into frequency-domain vectors, and the alternative criterion can easily use a combination of time-domain components and frequency-domain components.

[0098] Define the squared norm of the vector as the inner product of the vector with itself . The general form of the inner product is , where can be any positive definite matrix and its simplest form​​​​​​​​​​​​​​​​​​​​​​​ (in , The subintervals of these polynomial functions are specified by ) , In each subinterval, each basis function is... A polynomial function of degree 1, written as:

[0102] .

[0103] The smoothness of the function (which is a linear sum of B-spline basis functions) at the nodes is achieved using so-called multiple sequences. Control is performed on a sequence of integers greater than 0, where the values ​​are... Indicates at node place The first derivative is continuous. This means that... To give the maximum smoothness, and Only the continuity of the 0th derivative is given. Given a sequence of nodes and a multisequence, obtain the coefficients of the polynomial model iteratively, starting with a 0th-degree polynomial. The details of this process can be found in Carl de Boor's article "Bspline Basics" (ftp: / / ftp.cs.wisc.edu / Approx / bsplbasic.pdf).

[0104] exist Figure 9 The diagram shows the use of node sequences. and multiple sequences The assessment targets An example of a B-spline basis function for an elevation angle of degrees.

[0105] exist The azimuth angle in degrees is spatially related to (for any integer value) The meaning of points having the same azimuth angle is that the azimuth angle is periodic (e.g., cyclic), and in order to achieve efficient modeling in the azimuth dimension, periodic basis functions (i.e., ...) are used in the same way. It is very important. Figure 10 An example of such a periodic basis function is shown, where the function is plotted as a series of lines from 0 to 100. arrive The angle range is shown, and the part of the function outside this range is drawn using dashed lines.

[0106] We have designed a method for generating a set of periodic B-spline basis functions over an azimuth range of 0 to 360 degrees. Figure 11 The method is illustrated in the figure, which includes the following steps.

[0107] (Step 1) Specify from arrive A sequence of nodes within a degree range. The length of this node sequence is expressed as... .

[0108] (Step 2) Use less than degree The sum of the values ​​is greater than degree The value expands the node sequence in a periodic manner.

[0109] (Step 3) Use the extended node sequence and the extended multisequence of the node to generate an extended set of B-spline basis functions, using the standard method to generate the set of B-spline functions.

[0110] (Step 4) Select the extended basis functions starting from index 2. A series of extended basis functions, and mapping them periodically to... arrive The azimuth range in degrees.

[0111] This method provides arrive Within the range of degrees A set of periodic basis functions.

[0112] Each basis function in the azimuth angle is also A polynomial function of degree 1, and is written as:

[0113] .

[0114] Figure 12 The text shows the use of a length of... Node sequence The target of the evaluation An example of a periodic B-spline basis function for an azimuth angle of degrees.

[0115] 1.3 ITD Model Estimation

[0116] The general form of the ITD model is given by the following equation:

[0117] .

[0118] and These are B-spline basis functions for elevation and azimuth angles, respectively. It is the set of model parameters.

[0119] 1.3.1 Model Parameter Estimation

[0120] The model parameters are obtained by minimizing the least squares criterion. ,

[0121] ,

[0122] in,

[0123] .

[0124] It is aimed at ITD. It is a frequency-independent delay, either provided by the original database or estimated using the methods described in Section 1.1.

[0125] Applying Tikhonov regularization to avoid overfitting, the minimization criterion becomes:

[0126] ,

[0127] in, The size is The identity matrix and It has A zero-column vector with 0 elements.

[0128] Can The value is determined to make the matrix The condition number is less than 10 or some other value that leads to good model accuracy. As described in Section 1.2.1, by means of SVD at that time, through Obtain the model parameters, which are... A column vector of n elements.

[0129] 1.3.2. Definition of Basis Functions for Elevation and Azimuth

[0130] As the elevation angle moves from -90 degrees upwards to 90 degrees, the ITD increases from zero to its maximum value at 0 degrees elevation and then decreases back to zero. Based on this, it is natural to use basis functions that are zero at + / -90 degrees elevation. This requirement is equivalent to at least one smoothing condition at + / -90 degrees elevation. As explained in Section 1.2.2, the smoothing of the function at the nodes is achieved by multiple sequences. Control. Write each basis function as:

[0131] .

[0132] exist Figure 13A The diagram shows the use of node sequences. and multiple sequences The assessment targets An example of a B-spline basis function for an elevation angle of degrees.

[0133] Considering that the ITD may not be exactly zero at + / -90 degrees elevation angle due to asymmetry in the measurement setup and the object itself, using standard B-spline basis functions without smoothing conditions at the nodes + / -90 degrees remains a good choice. Figure 13B The diagram shows the use of node sequences. and multiple sequences The assessment targets An example of a standard B-spline basis function for an elevation angle of degrees.

[0134] As the azimuth moves along a circle, the change in ITD follows a sinusoidal shape, with zero ITD occurring at azimuths of 0 / 180 / 360 degrees and maximum ITD occurring at azimuths of 90 / 270 degrees. Similarly, a smoothing condition is satisfied at azimuths of 0 / 180 / 360 degrees. Furthermore, the ITD at azimuths between 180 and 360 degrees can be considered a mirror image of the ITD at azimuths between 0 and 180 degrees. Therefore, we use a set of basis functions for azimuth angles in two intervals [0, 180] and [180, 360]. Each basis function is written as:

[0135]

[0136] exist Figure 14A The diagram shows the initial use of node sequences. and multiple sequences The assessment targets An example of the B-spline basis function for the azimuth angle in degrees.

[0137] Considering that the ITD may not be exactly zero at azimuth angles of 0 / 180 / 360 degrees, standard B-spline basis functions can be used without smoothing conditions at the nodes of 0 / 180 degrees. Figure 14B An example of such a basis function is shown in the figure.

[0138] 1.4 Model Representation

[0139] Figure 15 A model representation of the HR filter dataset is shown. This representation includes a zero-delay HR filter model representation and an ITD model representation, both comprising basis functions and model parameters. The key to the modeling accuracy and computational efficiency of the modeling solution is the careful construction of a set of B-spline basis functions for modeling the angular variations of the HR filter set, which is simple enough to provide good computational efficiency and rich enough to provide good modeling accuracy.

[0140] For the zero-delay HR filter model, the following holds: Each elevation angle B-spline basis function; all contain Azimuth B-spline basis functions of a function A set; and all of them are sets; take The matrix represents two sets of model parameters. For the ITD model, there exists: Each elevation angle B-spline basis function; all contain Azimuth B-spline basis functions of a function A set; and as a set having A set of model parameters for a vector of elements.

[0141] Each set of B-spline basis functions consists of a sequence of nodes and polynomial model coefficients as a three-dimensional array. The first dimension corresponds to the order of the B-spline, the second dimension corresponds to the number of node intervals, and the third dimension corresponds to the number of basis functions.

[0142] or The number of elevation angles is much smaller than the number in the original HR filter dataset. or It is much smaller than the number of azimuth angles in the dataset. It is also smaller than the length or number of frequency bins of the original filter. Therefore, this model representation is efficient in representing HR filter datasets.

[0143] Furthermore, because the angle basis functions are continuous, the model representation can be used to generate HR filter pairs at any arbitrary location specified by the elevation and azimuth angles.

[0144] 2. HR filter generation

[0145] Figure 16 This is a block diagram of a system for generating zero-delay HR filter pairs (i.e., right-ear and left-ear filters) and their corresponding ITDs given a model representation. The model representation can be written to a binary file or a text file. The file is loaded via an API to obtain the model structure. The following describes how to use this model representation to obtain the HR filter pairs and ITDs at a specified location.

[0146] 2.1 Generating a zero-delay HR filter

[0147] Figure 17 This shows the position when given an HR filter model representation. The process of generating zero-delay HR filter pairs. As described in Section 1.2.2, the model of the elevation angle B-spline basis function set. Includes: node sequence Its specified sub-interval The function is a polynomial over the subinterval; and a 3D array indicating the model parameters. Involves assessment The elevation angle basis functions at the elevation angle angle The value at ( The steps are as follows:

[0148] (1) Find the satisfaction index ;as well as

[0149] (2) Evaluate the following formula: Elevation angle B-spline basis function at elevation angle Value at:

[0150] .

[0151] A similar process can be used to evaluate the set of azimuth B-spline basis functions at a given azimuth angle. value at ( ).

[0152] Once these sets of basis function values ​​are obtained, the position is obtained according to the following formula. Right ear zero-delay HR filter:

[0153] .

[0154] Based on this, the following formula is used to obtain the... The assessment is also clear:

[0155] .

[0156] The location is obtained according to the following formula. Left ear zero-delay HR filter:

[0157] .

[0158] The following formula is used to obtain the pair. Assessment:

[0159] .

[0160] 2.2 Generating ITD

[0161] Figure 18 An example is shown for positioning when given an ITD model representation. The process of generating ITD.

[0162] Following the process described in section 2.1, evaluate separately. The elevation angle basis functions at the elevation angle angle The value of the B-spline basis function set at a given azimuth angle and the value of the azimuth angle at a given azimuth angle value at ( Once the values ​​of the elevation and azimuth basis functions have been evaluated, the ITD is obtained according to the following formula:

[0163] .

[0164] As mentioned in section 5.1.1, we model the HR filter set as a combination of a minimum-phase system and a pure delay line. The delay of the right ear HR filter is:

[0165] .

[0166] The delay of the left ear HR filter is:

[0167] .

[0168] Please note, and The calculations should be consistent with the definition and coordinate system of the ITD used.

[0169] Figure 19 This is a flowchart illustrating process 1900 according to an embodiment. Process 1900 may begin at step s1902.

[0170] Step s1902 includes: generating a value for the elevation angle. and azimuth angle A filter pair at a specified location, the filter pair consisting of a right filter ( ) and left filter ( )constitute.

[0171] Step s1904 includes: filtering the audio signal using a right filter.

[0172] Step s1906 includes: filtering the audio signal using a left filter.

[0173] like Figure 20As shown, step s1902 includes: i) obtaining at least a first set of elevation basis function values ​​at the elevation angle (step s2002); ii) obtaining at least a first set of azimuth basis function values ​​at the azimuth angle (step s2004); iii) generating the right filter using: a) at least the first set of elevation basis function values, b) at least the first set of azimuth basis function values, and c) right filter model parameters (step s2006); and iv) generating the left filter using: a) at least the first set of elevation basis function values, b) at least the first set of azimuth basis function values, and c) left filter model parameters (step s2008).

[0174] In some embodiments, obtaining the first azimuth basis function value set includes: obtaining azimuth basis function values. A set, wherein the azimuth basis function values The set includes the first azimuth basis function value set. ,as well as ,in, ( =1 to , =1 to ,as well as =1 to ) is the set of parameters for the left model. ( =1 to , =1 to ,as well as =1 to ) is the set of parameters for the right model. ( =1 to Defined in elevation angle The set of basis function values ​​for the first elevation angle at that location, and ( =1 to ,as well as =1 to Defined in azimuth angle azimuth basis function values ​​at a set; and ( =1 to ) is of length The set of orthogonal basis vectors.

[0175] In some embodiments, obtaining the first set of elevation angle basis function values ​​includes: obtaining the elevation angle basis function values. A set, wherein the elevation angle basis function values The set includes the set of the first elevation angle basis function values. ,as well as ,in, ( =1 to , =1 to ,as well as =1 to ) is the set of parameters for the left model. ( =1 to , =1 to ,as well as =1 to ) is the set of parameters for the right model. ( =1 to ,as well as =1 to Defined in elevation angle The elevation angle basis function value at that location a set, and ( =1 to Define azimuth angle The set of first azimuth basis function values ​​at; and ( =1 to ) is of length The set of orthogonal basis vectors.

[0176] In some embodiments, obtaining the first set of elevation basis function values ​​includes: for each elevation basis function included in the first set of elevation basis functions, evaluating the elevation basis function at the elevation angle to generate an elevation basis function value corresponding to the elevation angle and the elevation basis function; and obtaining the first set of azimuth basis function values ​​includes: for each azimuth basis function included in the first set of azimuth basis functions, evaluating the azimuth basis function at the azimuth angle to generate an azimuth basis function value corresponding to the azimuth angle and the azimuth basis function.

[0177] In some embodiments, each elevation basis function included in the first elevation basis function set is a B-spline basis function, and each azimuth basis function included in the first azimuth basis function set is a periodic B-spline basis function.

[0178] In some embodiments, the process further includes: obtaining a model that at least represents the first set of elevation angle basis functions, wherein the model includes: a sequence ( ),in Its specified sub-interval }, where the elevation angle basis function is a polynomial over the subinterval; and a three-dimensional array of model parameters ( ).

[0179] In some embodiments, the first set of elevation angle basis functions includes the first... Elevation basis functions, evaluating each elevation basis function included in the first set of elevation basis functions at the elevation angle. Including: the elevation angle Assessment No. The elevation angle basis function, and in the elevation angle angle Evaluation of the first The elevation angle basis function includes the following steps: finding the basis function that satisfies the following conditions. The index u; and according to The assessment of the first The elevation angle basis function at the elevation angle The value at that location.

[0180] In some embodiments, the process further includes: obtaining a model that at least represents the first azimuth basis function set, wherein the model includes: a sequence ( ),in Its specified sub-interval The azimuth basis function is a polynomial over the subinterval; and the model parameters are a three-dimensional array. .

[0181] In some embodiments, the first azimuth basis function set includes the first Azimuth basis functions, in azimuth angle The evaluation of each azimuth basis function included in the first azimuth basis function set includes: in the azimuth angle... The assessment of the first Azimuth basis functions, and in azimuth angle The assessment of the first The azimuth basis functions include the following steps: finding those that satisfy... index ; and according to The assessment of the first Azimuth basis functions at this azimuth angle The value at that location.

[0182] In some embodiments, the process further includes: generating at least a first azimuth basis function set, wherein generating the first azimuth basis function set includes: generating a set of periodic B-spline basis functions over an azimuth range of 0 to 360 degrees. In some embodiments, generating the set of periodic B-spline basis functions over the azimuth range of 0 to 360 degrees includes: specifying arrive The length is within the range of degrees. A sequence of nodes; based on a length of The node sequence is used to generate an extended node sequence, wherein generating the extended node sequence includes: using a node sequence smaller than... degree The sum of the values ​​is greater than degree Each value expands in a periodic manner with a length of The node sequence; obtaining the extended multisequence of the nodes; generating an extended B-spline basis function set using the extended node sequence and the extended multisequence; selecting the extended basis functions starting from index 2. A continuous set of extended basis functions; and, periodically mapping the selected extended basis functions to... arrive The azimuth range in degrees.

[0183] In some embodiments, the process further includes: for elevation-azimuth angles Determine the time difference between ears ( In some embodiments, the process further includes: based on Determine right delay ; and based on Determine the left delay In some embodiments, filtering the audio signal using the right filter includes: using the right filter and a right delay. Filtering the audio signal; filtering the audio signal using the left filter includes: using the left filter and the left delay. The audio signal is filtered. In some embodiments, the right filter and... Filtering the audio signal includes calculation ), using the left filter and Filtering the audio signal includes calculation ),in () is a sound signal.

[0184] In some embodiments,

[0185] ;and

[0186] .

[0187] Figure 21 This is a block diagram of an HR filtering apparatus 2100 for implementing an HR filtering unit 400, according to some embodiments. That is, the apparatus 2100 is operatively capable of performing the processes disclosed herein. Figure 21As shown, device 2100 may include: a processing circuit (PC) 2102, which may include one or more processors (P) 2155, such as a general-purpose microprocessor and / or one or more other processors, such as application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), etc., said processors may coexist in a single housing or a single data center, or may be geographically distributed (i.e., device 2100 may be a distributed computing device); a network interface 2148, including a transmitter (Tx) 2145 and a receiver (Rx) 2147 for enabling device 2100 to send data to and receive data from other nodes connected to network 110 (e.g., an Internet Protocol (IP) network), wherein network interface 2148 is (directly or indirectly) connected to network 110 (e.g., network interface 2148 may be wirelessly connected to network 110, in which case network interface 2148 is connected to an antenna arrangement); and a local storage unit (also referred to as a "data storage system") 2108, which may include one or more non-volatile storage devices and / or one or more volatile storage devices. In embodiments where PC 2102 includes a programmable processor, a computer program product (CPP) 2141 may be provided. CPP 2141 includes a computer-readable medium (CRM) 2142 storing a computer program (CP) 2143, which includes computer-readable instructions (CRI) 2144. CRM 2142 may be a non-transitory computer-readable medium (e.g., magnetic media (e.g., hard disk), optical media, memory devices (e.g., random access memory, flash memory), etc.). In some embodiments, the CRI 2144 of the computer program 2143 is configured such that, when executed by PC 2102, the CRI causes device 2100 to perform the steps described herein (e.g., the steps described herein with reference to the flowchart). In other embodiments, device 2100 may be configured to perform the steps described herein without requiring code. That is, for example, PC 2102 may consist of only one or more ASICs. Therefore, the features of the embodiments described herein may be implemented in hardware and / or software.

[0188] The following is a summary of the various embodiments described in this article:

[0189] A1. A method for filtering audio signals, the method comprising: generating a signal for filtering based on elevation angle. and azimuth angle A filter pair at a specified location, the filter pair consisting of a right filter ( ) and left filter ( The method comprises: constructing a filter pair; filtering an audio signal using the right filter; and filtering the audio signal using the left filter, wherein generating the filter pair comprises: i) obtaining at least a first set of elevation basis function values ​​at the elevation angle; ii) obtaining at least a first set of azimuth basis function values ​​at the azimuth angle; iii) generating the right filter using: a) at least the first set of elevation basis function values, b) at least the first set of azimuth basis function values, and c) right filter model parameters; and iv) generating the left filter using: a) at least the first set of elevation basis function values, b) at least the first set of azimuth basis function values, and c) left filter model parameters.

[0190] A2. The method according to claim A1, wherein obtaining the first azimuth basis function value set comprises: obtaining azimuth basis function values A set, wherein the azimuth basis function values ​​are... The set includes the set of the first azimuth basis function values.

[0191] A3. The method according to claim A1, wherein generating the right filter includes calculating... And generating the left filter includes calculating ,in, ( =1 to , =1 to ,as well as =1 to ) is the set of parameters for the right model. ( =1 to , =1 to ,as well as =1 to ) is the set of parameters for the left model. ( =1 to Defined in elevation angle The set of basis function values ​​for the first elevation angle at that location, and ( =1 to ,as well as =1 to ) is defined in the azimuth angle azimuth basis function values ​​at a set, and ( =1 to ) is of length The set of orthogonal basis vectors.

[0192] A4. The method according to claim A1, wherein obtaining the first set of elevation angle basis function values ​​comprises: obtaining the elevation angle basis function values. A set, wherein the elevation angle basis function values ​​are... The set includes the first set of elevation angle basis function values.

[0193] A5. The method according to claim A1, wherein generating the right filter includes calculating... And generating the left filter includes calculating ,in, ( =1 to , =1 to ,as well as =1 to ) is the set of parameters for the right model. ( =1 to , =1 to ,as well as =1 to ) is the set of parameters for the left model. ( =1 to ,as well as =1 to Defined in elevation angle The elevation angle basis function value at the location a set, and ( =1 to Defined in azimuth angle The set of basis function values ​​for the first azimuth angle at that location, and ( =1 to ) is of length The set of orthogonal basis vectors.

[0194] A6. The method according to any one of claims A1 to A5, wherein each of the elevation angle basis function values ​​depends on the azimuth angle, and / or each of the azimuth angle basis function values ​​depends on the elevation angle.

[0195] A7. The method according to any one of claims A1 to A5, wherein obtaining the first set of elevation basis function values ​​comprises: for each elevation basis function included in the first set of elevation basis functions, evaluating the elevation basis function at the elevation angle to generate an elevation basis function value corresponding to the elevation angle and the elevation basis function; and obtaining the first set of azimuth basis function values ​​comprises: for each azimuth basis function included in the first set of azimuth basis functions, evaluating the azimuth basis function at the azimuth angle to generate an azimuth basis function value corresponding to the azimuth angle and the azimuth basis function.

[0196] A8. The method according to claim A7, wherein each elevation basis function included in the first elevation basis function set is a B-spline basis function, and each azimuth basis function included in the first azimuth basis function set is a periodic B-spline basis function.

[0197] A9. The method according to claim A7 or A8, further comprising: obtaining a model representing at least the first set of elevation angle basis functions, wherein the model comprises: a sequence ( ),in Its specified sub-interval }, where the elevation angle basis function is a polynomial over the subinterval; and a three-dimensional array of model parameters ( ).

[0198] A10. The method according to claim A9, wherein the first set of elevation angle basis functions includes the first... Elevation basis function, at the elevation angle Evaluating each elevation basis function included in the first set of elevation basis functions includes: at the elevation angle The assessment of the first The elevation angle basis function, and in the elevation angle angle The assessment of the first The elevation angle basis function includes the following steps: finding the basis function that satisfies the following conditions. index ; and according to The assessment of the first The elevation angle basis function at that elevation angle The value at that location.

[0199] A11. The method according to claim A7 or A8, further comprising: obtaining a model representing at least the first azimuth basis function set, wherein the model comprises: a sequence ( ),in Its specified sub-interval The azimuth basis function is a polynomial over the subinterval; and the model parameters are a three-dimensional array. .

[0200] A12. The method according to claim A11, wherein the first azimuth basis function set includes the first... Azimuth basis functions, in the azimuth angle The evaluation of each azimuth basis function included in the first azimuth basis function set includes: in the azimuth angle The assessment of the first Azimuth basis functions, and in the azimuth angle The assessment of the first The azimuth basis functions include the following steps: finding those that satisfy... index ; and according to The assessment of the first Azimuth basis functions at the azimuth angle The value at that location.

[0201] A13. The method according to any one of claims A7 to A12, wherein the step of obtaining the first azimuth basis function value set further includes generating the first azimuth basis function set.

[0202] A14. The method according to claim A13, wherein generating the first azimuth basis function set comprises: generating a periodic B-spline basis function set over an azimuth range of 0 to 360 degrees.

[0203] A15. The method according to claim A14, wherein generating the set of periodic B-spline basis functions over the 0 to 360 degree azimuth range comprises: specifying arrive The length is within the range of degrees. A sequence of nodes; based on a length of The node sequence is used to generate an extended node sequence, wherein generating the extended node sequence includes: using a node sequence smaller than... degree The sum of the values ​​is greater than degree Each value expands in a periodic manner with a length of The node sequence; obtaining the extended multisequence of the nodes; generating an extended B-spline basis function set using the extended node sequence and the extended multisequence; selecting the extended basis functions starting from index 2. A series of extended basis functions; and periodically mapping the selected extended basis functions to arrive The azimuth range in degrees.

[0204] A16. The method according to any one of claims A1 to A15, further comprising: for elevation-azimuth angle... Determine the time difference between ears ( ).

[0205] A17. The method according to claim A16 further includes: based on Determine right delay ; and based on Determine the left delay .

[0206] A18. The method according to claim A17, wherein filtering the audio signal using the right filter comprises: using the right filter and a right delay. Filtering the audio signal; and filtering the audio signal using the left filter includes: using the left filter and the left delay. The sound signal is filtered.

[0207] A19. The method according to claim A18, wherein the right filter and Filtering the audio signal includes calculation ), using the left filter and Filtering the audio signal includes calculation ),in, It is a sound signal.

[0208] A20. The method according to any one of claims A17 to A19, wherein,

[0209] ;and .

[0210] A21. The method according to any one of claims A7 to A15, wherein the azimuth basis function has a periodicity with a period of 360 degrees.

[0211] While various embodiments are described herein (including any appendices present), it should be understood that they are presented by way of example only and not as limiting. Therefore, the breadth and scope of this disclosure should not be limited by any of the exemplary embodiments described above. Furthermore, all possible variations and combinations thereof of the foregoing elements are included in this disclosure, unless otherwise indicated herein or otherwise obviously contradicted in the context.

[0212] Furthermore, although the process described above and shown in the accompanying drawings is presented as a series of steps, this is for illustrative purposes only. Therefore, it should be considered that some steps may be added, some steps may be omitted, the order of the steps may be rearranged, and some steps may be performed in parallel.

[0213] Abbreviations:

[0214] Augmented Reality (AR)

[0215] DOA arrival direction

[0216] FIR Finite Impulse Response

[0217] HR Head Related

[0218] HRIR head-related impulse response

[0219] HRTF header-related transfer functions

[0220] ILD interauricular strength difference

[0221] IPD interauricular phase difference

[0222] ITD interaural time difference

[0223] ITF interauricular transfer function

[0224] MAA Minimum Hearing Angle

[0225] MPEG Moving Picture Experts Group

[0226] MR Mixed Reality

[0227] MSE Mean Square Error

[0228] PCA principal component analysis

[0229] SAOC Spatial Sound Object Encoding

[0230] SH spherical harmonic function

[0231] SOFA Space-Oriented Acoustic Format

[0232] Singular Value Decomposition (SVD)

[0233] VR Virtual Reality

[0234] XR Extended Reality

[0235] References:

[0236] [1] Doris J. Kistler, Frederic L. Wightman, “A model of head-related transfer functions based on principal components analysis and minimum-phase reconstruction”, Journal of Acoustical Society of America, 91(3):1637-1647, March 1992.

[0237] [2] Fábio P. Freeland, Luiz WP Biscainho and Paulo SR Diniz, "Interpolation of Head-Related Transfer Functions (HRTFS): A multi-source approach", in 12th European Signal Processing Conference, pp. 1761-1764, Vienna, September 2004.

[0238] [3] Mengqiu Zhang, Rodney A. Kennedy and Thushara D. Abhayapala, "Empirical determination of frequency representation in spherical harmonics-based HRTF functional modeling", IEEE / ACM Transactions on Audio, Speech and Language Processing, Vol. 23(2), pp. 351-360, February 2015.

[0239] [4] Zamir Ben-Hur, David Lou Alon, Boaz Rafaely and Ravish Mehra, "Loudness stability of binaural sound with spherical harmonic representation of sparse head-related transfer functions", EURASIP Journal on Audio, Speech, and Music Processing, May 2019.

Claims

1. A method for generating elevation angles and azimuth A method for processing data sequences, the method comprising: Obtaining a first set of azimuth basis function values ​​for the azimuth angle, wherein obtaining the first set of azimuth basis function values ​​includes: for each azimuth basis function included in the first set of azimuth basis functions, estimating the azimuth basis function at the azimuth angle to generate azimuth basis function values ​​corresponding to the azimuth angle and the azimuth basis functions, and wherein each azimuth basis function included in the first set of azimuth basis functions is a periodic basis function; and The data sequences at the elevation and azimuth angles are generated using the first set of azimuth basis function values.

2. The method according to claim 1, wherein, Obtaining the first set of azimuth basis function values ​​includes: obtaining P sets of azimuth basis function values ​​of the azimuth, wherein the P sets of azimuth basis function values ​​include the first set of azimuth basis function values.

3. The method according to claim 1, wherein, Each of the aforementioned azimuth basis function values ​​depends on the elevation angle.

4. The method according to claim 2, wherein, Generating the data sequence includes calculating the data sequence in an elevation-anchored unfolded form: in (in To P, to (k=1 to K) is a set of model parameters; (in The elevation angle is defined as P). The first set of elevation angle basis function values; (in To P and to The azimuth angle is defined. The azimuth basis function values ​​of group P; (where k = 1 to K) is a set of regular orthogonal basis vectors of length N; P is the number of elevation angle basis function values; It is the number of azimuth basis function values ​​used in conjunction with the p-th elevation basis function value; K is the number of regular orthogonal basis vectors, where K ≤ N; and N is the length of the generated data sequence.

5. The method according to claim 1, wherein, Generating the data sequence includes calculating the data sequence in an azimuth-anchored unfolded form: in (in To Q, to (k=1 to K) is a set of model parameters; (in To Q, to The elevation angle is defined. The Q-group elevation angle basis function values; (in The azimuth angle is defined to Q. The first set of azimuth basis function values; (where k = 1 to K) is a set of regular orthogonal basis vectors of length N; Q is the number of azimuth basis function values; It is the number of elevation basis function values ​​used in conjunction with the q-th azimuth basis function value; K is the number of regular orthogonal basis vectors, where K ≤ N; and N is the length of the generated data sequence.

6. The method according to claim 1, further comprising obtaining a representation of the first set of azimuth basis functions, wherein the representation includes: sequence( ),in = ( , ... , This sequence specifies a sub-interval. }, where the azimuth basis functions are polynomials, and 3D power coefficient array The polynomial is described as a linear combination of powers of azimuth angles.

7. The method according to claim 6, wherein, The first set of azimuth basis functions includes the q-th azimuth basis function; opposite angle Estimating each azimuth basis function in the first set of azimuth basis functions includes: estimating the azimuth angle. Estimate the basis function of the q-th azimuth angle at position q; and opposite angle The estimation of the basis function for the q-th azimuth angle at a given location includes the following steps: Find index l such that ;as well as azimuth angle The estimated value of the basis function for the q-th azimuth angle at point q is: .

8. The method according to claim 1, wherein, The steps for obtaining the first set of azimuth basis function values ​​also include generating the first set of azimuth basis functions.

9. The method according to claim 8, wherein, The first set of azimuth basis functions is generated by generating a set of periodic B-spline basis functions within the azimuth range of 0 to 360 degrees.

10. The method according to claim 9, wherein, Generate a set of periodic B-spline basis functions within the azimuth angle range of 0 to 360 degrees, including: Specify a node sequence of length L within the range of 0 to 360 degrees; An extended node sequence is generated based on the node sequence of length L, wherein generating the extended node sequence includes: periodically expanding the node sequence of length L, wherein J values ​​are below 0 degrees and J-1 values ​​are above 360 ​​degrees. Obtain an extended multisequence consisting of 1s; A set of extended B-spline basis functions is generated using the extended node sequence and the extended multisequence; Starting from index 2, select L-1 consecutive basis functions from these extended basis functions; and The selected extended basis functions are periodically mapped to an azimuth range of 0 to 360 degrees.

11. The method according to claim 1, wherein, The azimuth basis function is periodic with a period of 360 degrees.

12. The method according to claim 1, wherein, Each azimuth basis function in the first set is a periodic B-spline basis function.

13. The method according to claim 4, wherein, right Perform a T-transform on each vector.

14. The method according to claim 5, wherein, right Perform a T-transform on each vector.

15. The method according to claim 1, wherein, Generating the data sequence includes calculating the data sequence in an expanded form: in (in To Q, (P, k=1 to K) is a set of model parameters; (in The elevation angle is defined as P). The first set of elevation angle basis function values; (in The azimuth angle is defined to Q. The first set of azimuth basis function values; (where k = 1 to K) is a set of regular orthogonal basis vectors of length N; Q is the number of azimuth basis function values; P is the number of elevation angle basis function values; K is the number of regular orthogonal basis vectors, where K ≤ N; and N is the length of the generated data sequence.

16. A method for generating a model for modeling a set of data sequences sampled at a set of elevation and azimuth angles, the method comprising: Specify the azimuth basis functions, wherein the azimuth basis functions are periodic basis functions; as well as The azimuth basis functions are used to determine a set of model parameters for the model.

17. The method according to claim 16, wherein, The model, defined in the form of elevation-anchored deployment, is as follows: in (in To P, to (k=1 to K) is a set of model parameters; (in The elevation angle is defined as P). The first set of elevation angle basis function values; (in To P and to The azimuth angle is defined. The azimuth basis function values ​​of group P; (where k = 1 to K) is a set of regular orthogonal basis vectors of length N; P is the number of elevation angle basis function values; It is the number of azimuth basis function values ​​used in conjunction with the p-th elevation basis function value; K is the number of regular orthogonal basis vectors, where K ≤ N; and N is the length of the generated data sequence.

18. The method according to claim 16, wherein, The model, defined in azimuth-anchored expansion form, is as follows: in (in To Q, to (k=1 to K) is a set of model parameters; (in To Q, to The elevation angle is defined. The Q-group elevation angle basis function values; (in The azimuth angle is defined to Q. The first set of azimuth basis function values; (where k = 1 to K) is a set of regular orthogonal basis vectors of length N; Q is the number of azimuth basis function values; It is the number of elevation basis function values ​​used in conjunction with the q-th azimuth basis function value; K is the number of regular orthogonal basis vectors, where K ≤ N; and N is the length of the generated data sequence.

19. The method of claim 16, wherein, The model is defined in the following expanded form: in (in To Q, (P, k=1 to K) is a set of model parameters; (in The elevation angle is defined as P). The first set of elevation angle basis function values; (in The azimuth angle is defined to Q. The first set of azimuth basis function values; (where k = 1 to K) is a set of regular orthogonal basis vectors of length N; Q is the number of azimuth basis function values; P is the number of elevation angle basis function values; K is the number of regular orthogonal basis vectors, where K ≤ N; and N is the length of the generated data sequence.

20. The method of claim 16, wherein the azimuth basis function is a periodic B-spline basis function.

21. A non-transitory computer-readable medium storing a computer program containing instructions that, when executed by a processing circuit of a device, cause the device to perform the method of claim 1.

22. A non-transitory computer-readable medium storing a computer program containing instructions that, when executed by processing circuitry of a device, cause the device to perform the method of claim 16.

23. An apparatus comprising: Processing circuitry; as well as A memory containing instructions executable by the processing circuitry, wherein the means is configured to perform the method of claim 1.

24. An apparatus comprising: Processing circuitry; as well as A memory containing instructions executable by the processing circuitry, wherein the means is configured to perform the method of claim 16.

25. The method according to claim 16, wherein, The azimuth basis function is periodic with a period of 360 degrees.

Images