HRTF data fusion and sparse prediction method based on binaural factors
Through the HRTF data fusion and sparse prediction method of binaural factors, the problems of high measurement cost and large data differences in HRTF data sets are solved, and the accurate prediction and audio playback of personalized HRTF parameters are realized, which improves the accuracy of immersive audio experience. degree and practicality.
Patent Information
- Application Number
- CN202411002774.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-25
- Publication Date
- 2024-11-29
AI Technical Summary
The measurement cost of existing HRTF data sets is high, time-consuming and data varied greatly, resulting in the performance of deep learning models on different data sets, limiting their wide application. The HRTF sparse measurement technology relies on the quality and distribution of measured data. , it is difficult to ensure the accuracy and completeness of the data.
Using HRTF data fusion and sparse prediction methods based on binaural factors, pre-processing the SOFA format data set, using machine learning classification algorithm to evaluate the data set correlation, eliminate low-association data sets, merge the remaining data sets, and use hidden The neural network model predicts personalized binaural factors (ILD and ITD), and combines the minimum phase model to generate personalized HRTFs, reducing measurement steps and improving audio playback accuracy.
By eliminating data set differences, a more comprehensive and diverse HRTF data set is built, personalized HRTF parameter prediction is achieved, which significantly improves the accuracy and user experience of audio playback, reduces measurement steps, and improves practicality and Convenience.
Smart Images

Figure CN119052713A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of audio signal processing technology, and in particular refers to a head-related transfer function (HRTF) data fusion and HRTF sparse prediction method based on binaural factors. The method customizes HRTF for users based on implicit neural networks and data set fusion, and replays it through headphones, so that users can have an immersive experience. The method is suitable for virtual reality (VR), augmented reality (AR) and other occasions that require 3D audio positioning technology, such as games, film production and virtual meetings. Background Art
[0002] With the rise of metaverse technology, virtual reality (VR) and augmented reality (AR) applications are rapidly becoming mainstream. The core of these technologies is to provide users with a highly immersive experience, and immersive audio technology is an important part of achieving this goal. Immersive audio technology can simulate the sound propagation in the real world, allowing users to feel the direction and distance of the sound, thereby enhancing the realism and presence of the experience. The importance of 3D audio technology in these applications is becoming increasingly prominent, making it critical to research and develop advanced audio processing technology.
[0003] Head-related transfer function (HRTF) is a set of functions that describe the propagation characteristics of sound around the head. It integrates the reflection, absorption and scattering effects of the human torso, neck, head and auricle on sound. These factors work together on the sound signal, making the sound heard by each individual have unique characteristics. HRTF is crucial for achieving accurate sound source localization and is a fundamental parameter in immersive audio technology. At present, HRTF datasets measured in the laboratory provide important support for the development of audio technology, but these datasets have some limitations. For example, the quality of the data may vary due to measurement conditions, equipment and individual differences. In addition, the cost of obtaining these datasets is high and the process is time-consuming, which limits its feasibility in large-scale applications. In order to address the limitations of existing datasets, researchers have proposed HRTF sparse measurement technology. This method obtains HRTF data at key locations and then uses interpolation methods to predict HRTFs at other locations. Although this method can significantly reduce the time and cost required for measurement, it is highly dependent on the quality and distribution of the measured data, which poses challenges to data accuracy and completeness.
[0004] With the development of deep learning technology, data-driven neural networks are often used for HRTF estimation. These methods train neural network models with a large amount of data to achieve accurate prediction of HRTF. However, there are many HRTF datasets, but the measurement standards and procedures of each dataset are different, resulting in large data differences. This difference makes it difficult for a single model to perform well on all datasets, limiting the widespread application of deep learning methods. In order to solve the above problems, binaural difference features (ILD and ITD) are used to avoid the differences caused by measurement equipment and environment and enhance the generalization ability of the model. Summary of the invention
[0005] The purpose of the present invention is to propose a HRTF data fusion and sparse prediction method based on binaural factors. The method first preprocesses the existing HRTF data set to obtain the inter-auditory level difference (ILD). Then, the preprocessed ILD is classified by a machine learning classification algorithm to evaluate the correlation between different data sets. A threshold of 50% classification accuracy is set, and the data sets above the threshold are eliminated, and the remaining data sets are merged to form a data set for training. Using an implicit neural network model, the sound source azimuth, horizontal angle and the personalized characteristics of the subject are used as input to predict personalized binaural factors (ILD and ITD). Through large-scale data training, the model can capture the complex relationship between the sound source orientation and binaural factors, and generate a personalized head-related impulse response (HRIR) in combination with the minimum phase model, significantly improving the accuracy of audio playback. This method can accurately predict personalized HRTF parameters according to the specific sound source orientation of the user, thereby providing a more accurate and personalized immersive audio experience.
[0006] The specific technical solution for achieving the purpose of the present invention is:
[0007] A binaural factor-based HRTF data fusion and sparse prediction method, characterized in that the method comprises the following specific steps:
[0008] Step 1: Preprocess the HRTF dataset in SOFA format;
[0009] Step 2: Use machine learning methods to evaluate the correlation between the HRTF data sets preprocessed in step 1 and directly fuse the data;
[0010] Step 3: Construct an implicit neural network for predicting ILD and ITD, and input the fused HRTF data in step 2 into the neural network to train the network models for predicting ILD and ITD respectively;
[0011] Step 4: Use the network model for predicting ILD in Step 3 to predict ILD, and use the predicted ILD to adjust the artificial head HRTF, so as to generate the HRTF amplitude spectrum of the subject; combine the amplitude spectrum with the ITD predicted by the network model for predicting ITD in Step 3 through the minimum-phase model, and apply Fourier transform to obtain the personalized HRTF;
[0012] Step 5: The user measures the HRTF at multiple random azimuths in space, calculates the ILD and ITD at multiple random azimuths, and predicts the personalized HRTF of the user's entire space according to Step 4.
[0013] Furthermore, Step 1 specifically includes:
[0014] Perform a 256-point fast Fourier transform on the HRIR in SOFA format to obtain the HRTF; filter the HRTF to keep the frequencies between 200 Hz and 16 kHz;
[0015] Unify the azimuth range to [0°, 360°), and define the azimuth angle of the right ear as θ = 360 - θ, so that the left ear and the right ear share the same model;
[0016] Perform a smoothing or normalization operation on the HRTF amplitude spectrum, and then calculate the ILD and ITD; among them, the smoothing algorithm is as follows:
[0017]
[0018] Among them, is the smoothed HRTF amplitude spectrum at the j-th frequency point; N is the number of sampling points; M is the smoothing factor, usually M < N; C(n) is the n-th coefficient in the Fourier series; the normalization algorithm is as follows:
[0019]
[0020] Among them, θ is the sound source azimuth angle, is the sound source elevation angle; is the HRTF value of a given subject measured from the source position θ and ; is the average HRTF value of the database from the same source position; is the normalized HRTF amplitude spectrum of the subject at this sound source position;
[0021] ILD is defined as:
[0022]
[0023] Among them, θ is the sound source azimuth angle, is the sound source elevation angle; is the HRTF amplitude spectrum value of the right ear after smoothing or normalization; is the left ear HRTF amplitude spectrum value after smoothing or normalization; is the calculated binaural level difference;
[0024] ITD is calculated from the average time difference of binaural sound pressure, specifically:
[0025]
[0026] Where θ is the azimuth of the sound source, is the elevation angle of the sound source; is the average time of the left ear signal; is the average time of the right ear signal; h1(t) is the impulse response of the left ear; h0(t) is the impulse response of the right ear; is the calculated interaural time difference.
[0027] Further, the step 2 specifically includes:
[0028] The ILD obtained after preprocessing in step 1 is classified by machine learning classification algorithm to obtain the correlation between different data sets; the data set classification accuracy rate is set to 50% as the threshold, the data sets above the threshold are eliminated, and the preprocessed data of the remaining data sets are directly merged to form a data set for network training.
[0029] Further, the step three specifically includes:
[0030] For each subject Using Neural Field Models Modeling; the neural field model uses the sinusoidal representation network architecture (SIREN), whose input is the sound direction Mapping Where K is the number of frequencies; In order to model the HRTF of multiple subjects, a D-dimensional latent vector z is introduced at the input of the neural field to represent the personalized characteristics of different subjects; By introducing, the HRTF field is defined as the frequency domain from the sound direction The mapping of latent code z to individual HRTF is specifically expressed as: The output is the ILD and ITD values of the subject at all frequencies at that position. During the training process, the implicit gradient origin network (IGON) is used as the generative model by co-optimizing the latent space and network weights. First, z is inferred by calculating the mean square error (MSE) gradient between the generated value and the actual value of the subject in direction x relative to the origin z.
[0031]
[0032] Then, we update the model G by fitting the ILD with the MSE loss,
[0033]
[0034] Among them, z is the updated latent vector, which is used to represent the personalized characteristics of the subject; z is the initial latent vector, which represents the initial hypothesized subject characteristics; To find the gradient of the initial potential vector z, the gradient represents the loss function Sensitivity to the latent vector z, i.e. how the latent vector needs to be adjusted to minimize the loss;
[0035] is the mean square error (MSE) loss function, which is used to measure the difference between the model output and the actual value; x is the set of sound source directions, including all sound source azimuth angles θ and elevation angles G(·,·,z) is the neural field model function, and the input is the direction of the sound source and the initial latent vector z, the model generates the predicted values of ILD and ITD; the symbol · represents the direction input, which is any combination of ; the training is iterated between equation (5) and equation (6) in small batches until convergence; the training set is the data set obtained by directly fusion of the data in step 2; the test set is the RIEC data set, and the ILD and ITD data of some directions are sampled by the downsampling algorithm; the logarithmic spectral distortion (LSD) is used as the evaluation indicator of model training.
[0036] Further, the step 4 specifically includes:
[0037] The network model used in step 3 to predict ILD is used to predict ILD, and the predicted ILD is used to adjust the amplitude spectrum of the artificial head HRTF. The logarithm of the amplitude spectrum is then taken, and the phase is calculated using the Hilbert transform to obtain the HRTF minimum phase result. The ITD predicted by the network model used in step 3 to predict ITD is used as a pure time delay, and the binaural impulse response function is obtained according to the minimum phase model in combination with the minimum phase result. The binaural impulse response in the time domain is converted into the frequency domain using the Fourier transform to obtain a personalized HRTF.
[0038] Further, the step five specifically includes:
[0039] In an anechoic chamber environment, a frequency sweep signal is used to obtain the HRTF of the subject in multiple random directions, and the ILD and ITD are calculated; a neural network is used to predict the ILD and ITD of the subject in all directions of space, and according to step four, a personalized HRTF of the subject in all directions of space is predicted.
[0040] The beneficial effect of the present invention is that multiple HRTF data sets are analyzed and integrated through machine learning methods, the differences between different data sets are eliminated, and a more comprehensive and diverse HRTF data set is constructed, thereby providing richer and more accurate training data for subsequent modeling. By using implicit neural networks, the sound source azimuth, horizontal angle and personalized characteristics of the subject are used as inputs, and personalized binaural factors (ILD and ITD) can be accurately predicted to meet the personalized needs of users and provide a more accurate and personalized immersive audio experience. Through training on large-scale data, the model can capture the complex relationship between the sound source azimuth and binaural factors, and generate personalized time-domain binaural playback head-related impulse response functions (HRIR) in combination with the minimum phase model, which significantly improves the accuracy of audio playback. The core advantage of this technology is that users only need to measure the HRTF of some directions, and by calculating the ILD and ITD in the direction, they can infer a full range of personalized HRTFs, which reduces cumbersome measurement steps and improves practicality and convenience. BRIEF DESCRIPTION OF THE DRAWINGS
[0041] Figure 1 Schematic diagram of HRTF amplitude spectrum smoothing results with different smoothing factors;
[0042] Figure 2 Schematic diagram of the confusion matrix of kernel SVM classification based on original ILD and with different smoothing factors;
[0043] Figure 3 This is the SIREN neural network diagram. DETAILED DESCRIPTION
[0044] The present invention is described in detail below with reference to the accompanying drawings and embodiments.
[0045] The present invention provides a binaural factor-based HRTF data fusion and sparse prediction method, comprising the following specific steps:
[0046] Step 1: Data preprocessing
[0047] The HRTF is obtained by taking a 256-point Fast Fourier Transform (FFT) of the HRIR. To reduce the effect of the frequency response limitation of the HRIR measurement equipment, the HRTF is filtered to keep frequencies between 200 Hz and 16 kHz.
[0048] Unify the azimuth range to [0°, 360°), and re - define the azimuth of the right ear as θ = 360 - θ, so that the left and right ears share the same model. Expand the azimuth to (-30°, 390°) by repeating the HRTF data within the azimuth ranges (0°, 30°) and (330°, 360°). This azimuth data is used for model training in subsequent steps. Retain the azimuth data in the range (0°, 360°) for model evaluation in subsequent steps.
[0049] Perform smoothing or normalization operations on the HRTF magnitude spectrum, and then calculate the ILD and ITD. Among them, the smoothing algorithm is as follows:
[0050]
[0051] Among them, is the smoothed HRTF magnitude spectrum at the j - th frequency point; N is the number of sampling points; M is the smoothing factor, usually M < N; C(n) is the n - th coefficient in the Fourier series. Figure 1 The smoothing results of the HRTF magnitude spectrum with different smoothing factors are given. Figure 1 (a) Taking the HUTUBS dataset as an example, the left side shows the HRTF magnitude spectrum of the 189th subject in the dataset at the azimuth of (0°, 0°), Figure 1 (b) is the average value of the HRTF magnitude spectra of all subjects in the dataset at the azimuth of (0°, 0°).
[0052] The normalization algorithm is as follows:
[0053]
[0054] Among them, θ is the sound source azimuth angle, is the sound source elevation angle; is the HRTF value of a given subject measured from the source position θ and ; is the average HRTF value of the database from the same source position. is the normalized HRTF magnitude spectrum of the subject at this sound source position.
[0055] ILD is defined as:
[0056]
[0057] Among them, θ is the sound source azimuth angle, is the sound source elevation angle; is the HRTF magnitude spectrum value of the right ear after smoothing or normalization; is the HRTF magnitude spectrum value of the left ear after smoothing or normalization; is the calculated inter - aural level difference.
[0058] ITD is calculated from the average time difference of binaural sound pressure, specifically:
[0059]
[0060] Where θ is the azimuth of the sound source, is the elevation angle of the sound source; is the average time of the left ear signal; is the average time of the right ear signal; h1(t) is the impulse response of the left ear; h0(t) is the impulse response of the right ear; is the calculated interaural time difference.
[0061] Step 2: Machine learning algorithms evaluate the correlation between different data sets
[0062] Different machine learning classification algorithms are trained to classify which database a given ILD comes from, thereby evaluating the correlation between different data sets after data preprocessing. Table 1 gives the python code and parameter settings of different machine learning classification algorithms.
[0063] Table 1 Python codes for different machine learning classification algorithms
[0064] Support Vector Machine svm_clf=SVC(kernel='rbf', gamma=0.004, C=0.005) Gradient Boosted Tree Classifier gbt_clf=GradientBoostingClassifier(random_state=42) Logistic Regression lr_clf=LogisticRegression(random_state=42,max_iter=1000) Naive Bayes nb_clf = GaussianNB() k-nearest neighbor algorithm knn_clf=KNeighborsClassifier(n_neighbors=5) Decision Tree Classifier dt_clf=DecisionTreeClassifier(random_state=42) Random Forest Classifier rf_clf=RandomForestClassifier(n_estimators=100,random_state=42)
[0065] The training was performed on a subset of 12 positions: 0°, ±30°, ±60°, ±90°, ±120°, ±150°, 180°, which were shared among the eight selected databases; the remaining two databases did not contain these common positions. A total of 144 subjects were used in this task, with 18 subjects extracted from each database (minimum database size). This sampling method was used to reduce bias towards any database. The total number of ILDs in this experiment was 12 (positions) × 18 (subjects) × 2 (ears) = 432 per database. Five cross-validations were performed and Figure 2 The confusion matrices of the original ILD and kernel SVM classification with different smoothing factors are reported in . Figure 2 (a) is a schematic diagram of the kernel SVM classification confusion matrix based on the original ILD, (b) is the kernel SVM classification confusion matrix when the smoothing factor M = 64, and (c) is the kernel SVM classification confusion matrix when the smoothing factor M = 8. The classification accuracy of the data set is set to 50% as the threshold, and the data sets above the threshold are eliminated, and the preprocessed data of the remaining data sets are directly merged to form the data set for network training.
[0066] Step 3: Neural Network Design Since the HRTF function is defined on a continuous sphere, a neural field can be used to represent it. A deep neural network is used to parameterize the function defined in the continuous domain. Unlike the discrete positions of the measured HRTF, the neural field is independent of the grid resolution and can be automatically differentiated with respect to the spatial coordinates. For each subject Using Neural Fields Model it. The transformation of the network is Where k is the number of frequencies. The specific structure of the network is the Sinusoidal Representation Network architecture (SIREN), which is a multi-layer perceptron (MLP) with a sinusoidal activation function.
[0067] In order to model the HRTF of multiple subjects, a D-dimensional latent vector z is introduced at the input of the neural field to represent the personalized characteristics of different subjects. With this modification, the HRTF field is defined as And the mapping of latent code z to individual HRTF. The implicit gradient origin network (IGON) is used as the generative model of the neural field, which is expressed as In the present invention, the output is the ILD and ITD values of the subject at all frequencies at that position. The specific structure of the neural network model is as follows Figure 3 shown.
[0068] The training is performed by co-optimizing the latent space and network weights. The IGON algorithm first infers z by calculating the mean squared error (MSE) gradient between the generated value and the actual value of the subject in all directions x relative to the origin z.
[0069]
[0070] Afterwards, G is updated by fitting the ILD with the MSE loss,
[0071]
[0072] Among them, z is the updated latent vector, which is used to represent the personalized characteristics of the subject; z is the initial latent vector, which represents the initial hypothesized subject characteristics; To find the gradient of the initial potential vector z, the gradient represents the loss function Sensitivity to the latent vector z, i.e. how the latent vector needs to be adjusted to minimize the loss.
[0073] is the mean square error (MSE) loss function, which is used to measure the difference between the model output and the actual value; x is the set of sound source directions, including all sound source azimuth angles θ and elevation angles G(·,·,z) is the neural field model function, and the input is the direction of the sound source and the initial latent vector z, the model generates the predicted values of ILD and ITD; the symbol · represents the direction input, which can be any combination of .
[0074] Training iterates between Eq. (5) and Eq. (6) in mini-batches until convergence. Initializing z at the origin is based on the assumption that the mean of the latent distribution is 0.
[0075] Step 4: Specific experimental steps
[0076] 1) Dataset: The publicly available datasets of the measured far-field HRTF of the subjects are provided in the SOFA format repository. The training set is obtained by fusion of different datasets. The test set is the RIEC dataset, and the HRTF data of some directions are sampled using the downsampling algorithm, and the downsampling factor is set to T=5.
[0077] 2) Evaluation index: Logarithmic spectral distortion (LSD) is used as an objective evaluation index. The specific calculation method is as follows:
[0078]
[0079] where j represents the frequency index. L and K are the number of spatial locations and frequency bins, respectively. and Denote the azimuth angle θ and the elevation angle The linear scale magnitude of the actual ILD and predicted ILD of the jth frequency in the direction. In the present invention, ITD estimation uses a similar evaluation index.
[0080] 3) Implementation details: The MLP network contains two hidden layers, each containing 1024 nodes. The latent vector z is a 32-dimensional vector (D=32). The Adam optimizer is used for optimization, and the batch size is set to 16. In each batch, padding and masking are applied for loss calculation because the ILD data from different datasets have data samples of different lengths. The initial learning rate is set to μ=0.0001 and is adjusted according to Gradually decay.
[0081] Step 5: Use the network model that predicts ILD to predict ILD, use the ILD to adjust the artificial head HRTF amplitude spectrum, then take the logarithm of the amplitude spectrum, and use the Hilbert transform to calculate the phase to obtain the HRTF minimum phase result; use the ITD predicted by the network model that predicts ITD in step 3 as the pure time delay, and combine the minimum phase result to obtain the binaural impulse response function according to the minimum phase model; use the Fourier transform to convert the binaural impulse response in the time domain into the frequency domain to obtain a personalized HRTF. Step 6: Measure the HRTF of the user's partial orientation
[0082] In an anechoic room environment, the HRTF of the subject in multiple directions is obtained using a frequency sweep signal, and the ILD and ITD are calculated. The ILD and ITD of the subject in all spatial directions are generated through a neural network, and finally calculated as a personalized HRTF.
[0083] Step 7: Subjective Experimental Verification
[0084] A subjective experiment was designed to verify the ability of the method of the present invention to improve the performance of virtual sound source localization. In the experiment, the playback sound used a broadband white noise source of 0.5-15kHz. The broadband noise consisted of two groups of 200ms Gaussian white noise, with a 100ms silence in between. Several volunteers with normal hearing participated in this subjective experiment, and a total of four planes were used for positioning experiments, namely, horizontal planes with pitch angles of 0 and 33.75° and vertical planes with azimuth angles equal to 0 and -20°. The HRTF of the original database, the HRTF of the artificial head, and the HRTF restored by the method of the present invention were used to convolve the broadband noise source and replay three groups of virtual sources, and the volunteers were asked to judge which of the spatial cue orientations of the virtual sources obtained by the artificial head and the method of the present invention was closer to the spatial orientation of the virtual sources obtained by the original database. Computer speakers and headphones were used to play the sound to the volunteers in the experiment. The improvement of the performance of virtual sound source localization was evaluated by calculating the mean absolute error (MAE) of the subjects. The experimental results show that the virtual source obtained by replaying the HRTF convolution broadband noise source restored by the method of the present invention has a significantly lower positioning error than the artificial head HRTF and the original database HRTF, indicating that the method of the present invention has the highest accuracy in sound source positioning. The results verify the superiority of the method of the present invention in virtual sound source positioning.
Claims
1. A HRTF data fusion and sparse prediction method based on binaural factors, It is characterized in that The method comprises the following specific steps: Step 1: Preprocess the HRTF dataset in SOFA format; Step 2: Use machine learning methods to evaluate the correlation between the HRTF data sets preprocessed in step 1 and directly fuse the data; Step 3: Construct an implicit neural network for predicting ILD and ITD, and input the fused HRTF data in step 2 into the neural network to train the network models for predicting ILD and ITD respectively; Step 4: Use the network model that predicts ILD in step 3 to predict ILD, and use the predicted ILD to adjust the artificial head HRTF to generate the subject's HRTF amplitude spectrum; combine the amplitude spectrum with the ITD predicted by the network model that predicts ITD in step 3 through the minimum phase model, and apply Fourier transform to obtain a personalized HRTF; Step 5: The user measures HRTF at multiple random positions in the space, calculates ILD and ITD at multiple random positions, and predicts personalized HRTF in all directions of the user space according to step 4.
2. The HRTF data fusion and sparse prediction method according to claim 1, It is characterized in that The step 1 specifically includes: Perform 256-point fast Fourier transform on the HRIR in SOFA format to obtain HRTF; filter the HRTF to keep the frequency between 200Hz-16kHz; The azimuth angle range is unified to [0°, 360°), and the azimuth angle of the right ear is defined as θ = 360-θ, so that the left ear and the right ear share the same model; The HRTF amplitude spectrum is smoothed or normalized, and then the ILD and ITD are calculated; the smoothing algorithm is as follows: wherein, is the smoothed HRTF amplitude spectrum at the j-th frequency point; N is the number of sampling points; M is the smoothing factor, M < N; C(n) is the n-th coefficient in the Fourier series; the normalization algorithm is as follows: Where θ is the azimuth of the sound source, is the elevation angle of the sound source; is the source position θ and The measured HRTF value for a given subject; is the average HRTF value from the database at the same source location; is the normalized HRTF amplitude spectrum of the subject at the sound source position; ILD is defined as: Where θ is the azimuth of the sound source, is the elevation angle of the sound source; is the right ear HRTF amplitude spectrum value after smoothing or normalization; is the left ear HRTF amplitude spectrum value after smoothing or normalization; is the calculated binaural level difference; ITD is calculated from the average time difference of binaural sound pressure, specifically: Where θ is the azimuth of the sound source, is the elevation angle of the sound source; is the average time of the left ear signal; is the average time of the right ear signal; h 2 (t) is the impulse response of the left ear; h 1 (t) is the impulse response of the right ear; is the calculated interaural time difference.
3. The HRTF data fusion and sparse prediction method according to claim 1, It is characterized in that The step 2 specifically includes: The ILD obtained after preprocessing in step 1 is classified by machine learning classification algorithm to obtain the correlation between different data sets; the data set classification accuracy rate is set to 50% as the threshold, the data sets above the threshold are eliminated, and the preprocessed data of the remaining data sets are directly merged to form a data set for network training.
4. The HRTF data fusion and sparse prediction method according to claim 1, It is characterized in that The step three specifically includes: For each subject Using Neural Field Models Modeling; the neural field model uses a sinusoidal representation network architecture, and its input is the sound direction Mapping Where K is the number of frequencies; In order to model the HRTF of multiple subjects, a D-dimensional latent vector z is introduced at the input of the neural field to represent the personalized characteristics of different subjects; By introducing, the HRTF field is defined as the frequency domain from the sound direction The mapping of latent code z to individual HRTF is specifically expressed as: The output is the ILD and ITD values of the subject at all frequencies at that position. During the training process, the implicit gradient origin network (IGON) is used as the generative model by co-optimizing the latent space and network weights. First, z is inferred by calculating the mean square error gradient between the generated value and the actual value of the subject in direction x relative to the origin z. Then, we update the model G by fitting the ILD with the MSE loss, Among them, z is the updated latent vector, which is used to represent the personalized characteristics of the subject; z is the initial latent vector, which represents the initial hypothesized subject characteristics; To find the gradient of the initial potential vector z, the gradient represents the loss function Sensitivity to the latent vector z, i.e. how the latent vector needs to be adjusted to minimize the loss; is the mean square error loss function, which is used to measure the difference between the model output and the actual value; x is the set of sound source directions, including all sound source azimuth angles θ and elevation angles G(·,·,z) is the neural field model function, and the input is the direction of the sound source and the initial latent vector z, the model generates the predicted values of ILD and ITD; the symbol · represents the direction input, which is any combination of ; the training is iterated between equation (5) and equation (6) in small batches until convergence; the training set is the data set obtained by directly fusion of the data in step 2; the test set is the RIEC data set, and the ILD and ITD data of some directions are sampled by the downsampling algorithm; the logarithmic spectral distortion is used as the evaluation indicator of model training.
5. The HRTF data fusion and sparse prediction method according to claim 1, It is characterized in that The step 4 specifically includes: The network model used in step 3 to predict ILD is used to predict ILD, and the predicted ILD is used to adjust the amplitude spectrum of the artificial head HRTF. The logarithm of the amplitude spectrum is then taken, and the phase is calculated using the Hilbert transform to obtain the HRTF minimum phase result. The ITD predicted by the network model used in step 3 to predict ITD is used as a pure time delay, and the binaural impulse response function is obtained according to the minimum phase model in combination with the minimum phase result. The binaural impulse response in the time domain is converted into the frequency domain using the Fourier transform to obtain a personalized HRTF.
6. The HRTF data fusion and sparse prediction method according to claim 1, It is characterized in that The step five specifically includes: In an anechoic chamber environment, a frequency sweep signal is used to obtain the HRTF of the subject in multiple random directions, and the ILD and ITD are calculated; a neural network is used to predict the ILD and ITD of the subject in all directions of space, and according to step four, a personalized HRTF of the subject in all directions of space is predicted.
Citation Information
Cited By
Personalized head-related transfer function up-sampling method and system
CN120111429A
A personalized head-related transfer function upsampling method and system
CN120111429B
Header-related transfer function data set fusion method and system
CN120217298A
A head-related transfer function data set fusion method and system
CN120217298B
