Audio simulation method and device, equipment and storage medium

Abstract

The invention discloses an audio simulation method, device and equipment and a storage medium, relates to the field of audio simulation, introduces an agent model based on a multilayer perceptron, and adopts a spherical harmonic function coefficient as a key bridge for connecting a micro sound source and a macro environment. Through offline training, the proxy model learns and solidifies a nonlinear mapping relation from excitation parameters (frequency and voltage) to complex sound field physical representation (spherical harmonic function coefficient). In real-time application, the trained model can instantaneously predict the spherical harmonic function coefficient of the sound source according to the driving parameters, and the spherical harmonic function coefficient is used for subsequent acoustic simulation and finally synthesized into audible audio. Therefore, a time-consuming microscopic physical simulation process is decoupled into offline training, so that a huge calculation gap of source-field direct coupling is bypassed, real-time full-link audio simulation is realized on the premise of ensuring high precision, and the calculation bottleneck is effectively solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of audio simulation, and more specifically, to an audio simulation method, device, equipment and storage medium. BACKGROUND

[0002] With the rapid development of consumer electronics and automotive industries, users' requirements for audio experience are increasingly improved. In particular, in application scenarios such as car audio and high-end smart sound boxes, high-fidelity and immersive spatial sound field effects have become the core competitiveness of products. Therefore, it is of great engineering and commercial value to accurately predict and optimize the final listening experience through audio simulation technology in the product design stage, thereby shortening the development cycle and reducing the trial and error cost.

[0003] The existing audio simulation process often faces the problem of efficient coupling between the micro source fine model and the macro environment model. Usually, engineers will use fine physical models such as finite element method to simulate micro sources such as loudspeakers to obtain their accurate vibration and radiation characteristics; at the same time, geometric acoustics methods such as ray tracing are used to simulate macro environments such as car cabins or indoor environments. However, it is extremely difficult to combine the two directly. A simplified coupling method is to extract the far-field directivity data of the micro source simulation and apply it to a simplified point source in the macro environment. The defect of this method is that it seriously ignores the complex near-field characteristics and key phase information of the sound source, which are crucial for accurately simulating the complex diffraction and reflection between sound waves and structures near the sound source (such as instrument tables and windshields), ultimately resulting in a significant loss of fidelity of the simulation results, which cannot truly restore the actual listening experience.

[0004] The above-mentioned defects are due to the inherent incompatibility of the calculation principle and the huge gap between the calculation scale and cost. If you try to use wave acoustics methods such as finite element method to model the entire macro environment, the number of grids will increase exponentially as the frequency increases, causing a scale disaster, which is currently beyond the reach of engineering practice in terms of computing resources and time cost. Therefore, although the simulation methods of different principles are used to process micro sources and macro environments respectively, a kind of intermediate data representation and coupling technology that can completely retain the physical details of the sound source (especially the direction and phase information) and be efficiently used by the macro simulation engine has always been a key technical bottleneck restricting the accuracy and efficiency of audio simulation. SUMMARY

[0005] To solve the problems in the background art, according to an aspect of the present application, an audio simulation method is provided, which comprises:

[0006] obtain a training dataset, each training sample in the training dataset being a {excitation parameter, sound field physical representation} data pair, the excitation parameter including frequency and voltage, and the sound field physical representation being a spherical harmonic coefficient vector;

[0007] train a multi-layer perceptron based proxy model based on the training dataset to obtain a trained proxy model;

[0008] obtain real-time driving parameters from an upper application, the real-time driving parameters including simulated frequency and voltage;

[0009] input the real-time driving parameters into the trained proxy model to obtain real-time predicted spherical harmonic coefficients;

[0010] perform macro-environmental acoustic simulation based on the real-time predicted spherical harmonic coefficients to obtain an acoustic impulse response;

[0011] perform perceptual synthesis of the acoustic impulse response and a dry audio signal to obtain audible audio.

[0012] According to another aspect of the present application, there is provided an audio simulation apparatus comprising:

[0013] a training dataset obtaining module configured to obtain a training dataset, each training sample in the training dataset being a {excitation parameter, sound field physical representation} data pair, the excitation parameter including frequency and voltage, and the sound field physical representation being a spherical harmonic coefficient vector;

[0014] a model training module configured to train a multi-layer perceptron based proxy model based on the training dataset to obtain a trained proxy model;

[0015] a real-time driving parameter obtaining module configured to obtain real-time driving parameters from an upper application, the real-time driving parameters including simulated frequency and voltage;

[0016] a real-time prediction module configured to input the real-time driving parameters into the trained proxy model to obtain real-time predicted spherical harmonic coefficients;

[0017] a macro-environmental acoustic simulation module configured to perform macro-environmental acoustic simulation based on the real-time predicted spherical harmonic coefficients to obtain an acoustic impulse response;

[0018] an audible audio generation module configured to perform perceptual synthesis of the acoustic impulse response and a dry audio signal to obtain audible audio.

[0019] According to another aspect of the present application, there is also provided a device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor implementing the steps of the above method when executing the program.

[0020] According to another aspect of the present application, a storage medium is also provided, which has a computer program stored thereon, the computer program being executed by a processor to perform the steps of the method described above.

[0021] Compared with the prior art, the audio simulation method, device, equipment and storage medium provided by the present application introduce an agent model based on a multi-layer perception machine, and adopt spherical harmonic function coefficients as a key bridge connecting a micro sound source and a macro environment. Through offline training, the agent model learns and solidifies a nonlinear mapping relationship from excitation parameters (frequency, voltage) to a complex sound field physical representation (spherical harmonic function coefficients). In real-time application, the trained model can instantaneously predict the spherical harmonic function coefficients of the sound source according to the driving parameters. The coefficients not only completely retain the directivity information of the sound source including the amplitude and the phase, solve the problem of low information fidelity caused by the traditional simplified method, but also have a standardized data format that can be efficiently called by a macro acoustic simulation engine, used for subsequent acoustic simulation and finally synthesized into audible audio. In this way, the time-consuming micro physical simulation process is decoupled from offline training, thereby bypassing the huge computing gap of the source-field direct coupling, realizing the real-time of the full-link audio simulation under the premise of ensuring high precision, and effectively solving the computing bottleneck. BRIEF DESCRIPTION OF DRAWINGS

[0022] The above and other objects, features and advantages of the present application will become more apparent from the following detailed description of embodiments of the present application taken in conjunction with the accompanying drawings. The accompanying drawings are provided to aid in understanding the embodiments of the present application, and constitute a part of the specification. The accompanying drawings, together with the specification, serve to explain the present application, and do not constitute a limitation on the present application. In the drawings, the same reference numerals generally refer to the same components or steps throughout the drawings.

[0023] Figure 1 A flowchart of an audio simulation method according to an embodiment of the present application.

[0024] Figure 2 A data flow diagram of an audio simulation method according to an embodiment of the present application.

[0025] Figure 3 A flowchart of step 1 in an audio simulation method according to an embodiment of the present application.

[0026] Figure 4 A flowchart of step 5 in an audio simulation method according to an embodiment of the present application.

[0027] Figure 5 A block diagram of an audio simulation device according to an embodiment of the present application. DETAILED DESCRIPTION

[0028] Embodiments of the present disclosure will be described below in greater detail with reference to the accompanying drawings. While certain embodiments of the present disclosure are shown in the drawings, it is understood that the present disclosure can be embodied in various forms and should not be interpreted as being limited to the embodiments set forth herein. Rather, these embodiments are provided so that the present disclosure can be more thoroughly and completely understood.

[0029] Based on the technical bottleneck in the foregoing technical content, the present application proposes an audio simulation method. Figure 1 A flowchart of the audio simulation method according to an embodiment of the present application. Figure 2 A data flow diagram of the audio simulation method according to an embodiment of the present application. As shown in Figure 1 and Figure 2 The audio simulation method according to an embodiment of the present application includes: step 1, obtaining a training data set, each training sample in the training data set being a {excitation parameter, sound field physical representation} data pair, the excitation parameter including frequency and voltage, and the sound field physical representation being a spherical harmonic function coefficient vector; step 2, training a multi-layer perception-based proxy model based on the training data set to obtain a trained proxy model; step 3, obtaining real-time driving parameters from an upper-layer application, the real-time driving parameters including simulated frequency and voltage; step 4, inputting the real-time driving parameters into the trained proxy model to obtain real-time predicted spherical harmonic function coefficients; step 5, performing macro-environment acoustic simulation based on the real-time predicted spherical harmonic function coefficients to obtain an acoustic impulse response; and step 6, performing perceptual synthesis of the acoustic impulse response and dry audio signals to obtain audible audio.

[0030] In step 1, a training dataset is obtained. Each training sample in the training dataset is a data pair of {excitation parameters, sound field physical representation}. The excitation parameters include frequency and voltage, and the sound field physical representation is a vector of spherical harmonic function coefficients. It should be understood that in audio product design, to accurately predict the listening experience of a sound source in a real environment, its physical characteristics need to be simulated. However, directly using high-precision microscopic physical simulation engines such as the finite element method for real-time, interactive analysis is extremely time-consuming; the calculation of a single frequency point can take several hours, creating a significant bottleneck in design processes requiring rapid iteration. To overcome this curse of scale and bypass the high cost of real-time computation, this application introduces a machine learning-based surrogate model. Based on this, a training dataset is first needed to provide high-quality data containing physical laws for constructing this surrogate model. By pre-generating a series of excitation parameters covering the working range of the sound source to data pairs representing the physical characterization of the sound field, time-consuming physical simulations can be completed offline and in one go. The simulation results can be refined into an efficient and standardized data format, thus laying the foundation for the training of subsequent proxy models, enabling them to learn and replace the complex physical simulation process, and ultimately achieving the goal of real-time simulation.

[0031] In one feasible embodiment Figure 3 This is a flowchart of step 1 in the audio simulation method according to an embodiment of this application. Figure 3 As shown, step 1 of this application, obtaining the training dataset, includes: step 11, performing gridded sampling in the simulation parameter space to obtain a series of discrete simulation task parameter points, the simulation task parameter points including frequency and voltage; step 12, driving the microphysics simulation engine for the simulation task parameter points to obtain the original sound pressure field data; step 13, performing sound field feature transformation on the original sound pressure field data to obtain the spherical harmonic function coefficient vector of the simulation task parameter points.

[0032] In the above embodiment, the execution process of step 1 is as follows: first, step 11 is processed. In a specific embodiment, for a to-be-simulated sound source, for example, the design specifications and expected operating range of a 6.5-inch mid-bass speaker for a high-end car audio system. First, the dimension and boundary of the simulation parameter space need to be determined. Here the excitation parameters are determined as the two core physical quantities that drive the speaker: frequency and voltage. Frequency determines the characteristics and pitch of the sound wave, while voltage directly controls the input power of the speaker's voice coil, which in turn affects the displacement of the diaphragm and the final sound pressure level. The boundary of the parameter space is set according to the design indicators of the speaker. For example, for this mid-bass speaker, its effective frequency response range may be set to 80 Hz to 5000 Hz. Its rated operating voltage range, according to its impedance, for example, 4 ohms and rated power, for example, 50 watts, can determine a safe driving voltage range, for example, from 0.1 volts, i.e. very low volume, to 14.14 volts, i.e. peak voltage near the rated power. These two ranges together define a two-dimensional simulation parameter space. Next, the two-dimensional parameter space is sampled by a grid. The core of the grid sampling is to independently sample each dimension of frequency and voltage, and then combine these sampling points to form a grid that covers the entire parameter space. For the frequency dimension, considering that the acoustic frequency response is more physically meaningful and more consistent with the perception characteristics of the human ear in the logarithmic coordinate system, logarithmic uniform sampling is adopted instead of linear sampling. A specific implementation is to divide by the octave of musical intervals. For example, the range of 80 Hz to 5000 Hz can be sampled with an accuracy of 1 / 24 octave. That is, starting from 80 Hz, the next sampling point has a frequency of the previous frequency multiplied by 2 raised to the power of (1 / 24), and so on, until it exceeds 5000 Hz. This way, more dense sampling points can be obtained in the low frequency band to capture more detailed acoustic details in this band, while the sampling points are relatively sparse in the high frequency band, which is consistent with the conventional practice of acoustic analysis. This process will generate a list containing hundreds of discrete frequency values. For the voltage dimension, since its effect on the vibration of the sound source is considered to be linear, i.e. within the linear operating range of the speaker, linear equidistant sampling can be used. For example, in the range of 0.1 volts to 14.14 volts, a fixed sampling step size such as 0.5 volts can be set. This way, a list of voltage values will be generated: [0.1, 0.6, 1.1,..., 14.1]. Finally, the lists of discrete sampling points generated in the above two dimensions are combined by Cartesian product to form the final grid-based simulation task parameter points. That is, each frequency value in the frequency list is paired with each voltage value in the voltage list. For example, if the frequency list has 200 points and the voltage list has 29 points, a total of 200*29=5800 simulation task parameter points will be generated.Each point is a {frequency, voltage} data pair, such as {80 hertz, 0.1 volt}, {80 hertz, 0.6 volt},..., {4988 hertz, 14.1 volt}, and so on.

[0033] Next, step 12 is processed. First, a microscopic physical model of the sound source is constructed. This involves creating a three-dimensional model of a real loudspeaker unit, for example, a 3-inch midrange speaker used in a car audio system, using computer-aided design software. The model includes all key components that significantly affect the acoustic performance, such as the diaphragm, suspension, dust cap, voice coil, spider, magnetic circuit system (including the magnet, T-iron, and gap plate), and basket. Subsequently, each component of the model is assigned precise material properties, such as the Young's modulus, Poisson's ratio, and density of the diaphragm; the material properties of the voice coil former and the electrical conductivity of the copper wire; and the residual magnetic flux density of the magnet. These parameters collectively determine the electromechanical characteristics of the sound source. Next, a simulation environment is constructed. In the simulation software, a sufficiently large computational domain, i.e., a spherical or cubic air domain, is created around the three-dimensional model of the sound source to simulate the space in which sound waves propagate. To ensure that the simulation results are not disturbed by reflections at the boundaries of the computational domain, thereby simulating an anechoic chamber or free-field environment, a Perfectly Matched Layer (PML) boundary condition is applied to the outermost layer of the air domain. The PML can absorb all incoming sound waves at the boundary, preventing the generation of false reflections. Meanwhile, within the air domain, a virtual closed sampling sphere is defined concentrically with the sound source, with a radius set in the far-field region of the sound source, for example, 1 meter, to ensure that the captured sound field has a stable directivity. The surface of this sampling sphere is pre-divided into a dense grid of points for subsequent accurate recording of sound pressure data. In a feasible embodiment, the microscopic physical simulation engine is COMSOL with AC / DC & Acoustics Module, a multi-physics coupling simulation software that integrates AC / DC and acoustic modules. Its internal architecture allows for bidirectional or unidirectional coupling calculations between multiple physical fields. When the simulation task is started, the engine performs a coupled, serialized calculation process for an input simulation task parameter point, such as {500 Hz, 1.0 V}: First, the AC / DC module applies the input driving voltage of 1.0 V and frequency of 500 Hz as excitation to the electrical port of the voice coil. The software solves Maxwell's equations in the frequency domain to calculate the alternating current flowing through the voice coil and the Lorentz force generated on the voice coil after the interaction of the current with the static magnetic field produced by the permanent magnet. This force is the original driving force for the vibration of the loudspeaker, and its magnitude and phase are related to the input voltage and frequency. Second, the Lorentz force distribution calculated in the previous step is applied as a load to the voice coil and the vibrating components connected to it, such as the diaphragm and suspension. The solid mechanics module solves the equations of elastic mechanics based on the material properties and geometric structure of these components to obtain the complex vibration displacement or velocity distribution of each point on the diaphragm surface at a frequency of 500 Hz. This distribution is complex and not simply a piston motion. Finally, the complex vibration velocity distribution of the diaphragm surface is passed to the acoustic module as an acoustic boundary condition.The acoustic module solves the Helmholtz equation (governing equation for steady wave acoustics) over the pre-defined air domain to calculate the complex sound pressure at every point in the entire sound field space. The end result of this simulation process is the raw sound pressure field data. Specifically, it refers to the complex sound pressure values recorded at all grid points on the pre-defined 1 meter radius virtual sampling sphere. This data is a collection of numerical values, each of which contains both amplitude and phase information, collectively precisely depicting the complete 3D form of the sound radiation from the source into space at 500 Hz, 1.0 volt drive. Finally, the simulation software is controlled by an automated script to repeat the complete simulation process for each parameter point in the simulation task parameter point list. Each time a calculation is completed, the resulting raw sound pressure field data is associated with and stored with the corresponding input parameter point.

[0034] Finally, step 13 is processed. The feature transformation is applying spherical harmonic expansion. Spherical harmonics are a set of standard orthogonal basis functions defined on a sphere, any square-integrable function on a sphere (such as the complex pressure distribution here) can be uniquely decomposed into a weighted sum of these basis functions. These weight coefficients, i.e. the spherical harmonic coefficients, can completely and losslessly encode the amplitude and phase information of the original sound field in all directions. A pressure distribution p(θ, φ) defined on a sphere, where θ is the zenith angle and φ is the azimuth angle, can be expressed as: p(θ, φ) = Σ[n=0 to N] Σ[m=-n to n] c_nm*Y_nm(θ, φ). Where N is the highest order of expansion, Y_nm(θ, φ) is the n-th order m-th spherical harmonic basis function, and c_nm is the corresponding complex spherical harmonic coefficient. The implementation of this step is to solve these coefficients c_nm. Using the orthogonality of spherical harmonics, each coefficient can be obtained by integrating the product of the pressure distribution and the conjugate of the corresponding basis function over the entire sphere: c_nm=∫[Ω]p(θ, φ)*Y_nm*(θ, φ)dΩ, where Y_nm*is the complex conjugate of Y_nm, and the integration is performed over the entire unit sphere Ω. In actual numerical calculation, since the original sound field data is given on discrete grid points, the above integral operation is approximated by a high-precision numerical summation. The specific process is as follows: First, the highest order N of the spherical harmonic expansion needs to be determined. The value is crucial, which determines the dimension of the feature vector and the ability to describe the spatial details of the sound field. The selection of order N needs to follow the spatial sampling theorem, which requires N > k*r, where k is the wave number of the sound wave (k = 2πf / c, f is the frequency, c is the sound speed), and r is the radius of the sampling sphere. For example, for the highest frequency of 5000 Hz covered by the present application, the sound speed c is 343 m / s, and the sampling radius r is 1 m, then the wave number k is about 91.6. Therefore, in order to ensure that the sound field information is not distorted at the highest frequency, the highest order N should be set to an integer greater than 91.6, for example, N = 92. To ensure the consistency of the vector dimension of all training samples, the highest order N = 92 is used for transformation for the simulation results of all frequencies. After determining the highest order N, the transformation process can begin. For a given original sound field data set corresponding to a {frequency, voltage} pair, the program will iterate through all the coefficients c_nm to be solved (n from 0 to 92, m from -n to n). For each coefficient c_nm to be solved, the program will again iterate through all I grid points on the sampling sphere. At the i-th grid point, read its corresponding complex pressure value p_i, get the spherical coordinates (θ_i, φ_i) of the point, and calculate the conjugate value Y_nm*(θ_i, φ_i) of the spherical harmonic basis function corresponding to the point. At the same time, it also needs to know the small spherical area element ΔΩ_i represented by the grid point.Then, multiply the three (p_i * Y_nm * (theta_i, phi_i) * deltaOmega_i) and accumulate the product. After traversing all the grid points, the sum is the approximation of the coefficient c_nm. After calculating the coefficients of all orders and degrees, (N+1)2 complex coefficients are obtained. In this example, N=92, so (92+1)2=8649 complex coefficients are obtained. These coefficients are arranged in a predefined order (for example, in the order of increasing n, then increasing m) to form a one-dimensional complex vector, that is, the spherical harmonic coefficient vector. It completely encapsulates the complete three-dimensional sound field information formed by the sound source at 1 meter under the driving of {500 Hz, 1.0 V} in an extremely compact manner.

[0035] Finally, the transformation process is automatically applied to all original sound pressure field data files generated in the previous step. Each data file is converted into a corresponding spherical harmonic coefficient vector through this process, thereby completing the construction of the entire training data set.

[0036] In step 2, a multi-layer perceptron-based surrogate model is trained based on the training data set to obtain a trained surrogate model. Accordingly, through the previous steps, a high-quality training data set containing a large number of discrete data points has been constructed. However, this data set itself is static and discrete, and cannot directly meet the real-time and continuous simulation requirements, because the frequency and voltage parameters of the driving sound source in actual applications can be any value, not just the grid points pre-set in the data set. If a simple interpolation method is used, the accuracy of the sound field, which is a highly nonlinear physical phenomenon, cannot be guaranteed. Therefore, in order to establish a continuous prediction calculation model that can learn and generalize the potential physical laws from discrete training data, thereby completely replacing the time-consuming first-principle-based simulation process with a lightweight, real-time callable mathematical function, the present application introduces and trains a surrogate model.

[0037] In a feasible embodiment, the execution process of step 2 of the present application is as follows: Before the training starts, the training data set needs to be preprocessed first. The complete data set is randomly divided into training set, validation set and test set according to a certain proportion, for example 8:1:1. The training set is used for learning and updating the model parameters; the validation set is used for monitoring the model performance during the training process, adjusting the hyperparameters and preventing overfitting; the test set does not participate in the training process at all, and is only used for objective evaluation of the generalization ability of the final model after the training is completed. In addition, due to the huge difference in the numerical range of the input features (frequency and voltage) and the output label (spherical harmonic function coefficient), the data needs to be normalized. For example, the two input features of frequency and voltage are mapped to the interval [0, 1] through the min-max scaling method. For the spherical harmonic function coefficient vector as the output label, since it is a complex vector, it needs to be split into real and imaginary parts, and combined into a longer real vector. For example, a complex vector with a dimension of 8649 will be converted into a real vector with a dimension of 17298.

[0038] Next, the specific architecture of the proxy model based on multilayer perception is defined. The model is a feedforward neural network, whose structure includes an encoder, i.e. an input layer, multiple hidden layers and a decoder, i.e. an output layer. Specifically, the number of neurons in the input layer corresponds to the dimension of the input features. In this embodiment, the input features are the normalized frequency and voltage, so the input layer contains 2 neurons. The hidden layer is the core of the model to learn the complex nonlinear mapping relationship. In order to have sufficient fitting ability, a deeper network structure can be designed, for example, containing 5 hidden layers. The number of neurons in each hidden layer can be set to, for example, [1024, 1024, 512, 512, 256]. Each layer of neurons is fully connected to all neurons of the previous and next layers. After each hidden layer, a nonlinear activation function, such as the rectified linear unit ReLU or its variants, is applied to give the model the ability to learn nonlinear relationships. The number of neurons in the output layer must be exactly the same as the dimension of the preprocessed output label. In this embodiment, the output label is the spherical harmonic function coefficient vector split into real and imaginary parts, with a dimension of 17298, so the output layer contains 17298 neurons.

[0039] The process of model training is an iterative optimization process. First, all the weight matrices and bias vectors in the hidden layers and the output layer of the model are randomly initialized. Then, the hyperparameters of the training process are set, including the learning rate such as 0.001, the batch size such as 128, and the training period such as 500. After the training starts, data is taken out from the training set in batches. For each batch of data, the following operations are performed: the input features in the batch, i.e., the normalized frequency and voltage, are sent to the input layer of the model, and the data flows through each hidden layer in turn, and after weighted summation and activation function transformation, the model's predicted spherical harmonic coefficient vector (in real form) is finally obtained in the output layer. The model's predicted value is compared with the true label (the true spherical harmonic coefficient vector) corresponding to the batch of data, and a predefined loss function is used to quantify the difference between them. For regression tasks, mean squared error (MSE) is selected as the loss function. According to the loss value, an optimization algorithm (such as the Adam optimizer) is used to calculate the gradient of the loss function with respect to each weight and bias in the model through the backpropagation algorithm. According to these gradient information, the optimizer adjusts the model's all weights and biases slightly according to the set learning rate, so that the model's next prediction can be closer to the true value.

[0040] This loop will go through all the batches in the training set to complete a training period. The entire training process will be repeated for hundreds of cycles as set. At the end of each cycle, the performance of the model will be evaluated on the validation set. If the loss on the validation set does not decrease for several consecutive cycles, an early stopping mechanism can be triggered to prevent overfitting, and the model parameters at this time are saved.

[0041] After training, the trained proxy model is obtained. It is a solidified multilayer perceptron network that contains the optimized weight and bias parameters. This model can now serve as an efficient computational function, receiving any {frequency, voltage} pair within the training range as input, and outputting a high-precision, predicted spherical harmonic coefficient vector through a single fast forward propagation calculation.

[0042] In particular, in the field of audio simulation, the core task of a surrogate model is to accurately and robustly predict a high-dimensional and extremely complex-structured output (e.g., a vector composed of thousands of spherical harmonic coefficients) from relatively simple inputs (e.g., two-dimensional frequency and voltage). These coefficients are not isolated numerical values, but collectively encode a three-dimensional sound field that follows the laws of wave physics, and thus have strong nonlinearity and correlation within them. If a standard multi-layer perceptron (MLP) is used as a surrogate model, it is essentially a general-purpose black-box function fitter. It will try to learn the complex mapping from input to high-dimensional output directly, but in the process it will ignore two crucial priori knowledge: one is that the high-dimensional spherical harmonic coefficient vector is not randomly distributed in space, but exists on an intrinsic low-dimensional smooth manifold; the second is that the sound field represented by these coefficients must comply with basic physical laws such as the Helmholtz equation. Therefore, in order to overcome the defects of traditional models that lack structural priori and physical constraints, resulting in poor generalization ability, large data requirements, and possibly making physically impossible predictions, the present application proposes a hybrid surrogate model based on physical information constraints and structured latent space learning, aiming to inject physical priori knowledge and data intrinsic structure into the model, thereby greatly improving the accuracy and robustness of prediction.

[0043] Further, in a feasible embodiment, training the surrogate model comprises:

[0044] obtaining a spherical harmonic coefficient vector and an excitation parameter vector, wherein the excitation parameter vector is a {frequency, voltage} pair. It should be understood that this step is the data preparation stage of model training, providing paired input and target output for supervised learning. It provides basic data for subsequent encoding, reconstruction and loss calculation, ensuring that the model learns the mapping relationship in the real physical world. In particular, the acquisition of these two data is the same as the acquisition process of the training data in the above embodiment.

[0045] The spherical harmonic coefficient vector is input into the sound field encoder of the proxy model to obtain a spherical harmonic coefficient latent space encoding vector. Correspondingly, considering that the spherical harmonic coefficient vector, although high-dimensional, describes the sound field of the same physical entity (such as a loudspeaker) under different excitations, these vectors are actually distributed on a low-dimensional smooth nonlinear manifold. For example, a slight change in frequency will only cause a smooth transition in directivity, which is reflected as a continuous and relevant change rather than a sudden change in the coefficient vector. Therefore, by means of a special sound field encoder (for example, a neural network composed of multiple fully connected layers and nonlinear activation functions), the high-dimensional spherical harmonic coefficient vector can be mapped to a latent space with lower dimension and more concentrated information. This achieves effective dimension reduction and feature extraction of the data, and converts the original complex data into a compact representation that is easier for the model to learn, i.e., the spherical harmonic coefficient latent space encoding vector. This not only avoids the difficulty of learning directly in a high-dimensional space, but also lays the foundation for subsequent structural consistency constraints, improving the learning efficiency and generalization ability of the model.

[0046] The excitation parameter vector is input into the encoder of the proxy model based on the multilayer perceptron to obtain an excitation parameter latent space encoding vector, wherein the spherical harmonic coefficient latent space encoding vector and the excitation parameter latent space encoding vector have the same length. It can be understood that the excitation parameters (frequency and voltage) are low-dimensional, but they are key factors driving the behavior of the sound source. Through the encoder of the proxy model based on the multilayer perceptron in the above embodiment, these excitation parameters can be mapped to a latent space with the same length as the spherical harmonic coefficient latent space encoding vector. This provides a unified latent space representation for the subsequent cross-reconstruction loss function, enabling the model to compare and correlate information from different sources (real sound field data and excitation parameters) in the same low-dimensional space, thereby facilitating the model to learn the intrinsic and structured relationship between the excitation parameters and the sound field characteristics.

[0047] The context vectors of the spherical harmonic coefficient vector and the spherical harmonic coefficient latent space encoding vector and the context vectors of the excitation parameter vector and the excitation parameter latent space encoding vector are calculated to obtain a sound field context vector and an excitation parameter context vector; and the sound field context vector and the excitation parameter context vector are cross-reconstructed to obtain a cross-reconstruction loss function, i.e.,

[0048]

[0049]

[0050]

[0051] wherein, is the spherical harmonic coefficient vector, , is the spherical harmonic order, is a spherical harmonic coefficient latent space encoding vector, is a soundfield encoder, is an excitation parameter vector, , and have the same length, and are, for example, , is an excitation parameter latent space encoding vector, is an encoder based on a multi-layer perceptron proxy model, is an auxiliary encoder, is a soundfield context vector, is an excitation parameter context vector, is a point-wise multiplication, is a computation of a two-norm, is a cross-reconstruction loss function. Accordingly, further, the computation of the cross-reconstruction loss can ensure that the mapping relationship of the two different sources of information (soundfield data and excitation parameters) learned by the model in the latent space has consistency and compatibility, thereby forcing the model to understand the deep structure behind the data. That is, this step is not simply a cross-entropy calculation, but rather a sophisticated structure consistency loss. In specific implementation, first, through an auxiliary encoder, the soundfield context vector and the excitation parameter context vector are calculated respectively based on the original vector and its latent space encoding. That is, the auxiliary encoder can be constructed as a multi-layer perceptron network. Its encoding process starts with concatenating its two inputs, i.e., the original data vector (e.g., the spherical harmonic coefficient vector) and its corresponding latent space encoding vector, to form a higher-dimensional combined feature vector. Subsequently, the combined feature vector is input into a network structure composed of one or more fully connected layers, and a non-linear activation function is applied after each layer. The final output of the multi-layer perceptron network is the context vector, which encapsulates the associated information needed for subsequent cross-reconstruction loss calculation. Subsequently, the latent space representation of one vector is reconstructed using the other vector, i.e., using to weight , using to weight , and calculating the difference between the two. Finally, the cross-reconstruction loss function is generated. By minimizing this loss, it can be ensured that the bidirectional mapping based on the information latent space conforms to the physical manifold structure of the high-dimensional spherical harmonic coefficient vector, avoiding the model producing physically unreasonable mapping directionality, greatly enhancing the model's generalization ability and the robustness of the prediction.

[0052] The spherical harmonic coefficient latent space encoding vector is input into the decoder of the surrogate model to obtain the spherical harmonic coefficients. It should be understood that during the training process, in order to calculate the physical loss, the excitation parameter latent space encoding vector needs to be input into the decoder of the surrogate model. The decoder (also a multi-layer perceptron network) is responsible for mapping the low-dimensional latent space encoding vector back to the high-dimensional spherical harmonic coefficient vector, generating the spherical harmonic coefficients predicted by the surrogate model based on the excitation parameters. These predicted coefficients are the model's estimate of the sound field characteristics of the sound source under a specific excitation, and they will be used as input for the next step of physical loss calculation to verify whether they meet the physical laws.

[0053] The spherical harmonic coefficients are subjected to sound field reconstruction based on the Helmholtz equation to obtain the physical loss function, i.e.,

[0054]

[0055]

[0056]

[0057] wherein, is the Laplacian operator, and is the wave number, is the frequency, e.g., 5000 Hz in step 13, is the sound speed, i.e., 343 m / s, is the amplitude coefficient related to the spherical harmonic coefficients and the sound source characteristics, is the second kind of spherical Hankel function (describing outward spherical waves), is the spherical basis function, is the zenith angle, is the azimuth angle, is the radius of the sampling sphere, e.g., 1 meter in step 13, is the highest order of expansion, e.g., 92 in step 13, is the sound pressure outside the sound source, is the reconstructed sound field of the spherical harmonic coefficients, is the physical loss function. Accordingly, in free space, the sound pressure field p outside any single-frequency sound source needs to satisfy the Helmholtz equation. In implementation, the latent space encoding vector of the excitation parameters is input into the decoder of the surrogate model to obtain the reconstructed spherical harmonic coefficients. Then, using the inverse spherical harmonic transform formula, the predicted sound field is reconstructed at multiple spatial sampling points around the sound source. Finally, the residual form of the Helmholtz equation is substituted into The loss function quantifies the deviation of the model prediction from the physical truth. By including this term in the total loss, the model is forced to not only fit the data distribution, but also to obey the physical laws, thus ensuring that its output is physically self-consistent and avoiding arbitrary, physically inconsistent interpolated outputs.

[0058] The gradient descent backpropagation of the agent model is performed with the sum of the cross-reconstruction loss function and the physical loss function as the total loss function, i.e.

[0059]

[0060] wherein, is the total loss function. Finally, the learning direction of the model is guided by the aforementioned two loss functions. In specific implementation, the cross-reconstruction loss function and the physical loss function are weighted and summed to form the total loss function. Then, an optimization algorithm such as gradient descent is used to calculate the gradient of the total loss function with respect to all trainable parameters (weights and biases of the encoder and decoder) in the model, and update these parameters in the direction of the gradient descent. Through continuous iteration of this process, the agent model is driven to find an optimal solution that can minimize both structural inconsistency and physical inconsistency. The trained agent model obtained finally, through the dual constraints of data and physics, can more accurately and robustly infer the complex, high-dimensional, and physically self-consistent spherical harmonic function coefficients from simple excitation parameters. Therefore, the optimized agent model can generate more accurate and physically self-consistent spherical harmonic function coefficients based on real-time driving parameters in the real-time inference stage, thereby significantly improving the accuracy and robustness of the prediction results.

[0061] In step 3, real-time driving parameters from the upper application are obtained, including the frequency and voltage of the simulation. It can be understood that the prequel has successfully constructed and trained a lightweight, high-precision agent model, which has solidified the complex physical behavior of the sound source in its network parameters and has the potential to replace time-consuming physical simulation. However, this trained model itself is a static, waiting-to-call computing entity. In order to transform it from an offline, trained asset into a dynamic tool that can serve real-time interactive design, a clear, immediate instruction is needed to trigger its prediction function. Therefore, the core purpose of this step of obtaining real-time driving parameters from the upper application is to provide a starting point and specific calculation target for the entire real-time simulation link.

[0062] In one possible embodiment, the execution of step 3 of the present application is as follows: in one application scenario, a car audio engineer is using a virtual cabin acoustic design software to evaluate the listening experience at the driver's position. The software interface may contain controls for setting the sound source signal, such as a text box for inputting a specific frequency and a slider for adjusting the volume.

[0063] When the engineer wants to evaluate the performance of the loudspeaker under a certain working condition, he interacts with the software interface. For example, the engineer types in the value 1200 in the frequency input box, representing that he wants to simulate a 1200 Hz sine wave signal. Then, he adjusts the volume slider to an intermediate position, which is pre-calibrated in the software as corresponding to a -6 dB attenuation.

[0064] At this moment, the step of acquiring real-time driving parameters is triggered. The internal logic of this step translates these user-friendly inputs from the graphical interface into the physical driving parameters required by the surrogate model in real time. The software directly reads the value 1200 in the frequency input box as the simulated frequency parameter. Therefore, the simulated frequency is 1200 Hz. The acquisition of the voltage parameter requires a pre-set conversion logic. The software needs a reference benchmark to interpret the meaning of -6 dB. For example, it can be pre-set that 0 dB corresponds to the maximum undistorted voltage that the power amplifier circuit driving the loudspeaker can output, such as 15 volts. According to the relationship between decibels and voltage amplitude, dB = 20*log10(V / V_ref), the software internally performs the calculation: V = V_ref*10^(dB / 20) = 15*10^(-6 / 20). The calculation result is approximately 7.52 volts. Therefore, the software takes this calculation result as the simulated voltage parameter, i.e., the simulated voltage is 7.52 volts.

[0065] After the above acquisition and conversion are completed, this step combines the two values into a structured data pair: {simulated frequency: 1200, simulated voltage: 7.52} as the real-time driving parameters.

[0066] In step 4, the real-time driving parameters are input into the trained surrogate model to obtain the real-time predicted spherical harmonic coefficients. It should be understood that in the real-time application stage, a specific simulation request from the upper application has been obtained, i.e., an explicit frequency and voltage combination. At the same time, a surrogate model that has been fully trained and can simulate the physical behavior of the sound source is also ready. However, the acoustic simulation of the macro environment (such as ray tracing) cannot directly use the two scalar parameters of frequency and voltage, and it needs a standardized data interface that can describe the complete three-dimensional radiation characteristics of the sound source. Therefore, in order to immediately convert the user's high-level, real-time request into a detailed, directly usable sound source description for the subsequent macro simulation engine, a fast prediction calculation needs to be performed. Thus, the real-time driving parameters are input into the trained surrogate model to provide key, high-fidelity sound source data for the entire real-time simulation link.

[0067] In a feasible embodiment, the execution process of step 4 of the present application is as follows: before performing the prediction, the trained surrogate model needs to be loaded first. The model is obtained in the offline training stage, and its specific form is a multilayer perceptron with a determined network structure and fixed parameters. All the weights and bias values inside have been optimized and determined through learning on a large amount of training data, and will not change in the real-time application stage. The model can be regarded as a highly complex, nonlinear mathematical function, and the input and output dimensions have been predefined.

[0068] When receiving the real-time driving parameters such as {1200 Hz, 7.52 V}, the input needs to be preprocessed in exactly the same way as during training to ensure that the input data is within the numerical range familiar to the model. That is, normalization processing. As in the training stage, the normalized range of frequency is [80 Hz, 5000 Hz], and the normalized range of voltage is [0.1 V, 15 V]. Then, the real-time input parameters will be converted to: normalized frequency = (1200-80) / (5000-80)≈0.2276, normalized voltage = (7.52-0.1) / (15-0.1)≈0.4980. The two normalized values [0.2276, 0.4980] form a two-dimensional input vector.

[0069] Next, the two-dimensional input vector is sent to the input layer of the trained surrogate model. The architecture of the surrogate model is exactly the same as defined during training, for example: an input layer containing 2 neurons, 5 hidden layers (the number of neurons is 1024, 1024, 512, 512, 256, and the rectified linear unit activation function is used), and an output layer containing 17298 neurons (using a linear activation function).

[0070] The data is computed once forward through the model. The specific process is as follows: the input vector [0.2276, 0.4980] is fed into the input layer. This vector is multiplied by the weight matrix of the first hidden layer, and the bias vector of this layer is added. The result is non-linearly transformed by the activation function of the first hidden layer. The transformed output is taken as the input of the second hidden layer, and the above multiplication, addition, and activation process is repeated. The data flows through all the hidden layers in turn, and each layer performs a complex non-linear feature extraction and mapping on the data. The output of the last hidden layer is fed into the output layer, multiplied by the weight matrix of the output layer, and the bias vector is added. Since the output layer is linear, its result is directly taken as the final output of the model.

[0071] The final output of the model is a one-dimensional real number vector with a dimension of 17298. This vector is the direct prediction result of the model on the spherical harmonic coefficients. However, the spherical harmonic coefficients themselves are complex numbers. Therefore, the original output vector needs to be decoded to restore it to a complex number form. This decoding process is exactly the opposite of the encoding process in the preprocessing of the training data. During training, a complex number vector with a dimension of 8649 is split into a real number vector with a dimension of 17298, i.e., the first 8649 elements are the real parts, and the last 8649 elements are the imaginary parts. Therefore, during decoding, the first 8649 elements of the 17298-dimensional real number vector output by the model are taken as the real parts of the predicted spherical harmonic coefficients, and the last 8649 elements are taken as the imaginary parts. Then, these two parts are recombined to generate a complex number vector with a dimension of 8649.

[0072] This finally generated complex number vector with a dimension of 8649 is the predicted spherical harmonic coefficient, which accurately and instantaneously describes the three-dimensional sound field radiation characteristics of the sound source under the driving of 1200 Hz and 7.52 volts.

[0073] In step 5, macro-environmental acoustic simulation is performed based on the real-time predicted spherical harmonic coefficients to obtain an acoustic impulse response. Accordingly, the previous steps have successfully converted the user's real-time simulation request into a high-fidelity spherical harmonic coefficient vector that accurately describes the inherent radiation characteristics of the sound source. However, this vector only describes the behavior of the sound source in a free field, and it does not contain any environmental information. In a real-world scenario, the sound perceived by a listener is the result of the superposition of the direct sound emitted by the sound source and the reflected sound after countless reflections, scattering, and absorption by various surfaces in the environment, such as walls, furniture, and car interiors. Therefore, in order to combine the accurate description of the sound source with the complex acoustic environment of the real world and calculate the complete acoustic transmission path from the sound source to a specific listener position, the application needs to perform macro-environmental acoustic simulation on the coefficients to simulate the propagation of sound waves in a specific scenario, and finally generate an acoustic impulse response that can fully represent the characteristics of the source-field-listener link.

[0074] In one possible embodiment, Figure 4 Figure 5 is a flowchart of step 5 of the audio simulation method according to an embodiment of the present application. As shown in Figure 5, step 5 of the present application, which is macro-environment acoustic simulation based on the real-time predicted spherical harmonic coefficients to obtain the acoustic impulse response, comprises: step 51, initializing a macro acoustic simulation engine to obtain an initialized scene context; step 52, spherical harmonic source instantiation on the sound source position and the real-time predicted spherical harmonic coefficients to obtain a sound source emission profile; step 53, directional ray tracing and hit data collection based on the sound source position, the sound source emission profile and the initialized scene context to obtain a listener hit log; and step 54, acoustic impulse response synthesis on the listener hit log to obtain the acoustic impulse response. Figure 4

[0075] ​In the above embodiment, the execution process of step 5 of the present application is as follows: first, step 51 is processed. It is worth mentioning that the macroscopic acoustic simulation engine is a computational framework based on the principles of geometric acoustics, especially the algorithm of sound ray tracing or its variants. The core function of this engine is to simulate the propagation path of sound energy in three-dimensional space. In the initialization stage, the engine performs a series of operations to build the scene context. First, the engine loads the three-dimensional geometric model. Taking a car audio simulation application as an example, this model is a high-precision three-dimensional model file of the car cabin, stored in OBJ or FBX format, for example. The model accurately describes all the main geometric structures inside the cabin, such as the instrument panel, A / B / C pillars, front and rear windshields, door linings, seats, carpets, and ceilings, etc., which are composed of a large number of triangular patches. After loading the geometric model, the engine does not directly use these original triangular patch data for calculation, because in the process of tracing millions of sound rays, the computational cost of intersecting with all the patches in the scene one by one is extremely high. Therefore, the engine performs a key preprocessing on the geometric data, that is, it constructs a spatial acceleration data structure. A common implementation is to construct a bounding volume hierarchy, namely BVH. This structure recursively divides the entire scene space into a series of nested, smaller bounding boxes. Each bounding box contains a part of the geometric patches. In subsequent sound ray tracing, a sound ray can first be tested for intersection with these hierarchical bounding boxes. If a sound ray does not intersect with a large parent bounding box, all the child bounding boxes and geometric patches contained in it can be safely ignored, greatly reducing unnecessary intersection calculations and significantly improving simulation efficiency. Next, the engine loads and assigns the acoustic properties of the materials. This is achieved through an external material library file, such as a JSON or XML file. This library file defines the various materials that may appear in the scene and provides frequency-dependent acoustic parameters for each material. For example, a material entry named seat_leather may contain the following data: at 125 Hz, the absorption coefficient is 0.15 and the scattering coefficient is 0.1; at 500 Hz, the absorption coefficient is 0.4 and the scattering coefficient is 0.2; at 2000 Hz, the absorption coefficient is 0.6 and the scattering coefficient is 0.3, and so on. In the three-dimensional geometric model, each surface or each triangular patch is pre-assigned a material identifier such as seat_leather. During initialization, the engine iterates through all the geometric patches, reads their material identifiers, then looks up the corresponding acoustic data in the material library, and associates these data with the corresponding geometric patches, storing them in memory. At the same time, this step also needs to define the listener information. In the upper-layer application, the user can specify one or more listener positions. For example, in a car audio simulation, a typical listener position is the driver's head position, whose coordinates are set to (x, y, z).To more realistically simulate the effect of a human head receiving a sound field, the listener is not defined as an infinitely small point, but as a geometric body of a certain size, for example a spherical receiver with a diameter of 17 cm. The engine stores the geometric information and the position of this receiver. Finally, the engine sets global parameters of the simulation, for example the air temperature and the relative humidity in the environment, which determine the speed of sound and the absorption effect of the air on high-frequency sound waves. For example, if a temperature of 22 degrees Celsius and a relative humidity of 50% are set, the engine calculates a speed of sound of approximately 344 m / s. After all the above operations, the geometric loading and the BVH construction, the assignment of material properties, the definition of the listener, the setting of the global parameters, a highly optimized memory data structure containing all this information is formed, which is ready for acoustic calculations at any time, this data structure is the initialized scene context.

[0076] Next, step 52 is processed. In one possible embodiment, the present application step 52, spherical harmonic source instantiation is performed on the sound source position and the real-time predicted spherical harmonic coefficients to obtain the sound source emission profile, including: step 521, inverse transform of the spherical harmonic function is performed on the real-time predicted spherical harmonic coefficients to obtain the complex sound pressure of the sound source in any direction on the unit sphere; step 522, the square modulus of the complex sound pressure of the sound source in any direction on the unit sphere is calculated to obtain the sound source emission profile. Specifically, first, step 521 is processed. It is worth mentioning that the inverse transform process is the mathematical inverse operation of the spherical harmonic expansion, which is to use the spherical harmonic series summation formula: p(θ,φ)=Σ[n=0to N]Σ[m=-n ton] c_nm*Y_nm(θ,φ). When this calculation is implemented numerically, a set of dense sampling points on the unit sphere need to be defined first. The density of the sampling points determines the spatial resolution of the reconstructed sound field. One possible implementation is to use an equiangular interval sampling grid. For example, the zenith angle θ can be set from 0 degrees to 180 degrees with a step size of 1 degree; the azimuth angle φ from 0 degrees to 360 degrees, also with a step size of 1 degree. In this way, 181*360=65160 discrete direction points are generated on the unit sphere. Next, for each of the 65160 direction points (θ_i,φ_i), a summation calculation is performed. In the calculation, c_nm is the complex value taken by index from the input 8649-dimensional coefficient vector, i.e. the real-time predicted spherical harmonic coefficients, while Y_nm(θ_i,φ_i) is the value of the standard spherical harmonic basis function at that direction point, which is a complex value that can be calculated in real time by a standard mathematical library function. By multiplying the coefficients of the corresponding order and degree with the basis function value, and adding up the products of all terms (from n=0, m=0 to n=92, m=92), the complex sound pressure value p_i at the direction point (θ_i,φ_i) can be obtained. This complex number p_i contains both the amplitude and phase information of the sound pressure. After traversing all 65160 direction points and completing the calculation, a data array is obtained, which stores all the sampling direction points on the unit sphere and their corresponding complex sound pressure values. Next, step 522 is processed. Geometric acoustics methods (such as ray tracing) essentially track the flow of sound energy, and their calculations do not directly handle the phase. Therefore, it is necessary to convert the complex sound pressure into the sound intensity or sound energy related to it. The input is the set of complex sound pressure values in each direction on the unit sphere generated by the previous sub-step. In specific implementation, each complex sound pressure value p_i in this set is traversed. For each p_i, the square modulus |p_i|² is calculated. Mathematically, the square modulus of a complex number z=a+bj is equal to a²+b², which is also equal to the product of the complex number and its complex conjugate. This calculation result is a non-negative real number, which is proportional in physical meaning to the sound power per unit area, i.e. the sound intensity, in that direction.After the square modulus calculation of the complex sound pressure value of all direction points, a new set of data is obtained, which maps each direction point on the unit sphere to a real number value. This mapping relationship, i.e. the sound intensity distribution of all directions, collectively constitutes the sound source emission profile. That is, the sound source emission profile can be imagined as a three-dimensional energy balloon, the surface of which bulges differently in different directions, accurately depicting the strength distribution of the energy radiation of the sound source under the current driving condition. For example, for a typical loudspeaker, in the mid-frequency band, its emission profile may be strongest in the front direction (axial direction), forming a main lobe, while the energy is weaker in the side and back directions.

[0077] Afterwards, step 53 is processed. First, the specific position of the sound source in the scene is obtained, which is specified by the upper-layer application. For example, the center coordinate of the left front door speaker in the car cabin. The core of this process is directional ray tracing. The simulation engine first launches a large number of virtual sound rays from the sound source position. To ensure the accuracy and convergence of the statistical results, the total number of launches is set to a very large value, for example, 10 million. Unlike traditional isotropic sound sources (energy radiates uniformly in all directions), the sound ray launch of this method has clear directivity. The initial launch direction of each sound ray is not randomly and uniformly distributed, but is probabilistically sampled according to the sound source emission profile. In specific implementation, the sound source emission profile is regarded as a three-dimensional probability density function. The engine adopts a method such as inverse transform sampling, which integrates the probability density function to obtain its cumulative distribution function, and then generates a uniformly distributed random number and maps it on the inverse function of the cumulative distribution function to determine a specific launch direction. This method ensures that the direction with higher energy intensity in the sound source emission profile has a higher probability of being selected as the launch direction. At the same time, the initial energy of each sound ray is also weighted according to the energy value of its launch direction in the profile, thereby efficiently concentrating computing resources in the main radiation area of the sound source. Once the sound ray is launched, it carries its initial energy and propagates along a straight line in the virtual three-dimensional space defined by the initialized scene context. During propagation, the sound ray continuously loses energy due to air absorption, and the amount of loss is related to the propagation distance, simulation frequency, and the air temperature and humidity set in the scene context. The simulation engine uses the spatial acceleration data structure constructed in the initialization step, such as the bounding volume hierarchy (BVH), to efficiently calculate the next intersection point of the sound ray with the geometric surfaces in the scene. When a sound ray intersects with a surface, the engine performs a series of physical interaction calculations. First, the acoustic material properties of the surface at the collision point corresponding to the current simulation frequency are queried from the scene context. The energy of the sound ray will be further attenuated according to the absorption coefficient of the surface. Subsequently, the remaining energy of the sound ray will be probabilistically determined according to the scattering coefficient of the surface to determine the reflection type. The engine generates a random number and compares it with the scattering coefficient: if the random number is less than the scattering coefficient, diffuse reflection occurs, and the remaining energy of the sound ray is scattered to a random direction in the hemisphere space of the collision point according to a diffuse reflection model (such as Lambert's cosine law); if the random number is greater than or equal to the scattering coefficient, specular reflection occurs, and the sound ray continues to propagate in a determined new direction according to the law of physical reflection with an equal reflection angle to the incident angle. During each propagation and interaction of the sound ray, the simulation engine continuously monitors whether the path of the sound ray intersects with the pre-set listener receiver, for example, a 17-centimeter-diameter sphere at the driver's head position. Once a sound ray successfully passes through or contacts the listener receiver, the event is considered a hit and is immediately recorded to form a new hit record.Each hit record contains a set of key information, such as: time of arrival, i.e., the total propagation time of a sound ray from its emission at the sound source to its arrival at the listener's receiver, computed by accumulating the path lengths of the segments of the sound ray's propagation in air and dividing by the speed of sound; energy of arrival, i.e., the remaining energy of a sound ray upon its arrival at the listener's receiver after experiencing multiple propagation and reflection attenuations; direction of arrival, i.e., the directional vector of a sound ray upon its entry into the listener's receiver, which is crucial for the subsequent generation of spatialized auditory sensations; and number of reflections, i.e., the number of reflections a sound ray has experienced before its arrival at the listener, with the number of reflections for the direct sound being 0. The lifetime of a sound ray is finite, and its tracing process ends when a pre-set termination condition is met. These conditions include: the energy of a sound ray decays below a pre-set threshold, e.g., below one millionth of its initial energy, i.e., -60 decibels, or its number of reflections exceeds a pre-set maximum depth, e.g., 15 times. By repeating the above tracing and hit detection for all the emitted ten million sound rays, a listener hit log containing all the successful hit events is eventually formed. This log is an exhaustive list, with each entry being a structured data record, e.g., a record in the log can contain: {time of arrival: 0.003 seconds, energy of arrival: 0.5 units, direction of arrival: [0.8, 0.6, 0], number of reflections: 1}, which precisely describes an acoustic path with one reflection and its physical attributes.

[0078] Finally, step 54 is processed. The synthesis process of the acoustic impulse response first needs to determine its basic parameters. This includes the sampling rate and the total duration, which are set according to the sampling rate requirement of subsequent audio processing or playback, and the total duration required to ensure the complete capture of acoustic events. The sampling rate is consistent with the sampling rate of subsequent audio processing or playback, for example, set to 48000 Hz. The total duration needs to be long enough to capture all important reflected sound, such as 1 to 2 seconds, or until the acoustic energy decays to a negligible level. For example, if the sampling rate is set to 48000 Hz and the total duration is 1.5 seconds, the final acoustic impulse response will be a digital sequence containing 72000 sample points. At the beginning of the synthesis process, a floating-point number array of length 72000 is initialized, with all elements set to zero. This array will serve as the carrier of the final acoustic impulse response. Subsequently, the program iterates through each hit record in the listener's hit log. For each record in the log, for example, a record is {arrival time: 0.003 seconds, arrival energy: 0.5 units, arrival direction: [0.8, 0.6, 0]}, the following operations are performed: First, according to the arrival time of the record and the preset sampling rate, the corresponding sample index of the acoustic event in the acoustic impulse response array is calculated. The calculation formula is: sample index = round(arrival time * sampling rate). For example, for an arrival time of 0.003 seconds, the sample index is round(0.003 * 48000) = round(144) = 144. Second, convert the arrival energy of the record to the sound pressure amplitude. In ray tracing, the energy of a sound ray is proportional to the square of the sound pressure. Therefore, in order to obtain the sound pressure amplitude, the square root operation needs to be performed on the arrival energy. For example, if the arrival energy is 0.5 units, the corresponding sound pressure amplitude is approximately sqrt(0.5) ≈ 0.707. Then, the calculated sound pressure amplitude is added to the position of the corresponding sample index in the acoustic impulse response array. If multiple sound rays arrive at the listener at very close time points (i.e., fall within the same sample time window), their sound pressure amplitudes will be superimposed at that sample point. For example, if in addition to the 0.707 amplitude above, another sound ray arrives at 0.00301 seconds with an energy converted to an amplitude of 0.2, the final value of the acoustic impulse response

[144] will be 0.707 + 0.2 = 0.907. It is worth noting that if the listener receiver is defined to have directionality (for example, simulating the head-related transfer function, HRTF, of a human ear), when adding the amplitudes, the sound pressure amplitude also needs to be spatialized according to the arrival direction information in the hit record. That is, for each arriving sound ray, its amplitude will be filtered according to its arrival direction through the preloaded HRTF, thereby generating independent impulse responses for the left and right ears, and finally obtaining a binaural acoustic impulse response.However, for a general mono acoustic impulse response, this step can be simplified as a direct amplitude accumulation. Repeat the above process for all hits in the listener hit log, and contribute the energy of each hit record to the corresponding time point of the acoustic impulse response array. When all hit records are processed, the array contains the complete time domain information from the direct sound to all early and late reflections. Finally, in order to ensure the usability of the acoustic impulse response, additional post-processing is needed. For example, normalize the whole impulse response so that its maximum amplitude does not exceed 1, in order to prevent clipping distortion in the subsequent convolution operation. The final generated digital sequence containing 72000 sample points is the acoustic impulse response, which is a time domain signal accurately reflecting the acoustic transmission characteristics of the sound source in a specific environment to the listener position.

[0079] In step 6, the acoustic impulse response is perceptually synthesized with the dry audio signal to obtain the audible audio. That is, in the macro-environment acoustic simulation stage, an accurate acoustic impulse response has been successfully generated. This impulse response is an abstract digital signal that contains all the propagation characteristics of sound waves from the sound source to the listener position, including direct sound, early reflections, and reverberation. However, this signal itself is silent and cannot be directly perceived by human hearing. In order to convert this abstract physical characteristic into a real listening experience that can be evaluated by hearing, so that engineers can intuitively judge the acoustic effect of the design scheme, the signal needs to be applied to the actual audio content. Therefore, the present application further synthesizes the acoustic impulse response with the dry audio signal to combine the acoustic environment characteristics with the actual audio content, thereby generating an audible audio that can be directly perceived by human hearing, as if it is playing in a simulated environment, realizing intuitive verification and evaluation of the simulation results.

[0080] In a feasible embodiment, step 6 of the present application, the acoustic impulse response is perceptually synthesized with the dry audio signal to obtain the audible audio, comprising: convolving the acoustic impulse response with the dry audio signal to obtain the audible audio.

[0081] In the above embodiment, the execution process of step 6 of the present application is as follows: It is worth mentioning that the dry audio signal is a pure, original audio signal that has not been subjected to any environmental acoustic processing. Its acquisition method can be various. For example, it can be a piece of music file (such as WAV or FLAC format) selected by the user from the local storage, a piece of pre-recorded human voice speech, or a synthetic signal for testing, such as pink noise or sweep signal.

[0082] The convolution operation is a mathematical operation that mixes one signal, the dry audio signal, with another signal, the acoustic impulse response, imprinting the characteristics of the second signal onto the first signal. In the field of acoustics, this means applying the acoustic characteristics of an environment (the impulse response) to the original sound (the dry audio signal), simulating the effect of sound propagation and reflection in a particular environment.

[0083] Due to the length of the dry audio signal, the computational load of direct convolution operation in time domain is very large and inefficient. Therefore, in actual implementation, the frequency domain convolution method based on fast Fourier transform (FFT) is adopted, also known as fast convolution. This method takes advantage of the principle that time domain convolution is equivalent to frequency domain multiplication, significantly improving the computational efficiency. The specific implementation process is as follows: First, the block size for FFT needs to be determined. In order to calculate efficiently, this block size is chosen as a power of 2, such as 4096 or 8192. In order to avoid signal truncation effects and ensure the integrity of the convolution result, the acoustic impulse response and the dry audio signal need to be zero-padded. Specifically, each processing block of the acoustic impulse response and the dry audio signal is padded to a length of two times the FFT block size minus one to accommodate the convolution result. Then, the dry audio signal is processed in blocks. For each processing block of the dry audio signal: a fixed length audio block is taken out from the dry audio signal, and it is zero-padded to a length required by FFT. The padded dry audio signal block and the entire acoustic impulse response are subjected to fast Fourier transform (FFT) respectively, converting them from time domain to frequency domain to obtain their respective complex spectra. In the frequency domain, the spectrum of the dry audio signal block is multiplied point by point with the spectrum of the acoustic impulse response. This operation is much less computationally intensive than time domain convolution. The multiplication result is subjected to inverse fast Fourier transform (IFFT) to convert it from the frequency domain back to the time domain, obtaining the convolution result of the audio block.

[0084] Since block processing can cause discontinuity between blocks, in order to obtain continuous final audio, techniques such as overlap-add or overlap-save are used. Taking the overlap-add method as an example, the convolution result of each processing block will be longer than the original block. When these results are spliced together, the overlapping parts of adjacent blocks will be added exactly, eliminating the artifacts caused by block processing and ensuring the smooth continuity of the final output audio.

[0085] The above process of blocking, transforming, multiplying and inverse transforming is repeated until the entire dry audio signal is processed. All processed audio blocks are spliced together by overlap-add method to form the final audible audio. This audio signal now contains all acoustic characteristics of the sound source propagating in the simulated environment, such as direct sound, early reflection sound, reverberation and possible spatialization effects (if the acoustic impulse response is binaural). For example, if the original dry audio signal is a pure human voice, after convolution, the listener will feel that this human voice is played in the car cabin, with the unique reverberation and reflection effects of the cabin. This audible audio can be directly played through speakers or earphones for users to evaluate the subjective listening experience, thereby verifying the accuracy of the simulation results and guiding product design optimization.

[0086] In particular, the audible audio provides engineers with an intuitive listening experience. However, it is difficult to make accurate quantitative comparisons and systematic design optimizations relying solely on subjective listening. In engineering practice, objective and repeatable acoustic indicators are often needed to judge performance. Therefore, in order to convert the subjective listening experience into a series of standardized and quantifiable objective data, thereby providing scientific basis for design iteration, performance verification and standard compliance check, the simulation results need to be post-processed and analyzed to extract recognized acoustic indicators from the simulation results, so as to realize objective evaluation of acoustic performance.

[0087] In one possible embodiment, the method further comprises the step of post-processing and analyzing the audible audio to obtain acoustic indicator analysis results, the acoustic indicator analysis results including sound pressure level, intelligibility and reverberation time at the listener position. It should be understood that although the audible audio provides an intuitive subjective experience, engineering design and optimization rely more on quantifiable objective standards. The analysis of these three core indicators is exactly to decompose the complex listening experience into key physical dimensions: sound loudness is objectively measured by sound pressure level to evaluate whether it meets the design target; the intelligibility of speech or music is used to judge the detail recognition degree, thereby quantifying the listening quality; and the spatial acoustic characteristics of the simulated environment are accurately represented by the reverberation time to evaluate the degree of sound coloring.

[0088] Specifically: first, the reverberation time is calculated. The reverberation time is the most important indicator to measure the strength of the space reverberation, which is represented by T60 or Early Decay Time (EDT). The calculation process directly uses the acoustic impulse response. First, the square operation is performed on each sample value of the acoustic impulse response signal to obtain its instantaneous energy. Then, the Shur inverse integration method is used to process the energy signal, that is, the cumulative sum is performed from the end of the signal to the front, so as to obtain a smooth energy decay curve. The energy decay curve is converted to decibel (dB) scale. In order to calculate T60, a relatively linear region is selected on the logarithmic decay curve, for example, the interval from -5 decibel decay to -35 decibel, and then the linear regression is performed on the data in the interval to fit a straight line. Finally, according to the slope of the straight line, the time required for energy decay of 60 decibels is extrapolated, which is T60. For example, if the slope of the fitted straight line indicates that 40 decibels are decayed per second, then T60 is 60 / 40=1.5 seconds.

[0089] Secondly, the clarity is calculated. The clarity indicator is used to quantify the ratio between the useful sound energy that arrives early and the harmful reverberation sound energy that arrives later. Common indicators include C50 (suitable for speech clarity) and C80 (suitable for music clarity). The calculation is also based on the acoustic impulse response. Taking the calculation of C80 as an example, first calculate the total energy of the acoustic impulse response within the first 80 milliseconds, which is obtained by squaring and integrating the impulse response signal from 0 to 0.08 seconds. Then, calculate all the energy from 80 milliseconds later to the end of the signal. Finally, the ratio of early energy to late energy is converted to decibel unit, that is, C80=10*log10(early energy / late energy). A higher C80 value, for example, greater than 2 decibels, indicates better music details and level.

[0090] Finally, the sound pressure level (SPL) is calculated. The sound pressure level reflects the loudness of the sound perceived at the listener's position. Unlike the previous two indicators, the calculation of sound pressure level needs to use the audible audio itself, because it depends on the intensity of the source signal and the transfer function of the environment. The calculation process first calculates the root mean square (RMS) value of the entire audible audio signal. Then, the RMS sound pressure value is converted to decibel sound pressure level using the standard formula: SPL=20*log10(RMS value of sound pressure / reference sound pressure), where the reference sound pressure is a standard value, which is 20 micro-pascal in air. It should be noted that in order to make the calculated sound pressure level have physical meaning, the entire simulation link needs to be calibrated, that is, the digital level of the dry audio signal needs to correspond to a known physical voltage or sound source output power.

[0091] After all the above calculations are completed, the resulting results are integrated into an acoustic index analysis report. For example, the report can show: {Listener position: Driver, Sound pressure level: 85.2 dB, Clarity C80: 2.5 dB, Reverberation time T60: 0.45 seconds}. This objective and quantitative data provides engineers with a scientific basis for evaluating and comparing different design schemes, thereby guiding subsequent optimization work.

[0092] It is worth mentioning that the predetermined parameters in the implementation process of the scheme are mainly set by combining theoretical basis, engineering experience and experimental optimization. Among them, the parameters related to physical simulation, such as the highest expansion order of spherical harmonics, are mainly determined according to the highest frequency of the target simulation and the required spatial resolution requirement. The architecture hyperparameters of the surrogate model, such as the network layer number, neuron number and latent space dimension of each encoder and decoder, are finally selected by systematic experiments to balance the model complexity and prediction accuracy. In addition, the key parameters in the training process, especially the weight coefficient for balancing the structure loss and the physical loss, are also adjusted to ensure that the model can stably converge and achieve the best performance.

[0093] In summary, the audio simulation method based on the embodiments of the present application is illustrated, which introduces a surrogate model based on multi-layer perceptron and uses spherical harmonic coefficients as the key bridge connecting micro sound sources and macro environment. Through offline training, the surrogate model learns and solidifies the nonlinear mapping relationship from excitation parameters (frequency, voltage) to complex sound field physical representation (spherical harmonic coefficients). In real-time applications, the trained model can instantaneously predict the spherical harmonic coefficients of the sound source according to the driving parameters. The coefficients not only completely retain the directivity information of the sound source containing amplitude and phase, solving the problem of low information fidelity caused by traditional simplified methods, but also have a standardized data format that can be efficiently called by a macro acoustic simulation engine for subsequent acoustic simulation and finally synthesized into audible audio. In this way, the time-consuming micro physical simulation process is decoupled from offline training, thereby bypassing the huge computational gap of source-field direct coupling, realizing real-time full-link audio simulation under the premise of ensuring high precision, and effectively solving the computational bottleneck.

[0094] Figure 5 A block diagram of an audio simulation device according to an embodiment of the present application is shown in FIG. 1. As shown in FIG. 1, the audio simulation device includes a micro acoustic simulation engine 101, a surrogate model 102, a macro acoustic simulation engine 103 and a sound field synthesis engine 104. Figure 5As shown, the audio simulation device 100 according to the embodiment of the present application comprises: a training data set acquisition module 110, configured to acquire a training data set, each training sample in the training data set being a {excitation parameter, sound field physical representation} data pair, the excitation parameter comprising a frequency and a voltage, and the sound field physical representation being a spherical harmonic function coefficient vector; a model training module 120, configured to train a multi-layer perception-based proxy model based on the training data set to obtain a trained proxy model; a real-time driving parameter acquisition module 130, configured to acquire a real-time driving parameter from an upper-layer application, the real-time driving parameter comprising a simulated frequency and a voltage; a real-time prediction module 140, configured to input the real-time driving parameter into the trained proxy model to obtain a real-time predicted spherical harmonic function coefficient; a macro-environment acoustic simulation module 150, configured to perform macro-environment acoustic simulation based on the real-time predicted spherical harmonic function coefficient to obtain an acoustic impulse response; and an audible audio generation module 160, configured to perform auditory synthesis of the acoustic impulse response and a dry audio signal to obtain audible audio.

[0095] Here, those skilled in the art can understand that the specific operations of each step in the above audio simulation device have been described in detail above with reference to the description of the audio simulation method, and therefore repeated descriptions thereof will be omitted. Figures 1 to 4

[0096] In the embodiments of the present application, a device is also provided, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned method when executing the program.

[0097] In the embodiments of the present application, a storage medium is also provided, having a computer program stored thereon, wherein the computer program is executed by a processor to implement the steps of the above-mentioned method.​

Claims

1. An audio simulation method, characterized in that, include: Obtain a training dataset, wherein each training sample in the training dataset is a data pair of {excitation parameters, sound field physical representation}, wherein the excitation parameters include frequency and voltage, and the sound field physical representation is a vector of spherical harmonic function coefficients; The agent model based on the multilayer perceptron is trained using the training dataset to obtain a trained agent model. Obtain real-time driving parameters from the upper-layer application, including the simulated frequency and voltage; The real-time driving parameters are input into the trained surrogate model to obtain the real-time predicted spherical harmonic function coefficients; Macroscopic environmental acoustic simulation is performed based on the real-time predicted spherical harmonic function coefficients to obtain the acoustic impulse response; Acoustic impulse response and dry audio signal are synthesized to obtain audible audio.

2. The audio simulation method according to claim 1, characterized in that, Obtain the training dataset, including: Grid sampling is performed in the simulation parameter space to obtain a series of discrete simulation task parameter points, including frequency and voltage. A microphysics simulation engine driven by parameter points is used to obtain the original sound pressure field data for the simulation task. The original sound pressure field data is subjected to sound field feature transformation to obtain the spherical harmonic function coefficient vector of the simulation task parameter points.

3. The audio simulation method according to claim 2, characterized in that, The microscopic physics simulation engine is COMSOL with AC / DC & Acoustics Module.

4. The audio simulation method according to claim 1, characterized in that, Macroscopic environmental acoustic simulation is performed based on the real-time predicted spherical harmonic function coefficients to obtain the acoustic impulse response, including: Initialize the macroscopic acoustic simulation engine to obtain the initialized scene context; The sound source location and the real-time predicted spherical harmonic function coefficients are used to instantiate a spherical harmonic function sound source to obtain a sound source emission profile; Based on the sound source location, sound source emission profile, and initialized scene context, directional sound ray tracking and hit data collection are performed to obtain the listener hit log; The acoustic impulse response is obtained by synthesizing the listener hit log.

5. The audio simulation method according to claim 4, characterized in that, The sound source emission profile is obtained by instantiating a spherical harmonic function source based on the sound source location and the real-time predicted spherical harmonic function coefficients, including: The inverse transformation of the spherical harmonic function coefficients predicted in real time is performed to obtain the complex sound pressure of the sound source in any direction on a unit sphere. The square modulus of the complex sound pressure from the sound source in any direction on a unit sphere is calculated to obtain the emission profile of the sound source.

6. The audio simulation method according to claim 1, characterized in that, The process of synthesizing an acoustic impulse response and a dry audio signal to obtain audible audio includes: performing a convolution operation on the acoustic impulse response and the dry audio signal to obtain the audible audio.

7. The audio simulation method according to claim 1, characterized in that, It also includes the step of: performing post-processing analysis on the audible audio to obtain acoustic index analysis results, the acoustic index analysis results including sound pressure level, intelligibility and reverberation time at the listener's location.

8. An audio simulation device, characterized in that, include: The training dataset acquisition module is used to acquire the training dataset, wherein each training sample in the training dataset is a data pair of {excitation parameters, sound field physical representation}, wherein the excitation parameters include frequency and voltage, and the sound field physical representation is a vector of spherical harmonic function coefficients. The model training module is used to train the agent model based on the multilayer perceptron on the training dataset to obtain the trained agent model. A real-time drive parameter acquisition module is used to acquire real-time drive parameters from the upper-layer application, including the simulated frequency and voltage. The real-time prediction module is used to input the real-time driving parameters into the trained surrogate model to obtain the spherical harmonic function coefficients for real-time prediction. The macroscopic environmental acoustic simulation module is used to perform macroscopic environmental acoustic simulation based on the real-time predicted spherical harmonic function coefficients to obtain the acoustic impulse response; The audible audio generation module is used to synthesize acoustic impulse response and dry audio signal to obtain audible audio.

9. An apparatus comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that... When the processor executes the program, it implements the steps of the method according to any one of claims 1 to 7.

10. A storage medium having a computer program stored thereon, characterized in that... When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images