Sound field simulation method and apparatus, device, medium, and program product

Abstract

A sound field simulation method and apparatus, a device, and a medium. The method comprises: acquiring audio input data, user parameters corresponding to a preset space, and system parameters corresponding to the preset space (S11); integrating the user parameters and the system parameters by means of a pulse generation algorithm to generate an initial pulse response (S12); cutting the initial pulse response on the basis of the system parameters to obtain early reflection and late reflection (S13); convolving the audio input data and the early reflection to obtain a convolution result (S14); inputting the audio input data into a target feedback delay network to obtain a network output (S15); and using the sum of the network output and the convolution result as audio output data (S16). The method is used for solving the problem of low sound field simulation precision.

Description

Sound field simulation method, device, equipment, medium and program product

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This disclosure claims priority to Chinese patent application No. 202311174904.1 filed on September 12, 2023, with Beijing Rockwell Technology Co., Ltd. as the applicant, and application name “A sound field simulation method, device and storage medium”, and Chinese patent application No. 202311174909.4 filed on September 12, 2023, with Beijing Rockwell Technology Co., Ltd. as the applicant, and application name “A sound field simulation method, device and storage medium”. The full text of the above applications is incorporated into this disclosure by reference. Technical Field

[0003] The present disclosure relates to the field of audio simulation technology, and in particular to a sound field simulation method, device, equipment, medium, and program product. Background Art

[0004] Sound field simulation helps audio producers adjust sound effects more precisely during recording and mixing. By simulating the characteristics of different enclosed spaces, including size, shape, reflection, and sound absorption, it can change the reverberation, echo, and sense of space of the sound, making the audio content more realistic.

[0005] Current methods for simulating sound fields are typically based on a Schroeder reverberator. In a Schroeder reverberator, a one-dimensional serial feedback structure is typically used. It utilizes the delay and mixing of reflected signals to simulate multiple reflections of sound, thereby producing a reverberation effect. The delay time and amplitude attenuation coefficient of each reflected signal are typically fixed and remain unchanged throughout the reverberation process. Therefore, due to the limitations of the one-dimensional structure, this method of simulating sound fields is generally limited in accuracy and realism, resulting in a certain gap between the simulated results and the actual spatial sound, and a poor user experience.

[0006] Summary of the Invention

[0007] The present disclosure provides a sound field simulation method, apparatus, device, medium, and program product, which can improve the accuracy of sound field simulation.

[0008] The present disclosure first provides a sound field simulation method, comprising:

[0009] Obtain audio input data, user parameters corresponding to the preset space, and system parameters corresponding to the preset space; use an impulse generation algorithm to integrate the user parameters and system parameters to generate an initial impulse response; cut the initial impulse response according to the system parameters to obtain early reflections and late reflections; convolve the audio input data with the early reflections to obtain a convolution result; input the audio input data into the target feedback delay network to obtain a network output; the sum of the network output and the convolution result is used as the audio output data; the audio output data is the simulated sound field corresponding to the preset space.

[0010] The disclosed embodiment can cut the initial impulse response according to the system parameters to obtain early reflections and late reflections, and convolve the audio input data with the early reflections in the initial impulse response, rather than convolving the audio input data with the entire initial impulse response, thereby saving a large amount of computing resources; in addition, the audio input data is input into the target feedback delay network to obtain the network output, and the sum of the network output and the convolution result is used as the audio output data, which solves the problem that there is a certain gap between the sound field simulated by using the Schroeder reverberator and the real sound, thereby improving the accuracy of the sound field simulation and further improving the user experience.

[0011] The present disclosure also provides a sound field simulation device, comprising:

[0012] an acquisition module configured to acquire audio input data, user parameters corresponding to the preset space, and system parameters corresponding to the preset space;

[0013] The generation module is configured to integrate the user parameters and the system parameters using an impulse generation algorithm to generate an initial impulse response;

[0014] The processing module is configured to cut the initial impulse response according to the system parameters to obtain early reflections and late reflections;

[0015] A convolution module is configured to convolve the audio input data with the early reflection to obtain a convolution result;

[0016] The processing module is further configured to input the audio input data into the target feedback delay network to obtain a network output;

[0017] The processing module is also configured to use the sum of the network output and the convolution result as audio output data; the audio output data is the sound field corresponding to the simulated preset space.

[0018] The present disclosure further provides an electronic device, comprising: a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program implements the aforementioned sound field simulation method when executed by the processor.

[0019] The present disclosure further provides a computer-readable storage medium, comprising: a computer program stored on the computer-readable storage medium, and the computer program implements the sound field simulation method as described above when executed by a processor.

[0020] The present disclosure also provides a computer program product, comprising: when the computer program product is run on a computer, the computer is enabled to implement the sound field simulation method as described above.

[0021] The technical solution provided by the present disclosure has the following advantages over the prior art: first, audio input data, user parameters corresponding to a preset space, and system parameters corresponding to the preset space are obtained, and the user parameters and system parameters are integrated using an impulse generation algorithm to generate an initial impulse response. The initial impulse response is then segmented according to the system parameters to obtain early reflections and late reflections, and the audio input data is convolved with the early reflections to obtain a convolution result. The audio input data is then input into a target feedback delay network to obtain a network output. Finally, the sum of the network output and the convolution result is used as the audio output data. In this way, the initial impulse response can be segmented according to the system parameters to obtain early reflections and late reflections, and the audio input data is convolved with the early reflections in the initial impulse response, rather than convolving the audio input data with the entire initial impulse response, saving a significant amount of computing power. Furthermore, the audio input data is input into the target feedback delay network to obtain a network output, and the sum of the network output and the convolution result is used as the audio output data. This solves the problem of a certain gap between the sound field simulated using a Schroeder reverberator and the real sound, thereby improving the accuracy of the sound field simulation and further improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0023] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0024] FIG1A is a schematic diagram of a flow chart of a sound field simulation method according to an embodiment of the present disclosure;

[0025] FIG1B is a schematic diagram of one of the interactive interfaces provided by an embodiment of the present disclosure;

[0026] FIG1C is a second flow chart of the sound field simulation method provided by an embodiment of the present disclosure;

[0027] FIG1D is a second schematic diagram of an interactive interface provided by an embodiment of the present disclosure;

[0028] FIG1E is a third schematic diagram of an interactive interface provided by an embodiment of the present disclosure;

[0029] FIG2 is a schematic diagram of an impulse response provided by an embodiment of the present disclosure;

[0030] FIG3 is a third flow chart of the sound field simulation method provided by an embodiment of the present disclosure;

[0031] FIG4 is a fourth flow chart of the sound field simulation method provided by an embodiment of the present disclosure;

[0032] FIG5 is a fifth flow chart of the sound field simulation method provided by an embodiment of the present disclosure;

[0033] FIG6 is a sixth flow chart of the sound field simulation method provided by an embodiment of the present disclosure;

[0034] FIG7 is a seventh flow chart of the sound field simulation method provided by an embodiment of the present disclosure;

[0035] FIG8 is a schematic diagram of a network structure of a feedback delay network according to an embodiment of the present disclosure;

[0036] FIG9 is a second schematic diagram of the network structure of the feedback delay network provided in an embodiment of the present disclosure;

[0037] FIG10 is a schematic structural diagram of a sound field simulation device provided by an embodiment of the present disclosure;

[0038] FIG11 is a schematic structural diagram of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0039] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features therein can be combined with each other in the absence of conflict.

[0040] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.

[0041] The sound field simulation technology in the related art is mainly divided into simulation methods based on physical models and simulation methods based on digital signal processing. The simulation method based on physical models usually uses wave equations, sound rays, statistical models and other methods to establish a preset space model to solve the sound field distribution of the preset space. However, these simulation methods based on physical models require high computational complexity and high storage capacity, and are difficult to calculate in real time, so they are rarely used in practical applications. The simulation method based on digital signal processing is usually based on reverberation theory, and uses the time delay and reflection coefficient between reflections to simulate the preset space audio. However, due to the limitations of the one-dimensional structure, the simulation method based on digital signal processing is usually limited in accuracy and authenticity, resulting in a certain gap between the simulation results and the real space sound, and a poor user experience.

[0042] In response to the above problems, embodiments of the present disclosure provide a sound field simulation method, apparatus, device, medium, and program product.

[0043] The sound field simulation method provided by the embodiment of the present disclosure first obtains audio input data, user parameters corresponding to the preset space, and system parameters corresponding to the preset space, and uses an impulse generation algorithm to integrate the user parameters and system parameters to generate an initial impulse response; then, the initial impulse response is cut according to the system parameters to obtain early reflections and late reflections, and the audio input data is convolved with the early reflections to obtain a convolution result; then, the audio input data is input into a target feedback delay network to obtain a network output; finally, the sum of the network output and the convolution result is used as the audio output data. Through the above operations, the audio input data can be processed with the early reflections in the initial impulse response instead of convolving the audio input data with the entire initial impulse response, which saves a lot of computing power resources and solves the problem that there is a certain gap between the sound field simulated by the Schroeder reverberator and the real sound, thereby improving the accuracy of the sound field simulation and further improving the user experience.

[0044] The sound field simulation method provided in the embodiments of the present disclosure can be implemented using a sound field simulation device, and the sound field simulation device provided in the embodiments of the present disclosure can be either hardware or software. For example, when the sound field simulation device is hardware, it can be various electronic devices capable of performing sound field simulation, including but not limited to in-vehicle equipment, smart vehicles, mobile phones, computers, etc., while when the sound field simulation device is software, it can be installed in the electronic devices listed above. It can include multiple software programs or software modules, or it can be composed of a single software program or software module, without specific limitation herein.

[0045] FIG1A is a flow chart of a sound field simulation method provided by an embodiment of the present disclosure. As shown in FIG1A , the sound field simulation method may include the following steps.

[0046] S11. Acquire audio input data, user parameters corresponding to the preset space, and system parameters corresponding to the preset space.

[0047] The preset space is equipment, facilities, or other objects with a closed space; for example, the preset space can be a vehicle with a closed compartment, or a room, etc., which is not limited in this disclosure.

[0048] Exemplarily, user parameters may include at least one of the size (Size), position (Position), material (Material), humidity (Moisture) and temperature (Temperature) of the preset space; wherein, the size may include parameters such as the length, width and height of the preset space, and the position may include the receiving position and the sound source position, and the receiving position may be the position where the user wants the sound to reach, or the position selected by the user.

[0049] Exemplarily, the system parameters may include at least one of the sampling rate (Sample Rate), reflection order (Reflection Order), pulse length (Impulse Kength), window function length (Window Size), filter bank center frequency (Filter Bank Center Frequency), and feedback delay network delay line length (Feedback Delay Lines Length).

[0050] Among them, the sampling rate refers to the number of times a signal is sampled per second in digital signal processing. The sampling rate in this disclosure is the sampling rate of the feedback delay network for audio input data; the reflection level refers to the number of reflections experienced by the sound wave in the feedback delay network (each reflection causes the path of the sound wave to change and the energy to decay); the pulse length refers to the length of the initial pulse signal used in the feedback delay network. The pulse length determines the time domain characteristics of the reflected signal. A shorter pulse length can produce a clear reflection effect, while a longer pulse length can produce a lasting reverberation effect.

[0051] The window function length refers to the length of the window function for processing the pulse signal. For example, in step S13, the window function can be used to cut the initial pulse response according to the pre-delay time (Pre-Delay Time) and the mixing time (Mixing Time) to obtain early reflections and late reflections; illustratively, the window function can be a Hanning window (Hanning Window), a rectangular window (Rectangular Window), a Hamming window (Heming Window), etc.

[0052] The center frequency of a filter bank refers to the center frequency of each filter in a group of filters used for frequency division and signal processing. In feedback delay networks, filter banks are often used to perform frequency domain analysis and processing on input signals. The center frequency of each filter determines its position in the spectrum and the filter's response to signals within a specific frequency range. The length of the delay line in a feedback delay network refers to the number of sampling points in each delay line. The length of the delay line determines the delay time of the reflected signal. Longer delay lines can produce longer delay times and reverberation effects.

[0053] In some embodiments, the audio input data may be real-time audio from any audio source, or pre-recorded audio; illustratively, the audio input data may include music, voice, and ambient sound.

[0054] In some embodiments, audio input data may be obtained first, and then user parameters corresponding to the preset space and system parameters corresponding to the preset space may be obtained.

[0055] In some embodiments, the user parameters may be parameters input by the user according to needs.

[0056] In some embodiments, the system parameters may be parameters related to a Feedback Delay Network (FDN), and the system parameters may be preset, for example, they may be default parameter values, or they may be parameter values ​​set by relevant personnel according to actual conditions.

[0057] In some embodiments, obtaining audio input data, user parameters corresponding to the preset space, and system parameters corresponding to the preset space can be achieved by:

[0058] In response to a target operation of the user on the interactive interface, audio input data, user parameters corresponding to the preset space, and system parameters corresponding to the preset space are acquired.

[0059] Among them, the interactive interface is used to realize interaction with the user.

[0060] In some embodiments, the interactive interface includes at least one of a spatial model and a write control corresponding to a preset space. For example, in the interactive interface shown in FIG1B , a spatial model 21 is included, and the user can change the user parameters corresponding to the preset space by dragging or rotating the spatial model 21. For another example, in the interactive interface shown in FIG1E , a spatial model 21 and a write control 22 are included, and the user can change the user parameters corresponding to the preset space by dragging or rotating the spatial model 21, and can also trigger the write control 22 to enter the user parameters in the corresponding target window.

[0061] In some embodiments, the preset space may include equipment, facilities or other objects with an enclosed space; for example, the preset space may be a vehicle with an enclosed compartment, or a room, etc., which is not limited in this application.

[0062] In some embodiments, the target operation may include one of a transmission operation and a download operation. In response to the user's target operation on the interactive interface, the method of obtaining audio input data may be: in response to the user's transmission operation, establishing a connection with the user device and receiving the audio input data sent by the user device; or, when it is determined that a network connection is established, downloading the audio input data in response to the user's download operation on the network resource.

[0063] In some embodiments, the user parameters are parameters input by the user according to needs.

[0064] In some embodiments, the interactive interface includes a spatial model corresponding to a preset space, and the target operation includes at least one of a touch operation, a voice operation, a gesture operation, and a line of sight operation.

[0065] In one embodiment, the interactive interface includes a space model corresponding to a preset space.

[0066] Accordingly, in response to the user's target operation on the interactive interface, the method of obtaining the user parameters corresponding to the preset space can be implemented in the following ways:

[0067] In response to the user's target operation on the interactive interface, the target position parameter and the target attribute parameter of the space model in the interactive interface are determined, and the sum of the target position parameter and the target attribute parameter is determined as the user parameter corresponding to the preset space.

[0068] First, as shown in FIG1C , in response to a user's target operation on the interactive interface, a method of determining a target position of the spatial model in the interactive interface may include the following steps:

[0069] S111 . When a target operation is detected, determine a target movement parameter according to the target operation.

[0070] The target movement parameter includes at least one of a target direction, a target distance, and a target angle.

[0071] In some embodiments, when the target operation is a touch operation, detecting the target operation and determining the target movement parameters according to the target operation can be implemented in the following manner:

[0072] When a touch event is detected, it is determined that a touch operation is detected, and then the target movement parameter is determined according to a predefined relationship between the touch event and the movement parameter.

[0073] In some embodiments, when the target operation is a voice operation, detecting the target operation and determining the target movement parameters according to the target operation can be achieved by:

[0074] First, a voice recognition service recognizes the user's voice and determines whether a voice operation is detected based on the voice recognition results. If the voice recognition results include a pre-stored voice command, the voice operation is detected; if the voice recognition results do not include a pre-stored voice command, the voice operation is not detected. Next, target movement parameters are determined based on a pre-defined relationship between the voice command and movement parameters.

[0075] In some embodiments, when the target operation is a gesture operation, detecting the target operation and determining the target movement parameters according to the target operation can also be achieved in the following manner:

[0076] A gesture recognizer recognizes user gestures and determines whether a gesture operation is detected based on the gesture recognition result; when the gesture recognition result includes a pre-stored gesture instruction, it is determined that the gesture operation is detected; when the gesture recognition result does not include the pre-stored gesture instruction, it is determined that the gesture operation is not detected; thereafter, the target movement parameter is determined based on the relationship between the pre-defined gesture instruction and the movement parameter.

[0077] In some embodiments, when the target operation is a gaze operation, the target operation may be detected by capturing and analyzing the user's eye movement data through an eye tracking system, and determining whether the gaze operation is detected based on the eye movement data. The target movement parameters are then determined based on a predefined relationship between the eye movement data and the movement parameters.

[0078] S112. Change the position of the spatial model in the interactive interface according to the target movement parameter.

[0079] In some embodiments, changing the position of the spatial model in the interactive interface according to the target movement parameters may include scaling the spatial model size, changing the shape of the spatial model, moving the position of the spatial model, and rotating the perspective of the spatial model according to the target movement parameters.

[0080] For example, in the interactive interface shown in FIG1B , a spatial model 21 is included, and the eight corners of the spatial model 21 are respectively the first corner A, the second corner B, the third corner C, the fourth corner D, the fifth corner a, the sixth corner b, the seventh corner c, and the eighth corner d. The corners of the spatial model 21 refer to the intersection positions of the three faces. When the target direction in the target movement parameters is the bd direction, the target distance is bb', and the target angle is 45° in the ba direction, then after the position of the spatial model in the interactive interface is changed according to the target movement parameters, the position of the spatial model in the interactive interface is as shown in FIG1D , and the eight corners are respectively the first corner A, the second corner B, the third corner C, the fourth corner D, the fifth corner a, the sixth corner b', the seventh corner c, and the eighth corner d.

[0081] S113: Determine the changed position of the spatial model in the interactive interface as the target position, and determine the position parameters corresponding to the target position as the target position parameters.

[0082] Afterwards, in response to the user's target operation on the interactive interface, the target attribute parameters of the spatial model in the interactive interface are determined.

[0083] In some embodiments, in response to a user's target operation on the interactive interface, determining the target attribute parameters of the spatial model in the interactive interface can be achieved by:

[0084] According to the target operation, the target attribute parameters of the spatial model in the interactive interface are obtained from the corresponding relationship.

[0085] The corresponding relationship is the corresponding relationship between the user operation and the spatial model attribute parameter; the user operation includes the target operation.

[0086] In one embodiment, the target operation is one of the user operations; illustratively, the user operation includes at least one of a touch operation, a voice operation, a gesture operation, and a sight operation.

[0087] In one embodiment, the attribute parameter includes at least one of material, humidity, and temperature.

[0088] In one embodiment, the corresponding relationship may be predetermined and stored.

[0089] Finally, the sum of the target position parameter and the target attribute parameter is determined as the user parameter corresponding to the preset space.

[0090] In some embodiments, the interactive interface includes a write control, and the target operation includes a write operation.

[0091] Accordingly, in response to the user's target operation on the interactive interface, the method of obtaining the user parameters corresponding to the preset space can also be implemented in the following ways:

[0092] In response to a user triggering operation on a write control in the interactive interface, a target window is displayed, and in response to the user's write operation, user parameters written by the user are obtained.

[0093] The target window is used to write user parameters.

[0094] In one embodiment, the user parameters include at least one of the size, position, material, humidity and temperature of the preset space, or the user parameters include at least one of the size, position, sound absorption coefficient, humidity and temperature of the preset space.

[0095] For example, in the interactive interface shown in FIG1E , the user can call out the target window by triggering the write control 22 and enter user parameters in the corresponding target window. Parameters such as the size, shape, and position of the write control 22 are not limited in this application.

[0096] In the above scheme, the user parameters corresponding to the user-defined preset space can be obtained according to the target operation performed by the user. That is, the user can define the physical conditions corresponding to the preset space by himself, and can even generate a material that does not exist in the physical world by defining the sound absorption coefficient, thereby enhancing the user's sense of participation and improving the user experience.

[0097] Finally, in response to the user's target operation on the interactive interface, the system parameters corresponding to the preset space are obtained.

[0098] S12. Use an impulse generation algorithm to integrate the user parameters and the system parameters to generate an initial impulse response.

[0099] For example, FIG2 is an impulse response. As shown in FIG2 , the horizontal axis represents the reverberation time in seconds (s), and the vertical axis represents the reverberation intensity. It can be seen that in the impulse response shown in FIG2 , the reverberation intensity gradually decreases as the reverberation time increases.

[0100] In some embodiments, as shown in FIG3 , a method of integrating user parameters and system parameters using an impulse generation algorithm to generate an initial impulse response may include the following steps:

[0101] S121. Generate a primary impulse response according to user parameters and system parameters using an image source algorithm.

[0102] In some embodiments, the user parameters include a sound receiving position and a sound source position, and the system parameters include a pulse length and a sampling rate. Thus, a method for generating a primary impulse response based on the user parameters and the system parameters using an image source algorithm may include: generating a primary impulse response based on equation (1) using an image source model (ISM).

[0103] Among them, r is used to represent the receiving position, r s Used to indicate the location of the sound source, h(r,r s ,n) is used to represent the primary impulse response at (r,r s ) at the value; q, j, k are the sets in p, and q is used to represent the mirror image on the x-axis, j is used to represent the mirror image on the y-axis, and k is used to represent the mirror image on the z-axis; m x , m y , m z is a set in m, and m x Used to indicate the number of reflections on the x-axis, m y Used to indicate the number of reflections on the y-axis, m z is used to represent the number of reflections on the z-axis; δ is the Dirac function, τ is used to represent the time of sound propagation, and τ is the continuous time, f s It is used to indicate the sampling rate, and d is used to indicate the distance that the sound propagates; and Used to express the sound absorption coefficient of two surfaces on the x-axis; and Used to express the sound absorption coefficient of two surfaces on the y-axis; and Used to indicate the sound absorption coefficient of two surfaces on the z-axis. LPF stands for Low-Pass Filter (LPF).

[0104] The time τ of sound propagation can be the pulse length in the system parameters, and the distance d of sound propagation can be the distance between the sound receiving position r and the sound source position r in the user parameters. s The Euclidean distance between the sound receiving position r and the sound source position r s The Euclidean distance between r and r s For example, when the coordinate of r is (x r ,y r , z r ), r s The coordinates of (x s ,y s , z s ), then r and r s The Euclidean distance between

[0105] S122: Generate an initial reverberation time and event occurrence rate according to user parameters and system parameters using a reverberation time algorithm.

[0106] The event rate is the probability of the primary impulse response event occurring.

[0107] In some embodiments, the user parameters include the size of the preset space, and the system parameters include the pulse length and the sampling rate.

[0108] First, the reverberation time algorithm can be used to calculate the event rate of the primary impulse response according to the size of the preset space, the pulse length, the sampling rate, and equation (2).

[0109] Here, μ represents the event rate, V represents the volume of the preset space, c represents the speed of sound, and t represents the time it takes for sound to propagate, with t being a discrete time. V can be calculated based on the size of the preset space. For example, if the preset space is a rectangular parallelepiped, V = U × O × P, where U is the length of the preset space, O is the width of the preset space, and P is the height of the preset space. t can be obtained by sampling the pulse length according to the sampling rate in the system parameters.

[0110] Afterwards, the initial reverberation time is calculated according to formula (3).

[0111] Where, Δt A (z) is used to represent the initial reverberation time, μ is used to represent the event occurrence rate, and z is a random number between 0 and 1.

[0112] S123 , integrating the primary impulse response, the initial reverberation time, and the event occurrence rate to obtain an initial impulse response.

[0113] In some embodiments, as shown in FIG4 , integrating the primary impulse response, the initial reverberation time, and the event occurrence rate to obtain the initial impulse response can be achieved by the following steps:

[0114] S1231. Generate a target sequence based on the event occurrence rate and the initial reverberation time using a Poisson process.

[0115] In some embodiments, the primary impulse response events are distributed at time points corresponding to the initial reverberation time according to the event occurrence rate of the primary impulse response events to obtain a target sequence.

[0116] S1232. Perform discrete transformation on the target sequence to obtain at least one subsequence.

[0117] In some embodiments, performing discrete transformation on the target sequence to obtain at least one subsequence may be achieved by any of the following methods:

[0118] A filter bank (the center frequency of which is the center frequency of the filter bank defined in the system parameters) is applied to the target sequence to divide the target sequence into at least one subsequence. The filter bank may include a quadrature mirror filter (QMF) filter bank, a finite impulse response (FIR) filter bank, an infinite impulse response (IIR) filter bank, and the like.

[0119] Performing Discrete Fourier Transform (DFT) on the target sequence to divide the target sequence into at least one subsequence.

[0120] For example, when the target sequence is h d (n), for h d (n) Applying a filter bank (eg, QMF, FIR, IIR, etc.) or performing a DFT transform to obtain at least one subsequence.

[0121] S1233. Perform weighted summation on at least one subsequence using the primary impulse response to obtain an initial impulse response.

[0122] Exemplarily, performing weighted summation on at least one subsequence using the primary impulse response to obtain the initial impulse response can be achieved in the following manner:

[0123] The energy envelope corresponding to the primary impulse response is used as a weight to perform weighted summation on at least one subsequence to obtain an initial impulse response.

[0124] In this solution, an image source algorithm is first used to generate a primary impulse response based on user and system parameters. A reverberation time algorithm is then used to generate an initial reverberation time and event rate based on these parameters. The primary impulse response, initial reverberation time, and event rate are then integrated to generate an initial impulse response. Generating an initial impulse response based on user and system parameters means that, from a user's perspective, the interface allows users to define the physical conditions corresponding to a preset space, thereby simulating the desired audio output data, improving the user experience.

[0125] In addition, a Poisson process is used to generate a target sequence based on the event rate and initial reverberation time. This target sequence is then discretely transformed to obtain at least one subsequence. Finally, the primary impulse response is used to perform a weighted summation of the at least one subsequence to obtain the initial impulse response. This operation, which uses the Poisson process to generate the target sequence and ultimately obtain the initial impulse response, reduces the impact of ray tracing on the sound field simulation and improves the accuracy of the sound field simulation.

[0126] In addition, the number of generated subsequences can be adjusted by controlling the number of filters in the filter group, thereby achieving a balance between computing power and realism. When computing power allows, the accuracy of the sound field simulation can be further improved by increasing the number of subsequences.

[0127] S13. Cut the initial impulse response according to the system parameters to obtain early reflections and late reflections.

[0128] Among them, early reflections are the set of sounds that reach the listener for the first time after the sound is emitted from the sound source, and late reflections are the set of sounds that reach the listener after experiencing at least one reverberation during the propagation process. Reverberation includes at least one of reflection, attenuation, and superposition of sound.

[0129] In some embodiments, before the initial impulse response is segmented according to the system parameters to obtain the early reflections and the late reflections, the following operations may be performed:

[0130] The initial impulse response is estimated using a time estimation algorithm to obtain the pre-delay time and mixing time.

[0131] The mixing time is the time it takes for the sound to be evenly distributed in the preset space after being emitted from the sound source and reflected and scattered.

[0132] In some embodiments, a pre-delay time estimation algorithm (Pre-Delay Time Estimation) and a mixing time estimation algorithm (Mixing Time Estimation) may be used to estimate the initial impulse response to obtain a pre-delay time (Pre-Delay Time) and a mixing time (Mixing Time).

[0133] In some embodiments, the system parameters include reflection orders; and the initial impulse response includes an initial reverberation time and an initial reverberation response.

[0134] As shown in FIG5 , the method of segmenting the initial impulse response according to the system parameters to obtain the early reflection and the late reflection may include the following steps:

[0135] S131: Determine an initial reverberation intensity set within a first time period in a preset impulse response as an early reflection.

[0136] The preset impulse response is an impulse response corresponding to the reflection order in the initial impulse response; the first time period is a time period between the starting initial reverberation time and the first initial reverberation time in the preset impulse response, and the length of the first time period is the same as the length of the pre-delay time.

[0137] In one embodiment, the preset impulse response may be a room impulse response (RIR). RIP is generally used to describe the correspondence between the reverberation time and reverberation intensity of sound propagating in a preset space and reaching a receiving point after various reflections, attenuation, and interference.

[0138] In this way, by adding pre-delay time and reflection levels, unnecessary delay time in the initial impulse response can be trimmed. These unnecessary delay times only have a delay effect but do not carry any acoustic characteristics associated with the preset space. Therefore, through the above operations, the accuracy of the sound field simulation can be improved.

[0139] S132: Determine an initial reverberation intensity set within a second time period in the preset impulse response as a late reflection.

[0140] The second time period is a time period between the first initial reverberation time and the second initial reverberation time in the preset impulse response, and the duration of the second time period is the same as the duration of the mixing time.

[0141] In this way, by adding mixing time and reflection series, the initial impulse response can be split using a complete window function instead of directly cutting the preset impulse response, which can improve the accuracy of the sound field simulation.

[0142] In the above scheme, the initial reverberation intensity set in the first time period of the preset impulse response is determined as the early reflection, and the initial reverberation intensity set in the second time period of the preset impulse response is determined as the late reflection. The reflection order based on the image source algorithm is adopted instead of the traditional direct truncation, which can better preserve the characteristics of the signal and thus improve the accuracy of the simulated sound field.

[0143] S14: Convolve the audio input data with the early reflection to obtain a convolution result.

[0144] First, before convolving the audio input data with the early reflections, the length of the audio input data and the length of the early reflections need to be aligned. In some embodiments, the audio input data or the early reflections can be zero-padded to make the length of the audio input data and the length of the early reflections the same. For example, zero samples can be added to the end of the audio signal to make its length the same as the length of the early reflections.

[0145] Afterwards, the audio input data with unified length is convolved with the early reflection to obtain the convolution result.

[0146] In some embodiments, the convolution operation can be implemented by time domain convolution or by frequency domain convolution. When implemented by frequency domain convolution, it is also necessary to convert the audio input data and early reflections from time domain signals to frequency domain signals, and then perform frequency domain convolution calculations to obtain the frequency domain convolution results, and then convert the frequency domain convolution results from frequency domain signals to time domain signals to obtain convolution results. Among them, the time domain-frequency domain signal conversion method can be to use Fourier transform (FT) algorithm or Fast Fourier Transform (FFT) algorithm to convert the audio input data and early reflections from time domain signals to frequency domain signals. The frequency domain-time domain signal conversion method can be to use Inverse Fourier Transform (IFT) algorithm to convert the frequency domain convolution results from frequency domain signals to time domain signals to obtain convolution results.

[0147] S15. Input the audio input data into the target feedback delay network to obtain the network output.

[0148] In one embodiment, the network output can be obtained by:

[0149] After the audio input data has been delayed for a preset time, it is input into the target feedback delay network to obtain the network output.

[0150] In one embodiment, the preset delay time can be pre-set.

[0151] In some embodiments, before inputting the audio input data into the target feedback delay network to obtain the network output, the sound field simulation method further includes executing the steps of:

[0152] The late reflections are processed to obtain the target feedback delay network.

[0153] By using the above method, the target feedback delay network is associated with the late reflection, which can improve the pertinence of the target feedback delay network in processing audio input data.

[0154] In one embodiment, processing the late reflections to obtain the target feedback delay network can be achieved by the steps shown in FIG6 :

[0155] S81. Analyze and process the late reflection to obtain a first coefficient and a second coefficient.

[0156] In some embodiments, as shown in FIG7 , analyzing and processing the late reflection to obtain the first coefficient and the second coefficient may include the following steps:

[0157] S811. Use a data processing algorithm to perform energy decay analysis on the late reflections to determine the secondary reverberation time and secondary reverberation intensity.

[0158] In some embodiments, the sound field simulation method also includes using a data processing algorithm to perform energy attenuation analysis on late reflections to determine the noise floor, which can be used to guide the selection of appropriate sound-absorbing materials and sound insulation measures, etc., to improve the desired acoustic environment in a preset space; it can also be used to guide the layout of speakers, microphones and other audio equipment to reduce the impact of noise on sound signals and improve sound quality.

[0159] In some embodiments, the data processing algorithm is an algorithm that can perform energy decay function (EDF) analysis on late reflections. For example, the data processing algorithm can be a Karjalainen algorithm or a Bayesian algorithm.

[0160] Energy attenuation analysis can be achieved by formula (4):

[0161] Among them, d K (t,θ) is used to represent the energy attenuation, N0 is used to represent the noise floor, T i Used to represent the i-th moment in the secondary reverberation time, A i It is used to indicate the i-th reverberation intensity in the secondary reverberation intensity, L is used to indicate the length of the late reflection, t is used to indicate the time of sound propagation, and t is a discrete time, f s Used to indicate the sampling rate.

[0162] S812: Perform level analysis on the secondary reverberation time, the secondary reverberation intensity, and the system parameters using an initial level analysis algorithm to obtain a first coefficient.

[0163] In some embodiments, the first coefficient can be calculated according to formula (5):

[0164] Where l(ω) is used to represent the first coefficient, L is used to represent the late reflection length, b(ω) is used to represent the center frequency of the filter bank, and A(ω) is used to represent the secondary reverberation intensity; f s Used to indicate the sampling rate, T 60 (ω) is used to represent the secondary reverberation time.

[0165] T 60 (ω) is RT 602. The parameter in the Absorption function, RT60 (Reverberation Time 60), is a parameter that measures the reverberation time in an audio environment. It refers to the time interval from the beginning of the sound to the decay of a certain intensity. The Absorption function describes the material's ability to absorb sound. It usually uses frequency as the independent variable and returns the sound absorption coefficient as the function value.

[0166] S813: Use a preset function to perform attenuation analysis on the secondary reverberation time and system parameters to obtain a second coefficient.

[0167] In some embodiments, the preset function may be RT 60 2Absorption function, the second coefficient can be calculated according to the following formula: A(ω) = Lγ dB (ω), where A(ω) is used to represent the second coefficient, L is used to represent the late reflection length, f s Used to indicate sampling rate, T 60 (ω) is used to represent the secondary reverberation time.

[0168] S82: Use the first coefficient as a level coefficient of the preset feedback delay network, and the second coefficient as an attenuation coefficient of the preset feedback delay network to obtain a target feedback delay network.

[0169] Among them, the level coefficient is used to control the level of the delay line output data, and the attenuation coefficient is used to control the attenuation speed of the sound on the delay line. The delay line is a component of the feedback delay network.

[0170] A feedback delay network typically consists of at least one delay line, a feedback matrix, at least one mixer, and a filter bank. The delay line is a linear buffer with a fixed delay time, typically used to store samples of audio input data and mix them with copies with different delay times. The feedback matrix is ​​used to feed the output of the delay line back to the input of the delay line. By adjusting the corresponding attenuation coefficients of the feedback matrix, the strength of the feedback signal and the complexity of the delay path can be controlled. The mixer is used to multiply the output signal of the delay line by the level coefficient to obtain the network output of the feedback delay network. The filter bank is used to further process the output signal of the delay line. For example, the filter bank can be used to adjust the frequency response and reverberation characteristics.

[0171] Figure 8 illustrates the network structure of a feedback delay network. As shown in Figure 8, the feedback delay network includes a delay line, a filter bank, a feedback matrix, a first mixer Sum1, and a second mixer Sum2. Thus, after audio input data x(n) enters the feedback delay network, it first passes through the delay line and filter bank to produce a first output. The first mixer Sum1 then adds a level coefficient to the first output to produce the network output y(n) of the feedback delay network. The second mixer Sum2 then adds an attenuation coefficient to the first output, which is then fed into the feedback matrix to produce a second output. The sum of the second output and the audio input data x(n) serves as the input to the next delay line.

[0172] For another example, Figure 9 shows the network structure of a feedback delay network, which includes four delay lines and eight mixers. As shown in Figure 9, the feedback delay network includes delay line Z1, delay line Z2, delay line Z3, delay line Z4, filter FB1, filter FB2, filter FB3, filter FB4, feedback matrix, mixer Sum11, mixer Sum12, mixer Sum13, mixer Sum14, mixer Sum21, mixer Sum22, mixer Sum23, and mixer Sum24. In this way, after the audio input data x(n) enters the feedback delay network, it is first distributed to the delay lines Z1-Z4 (for the sake of convenience, the x(n) distributed to Z1 to Z4 is called x1(n) to x4(n)), and then passes through the filter groups FB1 to FB4 to obtain the first result (including 4 first results, one filter corresponds to one first result), and then the level coefficient is added to the first result through the mixers Sum11 to Sum14 to obtain the network output y1(n) to y4(n) of the feedback delay network; in addition, the mixers Sum21 to Sum24 add the attenuation coefficient to the first result, input the feedback matrix, and obtain the second result (including 4 second results, each mixer in the mixers Sum21 to Sum24 corresponds to a second result), and the sum of the second result and the audio input data x1(n) to x4(n) is used as the input of the next delay line.

[0173] In the above scheme, late reflections are analyzed and processed to obtain a first coefficient and a second coefficient. The first coefficient is used as the level coefficient of the preset feedback delay network, and the second coefficient is used as the attenuation coefficient of the preset feedback delay network to obtain a target feedback delay network. Generating a target feedback delay network based on late reflections facilitates subsequent input of audio data after a preset delay time and then feeding it into the target feedback delay network to obtain the network output, without having to convolve the audio input data with the entire initial impulse response, thus saving a significant amount of computing power.

[0174] In addition, a data processing algorithm is used to perform energy decay analysis on late reflections to determine the secondary reverberation time and secondary reverberation intensity. An initial level analysis algorithm is then used to perform level analysis on the secondary reverberation time, secondary reverberation intensity, and system parameters to obtain the first coefficient. A preset function is then used to perform decay analysis on the secondary reverberation time and system parameters to obtain the second coefficient. In this way, energy decay analysis can more accurately simulate reverberation characteristics.

[0175] In some embodiments, inputting audio input data into a target feedback delay network to obtain a network output can be achieved by:

[0176] After the audio input data has been mixed for a certain time, it is input into the target feedback delay network to obtain the network output.

[0177] In one embodiment, the preset delay time may be a mixing time, that is, inputting the audio input data into the target feedback delay network to obtain the network output, which may be achieved by:

[0178] After the audio input data has passed the preset delay time, it is input into the target feedback delay network to obtain the network output.

[0179] In the above scheme, the audio input data is input into the target feedback delay network after the mixing time to obtain the network output, which can more accurately simulate the reverberation characteristics.

[0180] S16. The sum of the network output and the convolution result is used as the audio output data.

[0181] The audio output data is the simulated sound field corresponding to the preset space.

[0182] In the above scheme, the audio input data, the user parameters corresponding to the preset space, and the system parameters corresponding to the preset space are first obtained, and the user parameters and the system parameters are integrated using an impulse generation algorithm to generate an initial impulse response; then, the initial impulse response is cut according to the system parameters to obtain early reflections and late reflections, and the audio input data is convolved with the early reflections to obtain a convolution result; then, the audio input data is input into the target feedback delay network to obtain the network output; finally, the sum of the network output and the convolution result is used as the audio output data. In this way, firstly, audio digital output data can be generated according to the user parameters, system parameters, and audio input data corresponding to the preset space, that is, from the user's perspective, the user can define the physical conditions corresponding to the preset space through the interface, thereby simulating the audio digital output data that the user wants, thereby improving the user experience; secondly, in the present disclosure, the audio input data is convolved with the early reflections in the initial impulse response, rather than convolving the audio input data with the entire initial impulse response, thereby saving a large amount of computing power resources; finally, the audio input data is input into the target feedback delay network to obtain the network output, and the sum of the network output and the convolution result is used as the audio output data, which can solve the problem that there is a certain gap between the sound field simulated by the Schroeder reverberator and the real sound, improve the accuracy of the sound field simulation, and further enhance the user experience.

[0183] The embodiment of the present disclosure can divide the functional modules of the sound field simulation device according to the above method example. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one processing unit. The above integrated modules can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the embodiment of the present disclosure is schematic and is only a logical functional division. In actual implementation, there may be other division methods.

[0184] As shown in FIG10 , an embodiment of the present disclosure provides a structural diagram of a sound field simulation device 200 , which includes an acquisition module 201 , a generation module 202 , a processing module 203 , and a convolution module 204 ; wherein,

[0185] An acquisition module 201 is configured to acquire audio input data, user parameters corresponding to a preset space, and system parameters corresponding to the preset space;

[0186] The generating module 202 is configured to integrate the user parameters and the system parameters using an impulse generating algorithm to generate an initial impulse response;

[0187] The processing module 203 is configured to cut the initial impulse response according to the system parameters to obtain early reflections and late reflections;

[0188] The convolution module 204 is configured to convolve the audio input data with the early reflections to obtain a convolution result;

[0189] The processing module 203 is further configured to input the audio input data into the target feedback delay network to obtain the network output; the sum of the network output and the convolution result is used as the audio output data; the audio output data is the sound field corresponding to the simulated preset space.

[0190] In some embodiments, the generation module 202 is further configured to use an image source algorithm to generate a primary impulse response based on user parameters and system parameters; use a reverberation time algorithm to generate an initial reverberation time and an event occurrence rate based on user parameters and system parameters; integrate the primary impulse response, the initial reverberation time, and the event occurrence rate to obtain an initial impulse response; wherein the event occurrence rate is the probability of an event occurring in the primary impulse response.

[0191] In some embodiments, the generation module 202 is further configured to generate a target sequence based on an event occurrence rate and an initial reverberation time using a Poisson process; perform a discrete transformation on the target sequence to obtain at least one subsequence; and perform a weighted summation of the at least one subsequence using a primary impulse response to obtain an initial impulse response.

[0192] In some embodiments, the processing module 203 is further configured to use a time estimation algorithm to estimate the initial impulse response to obtain a pre-delay time and a mixing time.

[0193] In some embodiments, the system parameters include the number of reflection orders, and the initial impulse response includes the initial reverberation time and the initial reverberation response; the processing module 203 is further configured to determine the initial reverberation intensity set within a first time period in the preset impulse response as the early reflection; the preset impulse response is the impulse response corresponding to the number of reflection orders in the initial impulse response; the first time period is the time period between the starting initial reverberation time and the first initial reverberation time in the preset impulse response, and the duration of the first time period is the same as the duration of the pre-delay time;

[0194] The processing module 203 is further configured to determine the initial reverberation intensity set within a second time period in the preset impulse response as a late reflection; the second time period is a time period between the first initial reverberation time and the second initial reverberation time in the preset impulse response, and the duration of the second time period is the same as the duration of the mixing time.

[0195] In some embodiments, the processing module 203 is further configured to input the audio input data into the target feedback delay network after the mixing time to obtain a network output.

[0196] In some embodiments, the processing module 203 is further configured to process the late reflections to obtain a target feedback delay network.

[0197] In some embodiments, the processing module 203 is further configured to analyze and process the late reflection to obtain a first coefficient and a second coefficient; use the first coefficient as the level coefficient of the preset feedback delay network and the second coefficient as the attenuation coefficient of the preset feedback delay network to obtain a target feedback delay network.

[0198] In some embodiments, the processing module 203 is further configured to use a data processing algorithm to perform energy decay analysis on the late reflections to determine the secondary reverberation time and the secondary reverberation intensity; use an initial level analysis algorithm to perform level analysis on the secondary reverberation time, the secondary reverberation intensity, and the system parameters to obtain a first coefficient; and use a preset function to perform decay analysis on the secondary reverberation time and the system parameters to obtain a second coefficient.

[0199] In some embodiments, the acquisition module 201 is further configured to acquire audio input data, user parameters corresponding to the preset space, and system parameters corresponding to the preset space in response to the user's target operation on the interactive interface; the interactive interface is used to implement interaction with the user.

[0200] In some embodiments, the interactive interface includes a space model corresponding to the preset space.

[0201] The acquisition module 201 is further configured to determine the target position parameters and target attribute parameters of the spatial model in the interactive interface in response to the user's target operation on the interactive interface, and determine the sum of the target position parameters and the target attribute parameters as the user parameters corresponding to the preset space.

[0202] In some embodiments, the acquisition module 201 is also configured to determine the target movement parameters according to the target operation when the target operation is detected; change the position of the spatial model in the interactive interface according to the target movement parameters; determine the position of the changed spatial model in the interactive interface as the target position, and determine the position parameters corresponding to the target position as the target position parameters; the target movement parameters include at least one of the target direction, target distance, and target angle.

[0203] In some embodiments, the acquisition module 201 is further configured to acquire target attribute parameters of the spatial model in the interactive interface from the corresponding relationship according to the target operation; the corresponding relationship is the corresponding relationship between the user operation and the spatial model attribute parameters; the user operation includes the target operation.

[0204] In some embodiments, the interactive interface includes a write control, and the target operation includes a write operation;

[0205] The acquisition module 201 is further configured to display a target window in response to a user triggering operation on the write control in the interactive interface; the target window is used to write user parameters;

[0206] The acquisition module 201 is further configured to acquire the user parameters written by the user in response to the user's write operation.

[0207] The sound field simulation device provided in this embodiment can execute the sound field simulation method provided in the above method embodiment. Its implementation principle and technical effects are similar to those of the above method and will not be described in detail here.

[0208] FIG11 is a schematic structural diagram of an electronic device provided in an embodiment of the present disclosure.

[0209] As shown in FIG11 , an embodiment of the present disclosure provides an electronic device comprising: a processor 1201, a memory 1202, and a computer program stored in the memory 1202 and executable on the processor 1201. When executed by the processor 1201, the computer program implements the various processes of the sound field simulation method described in the above-described method embodiment. The same technical effects can be achieved, and to avoid repetition, details are omitted here.

[0210] An embodiment of the present disclosure provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the various processes of the sound field simulation method in the above-mentioned method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0211] The computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0212] An embodiment of the present disclosure provides a computer program product, which stores a computer program. When the computer program is executed by a processor, the various processes of the sound field simulation method in the above method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0213] Those skilled in the art will appreciate that embodiments of the present disclosure may be provided as methods, systems, or computer program products. Thus, the present disclosure may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present disclosure may take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program code.

[0214] In the present disclosure, a processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0215] In this disclosure, memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0216] In the present disclosure, computer-readable media includes permanent and non-permanent, removable and non-removable storage media. Storage media can be implemented by any method or technology to store information, and the information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory media such as modulated data and carrier waves.

[0217] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

[0218] The foregoing description is intended only to provide specific embodiments of the present disclosure, intended to enable those skilled in the art to understand and implement the present disclosure. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the embodiments described herein, but rather to be construed in the broadest manner consistent with the principles and novel features disclosed herein. Industrial Applicability

[0219] The present disclosure relates to a sound field simulation method, apparatus, device, medium, and program product. The method comprises the following steps: obtaining audio input data, user parameters corresponding to a preset space, and system parameters corresponding to the preset space; integrating the user parameters and the system parameters using an impulse generation algorithm to generate an initial impulse response; segmenting the initial impulse response according to the system parameters to obtain early reflections and late reflections; convolving the audio input data with the early reflections to obtain a convolution result; inputting the audio input data into a target feedback delay network to obtain a network output; generating the target feedback delay network from the late reflections through analysis and processing; and obtaining the sum of the network output and the convolution result as audio output data. The audio output data is the simulated sound field corresponding to the preset space.

Claims

1. A sound field simulation method, comprising: Acquire audio input data, user parameters corresponding to a preset space, and system parameters corresponding to the preset space; Using an impulse generation algorithm to integrate the user parameters and system parameters to generate an initial impulse response; Cutting the initial impulse response according to the system parameters to obtain early reflections and late reflections; Convolution processing is performed on the audio input data and the early reflection to obtain a convolution result; Inputting the audio input data into a target feedback delay network to obtain a network output; The sum of the network output and the convolution result is used as audio output data; the audio output data is the simulated sound field corresponding to the preset space.

2. The sound field simulation method according to claim 1, wherein: The step of integrating the user parameters and the system parameters using an impulse generation algorithm to generate an initial impulse response includes: generating a primary impulse response based on the user parameters and the system parameters using an image source algorithm; Generate an initial reverberation time and an event occurrence rate according to the user parameters and the system parameters using a reverberation time algorithm; wherein the event occurrence rate and the probability of occurrence of a primary impulse response event; The primary impulse response, the initial reverberation time and the event occurrence rate are integrated to obtain the initial impulse response.

3. The sound field simulation method according to claim 2, wherein: The step of integrating the primary impulse response, the initial reverberation time, and the event occurrence rate to obtain the initial impulse response includes: Using a Poisson process, a target sequence is generated according to the event occurrence rate and the initial reverberation time; Performing a discrete transformation on the target sequence to obtain at least one subsequence; The initial impulse response is obtained by performing weighted summation on the at least one subsequence using the primary impulse response.

4. The sound field simulation method according to claim 1, wherein: Before the initial impulse response is cut according to the system parameters to obtain early reflections and late reflections, the method further includes: The initial impulse response is estimated using a time estimation algorithm to obtain a pre-delay time and a mixing time.

5. The sound field simulation method according to claim 4, wherein: The system parameters include reflection levels, and the initial impulse response includes initial reverberation time and initial reverberation response; The cutting process of the initial impulse response according to the system parameter to obtain the early reflection and the late reflection includes: Determine an initial reverberation intensity set within a first time period in a preset impulse response as the early reflection; the preset impulse response is an impulse response corresponding to the reflection order in the initial impulse response; the first time period is a time period between a starting initial reverberation time and a first initial reverberation time in the preset impulse response, and the duration of the first time period is the same as the duration of the pre-delay time; An initial reverberation intensity set within a second time period in the preset impulse response is determined as the late reflection; the second time period is a time period between a first initial reverberation time and a second initial reverberation time in the preset impulse response, and the duration of the second time period is the same as the duration of the mixing time.

6. The sound field simulation method according to claim 4, wherein: The step of inputting the audio input data into a target feedback delay network to obtain a network output comprises: After the audio input data has passed the mixing time, it is input into a target feedback delay network to obtain the network output.

7. The method according to claim 1, wherein: Before inputting the audio input data into a target feedback delay network to obtain a network output, the method further includes: The late reflections are processed to obtain the target feedback delay network.

8. The sound field simulation method according to claim 7, wherein: The processing of the late reflection to obtain the target feedback delay network includes: Analyzing and processing the late reflection to obtain a first coefficient and a second coefficient; The first coefficient is used as a level coefficient of a preset feedback delay network, and the second coefficient is used as an attenuation coefficient of the preset feedback delay network to obtain the target feedback delay network.

9. The sound field simulation method according to claim 8, wherein: The analyzing and processing the late reflection to obtain the first coefficient and the second coefficient includes: performing an energy decay analysis on the late reflections using a data processing algorithm to determine a secondary reverberation time and a secondary reverberation intensity; Performing level analysis on the secondary reverberation time, the secondary reverberation intensity and the system parameter using an initial level analysis algorithm to obtain a first coefficient; A preset function is used to perform attenuation analysis on the secondary reverberation time and the system parameter to obtain a second coefficient.

10. The sound field simulation method according to claim 1, wherein: The obtaining of audio input data, user parameters corresponding to the preset space, and system parameters corresponding to the preset space includes: In response to a user's target operation on an interactive interface, audio input data, user parameters corresponding to a preset space, and system parameters corresponding to the preset space are acquired; the interactive interface is used to implement interaction with the user.

11. The sound field simulation method according to claim 10, wherein: The interactive interface includes a space model corresponding to the preset space; In response to the user's target operation on the interactive interface, obtaining user parameters corresponding to the preset space includes: In response to the user's target operation on the interactive interface, determining a target position parameter and a target attribute parameter of the spatial model in the interactive interface; The sum of the target position parameter and the target attribute parameter is determined as the user parameter corresponding to the preset space.

12. The sound field simulation method according to claim 11, wherein: In response to the user's target operation on the interactive interface, determining a target position parameter of the space model in the interactive interface includes: When a target operation is detected, a target movement parameter is determined according to the target operation; the target movement parameter includes at least one of a target direction, a target distance, and a target angle; changing the position of the spatial model in the interactive interface according to the target movement parameter; The position of the changed spatial model in the interactive interface is determined as the target position, and the position parameters corresponding to the target position are determined as the target position parameters.

13. The sound field simulation method according to claim 11, wherein: In response to the target operation of the user on the interactive interface, determining the target attribute parameters of the space model in the interactive interface includes: According to the target operation, the target attribute parameters of the space model in the interactive interface are obtained from the corresponding relationship; the corresponding relationship is the corresponding relationship between the user operation and the space model attribute parameters; the user operation includes the target operation.

14. The sound field simulation method according to any one of claims 10 to 13, wherein: The interactive interface includes a write control; the target operation includes a write operation; and in response to the user's target operation on the interactive interface, obtaining user parameters corresponding to the preset space includes: In response to the user's triggering operation on the write control in the interactive interface, a target window is displayed; the target window is used to write user parameters; In response to the writing operation of the user, a user parameter written by the user is acquired.

15. A sound field simulation device, comprising: An acquisition module, configured to acquire audio input data, user parameters corresponding to a preset space, and system parameters corresponding to the preset space; A generating module is configured to integrate the user parameters and the system parameters using an impulse generating algorithm to generate an initial impulse response; A processing module, configured to perform segmentation processing on the initial impulse response according to the system parameters to obtain early reflections and late reflections; A convolution module, configured to perform convolution processing on the audio input data and the early reflection to obtain a convolution result; The processing module is further configured to input the audio input data into a target feedback delay network to obtain a network output; The processing module is further configured to use the sum of the network output and the convolution result as audio output data; the audio output data is the simulated sound field corresponding to the preset space.

16. The sound field simulation device according to claim 15, wherein: The generation module is further configured to generate a primary impulse response according to the user parameters and the system parameters using an image source algorithm; generate an initial reverberation time and an event occurrence rate according to the user parameters and the system parameters using a reverberation time algorithm; integrate the primary impulse response, the initial reverberation time and the event occurrence rate to obtain the initial impulse response; wherein the event occurrence rate is the probability of occurrence of a primary impulse response event.

17. The sound field simulation device according to claim 16, wherein: The generation module is further configured to generate a target sequence according to the event occurrence rate and the initial reverberation time using a Poisson process; perform a discrete transformation on the target sequence to obtain at least one subsequence; and perform a weighted summation on the at least one subsequence using the primary impulse response to obtain the initial impulse response.

18. The sound field simulation device according to claim 15, wherein: The processing module is further configured to use a time estimation algorithm to estimate the initial impulse response to obtain a pre-delay time and a mixing time.

19. The sound field simulation device according to claim 18, wherein: The system parameters include reflection levels, and the initial impulse response includes initial reverberation time and initial reverberation response; The processing module is further configured to determine an initial reverberation intensity set within a first time period in a preset impulse response as the early reflection; the preset impulse response is an impulse response corresponding to the reflection order in the initial impulse response; the first time period is a time period between a starting initial reverberation time and a first initial reverberation time in the preset impulse response, and the length of the first time period is the same as the length of the pre-delay time; The processing module is further configured to determine an initial reverberation intensity set within a second time period in the preset impulse response as the late reflection; the second time period is a time period between a first initial reverberation time and a second initial reverberation time in the preset impulse response, and the duration of the second time period is the same as the duration of the mixing time.

20. The sound field simulation device according to claim 18, wherein: The processing module is further configured to input the audio input data into a target feedback delay network after the mixing time, to obtain the network output.

21. The sound field simulation device according to claim 15, wherein: The processing module is further configured to process the late reflection to obtain the target feedback delay network.

22. The sound field simulation device according to claim 21, wherein: The processing module is also configured to analyze and process the late reflection to obtain a first coefficient and a second coefficient; use the first coefficient as a level coefficient of a preset feedback delay network and the second coefficient as an attenuation coefficient of the preset feedback delay network to obtain the target feedback delay network.

23. The sound field simulation device according to claim 22, wherein: The processing module is further configured to use a data processing algorithm to perform an energy decay analysis on the late reflection to determine the secondary reverberation time and the secondary reverberation intensity; use an initial level analysis algorithm to perform a level analysis on the secondary reverberation time, the secondary reverberation intensity and the system parameter to obtain a first coefficient; and use a preset function to perform an decay analysis on the secondary reverberation time and the system parameter to obtain a second coefficient.

24. The sound field simulation device according to claim 15, wherein: The acquisition module is also configured to acquire audio input data, user parameters corresponding to the preset space, and system parameters corresponding to the preset space in response to the user's target operation on the interactive interface; the interactive interface is used to realize interaction with the user.

25. The sound field simulation device according to claim 24, wherein: The interactive interface includes a space model corresponding to the preset space; The acquisition module is also configured to determine the target position parameters and target attribute parameters of the spatial model in the interactive interface in response to the user's target operation on the interactive interface; and determine the sum of the target position parameters and the target attribute parameters as the user parameters corresponding to the preset space.

26. The sound field simulation device according to claim 25, wherein: The acquisition module is further configured to, when a target operation is detected, determine a target movement parameter according to the target operation; and change the position of the space model in the interactive interface according to the target movement parameter; Determining the position of the changed spatial model in the interactive interface as the target position, and determining the position parameters corresponding to the target position as the target position parameters; The target movement parameter includes at least one of a target direction, a target distance and a target angle.

27. The sound field simulation device according to claim 25, wherein: The acquisition module is further configured to acquire target attribute parameters of the space model in the interactive interface from a corresponding relationship according to the target operation; the corresponding relationship is a corresponding relationship between user operations and space model attribute parameters; and the user operation includes the target operation.

28. The sound field simulation device according to any one of claims 24 to 27, wherein: The interactive interface includes a write control; the target operation includes a write operation; The acquisition module is further configured to display a target window in response to the user's triggering operation on the write control in the interactive interface; the target window is used to write user parameters; The acquisition module is further configured to acquire the user parameters written by the user in response to the user's write operation.

29. An electronic device comprising: A processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the sound field simulation method according to any one of claims 1 to 14.

30. A computer-readable storage medium, comprising: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the sound field simulation method according to any one of claims 1 to 14 is implemented.

31. A computer program; the computer program comprises a computer-readable code, and when the computer-readable code is executed in an electronic device, a processor of the electronic device can implement the sound field simulation method as claimed in any one of claims 1 to 14.

32. A computer program product, comprising a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code, wherein when the computer-readable code runs in a processor of an electronic device, the processor in the electronic device implements the sound field simulation method as described in any one of claims 1 to 14 when executing the code.

Images