A sound effect processing system and method

Abstract

The present invention relates to a sound effect processing system and method. The corresponding calculation orders are configured for a low-order sound ray calculator and a high-order sound ray calculator respectively; according to the sound source position tracking information, the low-order sound ray calculator and the high-order sound ray calculator are respectively input to calculate and obtain each sound ray data; the sound ray decomposition output device decomposes and processes the sound ray according to the sound ray data provided by the low-order sound ray calculator and the high-order sound ray calculator to obtain a sound ray data subset A and a sound ray data subset B with different sound ray lengths; the loudspeaker audio driving signal corresponding to each sound ray is calculated respectively, and a complete loudspeaker audio driving signal is obtained after mixing. The present invention simultaneously achieves a large number of sound ray tracking and a small calculation delay.

Description

Technical Field

[0001] The invention relates to a sound effect processing system and method. Background Art

[0002] like Figure 1 As shown in the figure, the real-time processing of sound effects in a virtual acoustic environment requires the establishment of different sound effect models by tracking the sound rays emitted by the sound source. There are two key performance indicators in the process of sound ray tracking. The first is the maximum order of sound ray tracking. The higher the order, the more precise the model is, but the higher the order, the more sound rays need to be tracked, resulting in a longer calculation time. The second is the delay of sound ray update after the sound source position information is updated. The smaller the sound ray update delay, the better the real-time performance of the overall sound effect processing. However, the delay of sound ray update is limited by the calculation time of sound ray tracking. The longer the calculation time of sound ray tracking, the greater the delay of sound ray update.

[0003] Accurate sound modeling requires tracking a large number of sound lines, which brings a large amount of calculation. However, since the sound lines are independent of each other, multiple threads can be divided and parallel computing can be used to speed up the program calculation, making real-time calculation of the sound effect model possible. Currently, there are two main technical routes for tracking a large number of sound lines.

[0004] Technical route 1, such as EVERTims and other CPU-based parallel sound effect model real-time generation software (David Poirier-Quinot, Brian Katz, Markus Noisternig. EVERTims: Open source framework for realtime acoustics in architectural acoustics and virtual reality. 20th International Conference on Digital Audio Effects (DAFx-17), Sep 2017, Edinburgh, United Kingdom.), the advantage of this technology is that the calculation speed of a single CPU core is fast, but when the number of sound lines calculated in parallel is greater than the number of cores, each core needs to calculate multiple sound lines in series, which requires a longer calculation time, so the parallel calculation speed of this technology is limited by the number of cores. Due to the limitations of current CPU manufacturing technology, the more cores a CPU has, the higher the cost, so it is difficult to achieve simultaneous tracking of a large number of sound lines based on CPU parallel computing technology.

[0005] Technical route 2, for example, the real-time generation technology of room acoustic model based on GPU parallelism (L. Savioja, D. Manocha, and M. Lin, "Use of GPUs in room acoustic modeling and auralization," in Proc. Int. Symp. Room Acoust., Melbourne, Australia, Aug. 2010.), because the structure of GPU is suitable for parallel computing, the real-time generation technology of acoustic model based on GPU parallelism can generally allow a large number of sound lines to be tracked simultaneously, so the calculation speed is generally higher than the real-time generation technology of acoustic model based on CPU. However, the interaction delay between GPU and CPU program is large. This will increase the delay in the calculation of sound effect generation, which will have an adverse effect on the real-time performance of sound effect processing.

[0006] Although technical route 1 can calculate the trajectory of a single or a small number of sound rays quickly, it is difficult to achieve simultaneous tracking of a large number of sound rays due to the current technical level and cost limitations. That is, the calculation delay of technical route 1 is small, but the number of sound rays tracked is also small. Technical route 2 can achieve simultaneous tracking of a large number of sound rays. However, due to the limitation of GPU program interaction delay, there will be a large read and write delay. That is, the number of sound rays tracked by technical route 2 is large, but the calculation delay is also large.

[0007] In short, existing technologies cannot achieve real-time generation of sound effect models in virtual acoustic environments with a large number of sound ray tracing and a small computational delay at the same time. Summary of the invention

[0008] In order to solve the above technical defects, the present invention proposes a sound effect processing system and method. The present invention calculates sound rays to different orders respectively, and cooperates with the corresponding sound ray decomposition and mixing method based on the length of the sound ray, decomposes the sound ray calculation results into two parts, and outputs them to different sound effect processors for real-time sound effect processing, thereby optimizing the defects of the existing technology and achieving a larger number of sound ray tracking and a smaller calculation delay.

[0009] The technical solution to the above technical problem is: a sound effect processing system, including a sound ray real-time calculation module, a sound effect processor A, a sound effect processor B and a signal mixer; wherein the sound ray real-time calculation module is composed of a sound ray tracing model generator, a low-order sound ray calculator, a high-order sound ray calculator, and a sound ray synthesis output device;

[0010] Initialization phase:

[0011] The sound ray tracing model generator configures corresponding calculation orders for the low-order sound ray calculator and the high-order sound ray calculator according to the input sound field environment model;

[0012] Real-time operation stage;

[0013] After the sound source position tracking information is updated, the sound source position tracking information is respectively input into the low-order sound ray calculator and the high-order sound ray calculator for calculation to obtain the data of each sound ray;

[0014] The low-order sound ray calculator and the high-order sound ray calculator output corresponding sound ray data to the sound ray decomposition output device; the sound ray decomposition output device decomposes and processes the sound ray according to the sound ray data provided by the low-order sound ray calculator to obtain a sound ray data subset A, and outputs the sound ray data subset A to the sound effect processor A; the sound ray decomposition output device decomposes and processes the sound ray according to the sound ray data provided by the high-order sound ray calculator to obtain a sound ray data subset B, and outputs the sound ray data subset B to the sound effect processor B; wherein the sound ray length in the sound ray data subset A is less than the sound ray length in the sound ray data subset B;

[0015] The sound effect processor A performs real-time sound effect processing on the sound line data subset A, and calculates the speaker audio driving signal corresponding to the sound line data subset A; the sound effect processor B performs real-time sound effect processing on the sound line data subset B, and calculates the speaker audio driving signal corresponding to the sound line data subset B;

[0016] The signal mixer mixes the speaker audio driving signals output by the sound effect processor A and the sound effect processor B to obtain a complete speaker audio driving signal.

[0017] Further, the method for determining the calculation order of the low-order sound ray calculator is to increase the calculation order of the low-order sound ray calculator from 0 until a maximum calculation order N is determined. low , which satisfies the completion of 0~N low The time t for calculating the order sound rays low The expected sound ray calculation delay is not exceeded; the method for determining the calculation order of the high-order sound ray calculator is to increase the calculation order of the high-order sound ray calculator from 0 based on the update time interval Ts of the sound source position tracking information until a maximum calculation order N is determined. high , which satisfies the completion of 0~N high The time t for calculating the order sound rays high Not exceeding a certain percentage of the reference time Ts.

[0018] Furthermore, the sound ray decomposition outputter performs rectangular window truncation according to the sound ray length, and processes the sound ray data output by the low-order sound ray calculator and the high-order sound ray calculator respectively to obtain the sound ray data subset A and the sound ray data subset B. The sound ray length in the sound ray data subset A is less than or equal to t*c, and the sound ray length in the sound ray data subset B is greater than t*c, where t is the expected truncation time and c is the sound speed.

[0019] Furthermore, the audio frame length of the sound effect processor A is set at 8 to 128 sampling points, and the audio frame length of the sound effect processor B is set at 256 to 4096 sampling points.

[0020] Furthermore, the signal mixer delays the audio driving signal output by the sound effect processor B by dt and adds it to the audio driving signal output by the sound effect processor A. The method for determining dt is dt=t+t low +t a -t high -t b . Where t is the expected cutoff time, t a is the audio processing delay of sound effect processor A, t b is the audio processing delay of sound effect processor B.

[0021] The present invention also provides a sound effect processing method.

[0022] Initialization phase:

[0023] According to the input sound field environment model, corresponding calculation orders are configured for the low-order sound ray calculator and the high-order sound ray calculator respectively;

[0024] Real-time operation stage;

[0025] After the sound source position tracking information is updated, the sound source position tracking information is respectively input into the low-order sound ray calculator and the high-order sound ray calculator for calculation to obtain the data of each sound ray;

[0026] The low-order sound ray calculator and the high-order sound ray calculator output corresponding sound ray data to the sound ray decomposition output device; the sound ray decomposition output device decomposes and processes the sound ray according to the sound ray data provided by the low-order sound ray calculator to obtain a sound ray data subset A, and outputs the sound ray data subset A to the sound effect processor A; the sound ray decomposition output device decomposes and processes the sound ray according to the sound ray data provided by the high-order sound ray calculator to obtain a sound ray data subset B, and outputs the sound ray data subset B to the sound effect processor B; wherein the sound ray length in the sound ray data subset A is less than the sound ray length in the sound ray data subset B;

[0027] The sound effect processor A performs real-time sound effect processing on the sound line data subset A, and calculates the speaker audio driving signal corresponding to the sound line data subset A; the sound effect processor B performs real-time sound effect processing on the sound line data subset B, and calculates the speaker audio driving signal corresponding to the sound line data subset B;

[0028] The signal mixer mixes the speaker audio driving signals output by the sound effect processor A and the sound effect processor B to obtain a complete speaker audio driving signal.

[0029] Compared with the prior art, the present invention has the following significant advantages:

[0030] (1) The total system delay of the present invention is equivalent to that of the first technical route, both of which are t low +t a , but the maximum calculation order of the sound line in the present invention is N high , which is greater than the maximum calculation order of the sound rays in the technical route 1, so compared with the technical route 1, the present invention has the advantage of a larger number of sound ray tracing;

[0031] (2) The number of sound ray tracing processed by the present invention is equivalent to that of the second technical route, both of which are 0~N hig The number of order sound lines, but the total system delay of the present invention is t low +t a , which is less than the total system delay of technical route 2. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 It is a schematic diagram of a real-time sound processing system for a virtual acoustic environment in the prior art;

[0033] Figure 2 It is a schematic diagram of the sound effect processing system of the present invention. DETAILED DESCRIPTION

[0034] It is easy to understand that according to the technical solution of the present invention, without changing the essential spirit of the present invention, a person skilled in the art can imagine various implementation methods of the present invention. Therefore, the following specific implementation methods and drawings are only exemplary descriptions of the technical solution of the present invention, and should not be regarded as the whole of the present invention or as a limitation or restriction of the technical solution of the present invention.

[0035] Various exemplary embodiments of the present invention will now be described in detail with reference to the accompanying drawings. It should be noted that the relative arrangement of components and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present invention unless otherwise specifically stated.

[0036] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the invention, its application, or uses.

[0037] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered as part of the specification.

[0038] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not limiting. Therefore, other examples of the exemplary embodiments may have different values.

[0039] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0040] like Figure 2 As shown, the sound effect processing system of the present invention includes a sound ray real-time calculation module, a sound effect processor A, a sound effect processor B and a signal mixer. The sound ray real-time calculation module is composed of a sound ray tracing model generator, a low-order sound ray calculator, a high-order sound ray calculator, and a sound ray synthesis outputter, wherein the high-order sound ray calculator will call the GPU computing unit.

[0041] The sound effect processing system receives the sound field environment model in the initialization stage and receives the sound source position tracking information in the real-time processing stage. Different from the background technology that outputs a whole group of sound lines, the technical solution of the present invention outputs sound lines with shorter sound line lengths and sound lines with longer sound line lengths to different sound effect processors asynchronously.

[0042] The operation and processing flow of the sound effect processing system of the present invention are as follows:

[0043] 1. In the initialization phase, the sound ray tracing model generator configures the calculation orders corresponding to the two sound ray calculators for the low-order sound ray calculator and the high-order sound ray calculator according to the input sound field environment model. The calculation orders corresponding to the two sound ray calculators can be determined according to the calculation performance of the sound ray real-time calculation system and the complexity of the sound field environment model.

[0044] One implementation method for determining the calculation order of the low-order sound ray calculator is to increase the calculation order of the low-order sound ray calculator from 0 until a maximum calculation order N is determined. low , which satisfies the completion of 0~N low The time t for calculating the order sound rays low Do not exceed the expected ray calculation delay (i.e. the time it takes from the start of ray calculation to the completion of ray calculation).

[0045] One implementation method for determining the calculation order of the high-order sound ray calculator is to increase the calculation order of the high-order sound ray calculator from 0, based on the update time interval Ts of the sound source position tracking information (i.e., the spatial coordinates of the sound source position determined by a certain method, such as using a microphone array to estimate the sound source position), until a maximum calculation order N is determined. high , which satisfies the completion of 0~N high The time t for calculating the order sound rays high Not exceeding a certain proportion of the reference time Ts, for example, 50% to 80%, the proportion being set to allow a certain calculation margin).

[0046] The above setting of the calculation order of the low-order sound ray calculator and the high-order sound ray calculator generally makes the low-order sound ray calculator complete the calculation before the high-order sound ray calculator in time, that is, t low <th igh .

[0047] 2. In the real-time operation stage, after the sound source position tracking information is updated, the sound source position tracking information is used as the input parameter to call the low-order sound ray calculator and the high-order sound ray calculator to calculate and obtain the data of each sound ray, that is, the path of each sound ray and the corresponding transmission parameters, such as the spatial coordinates of the starting point, reflection point, and end point of each sound ray and the attenuation of sound energy.

[0048] 3. After the low-order sound ray calculator and the high-order sound ray calculator complete the calculation of the sound ray, they output the corresponding sound ray data to the sound ray decomposition output device. The sound ray decomposition output device will decompose and process the sound ray data obtained by the two sound ray calculators respectively, and output the decomposed sound ray data to different sound effect processors.

[0049] 3.1 After the low-order sound ray calculator completes the sound ray calculation, the sound ray decomposition output device will first decompose the sound ray according to the sound ray data of the low-order sound ray calculator to obtain a sound ray data subset A with a small sound ray length. One implementation method of the decomposition process is to perform a rectangular window truncation on the sound ray according to the sound ray length, for example, retaining the sound ray with a length less than or equal to t*c and removing the sound ray with a length greater than t*c (equivalent to retaining the part reaching the listener from 0 to t), where t is the expected truncation time parameter (generally equal to the time required from the sound emitted by the sound source at a certain moment to the appearance of the late reverberation sound corresponding to the sound at that moment, for example 200ms), and c is the speed of sound. Similarly, other window functions or more complex processing can be used for decomposition. The sound ray data subset A with a smaller sound ray length obtained after decomposition is output to the sound effect processor A.

[0050] 3.2 After the high-order sound ray calculator completes the sound ray calculation, the sound ray decomposition output device also decomposes the sound ray data of the high-order sound ray calculator to obtain a sound ray data subset B with a large sound ray length. One implementation of the decomposition process is to perform a rectangular window truncation on the sound ray according to the sound ray length, for example, retaining the sound ray with a length greater than t*c and removing the sound ray with a length less than or equal to t*c. Similarly, other window functions or more complex processing can be used for decomposition. The sound ray data subset B with a larger sound ray length obtained after decomposition is output to the sound effect processor B.

[0051] 4. The sound effect processor A receives the sound line data subset A output by the sound line decomposition output device for real-time sound effect processing, and calculates the audio driving signal corresponding to this part of the sound line fed to the speaker. According to the characteristics of human auditory perception, the sound effect processor A processes the direct sound and early reflected sound in the sound effect, and needs to process the sound with a lower audio processing delay ta (i.e., the time taken from the original audio signal inputting the sound effect processor to the time when the sound effect processor starts to output the speaker driving signal) is the target configuration. Since the smaller the audio frame length is in audio signal processing, the lower the system delay is, the smaller the audio frame length needs to be set when configuring the parameters of the sound effect processor A, for example, within the range of 8 to 128 sampling points, preferably set at 8 to 32 sampling points.

[0052] 5. The sound effect processor B receives the sound line data subset B output by the sound line decomposition output device for real-time sound effect processing, and calculates the audio driving signal corresponding to this part of the sound line fed to the speaker. The sound effect processor B mainly corresponds to the processing of late reverberation sound. Since there is a natural time interval (i.e., the expected cutoff time t) from the sound emitted by the sound source at a certain moment to the appearance of the late reverberation sound corresponding to the sound at that moment, the sound effect processor B is allowed to have a slightly higher audio processing delay t b , only needs to satisfy t b ≤(t+t low +t a -t high ). Since the sound line data subset B generally has a large number of sound lines, the processing of the late reverberation sound in the sound effect requires a large amount of calculation. Therefore, when configuring the parameters of the sound effect processor B, it is necessary to set a larger audio frame length, for example, within the range of 256 to 4096 sampling points, preferably set at 1024 to 2048 sampling points. The audio frame length of the sound effect processor B is greater than the audio frame length of the sound effect processor A, so t a <t b .

[0053] 6. The audio signals output by sound effect processor A and sound effect processor B are mixed by the signal mixer. The mixed output signal of the mixer is the complete speaker driving signal. The specific operation of mixing is to delay the audio signal output by sound effect processor B by dt and then add it to the audio signal output by sound effect processor A. The method for determining dt is dt = t + t low +t a -t high -t b .

[0054] According to the above processing flow, it can be seen that the present invention tracks 0~N high The total system delay, that is, the time from the start of calculating the sound line to the start of the sound effect processor to output the loudspeaker drive signal, is t low +t a .

[0055] The present invention also proposes a sound effect processing method, in which, in an initialization stage, corresponding calculation orders are configured for a low-order sound ray calculator and a high-order sound ray calculator respectively according to an input sound field environment model;

[0056] In the real-time operation stage, after the sound source position tracking information is updated, the sound source position tracking information is input into the low-order sound ray calculator and the high-order sound ray calculator for calculation to obtain the data of each sound ray.

[0057] The low-order sound ray calculator and the high-order sound ray calculator output corresponding sound ray data to the sound ray decomposition output device; the sound ray decomposition output device decomposes and processes the sound ray according to the sound ray data provided by the low-order sound ray calculator to obtain a sound ray data subset A, and outputs the sound ray data subset A to the sound effect processor A; the sound ray decomposition output device decomposes and processes the sound ray according to the sound ray data provided by the high-order sound ray calculator to obtain a sound ray data subset B, and outputs the sound ray data subset B to the sound effect processor B; wherein the sound ray length in the sound ray data subset A is less than the sound ray length in the sound ray data subset B;

[0058] The sound effect processor A performs real-time sound effect processing on the sound line data subset A, and calculates the speaker audio driving signal corresponding to the sound line data subset A; the sound effect processor B performs real-time sound effect processing on the sound line data subset B, and calculates the speaker audio driving signal corresponding to the sound line data subset B; the signal mixer mixes the speaker audio driving signals output by the sound effect processor A and the sound effect processor B to obtain a complete speaker audio driving signal.

[0059] The method for determining the calculation order of the low-order sound ray calculator is to increase the calculation order of the low-order sound ray calculator from 0 until a maximum calculation order N is determined. low , which satisfies the completion of 0~N low The time t for calculating the order sound rays low The expected sound ray calculation delay is not exceeded; the method for determining the calculation order of the high-order sound ray calculator is to increase the calculation order of the high-order sound ray calculator from 0 based on the update time interval Ts of the sound source position tracking information until a maximum calculation order N is determined. high , which satisfies the completion of 0~N high The time t for calculating the order sound rays high Not exceeding a certain percentage of the reference time Ts.

[0060] The sound line decomposition output device performs rectangular window truncation according to the sound line length, and processes the sound line data output by the low-order sound line calculator and the high-order sound line calculator respectively to obtain sound line data subset A and sound line data subset B. The sound line length in sound line data subset A is less than or equal to t*c, and the sound line length in sound line data subset B is greater than t*c, where t is the expected truncation time and c is the speed of sound. The audio frame length of sound effect processor A is set at 8 to 32 sampling points, and the audio frame length of sound effect processor B is set at 1024 to 2048 sampling points. The signal mixer delays the audio driving signal output by sound effect processor B by dt and adds it to the audio driving signal output by sound effect processor A. The method for determining dt is dt=t+t low +t a -t high -t b . Where t is the expected cutoff time, t a is the audio processing delay of sound effect processor A, t b is the audio processing delay of sound effect processor B.

[0061] The main difference between the present invention and the prior art is that the present invention uses a sound line decomposition and mixing method based on the sound line length, and uses two technologies to calculate the sound line to different orders respectively, and decomposes the sound line calculation results into two parts and outputs them to different sound effect processors for real-time sound effect processing. The total system delay of the present invention is equivalent to that of the technical route 1, both of which are t low +t a , but the maximum calculation order of the sound line in the present invention is N high , which is greater than the maximum calculation order of the sound line of technical route 1. The maximum calculation order of the sound line of technical route 1 is approximately equal to N low Therefore, compared with the first technical route, the present invention has the advantage of a larger number of sound ray tracing; the number of sound ray tracing processed by the present invention is equivalent to that of the second technical route, both of which are 0~N high The number of order sound lines, but the total system delay of the present invention is t low +t a , which is less than the total system delay of the second technical route. The sound effect processor of the second technical route is similar to the sound effect processor B in the present invention. The total system delay is approximately equal to t hig +t b .

Claims

1. A sound effect processing system, characterized in that: It includes a sound ray real-time calculation module, a sound effect processor A, a sound effect processor B and a signal mixer; wherein the sound ray real-time calculation module is composed of a sound ray tracing model generator, a low-order sound ray calculator, a high-order sound ray calculator and a sound ray decomposition output device; Initialization phase: The sound ray tracing model generator configures corresponding calculation orders for the low-order sound ray calculator and the high-order sound ray calculator according to the input sound field environment model; Real-time operation stage; After the sound source position tracking information is updated, the sound source position tracking information is respectively input into the low-order sound ray calculator and the high-order sound ray calculator for calculation to obtain the data of each sound ray; The low-order sound ray calculator and the high-order sound ray calculator output corresponding sound ray data to the sound ray decomposition output device; the sound ray decomposition output device decomposes and processes the sound ray according to the sound ray data provided by the low-order sound ray calculator to obtain a sound ray data subset A, and outputs the sound ray data subset A to the sound effect processor A; the sound ray decomposition output device decomposes and processes the sound ray according to the sound ray data provided by the high-order sound ray calculator to obtain a sound ray data subset B, and outputs the sound ray data subset B to the sound effect processor B; wherein the sound ray length in the sound ray data subset A is less than the sound ray length in the sound ray data subset B; The sound effect processor A performs real-time sound effect processing on the sound line data subset A, and calculates the speaker audio driving signal corresponding to the sound line data subset A; the sound effect processor B performs real-time sound effect processing on the sound line data subset B, and calculates the speaker audio driving signal corresponding to the sound line data subset B; The signal mixer mixes the speaker audio driving signals output by the sound effect processor A and the sound effect processor B to obtain a complete speaker audio driving signal.

2. The sound effect processing system according to claim 1, characterized in that: The method for determining the calculation order of the low-order sound ray calculator is to increase the calculation order of the low-order sound ray calculator from 0 until a maximum calculation order N is determined. low , which satisfies the completion of 0~N low The time t for calculating the order sound rays low The expected sound ray calculation delay is not exceeded; the method for determining the calculation order of the high-order sound ray calculator is to increase the calculation order of the high-order sound ray calculator from 0 based on the update time interval Ts of the sound source position tracking information until a maximum calculation order N is determined. high , which satisfies the completion of 0~N high The time t for calculating the order sound rays high Not exceeding a certain percentage of the reference time Ts.

3. The sound effect processing system according to claim 1, characterized in that: The sound ray decomposition outputter performs rectangular window truncation according to the sound ray length, and processes the sound ray data output by the low-order sound ray calculator and the high-order sound ray calculator respectively to obtain the sound ray data subset A and the sound ray data subset B. The sound ray length in the sound ray data subset A is less than or equal to t*c, and the sound ray length in the sound ray data subset B is greater than t*c, where t is the expected truncation time and c is the sound speed.

4. The sound effect processing system according to claim 1, characterized in that: The audio frame length of sound effect processor A is set at 8 to 128 sampling points, and the audio frame length of sound effect processor B is set at 256 to 4096 sampling points.

5. The sound effect processing system according to claim 2, characterized in that: The signal mixer delays the audio driving signal output by the sound effect processor B by dt and then adds it to the audio driving signal output by the sound effect processor A. The method for determining dt is dt=t+t low +t a -t high -t b , where t is the expected cutoff time, t a is the audio processing delay of sound effect processor A, t b is the audio processing delay of sound effect processor B.

6. A sound effect processing method, characterized in that: Initialization phase: According to the input sound field environment model, corresponding calculation orders are configured for the low-order sound ray calculator and the high-order sound ray calculator respectively; Real-time operation stage; After the sound source position tracking information is updated, the sound source position tracking information is respectively input into the low-order sound ray calculator and the high-order sound ray calculator for calculation to obtain the data of each sound ray; The low-order sound ray calculator and the high-order sound ray calculator output corresponding sound ray data to the sound ray decomposition output device; The sound ray decomposition output device decomposes the sound ray according to the sound ray data provided by the low-order sound ray calculator to obtain a sound ray data subset A, and outputs the sound ray data subset A to the sound effect processor A; The sound ray decomposition output device decomposes the sound ray according to the sound ray data provided by the high-order sound ray calculator to obtain a sound ray data subset B, and outputs the sound ray data subset B to the sound effect processor B; wherein the sound ray length in the sound ray data subset A is less than the sound ray length in the sound ray data subset B; The sound effect processor A performs real-time sound effect processing on the sound line data subset A, and calculates the speaker audio driving signal corresponding to the sound line data subset A; the sound effect processor B performs real-time sound effect processing on the sound line data subset B, and calculates the speaker audio driving signal corresponding to the sound line data subset B; The signal mixer mixes the speaker audio driving signals output by the sound effect processor A and the sound effect processor B to obtain a complete speaker audio driving signal.

7. The sound effect processing method according to claim 6, characterized in that: The method for determining the calculation order of the low-order sound ray calculator is to increase the calculation order of the low-order sound ray calculator from 0 until a maximum calculation order N is determined. low , which satisfies the completion of 0~N low The time t for calculating the order sound rays low The expected sound ray calculation delay is not exceeded; the method for determining the calculation order of the high-order sound ray calculator is to increase the calculation order of the high-order sound ray calculator from 0 based on the update time interval Ts of the sound source position tracking information until a maximum calculation order N is determined. high , which satisfies the completion of 0~N high The time t for calculating the order sound rays high Not exceeding a certain percentage of the reference time Ts.

8. The sound effect processing method according to claim 6, characterized in that: The sound ray decomposition outputter performs rectangular window truncation according to the sound ray length, and processes the sound ray data output by the low-order sound ray calculator and the high-order sound ray calculator respectively to obtain the sound ray data subset A and the sound ray data subset B. The sound ray length in the sound ray data subset A is less than or equal to t*c, and the sound ray length in the sound ray data subset B is greater than t*c, where t is the expected truncation time and c is the sound speed.

9. The sound effect processing method according to claim 6, characterized in that: The audio frame length of sound effect processor A is set at 8 to 128 sampling points, and the audio frame length of sound effect processor B is set at 256 to 4096 sampling points.

10. The sound effect processing method according to claim 7, characterized in that: The signal mixer delays the audio driving signal output by the sound effect processor B by dt and then adds it to the audio driving signal output by the sound effect processor A. The method for determining dt is dt=t+t low +t a -t high -t b , where t is the expected cutoff time, t a is the audio processing delay of sound effect processor A, t b is the audio processing delay of sound effect processor B.

Images