A method, medium, and system for training environmental sound spatialization simulation

Abstract

This invention provides a method, medium, and system for spatial simulation of environmental sound for training, belonging to the field of environmental sound spatial simulation technology. The invention establishes a database of environmental sound sources in a training field and performs spectral similarity clustering analysis on the sound sources. It initializes GPU devices and configures a CUDA stream processing architecture with a two-dimensional block mesh structure. Similar sound source groups are assigned to different CUDA streams for parallel processing. The first-dimensional block mesh performs sound wave propagation path calculation based on a ray tracing algorithm, while the second-dimensional block mesh performs precise spatial rendering including Doppler effect and three-dimensional positioning. Adaptive adjustment of calculation accuracy is achieved through distance threshold judgment. A circular queue scheduling algorithm is used to process sound source groups exceeding processing capacity. Finally, real-time stereo output is generated and the sound source status is updated, solving the technical problem of low efficiency in real-time spatial simulation of multiple sound sources in a training field.

Description

Technical Field

[0001] This invention belongs to the field of environmental sound spatialization simulation technology, and specifically relates to a method, medium and system for training environmental sound spatialization simulation. Background Technology

[0002] In the field of training simulation, environmental sound spatialization simulation technology is one of the key technologies for constructing realistic training environments. Traditional sound spatialization simulation methods mainly adopt a CPU-based serial computing architecture, using frequency domain transformation and spatial audio processing algorithms to achieve three-dimensional localization and stereo rendering of sound sources. This approach is widely used in flight simulators, driver training systems, and command training platforms. While traditional techniques perform well when processing single or small numbers of sound sources, they face limitations in computational resources and high algorithm complexity. In current training environments, due to the need to simultaneously process numerous sound sources generated by various equipment such as smoke command vehicles, smoke launch vehicles, and rocket launch vehicles, traditional serial computing methods cannot meet real-time requirements. The computational latency increases dramatically with the number of sound sources, leading to sluggish simulation system response. In other words, existing technologies suffer from low efficiency in real-time spatialization computation of multiple sound sources in training environments. Summary of the Invention

[0003] In view of this, the present invention provides a method, medium and system for spatial simulation of environmental sound for training, which can solve the technical problem of low efficiency in real-time spatialization calculation of multiple sound sources in training environment in the prior art.

[0004] The present invention is implemented as follows: The first aspect of the present invention provides a method for spatial simulation of environmental sound for training, comprising: establishing a training field environmental sound source database; collecting sound features of training equipment and performing frequency domain analysis to obtain spectral data; recording sound source spatial coordinate information and sound source motion parameters; performing similarity clustering analysis on the sound sources in the training field environmental sound source database; grouping the sound features of the training equipment using a clustering distance calculation method based on spectral data; calculating the spectral similarity distance between each sound source and other sound sources; when the spectral similarity distance is less than the clustering threshold, the sound sources are grouped into the same similar sound source group; initializing the GPU device and detecting the GPU computing power parameters; determining the number of CUDA streams based on the GPU computing power parameters; configuring two... The system employs a 3D block grid structure, allocating GPU global memory space and shared memory space for each CUDA stream. When the number of similar sound source groups exceeds the number of CUDA streams, a circular queue scheduling algorithm is used for multi-batch processing. Sound source data from similar sound source groups are loaded into the GPU global memory space. Each CUDA kernel function within the first-dimensional block grid independently processes a sound source within a similar sound source group, using a ray tracing algorithm to calculate the sound wave propagation path and obtaining acoustic parameters along the propagation path based on a terrain acoustic response library. The second-dimensional block grid uses a high-precision acoustic model to perform spatial audio rendering of the sound sources, including Doppler effect calculation and 3D localization processing. The CPU then mixes the spatial audio rendering results to generate stereo output and transmits it to the audio device in real time.

[0005] The sound characteristics of the training equipment specifically include the sound characteristics of various training equipment such as air smoke command vehicles, air smoke launch vehicles, and rocket launch vehicles. The spatial coordinate information of the sound source includes three-dimensional position coordinates and direction vectors, and the motion parameters of the sound source include velocity vectors and acceleration vectors.

[0006] Specifically, the spectral similarity distance calculation method is used to quantify the degree of similarity between different sound sources. The input includes the spectral data of two sound sources, and the output is a numerical distance representing the degree of similarity.

[0007] The clustering threshold is set according to the degree of spectral difference between the sound source types. When the sound source is a smoke command vehicle, the clustering threshold is set to 0.85; when the sound source is a smoke launch vehicle, the clustering threshold is set to 0.80; and when the sound source is a rocket launch vehicle, the clustering threshold is set to 0.75.

[0008] The two-dimensional block mesh structure includes a first-dimensional block mesh and a second-dimensional block mesh. The first-dimensional block mesh is used to perform the sound wave propagation path calculation task, and the second-dimensional block mesh is used to perform the acoustic spatialization rendering calculation task.

[0009] The circular queue scheduling algorithm is used to manage similar sound source groups that exceed the CUDA streaming processing capacity. The input includes the number of similar sound source groups and the number of CUDA streams, and the output is a batch processing scheduling sequence.

[0010] The ray tracing algorithm is used to calculate the propagation path and energy attenuation of sound waves in three-dimensional space. The inputs include the spatial coordinates of the sound source, the coordinates of the observation point, and the terrain geometry data. The outputs are the sound wave propagation time and the energy attenuation coefficient.

[0011] The terrain acoustic response library is a pre-established database containing acoustic characteristics of different terrain types, including reflection and absorption coefficients for plains, mountains, urban buildings, and forests.

[0012] Before the second-dimensional block grid performs spatial audio rendering, it also includes determining whether the distance between the sound source and the observation point is greater than a distance threshold. For sound sources whose distance is greater than the distance threshold, the first-dimensional block grid directly stores the simplified acoustic calculation results in the shared memory space and returns them to the CPU side.

[0013] The high-precision acoustic model is used to accurately locate and calculate the intensity of the sound source. The input includes the spatial coordinates of the sound source, the motion parameters of the sound source, and the coordinates of the observation point. The output is an audio signal containing spatial positioning information.

[0014] The distance threshold is a critical distance value used to determine whether a sound source requires precise calculation. Sound sources exceeding the distance threshold are processed using a simplified acoustic model, while those below the threshold are processed using a high-precision acoustic model. The output threshold is a volume level threshold used to determine the importance of a sound source, set to -20 dB. Sound sources exceeding the output threshold are marked as important sound sources and given priority processing. The default distance threshold is set to 500 meters.

[0015] The Doppler effect calculation is used to process the frequency changes of a moving sound source. The inputs include the velocity vector in the sound source motion parameters, the coordinates of the observation point, and the original frequency value. The output is the frequency value after correction by the Doppler effect.

[0016] The three-dimensional positioning process is used to convert mono sound source signals into stereo signals with spatial directionality. The inputs include sound source spatial coordinate information, observation point position coordinates, and sound source power value. The output is audio signal data of the left and right channels.

[0017] Specifically, the step of the CPU mixing the spatial audio rendering results involves marking the sound source as an important sound source and processing it preferentially if the volume level of the spatial audio rendering result is higher than the output threshold; otherwise, it is processed according to the standard procedure. After the CPU transmits the stereo output to the audio device in real time, it also includes updating the current status information of the sound source to the sound source database of the training environment to ensure that the accurate sound source status is obtained when calculating the next frame.

[0018] A second aspect of the present invention provides a computer-readable storage medium storing program instructions that, when executed in a computer, perform the aforementioned method for spatial simulation of environmental sound for training.

[0019] A third aspect of the present invention provides a system for spatial simulation of environmental sound for training, comprising the aforementioned computer-readable storage medium, wherein the system is any one of a computer, a server, or a microcontroller, the computer-readable storage medium is disposed within the system, and the system is provided with a microprocessor for executing program instructions stored in the computer-readable storage medium.

[0020] This invention employs a GPU-based parallel computing-based sound source similarity clustering method. By constructing a CUDA stream processing architecture with a two-dimensional block grid structure, it achieves efficient parallel computing for multiple sound sources. This method performs frequency domain analysis and similarity clustering on the sound features of training equipment, rationally distributing computational tasks across different CUDA streams. The first-dimensional block grid is dedicated to calculating sound wave propagation paths, while the second-dimensional block grid is dedicated to spatial rendering, forming a pipelined parallel processing mode that significantly improves computational efficiency. Simultaneously, a distance threshold judgment mechanism is used, employing simplified acoustic models for distant sound sources and high-precision acoustic models for nearby sound sources, reducing unnecessary computational overhead while ensuring the processing accuracy of key sound sources. In summary, this invention solves the technical problem of low real-time spatial computing efficiency for multiple sound sources in training environments mentioned in the background art. Attached Figure Description

[0021] Figure 1 This is a flowchart of the method of the present invention.

[0022] Figure 2 This is a schematic diagram of the network structure of the sound source feature extraction module in the high-precision acoustic model involved in this invention.

[0023] Figure 3 This is a diagram showing the correspondence between the spatial distribution of sound sources and terrain types in the embodiment.

[0024] Figure 4 The graph shows the variation of binaural hearing characteristic parameters with azimuth angle in the embodiment. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings.

[0026] like Figure 1 The diagram shown is a flowchart of a method for spatial simulation of environmental sound for training, provided by the first aspect of the present invention. This method includes the following steps:

[0027] S01. Establish a training field environment sound source database, collect sound characteristics of various training equipment including air smoke command vehicle, air smoke launch vehicle, and rocket launch vehicle, and perform frequency domain analysis on the sound characteristics of the training equipment to obtain spectrum data. At the same time, record the sound source spatial coordinate information and sound source motion parameters corresponding to the sound characteristics of the training equipment. The sound source spatial coordinate information includes three-dimensional position coordinates and direction vectors, and the sound source motion parameters include velocity vectors and acceleration vectors.

[0028] S02. Perform similarity clustering analysis on the sound sources in the sound source database of the training field environment. Use the clustering distance calculation method based on the spectrum data to group the sound features of the training equipment. Calculate the spectrum similarity distance between each sound source and other sound sources. When the spectrum similarity distance is less than the clustering threshold, the sound sources are classified into the same similar sound source group. The clustering threshold is set according to the degree of spectrum difference between the sound source types.

[0029] S03. Initialize the GPU device and detect the GPU computing power parameters. Determine the number of CUDA streams that can be started simultaneously based on the GPU computing power parameters. Configure a two-dimensional block grid structure within each CUDA stream. The two-dimensional block grid structure includes a first-dimensional block grid and a second-dimensional block grid. At the same time, allocate GPU global memory space and shared memory space for each CUDA stream. If the number of similar sound source groups exceeds the number of CUDA streams, use a circular queue scheduling algorithm for multi-batch processing. The first-dimensional block grid is used to perform sound wave propagation path calculation tasks, and the second-dimensional block grid is used to perform acoustic spatialization rendering calculation tasks.

[0030] S04. Load the sound source data in the similar sound source group assigned to each CUDA stream into the GPU global memory space. Each CUDA kernel function in the first dimension block grid independently processes the sound source in a similar sound source group. The sound wave propagation path of the sound source is calculated using a ray tracing algorithm, and the acoustic parameters on the propagation path are obtained based on the terrain acoustic response library. The terrain acoustic response library contains the reflection coefficient and absorption coefficient of different terrain types.

[0031] S05. For the sound wave propagation result calculated by the first-dimensional block grid, determine whether the distance between the sound source and the observation point is greater than a distance threshold. For a sound source with a distance greater than the distance threshold, the first-dimensional block grid directly stores the simplified acoustic calculation result in the shared memory space and returns it to the CPU side. For a sound source with a distance less than the distance threshold, the sound wave propagation result is transmitted to the second-dimensional block grid through the shared memory space for precise spatialization processing.

[0032] S06. The second-dimensional block grid reads the sound wave propagation results transmitted by the first-dimensional block grid from the shared memory space, and uses a high-precision acoustic model to perform spatial audio rendering on the sound source, including Doppler effect calculation and three-dimensional positioning processing. Each CUDA kernel function in the second-dimensional block grid processes a sound source and its associated spatial acoustic parameters.

[0033] S07. The second-dimensional block grid stores the spatial audio rendering result in the GPU global memory space and returns it to the CPU side. The CPU performs mixing processing on the spatial audio rendering result to generate the final stereo output. If the volume level of the spatial audio rendering result is higher than the output threshold, the CPU marks the sound source as an important sound source and processes it first; otherwise, it processes it according to the standard process.

[0034] S08. The CPU transmits the processed stereo output to the audio device in real time and updates the current status information of the sound source to the sound source database of the training field environment to ensure that the accurate sound source status can be obtained when calculating the next frame.

[0035] The ray tracing algorithm is used to calculate the propagation path and energy attenuation of sound waves in three-dimensional space. The inputs include the spatial coordinates of the sound source, the coordinates of the observation point, and the terrain geometry data. The outputs are the sound wave propagation time and the energy attenuation coefficient.

[0036] The spectral similarity distance calculation method is used to quantify the degree of similarity between different sound sources. The input includes the spectral data of the two sound sources, and the output is a numerical distance representing the degree of similarity.

[0037] The high-precision acoustic model is used to accurately locate and calculate the intensity of the sound source. The input includes the spatial coordinates of the sound source, the motion parameters of the sound source, and the coordinates of the observation point. The output is an audio signal containing spatial positioning information.

[0038] The Doppler effect calculation is used to process the frequency changes of a moving sound source. The inputs include the velocity vector, the coordinates of the observation point, and the original frequency value in the motion parameters of the sound source. The output is the frequency value after correction by the Doppler effect.

[0039] The three-dimensional positioning process is used to convert a mono sound source signal into a stereo signal with spatial directionality. The input includes the spatial coordinates of the sound source, the coordinates of the observation point, and the power value of the sound source. The output is the audio signal data of the left and right channels.

[0040] The circular queue scheduling algorithm is used to manage similar sound source groups that exceed the CUDA streaming processing capacity. The input includes the number of similar sound source groups and the number of CUDA streams, and the output is a batch processing scheduling sequence.

[0041] The GPU computing power parameters are performance indicators of the GPU hardware, including the number of stream multiprocessors, the number of CUDA cores, and memory bandwidth, which are used to determine the number of CUDA streams and the configuration parameters of the two-dimensional block mesh structure.

[0042] The clustering threshold is a critical value used to determine whether a sound source belongs to the same similarity group. The clustering threshold is set to 0.85 for sound sources such as air smoke command vehicles, 0.80 for sound sources such as air smoke launch vehicles, and 0.75 for sound sources such as rocket launch vehicles.

[0043] The distance threshold is a critical distance value used to determine whether a sound source needs to be precisely calculated. It is set to 500 meters. Sound sources exceeding the distance threshold are processed using a simplified acoustic model, while sound sources less than the distance threshold are processed using the high-precision acoustic model.

[0044] The output threshold is a volume level threshold used to determine the importance of a sound source, set to -20 dB. Sound sources with a volume level higher than the output threshold are marked as important sound sources and given priority processing privileges.

[0045] The terrain acoustic response library is a pre-established database containing acoustic characteristics of different terrain types, including a reflection coefficient of 0.1 and an absorption coefficient of 0.05 for plains, a reflection coefficient of 0.3 and an absorption coefficient of 0.15 for mountains, a reflection coefficient of 0.6 and an absorption coefficient of 0.25 for urban buildings, and a reflection coefficient of 0.2 and an absorption coefficient of 0.4 for forests.

[0046] The similar sound source group is a set of sound sources formed by classifying sound sources with similar spectral characteristics through the spectral similarity distance calculation method. It is used to achieve load balancing and computational efficiency optimization in CUDA parallel processing.

[0047] The two-dimensional block grid structure is a grid architecture used in the CUDA programming model to organize parallel computing tasks. It contains multiple computing blocks, and each computing block contains multiple CUDA kernel functions that are executed in parallel.

[0048] The CUDA kernel function is a basic computing unit that is executed in parallel on the GPU. Each CUDA kernel function independently processes the data tasks assigned to it, realizing large-scale parallel computing.

[0049] The GPU global memory space is a memory area on the GPU device used to store large amounts of data. It can be accessed by all CUDA kernel functions and is used to store sound source data and calculation results.

[0050] The shared memory space is a high-speed memory region on the GPU device used for data exchange between CUDA kernel functions within the same compute block, and its access speed is higher than that of the GPU global memory space.

[0051] The specific implementation methods of the above steps are described in detail below.

[0052] Step S01 involves establishing a complete process for creating a sound source database for the training environment. First, a multi-channel audio acquisition device is deployed at the training site. This device includes an omnidirectional microphone array and directional microphones. The omnidirectional microphone array captures omnidirectional sound information from all directions, while the directional microphones acquire the directional characteristics of the sound source. The acquisition frequency is set to 48 kHz, and the quantization precision is 24 bits to ensure high fidelity of the sound signal. Next, sound characteristics are acquired from training equipment such as smoke command vehicles, smoke launch vehicles, and rocket launch vehicles. Each type of equipment is recorded under different operating conditions, including startup, idling, acceleration, and braking. The recording time for each condition is no less than 30 seconds to ensure a complete sound cycle is captured. Subsequently, a Fast Fourier Transform (FFT) algorithm is used to perform frequency domain analysis on the acquired audio signals. The FFT converts time-domain signals into frequency-domain signals, extracting the energy distribution characteristics of each sound source in different frequency bands to form spectral data. This spectral data contains frequency response information within the range of 0 to 24000 Hz. Simultaneously, the three-dimensional position coordinates of each sound source are recorded. A combination of GPS and inertial measurement unit (IMU) positioning method is used to obtain the accurate location of the sound sources, achieving centimeter-level accuracy. The direction vector is obtained through digital compass measurement with an accuracy of 0.1 degrees. High-precision motion sensors are used to acquire the sound source motion parameters, recording the equipment's velocity and acceleration vectors in real time at a sampling frequency of 100 Hz to ensure time synchronization of the motion parameters.

[0053] Step S02 is a detailed implementation of the similarity clustering analysis of sound sources in the training environment sound source database. First, the spectral feature vector of each sound source is extracted. This vector consists of major frequency components, energy distribution ratio, and harmonic structure, forming data points in a multi-dimensional feature space. Next, the Euclidean distance algorithm is used to calculate the spectral similarity distance between sound sources. Euclidean distance quantifies the similarity between two feature vectors in multi-dimensional space; a smaller distance value indicates greater similarity in sound source features. Then, a cohesive hierarchical clustering algorithm is used to group the sound sources. This algorithm starts with each sound source as a separate cluster and gradually merges clusters with high similarity until a preset clustering threshold is reached. For sound sources like air-smoke command vehicles, since their engine characteristics are relatively stable, the clustering threshold is set to 0.85 to ensure that similar equipment is grouped together. For sound sources like air-smoke launch vehicles, considering the noise variations of the launch device, the clustering threshold is set to 0.80, allowing for a certain degree of acoustic difference. For sound sources such as rocket launch vehicles, due to the high complexity of their acoustic characteristics, a clustering threshold of 0.75 is set to ensure the capture of sound variations under different operating modes. The clustering results form several similar sound source groups, with sound sources within each group having similar spectral characteristics, which facilitates load balancing during subsequent parallel processing.

[0054] Step S03 is a complete process of initializing the graphics processing unit (GPU) device and configuring the parallel computing environment. First, the number and model of available GPU devices in the system are detected, and the computing power parameters of each device are obtained, including key indicators such as the number of stream multiprocessors, the number of unified computing device architecture (UCA) cores, memory bandwidth, and video memory capacity. Based on the GPU's computing power, the number of UCA streams that can be launched simultaneously is determined. Generally, the number of streams is set to twice the number of stream multiprocessors to fully utilize hardware resources. Next, a two-dimensional block grid structure is configured for each UCA stream. The size of the first-dimensional block grid is determined based on the number of similar sound source groups, with each block processing one sound source group. The number of threads within a block is set to a multiple of 32 to optimize memory access efficiency. The second-dimensional block grid is used for acoustic spatialization rendering calculations. The size of the blocks is dynamically adjusted according to the number of sound sources to be processed precisely, ensuring a uniform distribution of the computational load. Then, global GPU memory space is allocated for each UCA stream. The allocation size is determined based on the scale of the sound source data, generally reserving 1024 bytes of storage space for each sound source. Simultaneously, shared memory space is allocated for data exchange between threads within the block. The shared memory size is set to 48 kilobytes to meet the needs of acoustic parameter transfer. When the number of similar sound source groups exceeds the number of streams in the unified computing device architecture, a circular queue scheduling algorithm is used for multi-batch processing. The circular queue can ensure that all sound source groups receive equal processing opportunities and avoid resource contention.

[0055] The specific implementation of step S04 involves loading the sound source data into the graphics processor and performing a detailed calculation of the sound wave propagation path. First, the sound source data from the similar sound source groups allocated to each unified computing device architecture stream is transferred from the host memory to the global memory space of the graphics processor. Data transfer is performed asynchronously to avoid blocking other operations on the host side. The sound source data includes information such as three-dimensional coordinates, spectral characteristics, and motion parameters. The data format uses a compact structure to reduce memory usage and transmission time. Next, each unified computing device architecture kernel function within the first-dimensional block grid independently processes the sound sources within a similar sound source group. The kernel function uses a ray tracing algorithm to calculate the sound wave propagation path. The ray tracing algorithm starts from the sound source location and emits a virtual ray along the sound wave propagation direction. The intersection of the ray and the terrain geometry represents the sound wave reflection or scattering location. By recursively calculating multiple reflection paths, the complete sound wave propagation path is obtained. The algorithm input parameters include the sound source spatial coordinates, the observation point coordinates, and the terrain geometry data. The output results are the sound wave propagation time and energy attenuation coefficient. During the calculation, acoustic parameters along the propagation path are obtained based on a pre-established terrain acoustic response library. This library includes reflection and absorption coefficients for different terrain types: 0.1 for plains and 0.05 for mountains; 0.3 for urban areas and 0.15 for forests; and 0.25 for urban buildings. The ray tracing algorithm accurately simulates the propagation characteristics of sound waves in complex terrain, providing accurate acoustic parameters for subsequent spatialization processing.

[0056] The specific implementation of step S05 involves determining the processing strategy and allocating computing resources based on the distance between the sound source and the observation point. First, the Euclidean distance between each sound source and the observation point is calculated. This distance calculation is based on the coordinate difference in three-dimensional space, using square root operations to obtain an accurate spatial distance value. Next, the calculated distance is compared with a preset distance threshold of 500 meters. This threshold is determined based on the perceptual characteristics of the human ear for distant sound sources; sound sources beyond 500 meters have a relatively small impact on auditory perception. For sound sources with a distance greater than the distance threshold, the first-dimensional block grid directly uses a simplified acoustic calculation model for processing. This simplified model mainly considers distance attenuation and basic frequency response, resulting in lower computational complexity and faster processing speed. The simplified calculation results are directly stored in a designated area of ​​the shared memory space and returned to the central processing unit via a memory mapping mechanism, avoiding complex data transmission processes. For sound sources with a distance less than the distance threshold, the system transmits the sound wave propagation results calculated by the first-dimensional block grid to the second-dimensional block grid for precise spatialization processing. Data transmission is achieved through the shared memory space, and the transmitted content includes detailed information such as the sound wave propagation path, energy attenuation coefficient, and spectral characteristics. Shared memory read and write operations employ a synchronous mechanism to ensure data integrity and consistency, avoiding data errors caused by race conditions. This hierarchical processing strategy effectively balances computational precision and processing efficiency, prioritizing the allocation of limited computing resources to near-field sound sources that have a greater impact on auditory perception.

[0057] Step S06 is a detailed implementation of the high-precision spatial audio rendering process performed by the second-dimensional block mesh. First, the sound wave propagation results transmitted from the first-dimensional block mesh are read from the shared memory space. This reading operation is performed in a pipelined manner to ensure the continuity of data processing. Next, a high-precision acoustic model is used to render the spatial audio of the sound source. This high-precision acoustic model is built based on the head-related transfer function and the room impulse response function, and can accurately simulate the propagation and localization effects of sound in three-dimensional space. The model's input parameters include the spatial coordinates of the sound source, the motion parameters of the sound source, and the coordinates of the observation point. The output is an audio signal containing spatial localization information. Doppler effect calculation is an important component of the high-precision model, used to handle the frequency changes of moving sound sources. The calculation is based on the Doppler frequency shift principle. The input parameters include the sound source velocity vector, the coordinates of the observation point, and the original frequency value. The output is the frequency value corrected for the Doppler effect. The Doppler effect calculation considers the relative motion between the sound source and the observation point. The frequency increases when the sound source moves towards the observation point and decreases when it moves away. The frequency change is proportional to the relative velocity. 3D localization processing converts mono sound source signals into spatially directional stereo signals. The process is based on a binaural hearing model. Input parameters include the spatial coordinates of the sound source, the coordinates of the observation point, and the sound source power value. The output is the audio signal data for the left and right channels. The signal differences between the left and right channels include time delay difference and intensity difference. The time delay difference reflects the horizontal azimuth angle of the sound source, while the intensity difference reflects the distance and orientation information of the sound source. Within the second-dimensional block grid, each unified computing device architecture kernel function processes a sound source and its associated spatial acoustic parameters in parallel. Parallel processing significantly improves computational efficiency, meeting the requirements of real-time audio rendering.

[0058] Step S07 is a complete process for audio signal mixing and output management. First, the second-dimensional block grid stores the spatial audio rendering results in a designated area of ​​the graphics processor's global memory space. Storage employs a double-buffering mechanism to ensure that data writing does not affect reading operations. Next, the rendering results are transferred from the graphics processor's global memory to the host memory using direct memory access technology. This transfer is asynchronous, improving overall system performance. After receiving the spatial audio rendering results, the central processing unit (CPU) performs mixing processing, including multi-source signal superposition, dynamic range compression, and equalization adjustment, generating the final stereo output signal. The mixing algorithm uses a weighted average method, assigning different weight coefficients based on the importance of each sound source to ensure that important sound sources dominate the mixed result. During the mixing process, the CPU monitors the volume level of each spatial audio rendering result in real time, quantizing the volume level in decibels. When the volume level of the audio signal exceeds the output threshold, the CPU marks the corresponding sound source as an important sound source, setting the output threshold to -20 decibels, which is determined based on the characteristics of human hearing thresholds. Important sound sources receive priority in subsequent processing, including higher sampling precision, larger buffer allocation, and more frequent parameter updates. Sound sources with volume levels below the output threshold are processed according to standard procedures, using conventional processing parameters and resource allocation strategies. After mixing, the stereo output signal undergoes final format conversion and gain adjustment to ensure it meets the input requirements of the audio equipment.

[0059] Step S08 is a complete process of audio output transmission and sound source status update. First, the central processing unit (CPU) transmits the processed stereo output to the audio device in real time via the audio interface. The transmission uses a low-latency audio streaming protocol to ensure the real-time performance and continuity of the audio signal. The audio output sampling rate is set to 48 kHz, the bit depth to 16 bits, and the stereo format to meet the requirements of high-quality audio playback. A circular buffer mechanism is used during transmission, with a buffer size of 1024 sampling points, equivalent to approximately 21 milliseconds of audio data, ensuring real-time performance while avoiding audio interruptions. Next, the CPU updates the current status information of each sound source to the sound source database in the training environment. The status information includes the current position coordinates, motion parameters, audio characteristics, and processing markers of the sound source. Status updates use an incremental update method, updating only the parameters that have changed, reducing the overhead of database operations. The database update operation uses a transaction processing mechanism to ensure data consistency and integrity, avoiding data conflicts caused by concurrent access. The timestamp of the status information is recorded to the millisecond level, providing an accurate time reference for the calculation of the next frame. The system also maintains a state change history, recording the state change trajectory of each sound source within the last 10 seconds, used to predict the motion trend of the sound sources and optimize subsequent calculation parameters. After the update is complete, the system cleans up the temporary data of the current frame, releases memory resources, and prepares for the calculation of the next frame. The execution time of the entire update process is controlled within 5 milliseconds to ensure that it does not affect the real-time processing requirements of the next frame.

[0060] The high-precision acoustic model is the core technical component of this invention. Its detailed structure includes three main parts: a sound source feature extraction module, a spatial propagation simulation module, and an auditory perception modeling module. The sound source feature extraction module employs a deep convolutional neural network architecture, containing 8 convolutional layers and 3 fully connected layers. The convolutional layers extract the time-frequency features of the audio signal, while the fully connected layers are used for feature dimensionality reduction and classification. The network input is a 128-dimensional Mel-frequency cepstral coefficient feature vector, and the output is a 64-dimensional sound source feature code. The spatial propagation simulation module is built based on physical acoustic principles and includes a ray tracing engine, a reflection and scattering calculation unit, and an attenuation compensation unit, capable of accurately simulating the propagation process of sound waves in complex environments. The auditory perception modeling module uses a binaural hearing model, including a head-related transfer function database and a psychoacoustic masking model, outputting a spatial audio signal that conforms to the characteristics of human ear perception.

[0061] The process of establishing the training dataset for the high-precision acoustic model involves the following detailed steps. First, a standard sound source recording laboratory is established, featuring a fully anechoic design with background noise levels below 15 dB to ensure a pure recording environment. Next, high-quality audio samples are collected from different types of equipment under various operating conditions. Each type of equipment records at least 100 hours of audio data, covering the complete operating cycle from startup to shutdown. Then, data preprocessing is performed, including noise reduction, normalization, and segmentation, dividing long audio clips into 10-second segments to form training samples. Following this, data annotation is performed, including sound source type labels, operating state labels, and spectral feature labels. This annotation work is completed by professionals with acoustic backgrounds to ensure label accuracy. Next, data augmentation is performed by adding different types of environmental noise, changing the pitch and speed of the audio, and simulating different propagation distances to expand the training data to 10 times the size of the original data. Finally, the training set, validation set, and test set are divided in an 8:1:1 ratio to ensure the scientific rigor of model training and evaluation. Finally, a metadata index for the training data is established to record information such as the source, features, and labels of each sample, facilitating data retrieval and management during the model training process.

[0062] The simplified acoustic model employs a rule-based processing approach, primarily comprising a distance attenuation calculation module and a basic frequency response correction module. Its computational complexity is low, making it suitable for rapid processing of long-distance sound sources. The simplified acoustic model's structure consists of two main components: the distance attenuation calculation module and the basic frequency response correction module. The distance attenuation calculation module utilizes the spherical wave attenuation principle, modeling based on the physical law that sound intensity is inversely proportional to the square of the distance, resulting in constant-level computational complexity. The basic frequency response correction module employs a first-order low-pass filter structure to compensate for high-frequency attenuation caused by long-distance propagation; the filter parameters are adaptively adjusted according to the propagation distance. The model's input parameters include sound source intensity, propagation distance, and ambient temperature; the output is the audio signal after distance attenuation and frequency response correction.

[0063] The training steps for the simplified acoustic model include the following: First, sound source attenuation data are collected at different distances. Measurement points are set every 100 meters within a range of 500 to 2000 meters, and the sound intensity changes and spectral characteristics of the sound source are recorded at each point. Next, a theoretical calculation model is established, deriving the distance attenuation formula and frequency response correction parameters based on the principles of acoustic propagation. Then, model parameters are fitted using the least squares method to fit the collected experimental data and determine the coefficient parameters in the model. Following this, model validation is performed, testing the model's prediction accuracy under different environmental conditions to ensure that the error is controlled within an acceptable range. Finally, model parameters are optimized by adjusting the model structure and parameter settings based on the validation results, resulting in the final simplified acoustic model.

[0064] The key technical ideas of this invention are mainly reflected in the following aspects. The first technical idea is a sound source clustering and grouping strategy based on spectral similarity. This strategy achieves intelligent grouping by analyzing the spectral characteristics of sound sources. Compared with traditional random or fixed grouping methods, it can group sound sources with similar spectral characteristics into the same processing group. This ensures that each computing unit of the graphics processor processes sound sources with similar computational complexity during parallel computing, thereby achieving load balancing and significantly improving computational efficiency. It avoids the problem of some computing units being idle while others are overloaded, thus optimizing overall computational performance. The second technical idea is a two-level parallel architecture design for the graphics processor. By setting up a first-dimensional block grid and a second-dimensional block grid to handle sound wave propagation calculations and spatial rendering calculations respectively, compared with traditional single-level parallel processing methods, it can be specifically optimized according to the characteristics of different computing tasks. The first dimension focuses on large-scale parallel path tracing calculations, while the second dimension focuses on high-precision spatial audio processing. The two levels exchange data at high speed through shared memory, avoiding frequent global memory accesses and significantly reducing the impact of memory bandwidth bottlenecks on system performance. The third technical approach is a distance-adaptive processing strategy. By setting a distance threshold, it distinguishes between near and far sound sources, processing them using high-precision and simplified models respectively. Compared to traditional uniform processing methods, this approach prioritizes computational resources for near sound sources that have a greater impact on auditory perception. It significantly reduces overall computational complexity while ensuring the processing quality of key sound sources, achieving an optimal balance between processing accuracy and computational efficiency. The fourth technical approach is a real-time sound source state tracking and prediction mechanism. By maintaining historical records of sound source states and predicting motion trends, compared to traditional static processing methods, it can prepare the parameters and resources needed for the next frame's calculation in advance, reducing computational latency. Simultaneously, state prediction optimizes resource allocation strategies, improving the system's real-time response capability.

[0065] The synergistic effect of these technical approaches forms a complete and efficient sound spatialization simulation system, which has significant technical advantages compared to existing technologies. Clustering and grouping strategies provide an optimized task partitioning basis for parallel computing; a two-level parallel architecture achieves efficient utilization of computing resources; distance adaptive principles ensure the rational allocation of computing resources; and a real-time state tracking mechanism guarantees the system's response performance. These four technical approaches work together to form a complete technical system from data organization, computing architecture, resource allocation to performance optimization. This enables the entire system to simultaneously meet the requirements of real-time performance, accuracy, and efficiency when dealing with complex multi-sound source environments, offering comprehensive advantages over traditional methods, including strong processing power, low latency, and high accuracy.

[0066] It should be noted that the present invention also solves the following technical problems:

[0067] Technical problem of insufficient accuracy in sound wave propagation modeling in complex terrain environments. In practical training environments, terrain types are diverse, including plains, mountains, urban buildings, and forests. Traditional acoustic modeling methods often use a uniform propagation model, which cannot accurately reflect the impact of different terrains on sound wave propagation, leading to spatial distortion. This invention constructs a terrain acoustic response library, pre-calculating and storing reflection and absorption coefficients for different terrain types. For plains, the reflection coefficient is set to 0.1 and the absorption coefficient to 0.05; for mountains, 0.3 and 0.15; for urban buildings, 0.6 and 0.25; and for forests, 0.2 and 0.4. The ray tracing algorithm can dynamically obtain accurate acoustic parameters based on the actual terrain type when calculating the sound wave propagation path, significantly improving the accuracy of sound wave propagation modeling in complex terrain environments. Technical problem of high computational complexity in calculating the Doppler effect of moving sound sources. Traditional Doppler effect calculation methods typically use simplified linear models, which are difficult to accurately handle equipment sound sources with drastic acceleration changes, especially the complex Doppler effect generated by high-speed moving vehicles during acceleration and turning. This invention records the complete motion parameters of the sound source, including velocity and acceleration vectors, and achieves accurate Doppler effect calculation in a high-precision acoustic model of a second-dimensional block grid. Each CUDA kernel function independently processes the motion parameters of a sound source, enabling real-time tracking of changes in the motion state of the sound source and accurate calculation of frequency offset, ensuring that the spatialization effect of the moving sound source is realistic and reliable.

[0068] A second aspect of the present invention provides a computer-readable storage medium storing program instructions that, when executed in a computer, perform the aforementioned method for spatial simulation of environmental sound for training.

[0069] A third aspect of the present invention provides a system for spatial simulation of environmental sound for training, comprising the aforementioned computer-readable storage medium, wherein the system is any one of a computer, a server, or a microcontroller, the computer-readable storage medium is disposed within the system, and the system is provided with a microprocessor for executing program instructions stored in the computer-readable storage medium.

[0070] Specifically, the principle of this invention is as follows: The key to solving the technical problem of low efficiency in real-time spatialization computation of multiple sound sources in training environments lies in its innovative parallel computing architecture design and intelligent computing resource allocation strategy. First, by performing frequency domain analysis on the sound characteristics of the training equipment and establishing a spectral similarity distance calculation method, sound sources with similar spectral characteristics are grouped into similar sound source groups. This clustering method allows GPU parallel computing to fully leverage the advantages of batch processing; sound sources of the same type can be processed using the same computational kernel function, avoiding frequent context switching overhead. Second, the design of the two-dimensional block grid structure enables specialized division of computational tasks. The first-dimensional block grid focuses on sound wave propagation path calculation, using a ray tracing algorithm combined with a terrain acoustic response library to quickly obtain acoustic parameters; the second-dimensional block grid focuses on spatial audio rendering, including Doppler effect calculation and three-dimensional positioning processing. The two-dimensional block grids share a memory space for high-speed data transfer, forming an efficient computational pipeline. Furthermore, the introduction of a distance threshold mechanism enables adaptive adjustment of computational accuracy. Sound sources beyond 500 meters are processed using a simplified acoustic model, while only sources closer to this distance are included in the precise computation process. This hierarchical processing strategy significantly reduces computational load while maintaining auditory quality. The application of a circular queue scheduling algorithm ensures orderly multi-batch processing when the number of sound source groups exceeds CUDA's streaming processing capacity, avoiding resource contention and computational blockage.

[0071] The following provides a specific embodiment 1 of the present invention, and the specific implementation of each step in this embodiment 1 is described in detail below.

[0072] The specific implementation of step S01 involves using Fast Fourier Transform (FFT) to perform frequency domain analysis on audio signals when establishing the environmental sound source database for the training field. The FFT calculation formula is as follows:

[0073]

[0074] In the formula, X(k) is the k-th frequency component of the frequency domain signal; x(n) is the n-th sampling point of the time domain audio signal; N is the total number of sampling points; k is the frequency index, ranging from 0 to N-1; j is the imaginary unit; and π is pi. The spectral data is obtained by calculating the amplitude and phase of each frequency component:

[0075]

[0076] In the formula, |X(k)| is the amplitude of the k-th frequency component; Re[X(k)] is the real part of X(k); Im[X(k)] is the imaginary part of X(k). The spatial coordinates of the sound source are obtained through the Global Positioning System, and the three-dimensional position coordinates are represented by the vector P = [P x Py P z ] T , where P x P y P z These are the east-west, north-south, and vertical coordinate components, respectively, in meters.

[0077] The specific implementation of step S02 is to use the Euclidean distance algorithm to calculate the spectral similarity distance between sound sources. For the spectral feature vectors F of two sound sources p and q... p and F q The formula for calculating the similarity distance is:

[0078]

[0079] In the formula, d pq F represents the spectral similarity distance between sound sources p and q, and is dimensionless; p and F q F represents the M-dimensional spectral eigenvectors of sound sources p and q, respectively; p，m and F q，m These represent the m-th spectral feature components; M is the spectral feature dimension, with a value of 128; m is the index of the spectral feature component. The clustering judgment condition is:

[0080]

[0081] In the formula, C pq For clustering results, 1 indicates belonging to the same group, and 0 indicates belonging to different groups; θ cluster The clustering thresholds are 0.85 for smoke command vehicles, 0.80 for smoke launch vehicles, and 0.75 for rocket launch vehicles.

[0082] The specific implementation method of step S03 is the same as described above, and will not be repeated in detail here.

[0083] The specific implementation of step S04 involves using a ray tracing algorithm to calculate the sound wave propagation path. The formula for calculating the sound wave propagation time is:

[0084]

[0085] In the formula, t prop The total propagation time is in seconds; S is the number of propagation path segments, dimensionless; L s c is the length of the s-th path segment in meters; s is the path segment index; s The speed of sound in the s-th segment of the path is expressed in meters per second and is affected by temperature; it is calculated using the following formula:

[0086] c s =331.3 + 0.606T s ;

[0087] In the formula, T s Let be the temperature of the s-th path segment, in degrees Celsius. The constant 331.3 is in meters per second, and the constant 0.606 is in meters per second per degree Celsius. The formula for calculating the energy attenuation coefficient is:

[0088]

[0089] In the formula, A total α is the total energy decay coefficient, dimensionless; s β is the absorption coefficient of the s-th path segment, dimensionless, with a value of 0.05 for plains, 0.15 for mountainous terrain, 0.25 for urban areas, and 0.4 for forest terrain; s Let r be the reflection loss coefficient for the s-th path segment, dimensionless, with a value of 0.1 for plains, 0.3 for mountainous terrain, 0.6 for urban areas, and 0.2 for forests; r0 is the reference distance, taken as 1 meter; r s The propagation distance of the s-th segment of the path is expressed in meters.

[0090] The specific implementation of step S05 involves calculating the Euclidean distance between the sound source and the observation point. The distance calculation formula is:

[0091]

[0092] In the formula, D is the distance between the sound source and the observation point; P s =[P sx P sy P sz ] T P is the sound source location vector; o =[P ox P oy P oz ] T This is the observation point position vector; all coordinate components are in meters. Distance threshold D th Set to 500 meters, the processing strategy selection function is:

[0093]

[0094] In the formula, Strategy represents the result of the processing strategy selection, Simple represents simplified processing, and Precise represents precise processing.

[0095] The specific implementation of step S06 involves performing Doppler effect calculations and three-dimensional localization processing. The formula for calculating the frequency after Doppler effect correction is as follows:

[0096]

[0097] In the formula, fd f is the Doppler shifted frequency in Hertz; f0 is the original frequency of the sound source in Hertz; c is the speed of sound, taken as 343 meters per second; v o The velocity at the observation point is expressed in meters per second; v s φ represents the speed of the sound source, measured in meters per second. i φ is the angle between the velocity at the observation point and the direction of the sound source, expressed in radians. s The angle between the sound source velocity and the direction of the observation point is expressed in radians. In 3D positioning processing, the calculation formulas for the left and right channel signals are as follows:

[0098]

[0099] In the formula, S L and S R These are the complex signals for the left and right channels, respectively; S0 is the mono source signal; G L and G R τ represents the gain coefficients for the left and right channels, respectively, and is dimensionless; ω is the angular frequency, in radians per second; τ L and τ R These represent the time delays for the left and right channels, respectively, in seconds; j is the imaginary unit. The gain coefficient is calculated based on the sound source azimuth angle α.

[0100] G L =cos(α+π / 4), G R =sin(α+π / 4);

[0101] In the formula, α is the azimuth angle of the sound source relative to the observation point, in radians, ranging from 0 to 2π; π is pi, approximately equal to 3.14159.

[0102] The specific implementation of step S07 involves performing multi-source signal mixing processing. The formula for calculating the mixed signal is:

[0103]

[0104] In the formula, S mix The mixed stereo signal; N s The total number of sound sources; w i S is the weighting coefficient for the i-th sound source; i Let be the spatial audio signal of the i-th sound source. Volume level detection uses the decibel calculation formula:

[0105]

[0106] In the formula, L dB Volume level, in decibels; |S i | represents the amplitude of the i-th sound source signal; S ref The reference signal amplitude is set to 2 × 10.-5 Pascal. The criteria for identifying important sound sources are:

[0107]

[0108] In the formula, Important i The importance of the i-th sound source is marked, where 1 indicates an important sound source and 0 indicates a common sound source; L th The output threshold is set to -20 dB.

[0109] The specific implementation method of step S08 is the same as described above, and will not be repeated in detail here.

[0110] The specific implementation of the ray tracing algorithm involves updating the ray direction vector using the law of reflection when calculating the sound wave propagation path. When the sound wave encounters a terrain surface, the formula for calculating the reflected ray direction vector R is:

[0111] R = I - 2(I·N)N;

[0112] In the formula, I is the incident ray direction vector, a three-dimensional unit vector; N is the terrain surface normal vector, a three-dimensional unit vector; and R is the reflected ray direction vector, a three-dimensional unit vector. The reflection scattering calculation unit adopts the Rayleigh scattering model, and the scattering coefficient is calculated using the following formula:

[0113]

[0114] In the formula, σ scatter V is the scattering cross section, in square meters; λ is the wavelength of the sound wave, in meters; n is the refractive index of the scattering particle, dimensionless, ranging from 1.1 to 1.5; particle The volume of the scattered particles is in cubic meters. The attenuation compensation unit uses a frequency-dependent compensation algorithm, and the compensation coefficient is calculated using the following formula:

[0115]

[0116] In the formula, C comoensation α is the attenuation compensation coefficient, dimensionless; freq This is the frequency attenuation coefficient, with a value of 1.4 × 10⁻⁶. -11 ·f 1.5 The unit is per meter per hertz; f is the sound wave frequency, in hertz; D path The propagation path length is expressed in meters.

[0117] The specific implementation of the spectral similarity distance calculation method is to use weighted Euclidean distance, where the weight matrix W is a diagonal matrix, and the diagonal elements w mm The weighted distance is determined based on frequency importance. The formula is:

[0118]

[0119] In the formula, The weighted spectral similarity distance is dimensionless; W is an M×M dimensional weight matrix, with diagonal elements w mm The value range is from 0.1 to 2.0.

[0120] The specific implementation of the circular queue scheduling algorithm is as follows: when the number of similar sound source groups N... g Exceeding the number of CUDA streams N stream In this case, round-robin scheduling is used. The formula for calculating the scheduling sequence is:

[0121] Queue[k] = km mod N stream ;

[0122] In the formula, Queue[k] is the CUDA stream number assigned to the k-th similar sound source group; k is the sound source group index, with a value ranging from 0 to N. g -1; N stream The number of available CUDA streams; mod is the modulo operator.

[0123] The simplified acoustic model is implemented by using the spherical wave attenuation principle for distance attenuation calculation. The formula for calculating the signal strength after attenuation is:

[0124]

[0125] In the formula, I simple To simplify the signal strength calculation in the model; I0 is the initial intensity of the sound source; r0 is the reference distance, taken as 1 meter; D is the distance from the sound source to the observation point, in meters; α air The air absorption coefficient is taken as 2 × 10⁻⁶. -4 Per meter. The fundamental frequency response correction uses a first-order low-pass filter:

[0126]

[0127] In the formula, H simple (ω) is the frequency response function of the simplified model; ω c The cutoff angular frequency is ω, calculated based on the propagation distance D. c =2000π-10πD, in radians per second.

[0128] The specific implementation of the high-precision acoustic model involves using the head-related transfer function (HRF) for spatial positioning calculations. The frequency domain expression of the HRF is:

[0129] H(ω,θ,φ)=|H(ω,θ,φ)|·e jΦ(ω，θ，φ) ;

[0130] In the formula, H(ω, θ, φ) is the head-related transfer function under the conditions of frequency ω, azimuth angle θ, and elevation angle φ;

[0131] |H(ω, θ, φ)| is the amplitude response, dimensionless; Φ(ω, θ, φ) is the phase response, in radians; θ is the horizontal azimuth angle, in radians, ranging from 0 to 2π; φ is the vertical elevation angle, in radians, ranging from -π / 2 to π / 2. In the binaural hearing model, the formula for calculating the binaural time difference is:

[0132]

[0133] In the formula, ITD is the binaural time difference in seconds; a is the head radius, taken as 0.0875 meters; c is the speed of sound, taken as 343 meters per second; θ is the azimuth angle of the sound source, in radians. The formula for calculating the binaural intensity difference is:

[0134]

[0135] In the formula, ILD represents the binaural intensity difference, measured in decibels; |H L (ω, θ)| represents the amplitude response of the left ear head correlation transfer function; |H R (ω, θ)| represents the amplitude response of the right ear head correlation transfer function. The psychoacoustic masking model uses Bark critical band analysis, and the masking threshold is calculated using the following formula:

[0136] TM(f test ) = SPL masker -ΔTM(f test f masker );

[0137] In the formula, TM(f test () represents the test frequency f test The masking threshold, in decibels; SPL masker The sound pressure level of the masking tone, measured in decibels; ΔTM(f test f masker ) is the masking function, and its calculation formula is: Where z = 13arctan(0.76f) masker / 1000)+3.5arctan((f masker / 7500) 2 ) represents the Bark scale; f test For testing audio frequency, the unit is Hertz; f masker The masking frequency is expressed in Hertz; z is the Bark critical band, which is dimensionless.

[0138] The effects of the above formulas are explained below. Fast Fourier Transform Formula It achieves efficient conversion from the time domain to the frequency domain, reducing the computational complexity from O(N) compared to the traditional Discrete Fourier Transform. 2 The computation time is reduced to O(NlogN), significantly improving the real-time performance of spectrum analysis. (Euclidean distance formula d) pq =||F p -F q This quantifies the spectral similarity between sound sources and achieves intelligent grouping of sound sources through geometric distance measurement in a multi-dimensional feature space, improving the load balancing effect of subsequent parallel processing. (Doppler effect calculation formula) The relative motion between the sound source and the observation point was considered, accurately simulating frequency shift and enhancing the realism of moving sound sources. Three-dimensional positioning processing formula. and The stereo effect of spatial audio is achieved through the difference in gain and delay between two channels, providing a more realistic spatial auditory experience compared to traditional mono processing. The reflection law formula R = I⁻²(I·N)N in the ray tracing algorithm ensures the physical accuracy of sound wave propagation and can simulate multiple reflections in complex terrain environments compared to a simple line-of-sight propagation model. Rayleigh scattering formula. An acoustic scattering model based on the principle of optical scattering was established, which can accurately calculate the scattering loss of sound waves in non-uniform media, thus improving the accuracy of sound propagation prediction in complex environments. Attenuation compensation formula. Employing a frequency-dependent exponential compensation algorithm, this method effectively recovers high-frequency attenuation caused by long-distance propagation, improving the sound quality fidelity of distant sound sources. (Binaural time difference formula) A precise time delay calculation method was established based on the geometric model of the human head, achieving high-precision horizontal positioning and improving direction judgment accuracy by 30% compared to traditional stereo methods. (Binocular intensity difference formula) Distance perception is achieved by comparing the intensity of signals received by the left and right ears, thus enhancing the sense of depth in the sound field. The psychoacoustic masking formula TM(f) test ) = SPL masker -ΔTM(f test f masker Based on the characteristics of human auditory perception, audio signal processing was optimized. The masking effect reduced the processing burden of redundant information, improving the overall system efficiency by 25%. The circular queue scheduling algorithm, Queue[k] = kmodN, is used. stream This approach achieves balanced allocation of similar sound source groups across limited CUDA stream resources, avoiding resource contention and processing bottlenecks, and improving the overall parallel processing efficiency of the system. It also simplifies the spherical wave attenuation formula in the acoustic model. Based on the principles of physical acoustics, it achieves rapid processing of distant sound sources, reducing the amount of computation by 70% compared to high-precision models while maintaining acceptable accuracy loss.

[0139] To better understand and implement this invention, a specific application scenario is provided below as Example 2: A project requires sound simulation for 86 training devices across 12 categories, including smoke command vehicles, smoke launch vehicles, and rocket launch vehicles. The environmental terrain includes four typical types: plains, mountains, urban buildings, and forests. The technical team used the spatialized simulation method for training environment sound of this invention to construct a complete sound simulation system.

[0140] During the phase of establishing the environmental sound source database for the training ground, the technical team deployed 24 omnidirectional microphone arrays and 16 directional microphones at a pre-designated training base, setting the sampling frequency to 48kHz and the quantization precision to 24 bits. Sound was collected from 86 pieces of training equipment under four operating states: startup, idling, acceleration, and braking. Each state was recorded for 45 seconds, totaling approximately 46 hours of audio data. Using a combination of GPS and inertial measurement unit (IMU) for positioning, the sound source location accuracy reached 3 cm, and the direction vector measurement accuracy was 0.08 degrees. The motion parameter sampling frequency was set to 100Hz, recording the real-time speed and acceleration changes of the equipment.

[0141] Frequency domain analysis of the acquired audio signals was performed using Fast Fourier Transform (FFT) to extract frequency response information within the range of 0 to 24000 Hz. The main frequency components of the sound sources from smoke command vehicles were concentrated in the 200-800 Hz and 1200-2000 Hz frequency bands, while the characteristic frequency distribution of sound sources from smoke launch vehicles was in the 150-600 Hz and 1800-3500 Hz ranges. Sound sources from rocket launch vehicles exhibited significant energy distribution in the 300-1000 Hz and 2500-5000 Hz frequency bands. The technical team established a 128-dimensional spectral feature vector, covering the key frequency characteristics of various types of equipment.

[0142] In the sound source similarity clustering analysis phase, the technical team calculated the spectral similarity distances between 86 sound sources, forming an 86×86 distance matrix. Using the Euclidean distance algorithm and agglomerative hierarchical clustering method, clustering thresholds were set according to equipment type: 0.85 for smoke command vehicles, 0.80 for smoke launch vehicles, and 0.75 for rocket launch vehicles. The clustering results divided the 86 sound sources into 12 similar sound source groups, each containing 5-9 sound sources, as shown in Table 1.

[0143] Table 1. Statistical Table of Sound Source Clustering Results

[0144] 1 Air Smoke Command Vehicle Type A 8 0.87 2 Air Smoke Command Vehicle Type B 7 0.86 3 Air Smoke Launcher Type A 9 0.82 4 Air Smoke Launcher Type B 6 0.81 5 Air Smoke Launcher Type C 8 0.83 6 Rocket Launcher Type A 7 0.77 7 Rocket Launcher Type B 5 0.76 8 Rocket Launcher Type C 6 0.78 9 Auxiliary vehicle Type A 9 0.84 10 Auxiliary vehicle type B 7 0.85 11 Communication vehicles 8 0.88 12 Logistics support vehicles 6 0.86

[0145] Cluster analysis results show that the spectral characteristics of similar equipment have high similarity, with an average similarity of over 0.75, which provides a good foundation for load balancing in subsequent parallel processing.

[0146] During the GPU parallel computing environment configuration phase, the technical team determined that the system was equipped with an NVIDIA RTX A6000 graphics card, featuring 84 stream multiprocessors, 10752 CUDA cores, 48GB of video memory, and a memory bandwidth of 768GB / s. Based on the GPU's computing power, the number of CUDA streams was determined to be 16, capable of processing 12 similar sound source groups simultaneously. The remaining 4 CUDA streams were used for data transfer and result processing. A two-dimensional block mesh structure was configured for each CUDA stream, with the first dimension block mesh size set to 32×16 and the second dimension block mesh size set to 16×32 to ensure optimal memory access. Each CUDA stream was allocated 2GB of global memory space and 48KB of shared memory space.

[0147] During the ray tracing algorithm calculation phase, the technical team established a 3D geometric model containing detailed terrain information of the training base. The terrain acoustic response library includes plain terrain with a reflection coefficient of 0.1 and an absorption coefficient of 0.05, mountainous terrain with a reflection coefficient of 0.3 and an absorption coefficient of 0.15, urban buildings with a reflection coefficient of 0.6 and an absorption coefficient of 0.25, and forest terrain with a reflection coefficient of 0.2 and an absorption coefficient of 0.4. In a typical simulation calculation, 34 of the 86 sound sources were more than 500 meters away from the observation point and were processed using a simplified acoustic model; the remaining 52 sound sources were within 500 meters and were processed using a high-precision acoustic model.

[0148] The technical team set up an observation point at the training base command center, at coordinates (7500, 4200, 15), with various equipment distributed across different areas for mobile training. For example... Figure 3 As shown, the spatial distribution of sound sources in the training area covers four terrain types: 32 units in plains, 21 units in mountains, 18 units in urban areas, and 15 units in forests. In the Doppler effect calculations, the speed of the moving equipment ranges from 5 to 45 meters per second, and the equipment acceleration varies from 0.5 to 3.2 meters per second. 2 The frequency offset reaches a maximum of 8% of the original frequency.

[0149] In the 3D positioning processing stage, the technical team used a head-related transfer function for precise spatial audio rendering. The gain coefficients of the left and right channels were dynamically adjusted according to the azimuth angle of the sound source, with a time delay difference ranging from 0 to 0.8 milliseconds. Figure 4 As shown, the calculated results of binaural time difference and intensity difference can accurately reflect the spatial location information of the sound source, providing the observer with a realistic sense of direction and distance. The application of the psychoacoustic masking model significantly reduces the processing burden of redundant information, and the masking threshold calculation results are shown in Table 2.

[0150] Table 2 Calculation results of psychoacoustic masking threshold

[0151] 500 400 12.3 3.89 1000 800 15.7 7.94 2000 1600 18.4 13.89 4000 3200 21.8 20.15 8000 6400 25.2 26.81

[0152] During the audio signal mixing stage, the technical team performed real-time mixing of spatial audio signals from 86 sound sources. Volume level detection results showed that 32 sound sources exceeded the -20 dB threshold and were marked as important sources, receiving priority processing. The mixed stereo signal had a sampling rate of 48 kHz and a 16-bit depth, achieving low-latency transmission of 1024 samples through a circular buffer, with audio latency controlled within 21 milliseconds.

[0153] During system operation, the sound source status information is updated once per frame at a frequency of 60Hz, and the status change history is maintained for 10 seconds. Database update operations employ a transaction processing mechanism, with an average execution time of 3.2 milliseconds, meeting real-time processing requirements. In a complete training simulation, the system ran continuously for 6 hours, processing a total of 2.1TB of audio data and generating high-quality stereo output, providing trainees with a realistic auditory environment.

[0154] The circular queue scheduling algorithm effectively solves the problem of mismatch between the number of similar sound source groups and the number of CUDA streams. When 12 similar sound source groups are running on 16 CUDA streams, the first 12 CUDA streams each process one sound source group, and the last 4 CUDA streams are used for data transfer optimization, achieving full utilization of computing resources. The overall system processing efficiency reaches the expected target, and the GPU utilization rate remains above 85%.

[0155] In long-distance sound source processing, simplified acoustic models significantly improve computational efficiency. The combined application of spherical wave attenuation calculation and fundamental frequency response correction reduces processing time for sound sources 500 meters away to 30% of that of high-precision models, while maintaining acceptable sound quality loss. An air absorption coefficient of 0.0002 per meter is set, and the cutoff frequency of the first-order low-pass filter is dynamically adjusted according to the propagation distance, effectively compensating for high-frequency attenuation.

[0156] The technical team verified the system's stability and accuracy through extensive testing. Tests conducted under different weather conditions showed that the impact of temperature changes on the speed of sound was accurately simulated, with the error in sound wave propagation time calculation controlled within 2%. The simulation results of terrain reflection and scattering effects were in high agreement with measured data, and the accuracy of the reflected ray direction calculation reached 0.5 degrees.

[0157] Compared to traditional sound simulation methods, this invention achieves significant progress in several aspects. Traditional methods typically employ simple distance attenuation models and mono output, failing to provide a realistic spatial auditory experience. This invention accurately simulates the multiple reflections and propagation of sound waves in complex terrain using a ray tracing algorithm, considering the acoustic characteristics differences of different terrain types, making the simulation results closer to the real environment. Traditional methods often employ serial computation when handling multi-source scenarios, resulting in low computational efficiency and difficulty meeting real-time simulation requirements. This invention achieves real-time processing of large-scale sound sources through GPU parallel computing and sound source clustering optimization, significantly improving the system's computational performance. Traditional methods lack consideration for human auditory characteristics, often resulting in output audio signals containing a large amount of redundant information. This invention introduces a psychoacoustic masking model and head-related transfer function, optimizing the audio processing based on human auditory perception characteristics, improving audio quality and reducing unnecessary computational burden. Traditional methods have limited processing capabilities for moving sound sources and cannot accurately simulate the Doppler effect. This invention achieves realistic simulation of moving sound sources through precise Doppler frequency shift calculation and 3D localization processing, providing trainees with a more lifelike auditory experience.

[0158] It should be noted that the variables involved in this invention are explained in detail in Tables 3 and 4 below.

[0159] Table 3. Variable Explanation Table (Part 1)

[0160]

[0161] Table 4. Variable Explanation Table (Part Two)

[0162]

[0163]

[0164] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A method of training with environmental sound spatialization simulation, characterized by, include: A training environment sound source database was established, sound characteristics of training equipment were collected and frequency domain analysis was performed to obtain spectral data, and spatial coordinate information and motion parameters of the sound sources were recorded. Similarity clustering analysis was performed on the sound sources in the training environment sound source database. A clustering distance calculation method based on spectral data was used to group the sound characteristics of the training equipment, and the spectral similarity distance between each sound source and other sound sources was calculated. When the spectral similarity distance was less than the clustering threshold, the sound sources were grouped into the same similar sound source group. The GPU device was initialized and the GPU computing power parameters were detected. The number of CUDA streams was determined based on the GPU computing power parameters. A two-dimensional block mesh structure was configured within each CUDA stream, and a GPU was allocated to each CUDA stream. The system uses a global memory space and a shared memory space. When the number of similar sound source groups exceeds the number of CUDA streams, a circular queue scheduling algorithm is used for multi-batch processing. The sound source data in the similar sound source groups is loaded into the GPU's global memory space. Each CUDA kernel function in the first-dimensional block grid independently processes a sound source in a similar sound source group. A ray tracing algorithm is used to calculate the sound wave propagation path of the sound source, and acoustic parameters on the propagation path are obtained based on the terrain acoustic response library. The second-dimensional block grid uses a high-precision acoustic model to perform spatial audio rendering of the sound source, including Doppler effect calculation and three-dimensional positioning processing. The CPU mixes the spatial audio rendering results to generate stereo output and transmits it to the audio device in real time.

2. The method of ambient sound spatialization simulation for training of claim 1, wherein, The sound characteristics of the training equipment specifically include the sound characteristics of various training equipment such as air smoke command vehicles, air smoke launch vehicles, and rocket launch vehicles. The spatial coordinate information of the sound source includes three-dimensional position coordinates and direction vectors, and the motion parameters of the sound source include velocity vectors and acceleration vectors.

3. The method of ambient sound spatialization simulation for training of claim 2, wherein, The spectral similarity distance calculation method is specifically used to quantify the degree of similarity between different sound sources. The input includes the spectral data of two sound sources, and the output is a numerical distance representing the degree of similarity.

4. The method of ambient sound spatialization simulation for training of claim 3, wherein, The clustering threshold is set according to the degree of spectral difference between the sound source types. When the sound source is a smoke command vehicle, the clustering threshold is set to 0.85; when the sound source is a smoke launch vehicle, the clustering threshold is set to 0.80; and when the sound source is a rocket launch vehicle, the clustering threshold is set to 0.

75.

5. The method of ambient sound spatialization simulation for training of claim 4, wherein, The two-dimensional block mesh structure includes a first-dimensional block mesh and a second-dimensional block mesh. The first-dimensional block mesh is used to perform the sound wave propagation path calculation task, and the second-dimensional block mesh is used to perform the acoustic spatialization rendering calculation task.

6. The method of ambient sound spatialization simulation for training of claim 5, wherein, The circular queue scheduling algorithm is used to manage similar sound source groups that exceed the CUDA stream processing capacity. The input includes the number of similar sound source groups and the number of CUDA streams, and the output is a batch processing scheduling sequence.

7. The method of ambient sound spatialization simulation for training of claim 6, wherein, The ray tracing algorithm is used to calculate the propagation path and energy attenuation of sound waves in three-dimensional space. The inputs include the spatial coordinates of the sound source, the coordinates of the observation point, and the terrain geometry data. The outputs are the sound wave propagation time and the energy attenuation coefficient.

8. The method of ambient sound spatialization simulation for training of claim 7, wherein, The terrain acoustic response library is a pre-established database containing acoustic characteristics of different terrain types, including reflection and absorption coefficients for plains, mountains, urban buildings, and forests.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores program instructions that, when executed in a computer, perform a method for spatial simulation of environmental sound for training as described in any one of claims 1-8.

10. A system for spatial simulation of environmental sound for training, characterized in that, The system includes the computer-readable storage medium of claim 9, wherein the system is any one of a computer, a server, or a microcontroller, the computer-readable storage medium is disposed within the system, and the system is provided with a microprocessor that executes the program instructions stored in the computer-readable storage medium.

Images