Three-dimensional acoustic visualization rendering method and system based on sound field modeling

Abstract

The invention discloses a three-dimensional acoustic visualization rendering method and system based on sound field modeling. The method comprises the following steps: arranging a microphone array to collect a multi-channel time domain audio signal, and carrying out short-time Fourier transform to obtain frequency domain complex sound pressure data; performing inversion based on a beam forming algorithm to obtain complex sound pressure of each voxel point in a three-dimensional space, and constructing a four-dimensional acoustic voxel matrix; further utilizing a volume ray projection algorithm to map the volume element data into color and transparency, and realizing three-dimensional rendering and visual expression of the sound field; the system supports frequency screening, time evolution playback, transparency adjustment and three-dimensional visual angle control, and the visual analysis capability of the sound field is improved; and meanwhile, a structured data output interface is provided and can be linked with an AI algorithm, so that intelligent functions such as acoustic anomaly detection and tuning optimization are realized. The method has the capabilities of high-precision modeling, deep rendering expression and intelligent expansion, and is suitable for various acoustic application scenes such as intelligent sound boxes, vehicle-mounted sound boxes, conference systems and the like.

Description

Technical Field

[0001] This invention belongs to the field of acoustic signal processing and three-dimensional visualization, and particularly relates to a three-dimensional acoustic visualization rendering method and system based on sound field modeling, which is used to realize three-dimensional modeling, visualization, and sound effect analysis and optimization of sound fields. Background Technology

[0002] With the convergence of artificial intelligence, the Internet of Things, and spatial perception technologies, the acquisition, processing, and presentation of acoustic data are constantly evolving towards higher dimensions and stronger interactivity. In smart speakers, in-vehicle audio systems, remote video conferencing, and virtual reality (VR) and augmented reality (AR) environments, the spatial distribution characteristics of sound fields have a significant impact on user experience and system performance. To achieve accurate understanding and intervention of complex sound fields, it is urgent to break through the technical limitations of traditional two-dimensional sound pressure display methods and construct more realistic, intuitive, and multi-dimensional acoustic visualization tools.

[0003] Currently, mainstream acoustic visualization systems primarily rely on two-dimensional spectrograms, sound pressure heatmaps, or simplified three-dimensional vector maps. While these systems can represent basic sound pressure intensity distribution, they still have significant shortcomings in presenting spatial multi-band coupling, sound wave propagation dynamics, phase interference, and standing wave characteristics. Especially in scenarios such as indoor spatial acoustic tuning, sound system layout optimization, and auditory visual feedback, existing solutions struggle to meet the dual requirements of engineering-level interpretability and interactive analysis. From a data perspective, traditional methods often simplify the sound field to a two-dimensional intensity-space relationship, lacking the integration of the crucial frequency dimension, making it impossible for users to intuitively observe the three-dimensional distribution of specific frequency bands in space. Furthermore, because most visualization systems do not incorporate sound pressure phase information, coherent interference phenomena of sound waves are difficult to identify and reconstruct, which is key to analyzing acoustic problems such as standing waves and comb filtering. In addition, in terms of representation, traditional point, line, and surface displays cannot accurately convey the attenuation gradient of sound energy, the migration trajectory of energy centers, and spatial depth information, appearing flat and abstract, making it difficult to provide intuitive and reliable input support for non-professional users or AI systems.

[0004] More importantly, most existing acoustic visualization systems are used as closed tools, supporting only static displays and limited interaction. They lack standardized sound field data interfaces, making it difficult to efficiently integrate with AI algorithms and limiting their potential for expansion in emerging applications such as intelligent acoustic diagnostics and adaptive sound effect control. As acoustic systems increasingly demand automatic optimization, defect identification, and real-time feedback, building a sound field visualization system with high-dimensional modeling capabilities, dynamic rendering capabilities, and intelligent linkage capabilities has become one of the key technical challenges in improving the intelligence level of acoustic systems. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention provides a three-dimensional acoustic visualization rendering method and system based on sound field modeling. Its core lies in constructing an acoustic volumetric metadata model containing complex sound pressure information and using volumetric rendering technology adapted to acoustic physics to transform it into a three-dimensional image with depth perception and layering.

[0006] Specifically, the technical solution provided by this invention is as follows: A three-dimensional acoustic visualization rendering method based on sound field modeling includes the following steps: S1. Acquire audio signals from multiple spatial locations in the target space, wherein the audio signals are time-domain acoustic data synchronously acquired through a microphone array; S2. Preprocess and frequency domain transform the time-domain acoustic data to obtain time-frequency complex sound pressure spectrum data of each microphone acquisition channel; S3. Based on the complex sound pressure spectrum data and the spatial position parameters of each microphone, the sound field inversion of the target space is performed using a beamforming algorithm to construct an acoustic volume element matrix containing three-dimensional spatial coordinates, frequency information and complex sound pressure values, which is used to represent the joint structure of the sound field in the spatial and frequency dimensions. S4. Based on the acoustic volume element matrix, a volume rendering algorithm is used to perform three-dimensional sound field visualization processing, wherein frequency information is mapped to color attributes and sound pressure intensity is mapped to transparency attributes through a preset optical transfer function. S5. Output the rendered image in a three-dimensional visualization manner and support user interactive operations on the sound field image, including frequency filtering, time series playback, transparency adjustment and viewing angle control.

[0007] Furthermore, S3 includes the following steps: The target space is uniformly divided into several three-dimensional voxel points, and each voxel point corresponds to a unique spatial coordinate. The time-frequency complex sound pressure spectrum of each microphone acquisition channel was inverted to each voxel point to obtain the complex sound pressure of each voxel point at different frequency components; for voxel point (x,y,z), its complex sound pressure... The calculation formula is:

[0008] in, , It is a voxel point To the i The distance between the microphones, where c is the speed of sound, represents the distance from the voxel point to the first microphone. The propagation delay of each microphone, These are weighting coefficients. It is the phase compensation factor; For the first iThe time-frequency complex sound pressure spectrum of each microphone acquisition channel, where n is the total number of microphones; The complex sound pressure data of each voxel at different frequencies are organized in matrix form to construct a four-dimensional acoustic voxel matrix. Where the spatial dimensions x, y, and z are the position indices of the voxel points, and the frequency dimension... To analyze frequency points, the matrix element values ​​are complex sound pressure levels.

[0009] Furthermore, S4 employs a visualization rendering algorithm based on volumetric ray casting to transform acoustic volumetric matrix data containing spatial coordinates, frequencies, and complex sound pressure levels into a three-dimensional image representation; specifically, this includes the following steps: S401, Ray Generation and Sampling: A virtual ray is emitted for each pixel of the output image, penetrating the three-dimensional sound field data; the ray is then sampled along the ray path at fixed steps. Perform discrete sampling to generate a sequence of sampling points; S402. Acoustic parameter extraction: For each sampling point in the sampling point sequence, obtain its corresponding acoustic parameters from the acoustic voxel matrix, including sound intensity value and frequency value; if the sampling point does not have a corresponding element in the voxel matrix, it is obtained by interpolation based on its neighboring elements. S403, Optical Attribute Mapping: Converts the acoustic parameters of each sampling point into visual attributes, including color and transparency, through the optical transfer function; S404, Color Composition Calculation: Colors are accumulated and combined using rays in a backward-to-forward sequence. = +(1 )

[0010] = +(1 )

[0011] in, and These represent the composite color and transparency, respectively. and These represent the currently accumulated color and transparency, respectively. and These represent the color and transparency of the j-th sampling point, respectively.

[0012] Furthermore, the optical transfer function described in S403 includes a color mapping function: ,

[0013] in, Represents frequency The corresponding color value; s For saturation, Indicates a fully saturated color; v For brightness, Indicates the brightest color; For frequency The corresponding hue ranges from 0° (representing red) to 240° (representing blue); f max This is the maximum frequency value currently supported by the microphone.

[0014] Furthermore, the optical transfer function described in S403 includes a transparency mapping function;

[0015] in, This represents transparency, and its range is [0,1]. This is a scaling factor used to control the overall transparency level. This represents the sound intensity value at the j-th sampling point; Use the reference sound intensity or set as the sound intensity threshold to be used as a sound intensity threshold.

[0016] Preferably, in S5, the ray casting and color synthesis process is deployed in the fragment shader of the GPU: the voxel matrix is ​​uploaded to the GPU as a 3D texture, and the fragment shader is used to perform the integration calculation of millions of rays in parallel to achieve real-time rendering of the three-dimensional image.

[0017] A three-dimensional acoustic visualization rendering system based on the above method, the system mainly includes a data acquisition layer, a core processing layer and an application interaction layer; The data acquisition layer is used to acquire the raw audio signal of the target space and perform preprocessing, including a microphone array module, a signal synchronization module and a preprocessing module; The core processing layer is used to convert the acquired multi-channel audio signals into multi-dimensional sound field data suitable for rendering and intelligent analysis, including a spectrum extraction module, a sound field reconstruction module, and an acoustic volume matrix construction module. The application interaction layer is used to convert the acoustic volume matrix into image display and interactive data, and provides open interfaces to support linkage with external intelligent modules, including a 3D visualization module, an interactive control module, and an AI interface module.

[0018] Furthermore, the microphone array module consists of independent microphones arranged in a three-dimensional array structure, the position of each microphone is known and calibrated by spatial coordinates; the three-dimensional array structure is a uniform cubic array, a spherical array, or a non-uniform encrypted distribution array; the signal synchronization module adopts hardware synchronization or software synchronization to achieve time consistency of audio signal acquisition by each microphone acquisition channel; the preprocessing module is used to perform pre-emphasis filtering, noise suppression, and gain calibration on the acquired audio signal.

[0019] Furthermore, the interactive control module is equipped with a frequency selector, a transparency adjuster, and a viewing angle adjuster; The frequency selector allows users to select a specific frequency band for real-time sound field rendering. The transparency adjuster allows users to adjust the transparency of the rendered image. The user can rotate, scale, and move the rendered image using the view adjuster. The interactive control module is also equipped with a timeline slider, which allows users to browse the evolution of the sound field frame by frame in the temporal snapshot of the acoustic volume matrix.

[0020] Furthermore, the AI ​​interface module exports the acoustic volumetric matrix or its feature information in a structured form for use by artificial intelligence algorithms; the feature information refers to the spatial acoustic intensity heatmap, spectral tensor, phase interferogram, or temporal variation sequence generated after data extraction and encoding of the acoustic volumetric matrix.

[0021] This invention, by introducing a four-dimensional acoustic volumetric modeling mechanism, achieves for the first time a fusion expression of the sound field across time, space, and frequency dimensions. It overcomes the limitations of traditional two-dimensional sound pressure maps and static three-dimensional views in terms of dimensional representation, intuitively presenting the distribution and dynamic evolution of sound energy in a specific frequency band within three-dimensional space. Utilizing a volumetric rendering algorithm based on physical acoustic properties, the system realistically recreates the propagation path, energy attenuation, phase interference, and other physical phenomena of sound waves, enhancing the depth and layering of the visualization results and significantly improving the realism and interpretability of the sound field representation. Simultaneously, this invention constructs a complete interactive mechanism, supporting frequency band filtering, time-series playback, transparency adjustment, and spatial rotation, meeting users' needs for multi-angle and multi-dimensional sound field analysis. It also provides standardized structured data interfaces, offering rich acoustic feature inputs for AI algorithms and possessing excellent intelligent scalability. Compared to existing technologies, this invention achieves significant improvements in modeling accuracy, expressiveness, interactivity, and system openness, providing strong technical support for the analysis, diagnosis, and optimization of intelligent acoustic systems. Attached Figure Description

[0022] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof.

[0023] Figure 1 This is a schematic diagram of the architecture of the three-dimensional acoustic visualization rendering system provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the workflow of the three-dimensional acoustic visualization rendering system provided in the embodiment of the present invention. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, other embodiments obtained by those skilled in the art without creative effort are all within the scope of protection of the present invention.

[0025] Example 1 This embodiment provides a three-dimensional acoustic visualization rendering method based on sound field modeling, which mainly includes the following steps: 1. Acoustic data acquisition and preprocessing First, a microphone array is deployed in the target sound field region. This array consists of at least four independent microphones, preferably a three-dimensional array structure with adjustable structure and scalable array configuration, such as a uniform cubic array, a spherical array, or a non-uniformly distributed array. The position of each microphone should be known and calibrated using spatial coordinates to ensure that its positioning accuracy in three-dimensional space is controlled within the centimeter range.

[0026] Each microphone channel synchronously acquires the raw time-domain audio signal from the sound field. (i is the microphone index). To ensure data synchronization and timing consistency, either hardware or software synchronization methods can be used. Hardware synchronization uses a unified clock source to synchronize all sampling channels, ensuring consistent sampling start points and fixed sampling intervals. Software synchronization, on the other hand, uses signal alignment algorithms, such as cross-correlation time delay estimation (GCC-PHAT) or frame-level synchronization interpolation algorithms, to align and correct the acquired multi-channel signals, compensating for minute time differences caused by device clock drift or wiring variations.

[0027] The acquired audio signal then enters the signal preprocessing process, which mainly includes the following three steps: (1) Pre-emphasis: High-frequency components are boosted by a first-order high-pass filter, low-frequency energy is suppressed, and the recognizability and frequency domain resolution of high-frequency sound sources (such as speech, sharp object reflections, etc.) are improved.

[0028] (2) Denoising: Time-frequency domain noise reduction algorithms, such as spectral subtraction or Wiener filtering, are used to suppress background noise, wind noise, mechanical vibration noise and other interference signals. Spatial filtering algorithms for microphone arrays (such as MVDR beamforming) can be selected for spatial directional noise reduction.

[0029] (3) Gain Normalization: To address the differences in hardware gain and sensitivity between different channels, the amplitude of the audio signal in each channel is normalized. Common methods include RMS normalization or maximum value normalization to ensure the consistency of amplitude in subsequent sound pressure inversion.

[0030] Finally, in preparation for the frequency domain analysis and spatial reconstruction stages, the synchronized and preprocessed audio data is divided into frames and weighted by window functions to meet the time-frequency resolution requirements of the Short Time Fourier Transform (STFT). The frame length is typically 20–50 ms, the frame shift is 10–25 ms, and window functions such as Hamming windows, Heman windows, or Gaussian windows can be used.

[0031] 2. Three-dimensional spatial discretization and frequency domain transformation First, the region to be analyzed is discretized in three-dimensional space.

[0032] Specifically, the target space is divided into a uniform three-dimensional grid structure, with each voxel representing a spatial sampling point. The density of the spatial grid can be dynamically set according to the expected spatial resolution and hardware capabilities; for example, a 1m³ space can be divided into a 100×100×100 voxel array with a spatial sampling interval of 1cm. Each voxel is assigned a unique three-dimensional coordinate (x, y, z), which is used as the localization index for subsequent sound pressure data.

[0033] Subsequently, a short-time Fourier transform was performed on the acquired signals from each microphone channel.

[0034] This operation employs a sliding window mechanism, dividing the continuous audio signal into overlapping frames (e.g., frame length 32ms, frame shift 16ms). A window function (e.g., Hamming window) is applied to each frame, and the spectral representation is obtained through Fast Fourier Transform (FFT) to obtain the signal for each channel. The corresponding time spectrum This includes information about the amplitude of the sound pressure level. (For sound intensity and energy analysis) and phase information (Used for interference modeling), forming the time-frequency complex sound pressure spectrum of each microphone channel.

[0035] 3. Reconstruction of complex sound fields and construction of acoustic bulk elements Suppose the target space has been divided into several three-dimensional voxels, each voxel corresponding to a unique spatial coordinate. For each frequency component... By using a beamforming algorithm, the complex sound pressure spectrum of each microphone channel is inverted and mapped to each voxel point, and the complex sound pressure at that point at that frequency is estimated.

[0036] For a voxel (x,y,z), its complex sound pressure The calculation formula is:

[0037] in, , It is a voxel point The distance to the i-th microphone, where c is the speed of sound, representing the distance of the sound wave from that voxel point to the i-th microphone. The propagation delay of each microphone, These are weighting coefficients. It is the phase compensation factor.

[0038] The model essentially estimates the complex sound pressure level and phase of each voxel at a specific frequency by aligning and weighting the signals from each microphone channel, thereby forming a complete complex sound field reconstruction in space.

[0039] After the complex sound pressure level estimation is completed, the complex sound pressure level data of each spatial point (voxel) at different frequencies are organized in matrix form to construct a four-dimensional data structure called the Acoustic Voxel Matrix, denoted as […]. Where the spatial dimensions x, y, and z are the voxel position indices, and the frequency dimension... For frequency analysis; the data values ​​are sound pressure levels in complex form (including amplitude and phase).

[0040] 4. Sound field 3D volumetric rendering After constructing the acoustic volumetric matrix, the core step is 3D visualization, which transforms the four-dimensional volumetric data, including spatial coordinates, frequency, and complex sound pressure levels, into a 3D image representation that can be perceived and understood by the human eye. To this end, this embodiment employs a visualization rendering algorithm based on volume ray casting, achieving 3D visualization of the sound field energy through innovative optical transfer function design and efficient ray integration strategy.

[0041] 4.1 Optical Transfer Function Design First, the acoustic information corresponding to each voxel needs to be converted into color and transparency parameters required for image rendering. To this end, the following two types of optical transfer functions are designed: (1) Color mapping function Based on sound wave frequency The corresponding acoustic characteristics are mapped to the HSV color space, achieving the following pattern: Low frequencies (<200Hz): mapped to the red color scheme, representing heavy, fundamental low-frequency sound energy; Mid-frequency (200~2000Hz): Mapped to green tones, representing the core frequency band of human voices and major musical instruments; High frequency (>2000Hz): Mapped to blue to represent crisp, sharp high-frequency components.

[0042] This can be achieved using the following function:

[0043] in, Represents frequency f The corresponding color value; s Saturation, ranging from [0,1], is fixed at 1 in this embodiment, representing a fully saturated color; v Value is the brightness, ranging from [0,1]. In this embodiment, it is fixed at 1, representing the brightest color. For frequency f The corresponding hue ranges from red (0°) to blue (240°). When f When =0, h (0) = 0, corresponding to red; when f = f max hour, h ( f max )=240, corresponding to blue; f max This represents the maximum frequency value supported by the current sampling system. Therefore, this mapping causes the color to transition from red to yellow to green to cyan to blue as the frequency increases, forming an intuitive audio-color relationship.

[0044] (2) Transparency mapping function To reflect the visibility of sound energy, this embodiment defines the following typical function form:

[0045] in, This represents transparency, and its range is [0,1]. This is a scaling factor used to control the overall transparency level. This represents the sound intensity value of the current voxel. The reference sound intensity (baseline sound intensity) can be set as the minimum perceptible sound intensity, or it can be set as a sound intensity threshold to achieve a sound intensity limit. Only sound intensities exceeding a specific threshold will be displayed.

[0046] The above logarithmic mapping function can simulate the auditory characteristics of the human ear.

[0047] 4.2 Ray projection and color synthesis ① Ray generation and sampling: A virtual ray is emitted for each pixel of the output image, penetrating the three-dimensional sound field data; the ray is then used to generate a ray along the path with a fixed step size. Discrete sampling is performed to generate a sequence of sampling points.

[0048] ②Acoustic parameter extraction: For each sampling point in the sampling point sequence, obtain its corresponding acoustic intensity value I from the volume element matrix V. j and frequency value f j If a sampling point does not have a corresponding element in the voxel matrix, it is obtained by interpolation based on its neighboring elements.

[0049] ③ Optical attribute mapping: The optical transfer function described above is used to convert the acoustic parameters of each sampling point into visual attributes, namely, color. ,transparency .

[0050] ④ Color Combination Calculation: Colors are accumulated and combined using rays in a backward-to-forward sequence. = +(1 )

[0051] = +(1 )

[0052] in, and These represent the composite color and transparency, respectively. and These represent the currently accumulated color and transparency, respectively. and These represent the color and transparency of the current sampling point (the j-th sampling point), respectively. This synthesis strategy can effectively simulate the transmission and scattering process of light in a semi-transparent medium, giving the image a realistic sense of depth.

[0053] 4.3 Real-time GPU rendering To ensure rendering efficiency for large-scale data, this embodiment deploys the ray casting and color composition processes within the GPU's fragment shader. Specifically, the voxel matrix V is uploaded to the GPU as a 3D texture, and the fragment shader performs parallel integral calculations of millions of rays to achieve real-time rendering.

[0054] 5. Visual interactive control and dynamic rendering This step aims to empower users with active control over the visualization results of the 3D sound field, achieving a leap from static observation to dynamic exploration. By constructing a responsive visualization interaction module, users can not only customize the frequency range, time window, transparency, and viewing angle of the visualization, but also achieve dynamic playback and visual analysis of the sound field evolution process, greatly enhancing operability and immersion.

[0055] (1) Frequency dimension filtering and local rendering To meet diverse analytical needs, users can select the frequency range of interest in real time using the frequency selection slider in the graphical interface. The system then extracts the complex sound pressure subset of the corresponding frequency band from the complete four-dimensional acoustic volume matrix and re-renders it. It enables filtering of sound field distribution by frequency components and supports independent analysis of specific frequency bands (such as low-frequency standing waves and high-frequency noise).

[0056] (2) Time dimension switching and dynamic playback Considering the dynamic characteristics of sound fields, a time-axis-based dynamic sound field playback function is supported. Users can select a specific time point within the timing window by dragging the timeline slider. The corresponding acoustic volumetric matrix snapshot will be loaded from the cache in real time, triggering an instant rendering. If the user activates dynamic playback mode, snapshots of each time point will be sequentially switched and rendered at a fixed frame rate (e.g., 20fps), forming a 4D (spatial 3D + temporal 3D) visualization of the sound field. This implementation supports the observation and analysis of dynamic phenomena such as sound field propagation, sound source movement, and changes in interference modes.

[0057] (3) View control and transparency adjustment To enable users to flexibly observe complex three-dimensional sound field structures, users can freely rotate, scale, and translate the sound field image using a mouse or touch screen, and comprehensively explore the sound field distribution characteristics from any angle and scale.

[0058] Meanwhile, by adjusting the transparency parameters of the overall rendered object in real time using the transparency slider, users can observe the distribution layers of the internal structure in a high transparency state according to their analysis needs, and highlight energy-dense areas or simulate the visual effect of solid propagation paths in a low transparency state.

[0059] 6. Structured data output and linkage with intelligent systems In some embodiments, the underlying four-dimensional acoustic volume matrix is ​​further extracted in a structured manner and expressed in a standardized way, so that it has a characteristic data form that can be analyzed, processed and learned by artificial intelligence algorithms.

[0060] First, the complex sound pressure data of each voxel in the voxel matrix is ​​processed by dimension division to extract multiple key features that can be used for modeling, including: Frequency distribution map: Statistically analyze the energy intensity distribution of each spatial point at each frequency to construct a three-dimensional spectrum dataset, which is suitable for AI to perform spatial spectrum analysis, frequency band energy focusing or noise component identification, etc. Spatial energy density map: After merging according to a certain frequency or frequency band, the output is an acoustic energy heat map of the entire potential field, which is used to identify spatial anomalous areas such as standing waves, acoustic focal points, and enhanced reflection. Phase interference mode diagram: Extract phase information of volume element complex sound pressure, identify regions of constructive (enhancing) or destructive (attenuating) interference, and provide AI with a basis for judging physical phenomena such as comb filtering and acoustic modes; Dynamic time series: Encode sound field snapshots at different times to form sound field time series vectors, which can be used by AI for state recognition, trend prediction or time-varying feature modeling.

[0061] Then, the aforementioned feature data is encapsulated in a unified structured format (such as JSON, CSV, Tensor, HDF5, etc.) and output through a standardized AI interface module. This interface follows an open protocol, supports the input standards of mainstream AI training platforms (such as PyTorch, TensorFlow, ONNX, etc.), and can be directly accessed. An acoustic image recognition model based on convolutional neural networks (CNN) is used for automatic classification of acoustic defects; A sound field evolution prediction model based on recurrent neural networks (RNN / LSTM) is used for early warning of abnormal evolution trends. Sound field type clustering analysis based on clustering and dimensionality reduction algorithms is used for intelligent scene recognition and device adaptive configuration.

[0062] Simultaneously, it provides two-way interactive capabilities, not only supplying data to the AI ​​module but also receiving feedback results from the AI ​​model, such as judgment labels, weight adjustment suggestions, or parameter optimizations, and writing them back to the rendering control module. This achieves an intelligent closed-loop process from AI recognition to parameter adjustment and then to visual feedback. This mechanism makes the system not just a passive display tool, but also an actively evolving intelligent acoustic analysis platform.

[0063] For example, in a car audio system, this function can identify cabin standing waves and locate high-frequency interference based on sound field distribution, and automatically recommend EQ adjustment strategies based on AI recognition results, and display the adjusted sound field image through the user interface; or in the debugging of conference room audio systems, it can automatically identify abnormal sound focusing and feedback howling paths, helping engineers to quickly locate and optimize the sound pickup layout.

[0064] Example 2 Based on the above method, this embodiment provides a three-dimensional acoustic visualization rendering system based on sound field modeling, such as... Figure 1 As shown, the system is divided into a data acquisition layer, a core processing layer, and an application interaction layer. Each layer works together through standardized data interfaces to form a well-structured, fully functional, and highly scalable 3D acoustic modeling and visualization system.

[0065] 1. Data Acquisition Layer This layer is responsible for acquiring and preprocessing the raw acoustic signals, forming the physical sensing foundation of the entire system. Its output is a structured, synchronized, multi-channel time-domain audio signal, providing a high-fidelity input for the core processing layer. Its main components are as follows: (1) Microphone array module The microphone array module is responsible for deploying sound pressure sampling nodes in three-dimensional space. This module consists of multiple high-sensitivity microphones, with common array configurations including cubic arrays, rectangular planar arrays, and spherical arrays. The specific layout can be flexibly adjusted according to the application scenario (such as room sound field, vehicle cabin, conference space, etc.). Each microphone obtains precise spatial coordinates through three-dimensional calibration. Its position data will be used in subsequent sound field inversion to calculate the sound wave propagation path, propagation delay, and directivity factor, directly affecting the spatial accuracy of complex sound pressure estimation. To achieve high spatial resolution of the sound field, the microphone density needs to be designed based on the spatial scale and analysis frequency range to ensure that the Nyquist sampling conditions are met and spatial aliasing is avoided.

[0066] (2) Signal synchronization module The signal synchronization module is a crucial component for ensuring data timing consistency. In multi-channel audio systems, even if all microphones physically sample simultaneously, factors such as ADC conversion, channel delay, and hardware jitter can still introduce slight sampling time offsets, leading to waveform misalignment and phase distortion, which in turn severely interfere with subsequent spatial beamforming and interferometric analysis.

[0067] This embodiment supports two synchronization mechanisms: one is hardware synchronization, which uses a unified external clock source (such as a GPS clock, a 10 MHz master clock, etc.) or a multi-channel synchronous sampling card to enable all microphones to start and stop at the same time in the physical sampling layer; the other is software synchronization, which uses cross-correlation delay estimation (such as GCC-PHAT) to correct the delay of each channel signal in scenarios with limited hardware capabilities, and uses interpolation resampling technology to achieve time alignment at sub-sampling accuracy.

[0068] (3) Preprocessing module The acquired audio signals are processed by noise suppression, gain equalization, and pre-emphasis filtering, and frame division and window function weighting are performed to lay the foundation for subsequent frequency domain transformation and ensure the consistency and cleanliness of data quality.

[0069] 2. Core Processing Layer This layer is the core of the system's computation and modeling, responsible for transforming the acquired raw multi-channel audio signals into a multi-dimensional sound field data model with clear spatial structure, well-defined frequency characteristics, and suitability for rendering and intelligent analysis. It mainly includes the following modules: (1) Spectrum extraction module After the data acquisition layer outputs synchronized, noise-reduced, and weighted multi-channel audio signals, the core processing layer first enters the frequency domain analysis stage. The spectrum extraction module uses short-time Fourier transform to convert the time-domain signals of each microphone channel into complex spectra, thereby simultaneously recording the amplitude and phase characteristics of the sound pressure.

[0070] (2) Sound field reconstruction module The sound field reconstruction module is a key component of the core processing layer. By mapping multi-channel spectral information back to three-dimensional space, it estimates the complex sound pressure distribution at each voxel point in space, achieving the conversion from the microphone observation point to the spatial field distribution. The system divides the target sound field region into a uniform voxel grid, with each voxel point having unique three-dimensional coordinates. At each frequency, the system employs array signal processing and wave propagation models to invert the complex sound pressure of each channel to the voxel point location using beamforming formulas.

[0071] (3) Acoustic volumetric construction module The inversion results are organized into a four-dimensional acoustic voxel matrix, with each voxel recording its spatial location, frequency index, and corresponding complex sound pressure level, forming a highly structured and user-friendly digital sound field representation model. This matrix can be used directly for rendering or by AI models.

[0072] 3. Application Interaction Layer This layer, facing the user end and AI system, is responsible for converting acoustic voxels into image displays and interactive data, and provides open interfaces to support intelligent linkage. It mainly includes the following modules: (1) 3D visualization module This module employs volumetric rendering technology, specifically based on the volumetric ray casting method. It projects a ray from the observer's viewpoint onto each pixel in the image, traversing a three-dimensional voxel space, sampling complex sound pressure levels along the ray, and mapping frequencies to HSV colors and sound intensity to transparency α according to a predefined optical transfer function, thus fully visualizing the sound field structure. The color mapping strategy allows for a reddish tint to low frequencies and a bluish tint to high frequencies, intuitively distinguishing sound energy types; transparency control can highlight areas of concentrated energy or reveal hierarchical structures. To ensure rendering efficiency and responsiveness, the system utilizes parallel acceleration based on GPU fragment shaders, supporting high-frame-rate real-time rendering and maintaining stable performance even with high-dimensional data volumes (such as millions of voxels).

[0073] (2) Interactive control module The interactive control module provides the core entry point for user interface and multi-dimensional data exploration, enabling users to freely rotate, zoom, and move the 3D sound field image, observing the spatial energy distribution and its hierarchical relationships from different angles. The module offers a controllable frequency selector, allowing users to select a specific frequency band for local rendering in real time, facilitating focused analysis of low-frequency standing waves or high-frequency sharp noise. Simultaneously, through the timeline slider, users can browse the evolution of the sound field over time frame by frame in a time-series snapshot, observing dynamic phenomena such as sound source movement, interference spread, and energy diffusion. Furthermore, the module includes a global transparency adjuster, which can uniformly adjust the transparency of the entire image, aiding in the observation of local internal structures or shielding against background noise, further enhancing the freedom and depth of observation.

[0074] (3) AI Interface Module The AI ​​interface module enables bidirectional communication between this system and external intelligent analysis models. On one hand, this module can extract and encode features from the voxel matrix, generating standard structured data such as spatial sound intensity heatmaps, spectral tensors, phase interferograms, and temporal variation sequences, and outputting them in formats such as Tensor, CSV, and HDF5 for AI models to use for tasks such as classification, clustering, prediction, or parameter tuning and optimization. On the other hand, this module also accepts recognition results, tuning suggestions, or diagnostic labels from the AI ​​system, such as standing wave recognition results and EQ optimization parameters, and converts them into visual labels, adjustment instructions, or feedback annotations, directly affecting the visualization and control modules.

[0075] like Figure 2As shown, after the system starts, the data acquisition layer first completes the acquisition, synchronization, and preprocessing of multi-channel acoustic data to generate a high-quality input signal. Subsequently, the core processing layer performs frequency domain transformation and complex sound field inversion on the signal to construct a four-dimensional acoustic volume metadata model. This model is rendered in real-time as a three-dimensional sound field image at the application interaction layer, and the view and rendering parameters are dynamically updated and adjusted according to user operations or AI commands. Ultimately, the system achieves a closed-loop process from acoustic modeling and image representation to intelligent analysis, possessing highly visualized, interactive, and scalable characteristics.

[0076] The above system can execute the three-dimensional acoustic visualization rendering method based on sound field modeling as described in Embodiment 1, and has the corresponding functional modules and beneficial effects of the method. For technical details not described in detail in this embodiment, please refer to the three-dimensional acoustic visualization rendering method based on sound field modeling provided in Embodiment 1 of the present invention.

[0077] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented using software plus a general-purpose hardware platform, or of course, using hardware. Based on this understanding, the above technical solutions, in essence or the parts that contribute to the related technology, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0078] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; under the concept of the present invention, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of different aspects of the present invention as described above, which are not provided in detail for the sake of brevity; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A method for three-dimensional acoustic visualization rendering based on sound field modeling, characterized in that, The method comprises the steps of: S1, collecting audio signals of multiple spatial positions in a target space, wherein the audio signals are time-domain acoustic data synchronously collected by a microphone array; S2, preprocessing and frequency domain transforming the time-domain acoustic data to obtain time-frequency complex sound pressure spectrum data under each microphone collection channel; S3, based on the complex sound pressure spectrum data and the spatial position parameters of each microphone, using a beam forming algorithm to perform sound field inversion on the target space, constructing an acoustic voxel matrix containing three-dimensional spatial coordinates, frequency information and complex sound pressure values, for representing the joint structure of the sound field in the spatial and frequency dimensions; S4, based on the acoustic voxel matrix, using a volume rendering algorithm for three-dimensional sound field visualization processing, wherein the frequency information is mapped to the color attribute and the sound pressure intensity is mapped to the transparency attribute through a preset optical transfer function; S5, outputting the rendered image in a three-dimensional visualization manner, and supporting user interaction operation on the sound field image, including frequency filtering, time sequence playing, transparency adjustment and view angle control.

2. The three-dimensional acoustic visualization rendering method of claim 1, wherein, S3 comprises the steps of: uniformly dividing the target space into a plurality of three-dimensional voxel points, each voxel point corresponding to a unique spatial coordinate; The time-frequency complex sound pressure spectrum under each microphone acquisition channel is respectively inverted to each voxel point to obtain the complex sound pressure of each voxel point under different frequency components; for a voxel point (x, y, z), the calculation formula of the complex sound pressure is: ​ wherein, , is a voxel point to the distance of the first i microphone, c is the speed of sound, and represents the propagation time delay of the sound wave from the voxel point to the first microphone, is a weight coefficient, is a phase compensation factor; is the time-frequency complex sound pressure spectrum under the channel of the first i microphone, and n is the total number of microphones; The complex sound pressure data of each voxel point at different frequencies is organized in a matrix form to construct a four-dimensional acoustic voxel matrix wherein the spatial dimensions x, y, z are the position indexes of the voxel points, and the frequency dimension is the analysis frequency point, and the matrix element value is the complex sound pressure.

3. The three-dimensional acoustic visualization rendering method of claim 1, wherein, In S4, a visualization rendering algorithm based on volume ray casting is used to convert the acoustic voxel matrix data containing spatial coordinates, frequency and complex sound pressure values into a three-dimensional image representation; specifically comprising the steps of: S401, ray generation and sampling: emit a virtual ray for each pixel of the output image, penetrate the three-dimensional sound field data; along the ray path with fixed step length discrete sampling is performed to generate a sequence of sampling points; S402, acoustic parameter extraction: for each sampling point in the sampling point sequence, obtaining its corresponding acoustic parameters from the acoustic voxel matrix, including sound intensity value and frequency value; If the sampling point has no corresponding element in the voxel matrix, the element is calculated and obtained by interpolation method according to its adjacent elements; S403, optical attribute mapping: converting the acoustic parameters of each sampling point into visual attributes including color and transparency through an optical transfer function; S404, color synthesis calculation: color accumulation synthesis is performed in the order of rays from back to front = +(1 ) = +(1 ) wherein, and respectively represent the color and transparency after the synthesis, and respectively represent the color and transparency after the synthesis, and respectively represent the color and transparency of the jth sampling point.

4. The three-dimensional acoustic visualization rendering method of claim 3, wherein, The optical transfer function in S403 includes a color mapping function: ， wherein, represents a frequency corresponding color value; s is a saturation, represents a fully saturated color; v is a lightness, represents the brightest color; is a frequency corresponding hue, taking a value ranging from 0°, which represents the red color, to 240°, which represents the blue color; f max is a maximum frequency value supported by the current microphone.

5. The method of three-dimensional acoustic visualization rendering of claim 3, wherein, The optical transfer function in S403 includes a transparency mapping function; wherein, represents transparency, which ranges from [0, 1]; is a scaling factor for controlling the overall transparency strength; represents the sound intensity value of the jth sampling point; is a reference sound intensity or a sound intensity threshold set to be used as a sound intensity threshold.

6. The method of three-dimensional acoustic visualization rendering of claim 3, wherein, In S5, the ray projection and color synthesis process are deployed in the fragment shader of GPU for execution: uploading the voxel matrix to GPU as 3D texture, using fragment shader to perform integral calculation of millions of rays in parallel, realizing real-time rendering of three-dimensional image.

7. A three-dimensional acoustic visualization rendering system based on the method of any one of claims 1 to 6, characterized in that It comprises a data collection layer, a core processing layer and an application interaction layer; The data collection layer is used to obtain the original audio signal of the target space and perform preprocessing, including a microphone array module, a signal synchronization module and a preprocessing module; The core processing layer is used to convert the collected multi-channel audio signal into multi-dimensional sound field data suitable for rendering and intelligent analysis, including a spectrum extraction module, a sound field reconstruction module and an acoustic voxel matrix construction module; The application interaction layer is used to convert the acoustic voxel matrix into image display and interactive data, and provides an open interface to support linkage with external intelligent modules, including a three-dimensional visualization module, an interactive control module and an AI interface module.

8. The three-dimensional acoustic visualization rendering system of claim 7, wherein, The microphone array module is composed of microphones arranged in a three-dimensional array structure independently of each other, the positions of the microphones are known and calibrated by spatial coordinates; the three-dimensional array structure is a uniform cubic array, a spherical array or a non-uniformly distributed array; the signal synchronization module adopts a hardware synchronization or software synchronization mode to realize the time consistency of the audio signals obtained by each microphone acquisition channel; the preprocessing module is used for pre-emphasis filtering, noise suppression and gain calibration processing of the collected audio signals.

9. The three-dimensional acoustic visualization rendering system of claim 7, wherein, The interactive control module is provided with a frequency selector, a transparency adjuster and a view angle adjuster; Through the frequency selector, the user can select a certain frequency band for real-time sound field rendering; Through the transparency adjuster, the user can adjust the permeability of the rendered image; Through the view angle adjuster, the user can rotate, scale and move the rendered image; The interactive control module is also provided with a time axis slider, through which the user can frame by frame browse the evolution process of the sound field in the time sequence snapshot of the acoustic voxel matrix.

10. The three-dimensional acoustic visualization rendering system of claim 7, wherein, The AI interface module exports the acoustic voxel matrix or its feature information in a structured form for calling by an artificial intelligence algorithm; the feature information refers to a spatial sound intensity heat map, a frequency spectrum tensor, a phase interference map or a time sequence change sequence generated after data extraction and coding of the acoustic voxel matrix.

Images