Physical engine and mapping system for acousto-optic dynamic simulation of electronic fireworks

Abstract

The invention discloses a physical engine and mapping system for electronic firework acousto-optic dynamic simulation, and the system comprises a physical engine module which is used for receiving firework physical parameters and sound effect parameters inputted by a user, abstracting each firework unit into a particle system, generating a space-time state of each particle, and transmitting the space-time state of each particle to the physical engine module; the physical phenomenon of the firework explosion process is simulated in real time; the acousto-optic mapping module is used for converting the space-time state of the particles into acousto-optic control parameters; the spatial audio module is used for receiving the sound control parameters and the three-dimensional space positions of the particles in real time and performing real-time three-dimensional space rendering on the sound effect parameters so as to generate three-dimensional sound field signals; the environment adaptation module is used for dynamically adjusting the three-dimensional sound field signal through an audio rendering algorithm, so that the output audio effect can adapt to different physical spaces; particle motion is simulated through a physical engine, acousto-optic parameters are mapped, and dynamic simulation of high-fidelity electronic fireworks is realized in combination with adaptation of spatial audio and environment.

Description

Technical Field

[0001] This invention relates to the field of electronic fireworks technology, and more specifically to a physics engine and mapping system for dynamic simulation of the sound and light of electronic fireworks. Background Technology

[0002] Currently, in the field of electronic fireworks displays or multimedia performance arts, the main technical solutions for achieving synchronized sound and light effects include the following strategies: First, in existing professional editing software, technicians need to manually align the timestamps of audio and light animation. While this method can ensure the accuracy of performances in fixed scenes, it lacks flexibility. Second, audio-responsive LED intelligent control is used. Ambient sounds are captured through microphones, and the system triggers corresponding light modes based on preset thresholds. For example, high-frequency sounds trigger rapid light flashing, thus achieving basic sound and light interaction. Third, MIDI music signals are used to drive the system, precisely mapping the MIDI signals of the music to DMX light controllers, thereby achieving precise control of the light effects. However, this method is mainly suitable for the expression of instrument timbre and has limited ability to handle complex and varied explosion sound effects. Fourth, pre-recorded audio is matched with LED light animation through precise timeline alignment. However, this method lacks the ability to simulate physical processes in real time, resulting in mechanical and lifeless sound and light effects. Fifth, audio spectrum analysis tools (such as Fast Fourier Transform, FFT) are used to drive changes in the brightness of LED lights, achieving basic rhythmic synchronization between audio and light. However, due to the limitations of the technology itself, it can only capture rhythmic changes in audio and cannot accurately simulate the complex stereo sound field and dynamic decay process of light effects produced during a fireworks explosion. Sixth, drone formation light show technology is used, combined with GPS global positioning system and music rhythm control. Drone formations can perform complex flight maneuvers and display light effects. However, this sound and light linkage method mainly focuses on simple matching of drone positions and cannot fully reproduce the physical characteristics of fireworks explosions, such as shock wave propagation and particle diffusion.

[0003] In summary, existing technologies face numerous challenges and shortcomings. On one hand, there is inconsistency in the dynamic performance of sound and light. Existing solutions struggle to adjust sound and light parameters in real-time based on the actual physical processes of fireworks explosions, such as acceleration, blast radius, and particle dissipation rate, resulting in significant desynchronization between sound effects and animation. This is primarily due to the lack of a physics engine in the software, with lighting control relying excessively on linear interpolation or fixed keyframes, failing to adequately consider the impact of physical factors such as gravity and air resistance on the fireworks' trajectory. On the other hand, the spatial representation of sound effects is insufficient. Traditional solutions often employ mono or stereo sound effects, making it difficult to realistically reproduce the directional changes of fireworks in three-dimensional space, such as the Doppler effect during ascent and the omnidirectional diffusion of the explosion sound. Furthermore, the lack of integration of HRTF (Head-Related Transfer Function) or Ambisonic spatial audio technology in the software limits sound field rendering to a two-dimensional plane, lacking a sense of depth.

[0004] In summary, existing technologies for sound and light simulation suffer from insufficient physical realism, limited spatial expressiveness, poor real-time performance, and high interaction thresholds. The root cause lies in the limitations of hardware support, such as the lack of spatial audio hardware, and the deficiencies in algorithm design, such as the lack of a physics engine and parallel computing technology. Summary of the Invention

[0005] The purpose of this invention is to provide a physics engine and mapping system for dynamic simulation of electronic fireworks sound and light. By simulating particle motion through the physics engine, mapping sound and light parameters, and combining spatial audio with environmental adaptation, high-fidelity dynamic simulation of electronic fireworks can be achieved.

[0006] To achieve the above objectives, the present invention provides the following technical solution: A physics engine and mapping system for dynamic simulation of electronic fireworks sound and light includes: The module includes a physics engine module, an audio-visual mapping module, a spatial audio module, an environment adaptation module, and a user interaction module. The physics engine module is used to receive the physical parameters and sound effect parameters of the fireworks input by the user, and to abstract each firework unit into a particle system, generating the spatiotemporal state of each particle to simulate the physical phenomena of the fireworks explosion process in real time. The acousto-optic mapping module is used to convert the spatiotemporal state of particles into acousto-optic control parameters; wherein, light control parameters are generated based on the particle's position information, and sound control parameters are generated based on the particle's motion state. The spatial audio module is used to receive sound control parameters and the three-dimensional spatial position of particles in real time, and to perform real-time three-dimensional spatial rendering of sound effect parameters to generate a three-dimensional sound field signal. The environment adaptation module is used to dynamically adjust the three-dimensional sound field signal through the audio rendering algorithm so that the output audio effect can adapt to different physical spaces. The user interaction module allows users to customize the physical and sound parameters of fireworks and supports real-time preview of the fireworks explosion effect.

[0007] Furthermore, the physics engine module specifically refers to a particle physics engine accelerated by a GPU.

[0008] Furthermore, the spatiotemporal state of the particle includes: the particle's position and motion state; The motion states include: the particle's trajectory, explosion pattern, and particle diffusion.

[0009] Furthermore, the lighting control parameters include: the brightness, color, and flashing frequency of the light; The sound control parameters include: sound effect volume, pitch, timbre, and spatial position.

[0010] Furthermore, the spatial audio module employs high-order Ambisonic encoding and HRTF rendering technology.

[0011] Furthermore, the environment adaptation module collects ambient reflected sound through a microphone array and dynamically adjusts the three-dimensional sound field signal through an audio rendering algorithm.

[0012] Furthermore, the user interaction module has a visual parameter adjustment panel.

[0013] According to specific embodiments provided by the present invention, the present invention discloses the following technical effects: This invention allows users to customize physical and sound parameters through a user interaction module. Combined with a physics engine module, it abstracts fireworks units into particle systems and generates real-time spatiotemporal states, accurately simulating the entire explosion process involving physical factors such as gravity and drag. This solves the problem of traditional solutions relying too heavily on fixed keyframes, resulting in mechanical animations and a lack of dynamic details. The sound and light mapping module innovatively establishes a deep correlation between the spatiotemporal state of particles and sound and light control parameters. It uses position information to drive the lights and motion state to drive the sound, ensuring a high degree of unity between visual effects and physical motion, as well as real-time synchronization of sound and light. The spatial audio module, combined with an environment adaptation module, uses audio rendering algorithms to perform real-time 3D sound field rendering and dynamic environment adaptation based on the 3D spatial position of particles. This not only accurately presents the Doppler effect during ascent and the all-round diffusion of the explosion sound, but also allows the output sound effects to adapt to different physical spaces, effectively overcoming the limitations of traditional mono or stereo technology in spatial representation. The entire system supports real-time preview and flexible creation by users, while comprehensively improving the interactive freedom, physical accuracy, and auditory stereo effect of the performance, realizing a comprehensive upgrade of fireworks from physical simulation to sensory presentation. Attached Figure Description

[0014] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0015] The physical engine and mapping system for the dynamic simulation of electronic fireworks sound and light of the present invention will be further described below with reference to the accompanying drawings. Figure 1 This is a schematic diagram showing the connection of various modules in the physics engine and mapping system of the electronic fireworks sound and light dynamic simulation of the present invention. Detailed Implementation

[0016] The specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples. The following examples are for illustrative purposes only and are not intended to limit the scope of the invention.

[0017] To better understand the purpose, structure, and function of this invention, the invention will be described in further detail below with reference to the accompanying drawings.

[0018] This invention provides a physics engine and mapping system for dynamic simulation of the sound and light of electronic fireworks, comprising: The module includes a physics engine module, an audio-visual mapping module, a spatial audio module, an environment adaptation module, and a user interaction module. The physics engine module receives user-input physical and sound parameters for the fireworks, abstracts each fireworks unit into a particle system, and generates the spatiotemporal state of each particle to simulate the physical phenomena of the fireworks explosion process in real time. The particle motion state includes the trajectory of the fireworks particles, the explosion pattern, and particle diffusion. The physics engine module employs a GPU-accelerated particle physics engine.

[0019] The particle system includes: ① Ascending particles: affected by initial thrust, gravity, and air resistance, their parabolic trajectories are calculated. ② Exploding particles: based on preset explosion force parameters, they diffuse outwards in a spherical pattern (velocity decay conforms to 1 / r). 2 (Law). The physical parameters are adjustable (gravity coefficient, air density, explosion radius).

[0020] In this embodiment, the hardware of the physics engine module is NVIDIA Jetson AGX Orin (GPU computing power 200TOPS); the software is a dedicated physics simulator modified from the Unity engine (with integrated particle system plugin). The software part specifically uses the Runge-Kutta method to solve the algorithmic techniques used to simulate the particle motion trajectory of fireworks explosion, so as to achieve low latency and improve the efficiency of software and hardware resources. The Runge-Kutta method is essentially a technique that indirectly uses the Taylor series method, specifically as follows: Examine the difference According to the mean value theorem, there exists , so that: ; Using the given equation get: ; Here This is called an interval. The average slope on.

[0021] Euler's Formula Simply take points slope value As the average slope value The accuracy is naturally very low. The improved Euler formula... and It can be rewritten in the following average form: ; The improved Euler formula can be understood as: using and slope values ​​at two points and Take the arithmetic mean as the average slope ,and slope value at Then through book information To make predictions.

[0022] Learn from this process: If we manage to... Predict the slope values ​​at several points, then take a weighted average of them to obtain the average slope. This allows for the construction of calculation formulas with higher accuracy. This is the basic idea behind the Runge-Kutta method.

[0023] By generalizing and improving the Yura method, we can obtain the second-order Runge-Kutta method, the third-order Runge-Kutta method, and the fourth-order Runge-Kutta method (derivation omitted). Only the calculation formula for the fourth-order Runge-Kutta method is given here: ; Each step of the fourth-order Runge-Kutta method requires four calculations of the function value. It can be proven that its truncation error is The proof is rather complicated, so I will not go into details.

[0024] Looking at each step individually, the smaller the step size, the smaller the truncation error. However, as the step size decreases, the number of steps required to complete a certain solution range increases. The increase in the number of steps not only introduces a larger computational load but may also lead to a serious accumulation of rounding errors. To balance the contradiction between accuracy and computational load, the Runge-Kutta method with variable step size can be obtained by doubling or halving the step size.

[0025] The sound-light mapping module is used to convert the spatiotemporal state of particles into sound-light control parameters. Specifically, it generates light control parameters based on the particle's position information and sound control parameters based on the particle's motion state. The sound control parameters include the volume, pitch, timbre, and spatial position of the sound effect. The light control parameters include the brightness, color, and flashing frequency of the light. The sound-light mapping module specifically outputs the particle's spatiotemporal state using the DMX512 protocol to the LED driver circuit.

[0026] The spatial audio module is used to receive sound control parameters and the three-dimensional spatial position of particles in real time, and to perform real-time three-dimensional spatial rendering of sound effect parameters to generate a three-dimensional sound field signal. The spatial audio module employs high-order Ambisonic encoding and HRTF rendering technology. It supports 8-channel output DSP processors (such as TITAS6424); real-time data is transmitted via TSN (Time-Sensitive Network) to ensure that the audio-visual synchronization error is <5ms.

[0027] In this embodiment, for each particle's spatial coordinates (x, y, z) (where x represents the forward / backward displacement, y represents the left / right displacement, and z represents the vertical displacement, in meters), it is converted into a B-format four-channel signal using the following formula: W=√2 / 2·I / r 2 (W represents the omnidirectional pressure field component, I represents the particle acoustic intensity 0-1, r=√(x) 2 +y 2 +z 2 (where ) is the distance from the particle to the listener. X = x / r·I / r 2 (X represents the forward and backward direction components); Y = y / r·I / r 2 (Y represents the left-right component); Z = z / r·I / r 2 (Z represents the vertical component); When r < 1 meter, perform near-field correction on the W component: W = W·(0.5 + 0.5 / r).

[0028] Among them, HRTF personalization processing: Using the MIT KEMAR HRTF database (containing measurement data for 45 head shapes), the matching was achieved through the following steps: (1) The user uploads a photo of the auricle and extracts a 6-dimensional feature vector, such as the height of the concha cavity H and the width of the helix W. (2) Calculate the Euclidean distance between the feature vector and the database sample: d = √∑(Fi - Di) 2 (Fi represents user features, and Di represents database sample features); (3) Select the 5 HRTF data sets with the smallest distance and perform weighted fusion, with weight w=1 / (d+ε); Multi-channel output mapping: For a standard 5.1.4 Dolby Atmos system, the decoding matrix is:

[0029] Where M is a 10×4 decoding matrix, and its key elements include: L = 0.707W + 0.5X + 0.5Y (left front); R = 0.707W - 0.5X + 0.5Y (Right front); Tfl=0.707W+0.25Z (left top position); Other channels are configured according to similar rules to ensure energy conservation in all directions.

[0030] In summary, the connection relationships between the steps are as follows: The particle physics engine outputs coordinates (x, y, z) in real time → a B-format encoder generates W, X, Y, and Z signals → an HRTF convolution processor (4 sets of FIR filters for each of the left and right ears) → multi-channel decoding matrix allocation → a power amplifier drives the speaker array. The entire process latency is controlled within 8.2ms, including 0.2ms for coordinate encoding, 5ms for HRTF convolution, and 3ms for decoding and mapping.

[0031] The environment adaptation module dynamically adjusts the three-dimensional sound field signal through an audio rendering algorithm, so that the output audio effect can adapt to different physical spaces; the environment adaptation module collects ambient reflected sound through a microphone array, specifically: a 4-microphone circular array (sampling rate 48kHz).

[0032] The user interaction module has a visual parameter adjustment panel, which allows users to customize the physical and sound parameters of the fireworks and supports real-time preview of the effect. Specifically, it is a 7-inch touch screen (supporting gesture zooming of the fireworks trajectory).

[0033] The system in this invention also has the following technical effects: (1) The physical realism is significantly improved. Through the GPU-accelerated particle physics engine, the nonlinear dynamic characteristics of fireworks explosion (such as shock wave diffusion and random particle motion) are accurately simulated. Compared with traditional linear interpolation animation, the realism is improved by more than 300% (based on user perception test). The sound and light parameters are dynamically bound. The spatial distribution of the explosion particles directly drives the sound pressure level and spectral characteristics of the sound effect, achieving millimeter-level synchronization accuracy (the error of traditional solutions is ≥50ms). (2) Accurate three-dimensional sound field reproduction, using 7th-order Ambisonic encoding + HRTF personalized rendering, supports the audience to perceive the precise location of the fireworks sound source at any position within a 20m×20m area (horizontal error ≤2°), overcoming the positioning ambiguity problem of stereo technology. In addition, an environmental adaptive algorithm is used to automatically compensate for the reverberation differences of different venues (such as outdoor vs. indoor) through real-time acoustic feedback, reducing manual sound adjustment time by 90%; (3) Breakthrough improvement in computational efficiency: Parallel particle computing is adopted, achieving a processing speed of 1500 particles / millisecond on NVIDIA Jetson AGX Orin, which is 20 times faster than the CPU serial solution, and supports the simultaneous rendering of 50 independent firework units. Resource consumption has been optimized, with the physics engine memory usage controlled within 500MB (Unity native particle system requires ≥2GB), making it suitable for embedded device deployment; (4) Revolutionary simplification of user interaction: The design features a visual parameter panel, allowing users to adjust parameters such as explosive force and particle lifetime by dragging sliders (e.g., "explosive intensity" corresponds to a Newton force value range of 10-50N), significantly improving design efficiency. Real-time preview function is provided, showing the effect change within 0.5 seconds after parameter modification, without the need to recompile the code (traditional DMX systems require minutes of waiting time). (5) Energy saving, environmental protection and cost optimization, power consumption reduction, the GPU acceleration solution reduces energy consumption by about 70% compared with server-level CPU clusters (actual system operating power ≤150W). Almost zero consumables, completely digital sound and light generation, no need for gunpowder, smoke agents and other polluting materials, completely eliminating PM2.5 and noise pollution of traditional fireworks; this invention, through the innovative combination of physics engine + spatial audio + low latency architecture, comprehensively surpasses existing sound and light synchronization technology in terms of artistic expression, technical performance, user experience and environmental protection, setting a new standard for the field of electronic fireworks.

[0034] The above description of the disclosed embodiments enables those skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An electronic fireworks sound, light dynamic simulation physical engine and mapping system, characterized in that, include: The module includes a physics engine module, an audio-visual mapping module, a spatial audio module, an environment adaptation module, and a user interaction module. The physics engine module is used to receive the physical parameters and sound effect parameters of the fireworks input by the user, and to abstract each firework unit into a particle system, generating the spatiotemporal state of each particle to simulate the physical phenomena of the fireworks explosion process in real time. The acousto-optic mapping module is used to convert the spatiotemporal state of particles into acousto-optic control parameters; wherein, light control parameters are generated based on the particle's position information, and sound control parameters are generated based on the particle's motion state. The spatial audio module is used to receive sound control parameters and the three-dimensional spatial position of particles in real time, and to perform real-time three-dimensional spatial rendering of sound effect parameters to generate a three-dimensional sound field signal. The environment adaptation module is used to dynamically adjust the three-dimensional sound field signal through the audio rendering algorithm so that the output audio effect can adapt to different physical spaces. The user interaction module allows users to customize the physical and sound parameters of fireworks and supports real-time preview of the fireworks explosion effect.

2. The physics engine and mapping system for dynamic simulation of electronic fireworks sound and light according to claim 1, characterized in that, The physics engine module is specifically a particle physics engine accelerated by a GPU.

3. The physics engine and mapping system for dynamic simulation of electronic fireworks sound and light according to claim 1, characterized in that, The spatiotemporal state of the particle includes: the particle's position and motion state; The motion states include: the particle's trajectory, explosion pattern, and particle diffusion.

4. The physics engine and mapping system for dynamic simulation of electronic fireworks sound and light according to claim 1, characterized in that, The lighting control parameters include: brightness, color, and flashing frequency of the light; The sound control parameters include: sound effect volume, pitch, timbre, and spatial position.

5. The physics engine and mapping system for dynamic simulation of electronic fireworks sound and light according to claim 1, characterized in that, The spatial audio module employs high-order Ambisonic encoding and HRTF rendering technology.

6. The physics engine and mapping system for dynamic simulation of electronic fireworks sound and light according to claim 1, characterized in that, The environment adaptation module collects ambient reflected sound through a microphone array and dynamically adjusts the three-dimensional sound field signal through an audio rendering algorithm.

7. The physics engine and mapping system for dynamic simulation of electronic fireworks sound and light according to claim 1, characterized in that, The user interaction module has a visual parameter adjustment panel.

Images