AI adaptive ADSR envelope dynamic modulation method and system in spatial audio scene

Abstract

The invention provides an AI adaptive ADSR envelope dynamic modulation method and system in a spatial audio scene. The method comprises the following steps: acquiring spatial information; inputting the acquired spatial information parameters into a lightweight AI model so as to predict ADSR envelope parameters; according to the ADSR envelope parameter predicted by the lightweight AI model, dynamically generating an ADSR parameter suitable for the current spatial audio scene, and performing real-time amplitude modulation processing on the audio signal; and continuously optimizing the parameter mapping relation of the lightweight AI model through a reinforcement learning network based on user feedback and the change condition of the space environment so as to adjust the ADSR parameter to the preference requirement required by the user. According to the invention, a system based on spatial perception-AI decision-ADSR dynamic modulation-auditory feedback optimization is constructed, and intelligent cooperation of spatial audio ADSR envelope, spatial position, environmental characteristics and user preference is realized, so that friendly experience and immersion of a user to spatial audio are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of audio signal processing, in particular to an AI adaptive ADSR envelope dynamic modulation method and system in a spatial audio scene, which can be applied to virtual musical instruments, VR / AR, games, smart homes and other spatial audio scenes. By real-time sensing of spatial position, environmental acoustic characteristics and user preferences, the timbre envelope parameters of audio are dynamically optimized to improve immersion and auditory experience. BACKGROUND

[0002] Traditional spatial audio technology mainly achieves spatial rendering by adjusting volume, azimuth angle and direct / reflected sound ratio, which has the following defects: Timbre envelope and spatial information are split: ADSR parameters (attack, decay, sustain, release time) are fixed and cannot dynamically change with sound source position, motion speed or environmental acoustic characteristics (such as reverberation time RT60, signal-to-noise ratio SNR), resulting in a lack of immersion when the sound position changes but the timbre remains unchanged.

[0003] Lack of user adaptive mechanism: the differences in perception of spatial audio by different users and personal preferences are not considered, and existing technologies cannot adjust parameter mapping rules in real time according to user feedback.

[0004] Limited computing resources: mobile devices have limited computing power, and traditional complex audio processing models cannot run in real time. SUMMARY

[0005] The present application provides an AI adaptive ADSR envelope dynamic modulation method and system in a spatial audio scene, which can solve the problems of split ADSR envelope and spatial information, fixed modulation parameters, and inability to adapt to user auditory preferences in traditional audio. A system based on "spatial perception-AI decision-ADSR dynamic modulation-auditory feedback optimization" is constructed, realizing the intelligent collaboration of spatial audio ADSR envelope and spatial position, environmental characteristics, and user preferences, thereby improving the user-friendly experience and immersion of spatial audio.

[0006] The present application achieves the above-mentioned purposes through the following technical solutions: An AI adaptive ADSR envelope dynamic modulation method in a spatial audio scene, comprising: obtaining spatial information, which at least includes the distance between the sound source and the listener, the azimuth angle, the motion speed, and the environmental acoustic parameters; inputting the obtained spatial information parameters into a lightweight AI model to predict ADSR envelope parameters, which at least include attack time attack, decay time decay, sustain level sustain, and release time release; According to the ADSR envelope parameters predicted by the lightweight AI model, the ADSR parameters suitable for the current spatial audio scene are dynamically generated, and real-time amplitude modulation processing is performed on the audio signal; Based on user feedback and changes in the spatial environment, the parameter mapping relationship of the lightweight AI model is continuously optimized through the reinforcement learning network to adjust the ADSR parameters to the user's desired preference requirements.

[0007] According to the AI adaptive ADSR envelope dynamic modulation method in the spatial audio scene provided by the application, when obtaining spatial information, an ultra-wideband positioning device is used to obtain distance information between the sound source and the listener; The azimuth angle and motion state data of the sound source are collected by using an inertial measurement unit sensor, wherein the azimuth angle data is used to determine the spatial direction of the sound source relative to the listener, and the motion state data is used to reflect the moving speed and trajectory of the sound source; The environmental acoustic parameters are obtained by additional acoustic sensors or based on existing audio signal analysis, and at least include reverberation time, signal-to-noise ratio, which are used to characterize the acoustic characteristics of the space environment; The distance, azimuth angle, motion speed and environmental acoustic parameters collected by the ultra-wideband positioning device, inertial measurement unit sensor and acoustic sensor are fused to form a complete spatial information set.

[0008] According to the AI adaptive ADSR envelope dynamic modulation method in the spatial audio scene provided by the application, the construction of the lightweight AI model includes the following steps: Input layer construction: design the input layer to receive the preprocessed spatial information features, which include the distance normalized value between the sound source and the listener, the azimuth angle normalized value, the motion speed normalized value and the standardized value of the environmental acoustic parameters, to form a fixed-dimensional input vector; CNN layer design: connect the CNN layer after the input layer, and use one-dimensional convolution structure to extract spatial feature relationships; biLSTM layer integration: connect the biLSTM layer after the CNN layer, which is a bidirectional long short-term memory network, which uses its bidirectional time series processing capability to capture the dynamic characteristics of the spatial information changing with time; Fully connected output layer construction: connect the fully connected output layer after the biLSTM layer, which contains 4 neurons, respectively corresponding to the prediction output of the attack, decay, sustain and release parameters of the ADSR envelope; Among them, the transfer learning strategy is adopted, and the initial training of the model is completed based on a large-scale pre-training data set on the server side; through the phased training method of freezing the CNN layer parameters and only training the biLSTM and fully connected layers, the model convergence is accelerated and overfitting is prevented.

[0009] According to the AI adaptive ADSR envelope dynamic modulation method in a spatial audio scene provided by the application, the normalized spatial information features and user preference features are spliced into a 12-dimensional input vector in the input layer; the input vector is input into the CNN layer after normalization processing, so as to ensure that each feature dimension is in the same numerical range; In the CNN layer, a single-layer one-dimensional convolution is used to extract features from the input vector, the convolution kernel size is 3*1, the step is 1, and the output channel number is 8; the static correlation features between the spatial parameters are captured through the convolution operation, and the spatial feature map is generated; In the biLSTM layer, the time sequence features are captured, the spatial feature map output by the CNN layer is unfolded according to the time step and input into the biLSTM layer; wherein the number of biLSTM hidden layer units is set to 32, the dynamic features of the spatial information changing with time are captured through forward and backward propagation, and the time sequence feature vector is output; In the full connection layer, the time sequence feature vector output by the biLSTM layer is input into the full connection layer, the layer includes 4 neurons, respectively corresponding to the attack, decay, sustain and release parameters of the ADSR envelope; the full connection layer adopts a linear activation function and directly outputs the normalized ADSR parameter prediction value; The normalized ADSR parameter prediction value is converted into the actual ADSR parameter according to the preset dynamic mapping rule; When deployed on a mobile terminal, the reinforcement learning network dynamically fine-tunes the full connection layer weight according to the real-time feedback of the user, so as to realize the personalized adaptation of the ADSR parameter.

[0010] According to the AI adaptive ADSR envelope dynamic modulation method in a spatial audio scene provided by the application, the dynamic mapping rule of the ADSR envelope parameter is realized by the following formula: attack time = attack base × (1 + Σ(w i × F i )) Wherein, attack base is the reference value of the original attack time of the audio signal; F i is the normalized spatial feature variable, including distance D, azimuth A and motion speed V, each variable is normalized to the interval [0, 1]; w i is the weight coefficient corresponding to the spatial feature variable, and the specific value range is: The distance weight w D ∈ [0.3, 0.5], representing the positive influence of the sound source distance on the attack time; The azimuth weight w A∈ [0.1, 0.3], representing the secondary positive influence of sound source direction on attack time; Speed weight w V ∈ [-0.4, -0.2], representing the negative influence of sound source movement on attack time; The model realizes dynamic adjustment of attack time with sound source position and movement state through linear combination of spatial feature variables, and the weight coefficients satisfy the constraint condition Σ|w i | ≤ 1 to maintain physical rationality.

[0011] According to the AI adaptive ADSR envelope dynamic modulation method in a spatial audio scene provided by the application, when dynamically generating ADSR parameters suitable for the current spatial audio scene, the normalized ADSR parameter prediction value output by the lightweight AI model is converted into actual physical values, including attack time, decay time, sustain level, and release time, through a linear mapping or nonlinear mapping function; wherein the ADSR parameters between consecutive frames are first-order low-pass filtered to eliminate audio distortion caused by parameter mutation.

[0012] According to the AI adaptive ADSR envelope dynamic modulation method in a spatial audio scene provided by the application, when performing real-time amplitude modulation processing on the audio signal, the following real-time amplitude modulation engine is realized: Envelope generator: using piecewise linear interpolation method to generate ADSR envelope curve in real time, updating control point coordinates once per frame; Multiplier module: multiply the input audio signal with the generated ADSR envelope curve point by point to realize amplitude modulation, the formula is: output_sample(t) = input_sample(t) × envelope_value(t) Wherein, envelope_value(t) is the ADSR envelope value at the current time; Insert an automatic gain control module after modulation to maintain the RMS level of the output signal within the target range; Generate independent ADSR modulation signals for multiple sound sources in space; Merge each sound source signal through a weighted mixing algorithm, the mixing formula is: mixed_signal = Σ(source_signal_i × (1 / (1 + 0.5×Di))) Wherein, Di is the distance parameter of the i-th sound source.

[0013] According to the AI adaptive ADSR envelope dynamic modulation method in a spatial audio scene provided by the application, reinforcement learning network is used to realize continuous optimization of ADSR parameter mapping rules, and the method specifically includes the following functional modules. The agent module includes a policy network and a parameter modulator, wherein the policy network is composed of a single fully connected layer and a ReLU activation function, the input is a 12-dimensional environment state vector, and the output is a discrete adjustment amount of four ADSR parameters; the parameter modulator applies the adjustment amount output by the policy network to the prediction result of the CNN+biLSTM model to generate the final ADSR parameter: ADSR_final = ADSR_CNN+biLSTM x (1 + ΔRL) Where ΔRL is the adjustment amount output by the policy network, and |ΔRL| is limited by a hard constraint ≤ 20% to avoid timbre mutation; The environment state module defines the input state vector as a 12-dimensional structure, including the following features: Spatial information: normalized distance, azimuth angle, velocity, RT60; Model prediction value: attack, decay, sustain, and release parameters output by the last round of CNN+biLSTM; User feedback: last round of user feedback label and volume adjustment amount; Environmental adaptation degree: audio distortion degree and reverberation matching degree; The action module defines the action space as a 20-dimensional discrete vector composed of 5-level adjustment amounts of four ADSR parameters: Action = [Δattack, Δdecay, Δsustain, Δrelease] Where each Δ ∈ {-20%, -10%, 0, 10%, 20%}, and the mobile terminal calculation complexity is reduced by discretization; The reward calculation unit designs a reward function as a multi-objective weighted sum and normalizes it to the [-1, 1] interval: Reward = w1·Ruser + w2·Renv + w3·Rstable Where Ruser is the user preference reward, Renv is the environment adaptation reward, and Rstable is the stability penalty: The reinforcement learning optimization module uses a lightweight deep Q network, and the network structure is input layer→single layer FC→output layer. The DQN is pre-trained based on historical user feedback data on the server side to generate initial weights. The weight of the last layer of FC is only fine-tuned at the mobile end, and the parameter quantity is controlled within 50kb, ensuring real-time performance.

[0014] An AI adaptive ADSR envelope dynamic modulation system in a spatial audio scene comprises: A spatial information acquisition module is configured to acquire the distance, azimuth, motion speed of a sound source and a listener, and environmental acoustic parameters; An AI model prediction module adopts a hybrid structure of a lightweight CNN and a biLSTM, receives spatial information and predicts ADSR envelope parameters; An amplitude modulation module performs real-time amplitude modulation on an audio signal according to the predicted ADSR envelope parameters; A reinforcement learning optimization module continuously optimizes the parameter mapping relationship of the AI model based on user feedback and environmental changes through a deep Q network.

[0015] According to the AI adaptive ADSR envelope dynamic modulation system in a spatial audio scene provided by the application, the reinforcement learning optimization module comprises: An agent module comprises a parameter modulator and a policy network, and is configured to generate an adjustment amount of ADSR parameters; An environmental state module provides spatial information, environmental acoustic parameters, user interaction states and distortion degrees as decision feedback; A reward calculation unit calculates a reward value according to user preferences, environmental adaptability and stability penalties; The reinforcement learning optimization module adopts a deep Q network for offline pre-training and online fine-tuning to optimize the parameter mapping relationship of the AI model.

[0016] Therefore, compared with the prior art, the AI adaptive ADSR envelope dynamic modulation method and system in a spatial audio scene provided by the application significantly improve the immersion, personalized adaptation capability and computing efficiency of spatial audio, and have the following beneficial effects: 1、The application forms a mapping between audio and spatial information, including parameters such as azimuth, distance, motion speed, so that users have a better sense of immersion in spatial audio and a better effect of restoring natural audio in space.

[0017] 2、The application provides an AI decision module that uses a lightweight prediction model to intelligently output ADSR target parameters in combination with user preference features to adapt to space and user needs.

[0018] 3、The application dynamically optimizes the model based on user feedback, introduces reinforcement learning, and real-time corrects the model weight according to the scoring mechanism marked by the interactive interface, such as "timbre adaptation degree", so that the ADSR parameters gradually match the user's hearing preferences.

[0019] The application will be described in further detail below with reference to the drawings and specific embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0020] Figure 1 is a flow chart of an embodiment of an AI adaptive ADSR envelope dynamic modulation method in a spatial audio scene according to the application.

[0021] Figure 2 is a construction principle diagram of a lightweight AI model in an embodiment of an AI adaptive ADSR envelope dynamic modulation method in a spatial audio scene according to the application.

[0022] Figure 3 is a principle diagram of an embodiment of an AI adaptive ADSR envelope dynamic modulation system in a spatial audio scene according to the application.

[0023] Figure 4 is a principle diagram of a reinforcement learning optimization module in an embodiment of an AI adaptive ADSR envelope dynamic modulation system in a spatial audio scene according to the application. DETAILED DESCRIPTION

[0024] In order to make the objectives, technical solutions and advantages of the present application clearer, the technical solutions in the present application will be described clearly and completely below with reference to the drawings in the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0025] Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are they necessarily mutually exclusive of one another. It is explicitly understood that the embodiments described herein can be combined with each other, implicitly or explicitly, when possible.

[0026] An AI adaptive ADSR envelope dynamic modulation method in a spatial audio scene Referring to Figure 1 The embodiment provides an AI adaptive ADSR envelope dynamic modulation method in a spatial audio scene, which comprises the following steps. Step S1, acquiring spatial information, the spatial information at least comprising a distance between a sound source and a listener, an azimuth angle, a motion speed and an environmental acoustic parameter; Step S2, input the obtained spatial information parameters into a lightweight AI model to predict ADSR envelope parameters, including at least attack time, decay time, sustain level, and release time; Step S3, dynamically generate ADSR parameters suitable for the current spatial audio scene according to the ADSR envelope parameters predicted by the lightweight AI model, and perform real-time amplitude modulation processing on the audio signal; Step S4, based on user feedback and changes in the spatial environment, continuously optimize the parameter mapping relationship of the lightweight AI model through a reinforcement learning network to adjust the ADSR parameters to the user's desired preferences.

[0027] In the above step S1, when obtaining spatial information, an ultra-wideband positioning device is used to obtain distance information between the sound source and the listener; The azimuth angle and motion state data of the sound source are collected by an inertial measurement unit sensor, where the azimuth angle data is used to determine the spatial direction of the sound source relative to the listener, and the motion state data is used to reflect the moving speed and trajectory of the sound source; The environmental acoustic parameters are obtained by additional acoustic sensors or based on existing audio signal analysis, including at least reverberation time and signal-to-noise ratio, which are used to represent the acoustic characteristics of the spatial environment; The distance, azimuth angle, motion speed and environmental acoustic parameters collected by the ultra-wideband positioning device, inertial measurement unit sensor and acoustic sensor are fused to form a complete set of spatial information.

[0028] In the above step S2, as shown in Figure 2 The construction of the lightweight AI model includes the following steps: Input layer construction: design the input layer to receive preprocessed spatial information features, including distance normalized values, azimuth angle normalized values, motion speed normalized values, and standardized values of environmental acoustic parameters (such as RT60, SNR, etc.), forming a fixed-dimensional input vector; CNN layer design: connect the CNN layer after the input layer, use one-dimensional convolution structure to extract spatial feature relationships, where the convolution kernel size and number are optimized to balance feature extraction ability and computational complexity, ensuring real-time operation on mobile devices; This layer captures static correlation features between distance, azimuth angle and other spatial parameters through convolution operation; biLSTM layer integration: A bidirectional long short-term memory network, i.e., a biLSTM layer, is connected after the CNN layer. It utilizes its bidirectional temporal processing capability to capture the dynamic features of spatial information changing over time, thus overcoming the limitations of traditional LSTM unidirectional modeling. The number of hidden units in the biLSTM layer has been optimized to maximize the efficiency of temporal information capture under parameter constraints. Construction of the fully connected output layer: A fully connected output layer is connected after the biLSTM layer. This layer contains 4 neurons, which correspond to the predicted output of the four parameters of ADSR envelope: attack, decay, sustain, and release. The activation function of the output layer is a linear function to ensure that the parameter values ​​are continuously adjustable. Among them, a transfer learning strategy was adopted to complete the initial training of the model on the server side based on a large-scale pre-training dataset (100,000 time-series samples covering different distances, orientations, speeds and environmental conditions); by freezing the CNN layer parameters and training only the biLSTM and fully connected layers in a phased training method, the model convergence was accelerated and overfitting was prevented. Furthermore, by using model structure compression (such as replacing standard convolution with depthwise separable convolution), parameter pruning, and quantization techniques, the overall number of parameters can be strictly controlled to within 500kb to meet the deployment requirements of mobile devices; the final mean squared error (MSE) loss value of the model on the validation set is less than 0.001 (normalized ADSR parameters).

[0029] In the input layer, the standardized spatial information features (distance D, azimuth A, motion speed V, environmental acoustic parameter RT60 / SNR) and user preference features (historical feedback labels, volume adjustment) are concatenated into a 12-dimensional input vector; the input vector is then normalized and fed into the CNN layer to ensure that each feature dimension is within the same numerical range. In the CNN layer, a single-layer one-dimensional convolution (1D CNN) is used to extract features from the input vector. The convolution kernel size is 3×1, the stride is 1, and the number of output channels is 8. The convolution operation captures the static correlation features between spatial parameters such as distance and azimuth, such as the interaction between distance and azimuth, and generates a spatial feature map. In the biLSTM layer, temporal feature capture is performed by expanding the spatial feature map output by the CNN layer by time step and inputting it into the biLSTM layer. The number of hidden units in the biLSTM layer is set to 32. The forward and backward propagation are used to capture the dynamic features of spatial information changing over time and output temporal feature vectors. In the fully connected layer, the temporal feature vector output by the biLSTM layer is input into the fully connected layer. This layer contains 4 neurons, which correspond to the four parameters of ADSR envelope: attack, decay, sustain, and release. The fully connected layer uses a linear activation function to directly output the normalized ADSR parameter prediction values ​​(range [0,1]). The normalized predicted values ​​are converted into actual ADSR parameters according to a preset dynamic mapping rule; Limiters constrain parameter ranges, such as attack time ≥ 0ms and sustain level ≤ 0dB, to prevent outliers from causing audio distortion. When deployed on mobile devices, reinforcement learning networks dynamically fine-tune the weights of fully connected layers based on real-time user feedback to achieve personalized adaptation of ADSR parameters.

[0030] In this embodiment, the dynamic mapping rule for the ADSR envelope parameters is implemented using the following formula: attack time = attack base × (1 + Σ(w i × F i )) Among them, attack base Fi represents the baseline value of the original attack time of the audio signal; Fi represents the standardized spatial feature variables, including distance D, azimuth angle A, and motion speed V, with each variable normalized to the [0,1] interval; wi represents the weighting coefficients of the corresponding spatial feature variables, which are optimized and determined through acoustic experiments and subjective listening tests, and their specific value range is as follows: Distance weight w D ∈[0.3, 0.5] (example value 0.4), representing the positive impact of sound source distance on attack time; Azimuth weight w A ∈[0.1, 0.3] (example value 0.2), representing the minor positive influence of the sound source direction on the attack time; Speed ​​weight w V [∈-0.4, -0.2] (example value -0.3), characterizing the negative impact of sound source motion on attack time; For example, the dynamic mapping rule for ADSR envelope parameters is expressed as: attack time = attack base × (1+0.4D+0.2A 0.3V) Where base is the original value, D is the standardized distance, A is the standardized azimuth angle, and V is the standardized velocity.

[0031] This weighted model achieves dynamic adjustment of the attack time as the sound source position and motion state by linearly combining spatial feature variables, and the weighting coefficients satisfy the constraint condition Σ|w i | ≤ 1 to maintain physical plausibility; The mapping rules for the other ADSR parameters (decay, sustain, release) adopt a linear weighted structure similar to that of the attack parameters, but the weight coefficients are set independently according to acoustic principles. All parameter mapping results are constrained within a reasonable physical range by limiters (e.g., attack time ≥ 0ms) to avoid outliers causing audio distortion.

[0032] In step S3 above, the normalized ADSR parameters (range [0,1]) output by the lightweight AI model are converted into actual physical values ​​through a linear or nonlinear mapping function, including attack time, decay time, sustain level, and release time. In this process, the ADSR parameters between consecutive frames are subjected to first-order low-pass filtering to eliminate audio distortion caused by parameter abrupt changes.

[0033] Real-time amplitude modulation processing of audio signals is achieved through the following real-time amplitude modulation engine: Envelope generator: The piecewise linear interpolation method is used to generate ADSR envelope curves in real time, and the control point coordinates are updated once every frame (e.g., 10ms). Multiplier module: Multiplies the input audio signal point by point with the generated ADSR envelope curve to achieve amplitude modulation, using the following formula: output_sample(t) = input_sample(t) × envelope_value(t) Where envelope_value(t) is the ADSR envelope value at the current time; After modulation, an automatic gain control (AGC) module is inserted to maintain the RMS level of the output signal within the target range, such as -20dBFS to -6dBFS, to avoid volume fluctuations. Generate independent ADSR modulation signals for multiple sound sources in space; The signals from various sound sources are combined using a weighted mixing algorithm, such as distance-based attenuation weighting. The mixing formula is as follows: mixed_signal =Σ(source_signal_i × (1 / (1 + 0.5×Di))) Where Di is the distance parameter of the i-th sound source.

[0034] When deployed on mobile devices, the ARM NEON instruction set or GPU parallel computing is used to accelerate envelope generation and multiplication operations; fixed-point arithmetic is used to replace floating-point arithmetic, reducing computational complexity by more than 40% to meet real-time requirements, with frame processing time ≤5ms.

[0035] In step S4 above, a reinforcement learning network is used to continuously optimize the ADSR parameter mapping rule, which specifically includes the following functional modules and implementation steps: The agent module includes a policy network and a parameter modulator. The policy network consists of a single fully connected layer and a ReLU activation function. Its input is a 12-dimensional environment state vector, and its output is discrete adjustments for four ADSR parameters, each ∈ {-20%, -10%, 0, 10%, 20%}. The parameter modulator applies these adjustments from the policy network to the prediction results of the CNN+biLSTM model to generate the final ADSR parameters. ADSR_final = ADSR_CNN+biLSTM×(1 +ΔRL) Wherein, ΔRL is the adjustment amount of the policy network output, and is limited by hard constraints to |ΔRL| ≤ 20% to avoid abrupt changes in timbre; The environment state module, whose input state vector is defined as a 12-dimensional structure, contains the following features: Spatial information (4D): Normalized distance, azimuth, velocity, RT60; Model predictions (4 dimensions): attack, decay, sustain, and release parameters output by the previous CNN+biLSTM round; User feedback (2-dimensional): Previous round of user feedback labels (positive +1.0 / no feedback 0 / negative -1.0), volume adjustment amount (not adjusted +0.2 / increase +0.1 / decrease -0.1); Environmental adaptability (2D): audio distortion (≤1%), reverberation matching (based on RT60 difference normalized to [-0.5, 0.5]); The action module, whose action space is defined as a 20-dimensional discrete vector, consists of a combination of 5 levels of adjustment values ​​for 4 ADSR parameters: Action = [Δattack, Δdecay, Δsustain, Δrelease] In this case, each Δ∈{-20%, -10%, 0, 10%, 20%} is discretized to reduce the computational complexity on mobile devices; The reward calculation unit uses a reward function designed as a multi-objective weighted sum, normalized to the interval [-1, 1]. Reward = w1·Ruser + w2·Renv + w3·Rstable Ruser is a reward system for user preferences: +1.0 for positive feedback, -1.0 for negative feedback, and 0 for no feedback; +0.2 for no volume adjustment, +0.1 for increasing volume, and -0.1 for decreasing volume. Renv is the environment adaptation reward: RT60 matching degree: the smaller the difference between the ADSR release time and the environment RT60, the higher the reward (range [-0.5, 0.5]); SNR matching degree: +0.3 when the modulated SNR>30dB, -0.3 when <20dB; Rstable is the stability penalty: if the ADSR adjustment exceeds ±20%, it is considered an extreme adjustment (|ΔRL|>20%): -0.2; if the audio distortion is >1%: -0.3; Finally, the reward needs to be normalized to the (-1, +1) range to avoid gradient explosion.

[0036] The reinforcement learning optimization module employs a lightweight deep Q-network (DQN), with the following network structure: Input layer (12-dimensional) → Single-layer fully connected (64 neurons) → Output layer (20-dimensional action Q-value) The training mode is offline pre-training + online fine-tuning: On the server side, DQN is pre-trained based on historical user feedback data to generate initial weights; On mobile devices, only the weights of the last FC layer are fine-tuned, with the number of parameters kept within 50kb to ensure real-time performance. Further, initialize the DQN network weights and load the pre-trained model from the server. Real-time acquisition of environmental state vectors on mobile devices, input into DQN to obtain the optimal action (ADSR adjustment amount). The final ADSR parameters are generated by the parameter modulator, and audio amplitude modulation is performed. Rewards are calculated based on user feedback and environmental adaptability, and the Q-value table of DQN is updated. User interaction data is regularly uploaded to the server for model iteration and optimization.

[0037] Furthermore, the spatial coordinates (azimuth, distance, and height) of each sound source are collected in real time using a microphone array or spatial positioning sensor, and the spatial distance between any two sound sources is calculated.

[0038] when d ij ≤ D threshold(When the preset space is close to a threshold, such as 1 meter), the sound source is determined. i With the sound source j When in a state of spatial proximity, the dynamic ADSR cooperative modulation mechanism is triggered.

[0039] Priority weights are assigned based on the type of sound source (speech, music, sound effects). W type Among them, speech weight W voice ≥ W music ≥ W effec t; Personalized weighting adjustment coefficients are generated by combining user historical preference data (such as explicit feedback ratings and implicit operation records). α (0.8≤) α ≤1.2), the final priority weight is W = α · W type.

[0040] For the sound source with the highest priority weight (such as speech), increase the sustain level in its ADSR parameter. L sustain The formula is adjusted as follows: L sustain ′= L sustain ·(1+ β · W ) in, β The enhancement coefficient is 0.1 < β <0.3; For sound sources with lower priority (such as ambient sound effects), shorten their release time. T release The formula is adjusted as follows: T release ′= T release ·(1 gamma ·(1 W )) in, gamma The suppression coefficient is 0.2 < gamma <0.5; When the priority weights of multiple sound sources are close to (Δ W When <0.1), priority is given to ensuring the sustain level of high-frequency sound sources (such as human voices), while stronger release time suppression is applied to low-frequency sound sources (such as drum sounds).

[0041] Collect explicit feedback (such as "clear / fuzzy" rating) or implicit feedback (such as volume adjustment behavior) from users through the interactive interface. The reward function is designed based on a dynamic priority weight allocation strategy using reinforcement learning algorithms (such as DQN): R = R clarity + lambda R stability in, R clarity The clarity bonus is positively correlated with user feedback. R stability As a stability penalty, it is negatively correlated with the magnitude of change in the ADSR parameter. lambda The balance coefficient, 0 < lambda <0.5.

[0042] It is evident that by using spatial proximity detection and priority weight allocation, the sustain level of the main sound source (such as dialogue) is ensured to be prominent, while the release time of background sound sources (such as wind sound) is shortened, thereby reducing the spectral masking effect.

[0043] An embodiment of an AI adaptive ADSR envelope dynamic modulation system in a spatial audio scenario like Figure 3 As shown, an AI adaptive ADSR envelope dynamic modulation system for spatial audio scenarios includes: The spatial information acquisition module is used to obtain the distance, azimuth, speed of movement, and environmental acoustic parameters between the sound source and the audience. The AI ​​model prediction module adopts a lightweight CNN and biLSTM hybrid structure to receive spatial information and predict ADSR envelope parameters. The amplitude modulation module performs real-time amplitude modulation on the audio signal based on the predicted ADSR envelope parameters. The reinforcement learning optimization module continuously optimizes the parameter mapping relationship of the AI ​​model through a deep Q-network based on user feedback and environmental changes.

[0044] In this embodiment, as Figure 4 As shown, the reinforcement learning optimization module includes: The agent module, which includes a parameter modulator and a policy network, is used to generate adjustments to the ADSR parameters. The environmental status module provides spatial information, environmental acoustic parameters, user interaction status, and distortion as decision feedback. The reward calculation unit calculates the reward value based on user preferences, environmental adaptability, and stability penalties. The reinforcement learning optimization module uses a deep Q-network for offline pre-training and online fine-tuning to optimize the parameter mapping relationship of the AI ​​model.

[0045] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0046] The above embodiments are merely preferred embodiments of the present invention and should not be construed as limiting the scope of protection of the present invention. Any non-substantial changes and substitutions made by those skilled in the art based on the present invention shall fall within the scope of protection claimed by the present invention.

Claims

1. A method for AI adaptive ADSR envelope dynamic modulation in spatial audio scenarios, characterized in that, include: Acquire spatial information, which includes at least the distance between the sound source and the listener, azimuth angle, speed of movement, and environmental acoustic parameters. The acquired spatial information parameters are input into a lightweight AI model to predict ADSR envelope parameters, which include at least attack time, decay time, sustain level, and release time. Based on the ADSR envelope parameters predicted by the lightweight AI model, ADSR parameters suitable for the current spatial audio scene are dynamically generated, and the audio signal is subjected to real-time amplitude modulation processing. Based on user feedback and changes in the spatial environment, the parameter mapping relationship of the lightweight AI model is continuously optimized through a reinforcement learning network to adjust the ADSR parameters to meet the user's desired preferences.

2. The method according to claim 1, characterized in that: When acquiring spatial information, an ultra-wideband positioning device is used to obtain distance information between the sound source and the audience; Using inertial measurement unit sensors, the azimuth angle and motion state data of the sound source are collected. The azimuth angle data is used to determine the spatial direction of the sound source relative to the listener, while the motion state data is used to reflect the moving speed and trajectory of the sound source. Environmental acoustic parameters are obtained through additional acoustic sensors or based on analysis of existing audio signals, including at least reverberation time and signal-to-noise ratio, and are used to characterize the acoustic properties of the space environment. The distance, azimuth, speed, and environmental acoustic parameters collected by ultra-wideband positioning devices, inertial measurement unit sensors, and acoustic sensors will be fused to form a complete set of spatial information.

3. The method according to claim 1, characterized in that, Building a lightweight AI model involves the following steps: Input layer construction: The input layer is designed to receive preprocessed spatial information features, which include normalized values ​​of the distance between the sound source and the listener, normalized values ​​of the azimuth angle, normalized values ​​of the motion speed, and standardized values ​​of environmental acoustic parameters, forming a fixed-dimensional input vector. CNN layer design: A CNN layer is connected after the input layer, and a one-dimensional convolutional structure is used to extract spatial feature relationships; biLSTM layer integration: A bidirectional long short-term memory network, i.e., a biLSTM layer, is connected after the CNN layer to capture the dynamic features of spatial information changing over time by utilizing its bidirectional temporal processing capabilities. Construction of the fully connected output layer: The biLSTM layer is followed by a fully connected output layer, which contains 4 neurons, corresponding to the predicted output of the four parameters of ADSR envelope: attack, decay, sustain, and release. Among them, a transfer learning strategy is adopted to complete the initial training of the model on the server side based on a large-scale pre-trained dataset; by freezing the CNN layer parameters and training only the biLSTM and fully connected layers in a phased training method, the model convergence is accelerated and overfitting is prevented.

4. The method according to claim 3, characterized in that: In the input layer, the standardized spatial information features and user preference features are concatenated into a 12-dimensional input vector; the input vector is then normalized and fed into the CNN layer to ensure that each feature dimension is within the same numerical range. In the CNN layer, a single-layer one-dimensional convolution is used to extract features from the input vector. The convolution kernel size is 3×1, the stride is 1, and the number of output channels is 8. The convolution operation captures the static correlation features between spatial parameters and generates a spatial feature map. In the biLSTM layer, temporal feature capture is performed by expanding the spatial feature map output by the CNN layer by time step and inputting it into the biLSTM layer. The number of hidden units in the biLSTM layer is set to 32. The forward and backward propagation are used to capture the dynamic features of spatial information changing over time and output temporal feature vectors. In the fully connected layer, the temporal feature vector output by the biLSTM layer is input into the fully connected layer. This layer contains 4 neurons, which correspond to the four parameters of ADSR envelope: attack, decay, sustain, and release. The fully connected layer uses a linear activation function to directly output the normalized ADSR parameter prediction values. The normalized ADSR parameter prediction values ​​are converted into actual ADSR parameters according to a preset dynamic mapping rule; When deployed on mobile devices, the weights of the fully connected layers are dynamically fine-tuned based on real-time user feedback through a reinforcement learning network, enabling personalized adaptation of ADSR parameters.

5. The method according to claim 4, characterized in that, The dynamic mapping rule for ADSR envelope parameters is implemented through the following formula: attack time = attack base × (1 + Σ(w i × F i )) Among them, attack base F is the baseline value for the original attack time of the audio signal; i The standardized spatial characteristic variables, including distance D, azimuth A, and velocity V, are all normalized to the interval [0,1]. i These are the weighting coefficients for the corresponding spatial feature variables, and their specific value range is as follows: Distance weight w D ∈ [0.3, 0.5], representing the positive impact of sound source distance on attack time; Azimuth weight w A ∈ [0.1, 0.3], representing the minor positive influence of the sound source direction on the attack time; Speed ​​weight w V ∈ [-0.4, -0.2], representing the negative impact of sound source motion on attack time; The model achieves dynamic adjustment of the attack time as the sound source position and motion state by linearly combining spatial feature variables, and the weighting coefficients satisfy the constraint Σ|w i | ≤ 1 to maintain physical plausibility.

6. The method according to claim 1, characterized in that: When dynamically generating ADSR parameters suitable for the current spatial audio scene, the normalized ADSR parameter prediction values ​​output by the lightweight AI model are converted into actual physical values ​​through a linear or nonlinear mapping function, including attack time, decay time, sustain level, and release time. Among them, the ADSR parameters between consecutive frames are subjected to first-order low-pass filtering to eliminate audio distortion caused by parameter abrupt changes.

7. The method according to claim 6, characterized in that, Real-time amplitude modulation processing of audio signals is achieved through the following real-time amplitude modulation engine: Envelope generator: The ADSR envelope curve is generated in real time using piecewise linear interpolation, and the control point coordinates are updated once per frame; Multiplier module: Multiplies the input audio signal point by point with the generated ADSR envelope curve to achieve amplitude modulation, using the following formula: output_sample(t) = input_sample(t) × envelope_value(t) Where envelope_value(t) is the ADSR envelope value at the current time; An automatic gain control module is inserted after modulation to maintain the RMS level of the output signal within the target range; Generate independent ADSR modulation signals for multiple sound sources in space; The signals from each sound source are combined using a weighted mixing algorithm. The mixing formula is as follows: mixed_signal = Σ(source_signal_i × (1 / (1 + 0.5×Di))) Where Di is the distance parameter of the i-th sound source.

8. The method according to any one of claims 1 to 7, characterized in that, A reinforcement learning network is used to continuously optimize the ADSR parameter mapping rule, which includes the following functional modules: The agent module includes a policy network and a parameter modulator. The policy network consists of a single fully connected layer and a ReLU activation function, taking a 12-dimensional environment state vector as input and outputting discrete adjustments for four ADSR parameters. The parameter modulator applies these adjustments from the policy network to the prediction results of the CNN+biLSTM model to generate the final ADSR parameters. ADSR_final = ADSR_CNN+biLSTM×(1 +ΔRL) Wherein, ΔRL is the adjustment amount of the policy network output, and is limited by hard constraints to |ΔRL| ≤ 20% to avoid abrupt changes in timbre; The environment state module, whose input state vector is defined as a 12-dimensional structure, contains the following features: Spatial information: normalized range, azimuth, velocity, RT60; Model predictions: the attack, decay, sustain, and release parameters output by the previous round of CNN+biLSTM; User feedback: Previous round of user feedback tags and volume adjustment; Environmental adaptability: audio distortion, reverb matching; The action module, whose action space is defined as a 20-dimensional discrete vector, consists of a combination of 5 levels of adjustment values ​​for 4 ADSR parameters: Action = [Δattack, Δdecay, Δsustain, Δrelease] In this case, each Δ∈{-20%, -10%, 0, 10%, 20%} is discretized to reduce the computational complexity on mobile devices; The reward calculation unit uses a reward function designed as a multi-objective weighted sum, normalized to the interval [-1, 1]. Reward = w1·Ruser + w2·Renv + w3·Rstable Where Ruser is the user preference reward, Renv is the environment adaptation reward, and Rstable is the stability penalty: The reinforcement learning optimization module adopts a lightweight deep Q-network with a network structure of input layer → single-layer FC → output layer. On the server side, DQN is pre-trained based on historical user feedback data to generate initial weights; On mobile devices, only the weights of the last fully connected (FC) layer are fine-tuned, with the number of parameters kept below 50kb to ensure real-time performance.

9. An AI adaptive ADSR envelope dynamic modulation system for spatial audio scenarios, characterized in that, include: The spatial information acquisition module is used to obtain the distance, azimuth, speed of movement, and environmental acoustic parameters between the sound source and the audience. The AI ​​model prediction module adopts a lightweight CNN and biLSTM hybrid structure to receive spatial information and predict ADSR envelope parameters. The amplitude modulation module performs real-time amplitude modulation on the audio signal based on the predicted ADSR envelope parameters. The reinforcement learning optimization module continuously optimizes the parameter mapping relationship of the AI ​​model through a deep Q-network based on user feedback and environmental changes.

10. The system according to claim 9, characterized in that, The reinforcement learning optimization module includes: The agent module, which includes a parameter modulator and a policy network, is used to generate adjustments to the ADSR parameters. The environmental status module provides spatial information, environmental acoustic parameters, user interaction status, and distortion as decision feedback. The reward calculation unit calculates the reward value based on user preferences, environmental adaptability, and stability penalties. The reinforcement learning optimization module uses a deep Q-network for offline pre-training and online fine-tuning to optimize the parameter mapping relationship of the AI ​​model.

Images