Audio signal adaptive adjustment method and device, electronic equipment and storage medium

Abstract

The invention relates to the technical field of audio adjustment, and provides an audio signal adaptive adjustment method and device, electronic equipment and a storage medium. The method comprises the following steps: performing joint feature extraction on environmental noise data and audio signals to obtain multi-dimensional feature data, performing loudness gain prediction according to the multi-dimensional feature data to obtain a predicted loudness gain value, and performing loudness simulation on the audio signals to obtain predicted loudness. Adaptive filtering and gain calculation are carried out on the predicted loudness to obtain a real-time loudness compensation value, and data fusion is carried out on current scene information and user historical adjustment data to obtain a user preference gain value; and performing gain optimization on the audio signal according to the predicted loudness gain value, the user preference gain value and the real-time loudness compensation value to obtain a gain optimized audio signal. According to the invention, through multi-dimensional feature fusion, deep learning time sequence prediction, real-time adaptive compensation and user habit learning, accurate adaptive adjustment of the audio signal is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of audio adjustment, and in particular to an audio signal adaptive adjustment method and device, electronic equipment and a storage medium. BACKGROUND

[0002] Audio signal adjustment is crucial to improve the user experience of an audio system. It not only ensures the stability and comfort of audio output in different environments, but also avoids auditory fatigue or information omission caused by excessively large or small loudness. In traditional loudness adjustment methods, one of the most commonly used methods is the Automatic Gain Control (AGC) technology.

[0003] The AGC technology mainly adjusts the output gain by monitoring the intensity of the input signal in real time, so that the overall loudness is maintained at a relatively constant level. This method is simple and easy to implement, and is widely used in many portable devices and communication systems. However, traditional AGC often relies on a single signal intensity indicator for gain adjustment, and cannot distinguish the influence of different frequency bands and multiple source noises, which can easily lead to inaccurate adjustment or distortion in a noisy environment. At the same time, it lacks analysis of user habits and scene information, and cannot dynamically optimize according to the actual listening needs of the user, thereby affecting the overall auditory experience. SUMMARY

[0004] Therefore, the present application provides an audio signal adaptive adjustment method, device, electronic equipment and storage medium to solve the problem of lack of fine adjustment of complex environments and personalized needs.

[0005] The first aspect of the present application provides an audio signal adaptive adjustment method, the method comprising: joint feature extraction processing of detected environmental noise data and received audio signals to obtain multi-dimensional feature data, and loudness gain prediction processing of a double-layer LSTM network according to the multi-dimensional feature data to obtain a predicted loudness gain value; loudness simulation processing of the audio signal to obtain a predicted loudness, and adaptive filtering and gain calculation processing of the predicted loudness to obtain a real-time loudness compensation value; data fusion processing of detected current scene information and pre-set user historical adjustment data to obtain a user preference gain value; gain optimization processing of the audio signal according to the predicted loudness gain value, the user preference gain value and the real-time loudness compensation value to obtain a gain-optimized audio signal.

[0006] In an optional implementation, the multidimensional feature data includes audio features and environmental noise features; joint feature extraction processing of the detected environmental noise data and the received audio signal to obtain the multidimensional feature data includes: The detected environmental noise data and the received audio signal are subjected to frame segmentation and window function weighting to obtain weighted noise data and weighted audio data; The weighted noise data is subjected to Fast Fourier Transform and Wavelet Transform denoising processing to obtain noise spectrum data, and the weighted audio data is subjected to Fast Fourier Transform processing to obtain audio spectrum data. Power spectral density is calculated from the noise spectrum data to obtain energy spectral data, and short-time energy, zero-crossing rate, and Mel frequency cepstral coefficients are calculated from the audio spectrum data to obtain initial audio characteristics. The energy spectrum data is logarithmically transformed, and the transformed energy spectrum data and the initial audio features are normalized to obtain environmental noise features and audio features.

[0007] In an optional implementation, the loudness gain prediction processing using a two-layer LSTM network based on the multidimensional feature data to obtain the predicted loudness gain value includes: The multidimensional feature data are arranged into time-series feature data according to the time order; The temporal feature data is processed by a two-layer LSTM network using a preset deep learning model to predict the loudness gain value.

[0008] In an optional implementation, the step of performing loudness simulation processing on the audio signal to obtain predicted loudness, and performing adaptive filtering and gain calculation processing on the predicted loudness to obtain a real-time loudness compensation value includes: The audio spectrum data is processed for energy calculation and loudness simulation according to preset standard rules to obtain the predicted loudness. Based on the predicted loudness and the environmental noise characteristics, a frequency band difference compensation analysis is performed to obtain compensation requirement data between each frequency band. The compensation demand data is subjected to adaptive filtering and adjustment to obtain a compensation gain curve; The real-time loudness compensation value is obtained by calculating the gain of the compensation gain curve using a preset K-weighted filtering model.

[0009] In an optional implementation, the current scene information includes sensor datasets, current time information, and device status data; the step of performing data fusion processing on the detected current scene information and preset user historical adjustment data to obtain a user preference gain value includes: The sensor data is processed to extract physical environment features to obtain a set of physical environment parameters; The current time information and the device status data are subjected to time-domain feature extraction processing to obtain a time-domain status parameter set; The physical environment parameter set and the time domain state parameter set are normalized to obtain the scene parameter set. The preset historical user adjustment data is subjected to exponential weighting to obtain a smoothed historical adjustment value. The scene matching weight is obtained by performing scene matching on the scene parameter set and the smooth historical adjustment value through a preset user habit learning model. The user preference gain value is obtained by weighting and fusing the scenario matching weight and the historical adjustment weight in the user's historical adjustment data using the user habit learning model.

[0010] In an optional implementation, the step of performing gain optimization processing on the audio signal based on the predicted loudness gain value, the user preference gain value, and the real-time loudness compensation value to obtain a gain-optimized audio signal includes: The predicted loudness gain value, the user preference gain value, and the real-time loudness compensation value are weighted and averaged to obtain a preliminary comprehensive gain value. The preliminary integrated gain value is subjected to exponential smoothing and limiting processing to obtain the gain correction parameter; The audio spectrum data is divided into frequencies to obtain audio data for each frequency band. The audio data is weighted and optimized according to the gain correction parameters to obtain frequency domain gain adjustment data. The frequency domain gain adjustment data is subjected to inverse Fourier transform processing to obtain the gain-optimized audio signal.

[0011] A second aspect of this application provides an audio signal adaptive adjustment device, the device comprising: The sensor analysis module is used to perform joint feature extraction processing on the detected environmental noise data and the received audio signal to obtain multi-dimensional feature data, and to perform loudness gain prediction processing on the multi-dimensional feature data using a two-layer LSTM network to obtain the predicted loudness gain value. The prediction compensation module is used to perform loudness simulation processing on the audio signal to obtain predicted loudness, and to perform adaptive filtering and gain calculation processing on the predicted loudness to obtain real-time loudness compensation value. The preference gain module is used to perform data fusion processing on the detected current scene information and the preset user historical adjustment data to obtain the user preference gain value. The signal conditioning module is used to perform gain optimization processing on the audio signal based on the predicted loudness gain value, the user preference gain value, and the real-time loudness compensation value, so as to obtain a gain-optimized audio signal.

[0012] A third aspect of this application provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the audio signal adaptive adjustment method as described above.

[0013] A fourth aspect of this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the audio signal adaptive adjustment method as described above.

[0014] In summary, this application includes at least the following beneficial technical effects: 1. Joint feature extraction is performed on audio signal and environmental noise features to construct multi-dimensional temporal feature data. A two-layer LSTM network is then used for prediction processing, which can effectively capture the temporal dynamic changes in the audio signal and achieve accurate estimation of the predicted loudness gain value.

[0015] 2. The predicted loudness is obtained through loudness simulation processing, and adaptive filtering and gain calculation are performed by combining the K-weighted filtering model. The compensation gain curve can be calculated in real time according to the compensation requirements of each frequency band, thereby realizing dynamic and real-time adjustment of the audio signal and effectively mitigating the impact of environmental noise on audio playback effect.

[0016] 3. A user habit learning module has been introduced. By fusing sensor data, current time, device status and user historical adjustment data, scene parameters and user preferences are extracted to obtain user preference gain values. Based on objective noise and signal characteristics, it can also take into account the user's personalized experience, thereby improving the overall auditory comfort.

[0017] 4. By weighted smoothing and fusion of predicted loudness gain, user preference gain and real-time loudness compensation value, and by using exponential smoothing and limiting processing, the stability and continuity of gain correction parameters are ensured, abrupt changes and overshoots during the adjustment process are reduced, and the system can better adapt to dynamically changing environments, ensuring stable and optimized audio signal output. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart of an audio signal adaptive adjustment method provided in an embodiment of this application; Figure 2 This is a functional block diagram of an audio signal adaptive adjustment device provided in an embodiment of this application; Figure 3 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0020] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0021] like Figure 1 The diagram shows a flowchart of an audio signal adaptive adjustment method provided in an embodiment of this application. The audio signal adaptive adjustment method provided in this embodiment includes the following steps.

[0022] Step S1: Perform joint feature extraction processing on the detected environmental noise data and the received audio signal to obtain multidimensional feature data, and perform loudness gain prediction processing on the multidimensional feature data using a two-layer LSTM network to obtain the predicted loudness gain value.

[0023] It should be understood that the environmental noise data of the target area acquired by acoustic sensors (e.g., microphone arrays) is a continuously changing time-domain signal. In order to accurately analyze the noise spectrum characteristics and reduce the errors introduced by edge effects, the continuously acquired environmental noise signal is localized and smoothed by framing and window function weighting, thereby ensuring that each frame of data has good spectral characteristics during frequency domain transformation.

[0024] First, the continuously acquired environmental noise data is segmented into frames of fixed length. Frame segmentation effectively transforms the time-domain signal into local signals within several time windows, making the signal within each frame relatively stable, thus meeting the requirements of subsequent window function weighting and spectral analysis. Specifically, assuming the continuously acquired environmental noise signal is x(n) (where n is the index of the time-domain sampling point), a fixed length L (e.g., 1022 sampling points) needs to be selected as the length of each frame. Simultaneously, to reduce boundary effects between adjacent frames, a certain inter-frame overlap rate can be set. For example, an inter-frame overlap rate of 50% means that 50% of the data in each frame overlaps with the previous frame. Frame segmentation divides the entire environmental noise signal into a series of relatively stable local signals, with relatively stable signal characteristics within each frame, which is beneficial for subsequent time-frequency analysis.

[0025] Furthermore, to reduce spectral leakage caused by signal truncation during framing, each frame of data needs to be weighted using a window function. A window function is a function that weights a signal in the time domain, aiming to gradually attenuate the signal amplitude at the edges of each frame, thereby reducing spectral leakage introduced by truncation. Commonly used window functions include the Hamming window, Hanning window, and Blackman window. This embodiment uses the Hamming window function. The signal x of the m-th frame obtained through window function weighting is... w The (m,n) (i.e., weighted noise data) is processed by a selected window function w(n) (i.e., Hamming window) to smoothly transition the signal amplitude at frame edges to 0, thereby reducing spectral leakage caused by edge discontinuities during the FFT process. Window function weighting can effectively reduce boundary effects and avoid the spectral spread problem introduced by sudden signal truncation.

[0026] After obtaining the weighted noise data, to improve the signal-to-noise ratio of the spectral data, a Fast Fourier Transform (FFT) is needed to convert the weighted noise data into a frequency domain representation. The FFT converts a time-domain signal into a frequency-domain signal, obtaining the energy distribution of each frame of data across different frequency components. Simultaneously, since environmental noise data may contain transient interference or noise impulses, wavelet transform denoising is performed on the FFT-generated data to improve signal quality. This multi-scale denoising of the frequency domain data yields smoother spectral data that more accurately reflects the noise spectrum. For the weighted noise data x... w Performing a Fast Fourier Transform on (m,n) yields the frequency domain representation X(m,k). Specifically, for the weighted noise data x of the m-th frame... w (m,n) (n is the intra-frame sampling point number, ranging from 0 to L-1); (Number of sampling points per frame), calculated by comparing X(m,k) with the Fourier transform kernel function from n=0 to n=L-1. The sum of the products of (j represents the imaginary unit; k is the frequency component index, ranging from 0 to L-1) is used to obtain the first... Frame in The complex spectral values ​​X(m,k) at each frequency component are obtained. This converts the time-domain weighted noise data into frequency-domain data, thereby obtaining the amplitude and phase information of each frequency component.

[0027] To eliminate abnormal spectral components in frequency domain data that may be caused by transient interference, impulse noise, or other non-stationary signals, wavelet transform is applied to the frequency domain data after FFT transformation for denoising. Wavelet transform decomposes the signal into different scales, thresholds the coefficients at each scale, and then reconstructs the signal, thereby removing noise components. Specifically, assuming that the spectral data X(m,k) of the m-th frame is subjected to discrete wavelet transform, it is decomposed into coefficients {C} at several scales. m,s (k)} where S represents the scale index. During the denoising process, the wavelet coefficients are processed using a thresholding method as follows: Set a threshold T (which can be predetermined based on the signal-to-noise ratio), and assign coefficients C to each scale. m,s (k) Perform soft thresholding as shown in the following formula: Among them, C m,s (k) represents the original wavelet coefficients of the m-th frame at scale s. sgn(.) represents the sign function, which ensures that the signs of the processed coefficients are consistent with the original coefficients when thresholding the wavelet coefficients, as the original coefficients may be positive or negative. This maintains the polarity of the signal during subsequent inverse wavelet transform reconstruction of the denoised noise spectrum data, preventing erroneous changes to the signal's phase and other related information, and allowing the denoised signal to more accurately reflect the characteristics of the original signal. T is a preset threshold that determines the strictness of noise removal. Max(.,0) ensures the result is non-negative.

[0028] For each scale coefficient after thresholding Perform inverse wavelet transform to reconstruct the denoised noise spectrum data. .

[0029] It should be understood that noise spectrum data It contains the amplitude and phase information of each frame at each frequency component. By converting the frequency domain complex signal into energy distribution data, the strength distribution of the noise signal can be reflected more intuitively. Therefore, it is necessary to perform power spectral density calculation on the noise spectrum data to obtain energy spectrum data. Specifically, the denoised spectrum data is first processed... The complex spectral value is converted into a real number reflecting the energy of the corresponding frequency component by taking the square of the modulus. Then, it is normalized by dividing by the number of FFT points N, and finally the energy spectrum data P(m,k) of the m-th frame at frequency k is obtained. This more intuitively reflects the strength distribution of the noise signal and ensures that the energy spectrum numerical scale is reasonable and the overall value does not fluctuate greatly due to the frame length or the number of FFT points.

[0030] The calculated energy spectrum data P(m,k) reflects the energy information contained in each frequency component of each frame, providing the basic data for subsequent logarithmic transformation and normalization. Energy spectrum data can objectively describe the energy distribution of environmental noise signals in the frequency domain and serves as an important component of environmental noise characteristics.

[0031] Directly using linear energy spectrum data can easily lead to an excessively large dynamic range, affecting the stability of subsequent processing algorithms. Therefore, after obtaining the energy spectrum data P(m,k), logarithmic transformation and normalization are required. Logarithmic transformation compresses the data to a more reasonable range, making it more consistent with the human ear's perception of sound loudness. Normalization ensures consistency across different frames and frequency bands, providing a unified scale for data fusion and the input to deep learning models.

[0032] The logarithmic transformation employed in this application converts energy spectrum data from a linear to a logarithmic scale, compressing the dynamic range of the data and thus better aligning with human auditory perception characteristics. This also facilitates subsequent data fusion and comparison. Specifically, the energy spectrum data P(m,k) is first obtained, and then a base-10 logarithmic operation is performed on the energy spectrum data P(m,k) to obtain log... 10 (P(m,k)), and finally multiply the result of this logarithmic operation by a scaling factor of 10 to obtain the transformed logarithmic value L(m,k). Through this logarithmic transformation, the energy spectrum data can be transformed from a linear scale to a logarithmic scale, thereby compressing the dynamic range of the data and making it more consistent with the perceptual characteristics of the human ear. It also facilitates subsequent data fusion and comparison.

[0033] Furthermore, based on the maximum and minimum values ​​in the logarithmic data L(m,k), each data point in the logarithmic data L(m,k) is mapped to a fixed interval (e.g., [0,1]) to ensure a consistent numerical scale between data from different frames and frequency bands. Through logarithmic transformation and normalization of the energy spectrum data, the obtained environmental noise characteristics can objectively reflect the energy distribution of environmental noise in the frequency domain. Moreover, after logarithmic and normalization transformations, the data range is compressed and unified, providing standardized and highly comparable data input for subsequent fusion with audio signal characteristics.

[0034] It should be understood that the audio signal is the raw audio content input into the system, that is, the time-domain digitized audio data obtained through the audio acquisition module (such as an audio interface, microphone, or signal from a digital media player). Since both the audio signal and ambient noise data are time-varying and continuous signals, the system needs to segment the audio signal and ambient noise data into smaller segments, smooth them, and then convert them into frequency domain data in order to obtain stable and accurate time-frequency data. Therefore, both the audio signal and ambient noise data require framing, window function weighting, and Fast Fourier Transform (FFT) processing. The framing, window function weighting, and FFT processing of the audio signal will not be elaborated upon here; please refer to the above-mentioned framing, window function weighting, and FFT processing of the ambient noise data for details.

[0035] After the audio signal is processed by framing, window function weighting and fast Fourier transform to obtain audio spectrum data Y(m,k), multi-angle audio characteristics can be obtained by calculating short-time energy, zero-crossing rate and MFCC of the audio spectrum data. These characteristics play a key role in describing audio content, distinguishing speech from noise and subsequent pattern recognition.

[0036] Short-time energy reflects the energy level of an audio signal over a period of time and is an important indicator of signal strength. It should be understood that weighted audio data y... m (m,n) represents a discrete-time signal. This is achieved by weighting the audio data y for each frame. m By calculating the squared amplitude of (m,n), the received audio energy at each sampling point can be obtained. Finally, by summing the received audio energies at all sampling points, the short-time energy E(m) of the weighted audio data of that frame can be obtained.

[0037] Zero-crossing rate refers to the number of times an audio signal crosses the zero axis within a certain period of time, reflecting the signal's frequency characteristics and noise level. First, consider the weighted audio data y of the m-th frame. m (m,n) uses the sign function sgn(.) (input x, output 1 or -1, used to determine the sign of the sample point value) to obtain sgn(y m The absolute value of the difference between the sign function values ​​of adjacent sampling points (m, n) is calculated. If the absolute value of the difference is 2, it means that the sign of the adjacent sampling points has changed, that is, the signal has crossed the zero axis once; if it is 0, the sign has not changed. Then, the absolute values ​​of this difference are summed for all adjacent sampling points from n=1 to n=L-1 in the m-th frame, and then multiplied by 1 / 2L to normalize the statistical result to a suitable interval, finally obtaining the zero-crossing rate Z(m) of the m-th frame.

[0038] Mel-frequency cepstral coefficients are a commonly used feature in speech signal processing. Their calculation involves mapping spectral data to a Mel scale, followed by logarithmic and discrete cosine transforms. Specifically, the spectrum |Y(m,k)| is passed through a set of Mel filters to obtain the output energy S of each filter. j (m), thus performing a logarithmic operation on the output of each filter to obtain logS j (m) is used to transform the linear relationship of energy into a logarithmic relationship that is more consistent with human perception. Then, a discrete cosine transform is performed to obtain the Mel frequency cepstral coefficients. It should be understood that... This is the kernel function of the discrete cosine transform. By using logS... j The product of (m) and the kernel function is summed over the range j from 1 to N to obtain the j-th Mel frequency cepstral coefficient C in the m-th frame. j (m) is used to convert the energy of the logarithmic filter into cepstral coefficients, thereby describing the spectral envelope information of the audio.

[0039] The short-time energy E(m), zero-crossing rate Z(m), and Mel frequency cepstral coefficients {C} are used to calculate the short-time energy E(m), zero-crossing rate Z(m), and Mel frequency cepstral coefficients {C}. j (m)} serves as the initial audio features, each reflecting the time and frequency domain information of the signal, providing a foundation for subsequent standardization. The initial audio features can be expressed as: F init (m)=[E(m),Z(m),C1(m),……,C j (m)].

[0040] In obtaining the initial audio features F init Following (m), due to significant differences in the dimensions, value ranges, and dynamic ranges of different features, direct use may affect the stability of subsequent data fusion and deep learning models. Therefore, it is essential to normalize the initial features, mapping them to a uniform numerical range (e.g., [0,1]) to improve the comparability between features and the efficiency of model processing. For example, the maximum and minimum values ​​in the initial audio features are obtained. By calculating the difference between each frame value and the minimum value in the initial audio features, and the ratio between the difference between the maximum and minimum values, the normalization of all frame values ​​of the initial audio features is achieved. After normalization, all features are within the same numerical range, which helps avoid weight learning bias caused by different feature scales in subsequent deep learning models. This standardized data ensures that multidimensional feature data will not be distorted due to scale differences during fusion, thus making subsequent model training and data fusion more stable and reliable.

[0041] After obtaining standardized audio and environmental noise features, they need to be fused to construct a multi-dimensional feature dataset that comprehensively reflects both the audio content and the background noise. Data fusion aims to combine features from different sources according to certain rules to form a feature vector that simultaneously describes the intrinsic information of the audio signal and the influence of external noise. Data fusion can be either direct concatenation or weighted fusion. Direct concatenation connects audio and environmental noise features along the feature dimension. For example, it directly concatenates audio and environmental noise features of the same frequency band to form a higher-dimensional vector (i.e., multi-dimensional feature data). Weighted fusion assigns weights to the audio and noise features respectively and then sums them. By adjusting the corresponding weight coefficients, the proportion of audio signal and environmental noise in the fused features can be controlled, allowing the resulting multi-dimensional feature data to better reflect the combined impact of audio content and noise environment in real-world application scenarios.

[0042] By jointly processing audio features and environmental noise features, more comprehensive and accurate descriptive information can be obtained, enabling subsequent deep learning models and gain adjustment algorithms to simultaneously consider the dynamic changes in audio content and the influence of background noise. The fused multidimensional feature data includes not only the energy, spectral shape, and time-domain statistical indicators of the audio signal, but also the energy distribution information of environmental noise in each frequency band, providing a rich data input foundation for the entire relative loudness adaptive adjustment method.

[0043] Many sound signals exhibit strong time dependence. In audio content such as speech and music, instantaneous changes in the signal often carry rich information about prosody, rhythm, and emotion. Directly processing unordered frame data cannot reveal these temporal relationships. Time-series processing using multidimensional feature data is a prerequisite for deep learning models to accurately capture the temporal dynamics of input signals. Multidimensional feature data is typically generated by fusing features from audio signals and environmental noise; its structure is a matrix or vector sequence, with each frame containing several dimensions of feature information. Since the original multidimensional feature data may be stored frame by frame, directly inputting it into a deep learning model requires first arranging it chronologically and constructing it into continuous temporal feature data.

[0044] The processed data retains the sequential information of features for each frame, enabling the deep learning model to learn the trend of feature changes over time during subsequent temporal modeling. This process ensures that all frame data are correctly arranged according to the sampling order, without disrupting the temporal structure of the signal, laying a solid foundation for the subsequent input of the two-layer LSTM network. Through this processing, the temporal feature data contains both a static description of the fused audio and noise features and information about the dynamic evolution of the signal in the time dimension, which is crucial for capturing instantaneous loudness changes in audio signals.

[0045] After obtaining the temporal feature data, the first layer of the LSTM network is processed using a pre-defined deep learning model. Long Short-Term Memory (LSTM) is a special type of recurrent neural network with memory capabilities, suitable for processing sequence data with long-term dependencies. The purpose of the first LSTM layer is to capture short-term dynamic changes from the temporal features and extract preliminary temporal patterns.

[0046] For example, suppose the time series feature data is F sep ={f(1), ..., f(M)}. At each time step, the first LSTM unit receives the input vector and the hidden state h from the previous time step. t (t-1) and the memory cell state. The LSTM cell updates the hidden state and memory state at the current time step through a series of gating mechanisms (input gate, forget gate, and output gate). Specifically, input gate i... t f(t) controls how much of the current input information can be stored in the memory unit. This is determined by the current input f(t) and the hidden state h from the previous time step. t (t-1) is weighted (using the input weight matrix and the hidden state recursive weight matrix) and combined with a bias, then processed by an activation function to obtain a value between 0 and 1. This value determines how many of the current input candidate memories can enter the memory unit to update the state. Forget gate f t f(t) is used to determine how much information in the memory unit from the previous time step needs to be forgotten. Similarly, for the current input f(t), the previous hidden state h... t (t-1) undergoes weighting and biasing, and after activation, a value between 0 and 1 is obtained. This value represents the proportion of the memory unit's state retained from the previous time step; the smaller the value, the more information is forgotten. Output gate o t (t) controls how much information in the memory unit can be output to the current hidden state. For the current input f(t), the previous hidden state h... t(t-1) undergoes weighting, biasing, and activation function processing to obtain values ​​between 0 and 1, thereby adjusting the contribution of the memory unit state to the current hidden state. The candidate memory unit generates candidate memory content for the current input, which integrates the previous hidden state h of the current input f(t). t The information in (t-1), after being processed by the hyperbolic tangent function, outputs information ranging from -1 to 1, providing new candidate information for updating memory cells. Memory cell state c t (t) integrates the information retained by the memory unit from the previous time step (controlled by the forget gate) and the new candidate memory information currently input (controlled by the input gate). It is the core carrier of long-term memory in LSTM, recording key information of the sequence. Hidden state h t The output state h1(t) at the current time step is controlled by the output gate, which controls the output of the memory cell state. It carries the filtered, valid information for the current time step and is passed to the next time step for subsequent computation and feature transfer. By controlling the flow of information in the time dimension through a gating mechanism, useful information is retained while unnecessary parts are discarded, enabling the first-layer LSTM to capture the dynamic features of short-term changes in the input sequence. The output hidden state h1(t) after processing by the first-layer LSTM serves as the input for the next time step and also provides a preliminary temporal feature representation for the input of the second-layer LSTM. This process ensures the full transmission of instantaneous information in the time-series data, laying the foundation for subsequent, deeper temporal modeling.

[0047] While the first layer can capture short-term temporal changes, audio signals and background noise often exhibit patterns over longer time scales. Further processing of the first-layer LSTM network's output by the second-layer LSTM network integrates cross-frame information, captures long-term dependencies, and globally models the dynamic changes throughout the sequence. The main task of the second-layer LSTM network is to integrate and model the short-term temporal features extracted by the first layer at a deeper level, thereby capturing global dynamic trends and higher-level temporal patterns. This process helps enhance prediction capabilities and provides a more abstract feature representation for the final fully connected layer mapping. The second-layer LSTM network uses a similar gating mechanism to the first-layer LSTM network for updates. The computational processing of the second-layer LSTM network will not be elaborated upon here; please refer to the computational processing of the first-layer LSTM network for details. By further abstracting the output data of the first layer through the second-layer LSTM network, local temporal features are integrated into higher-level global temporal patterns. The second-layer LSTM can capture dependencies over longer time periods, resulting in hidden state data containing rich contextual information, which plays a decisive role in predicting the output.

[0048] While the hidden state data obtained after passing through a two-layer LSTM network contains rich temporal information, its high dimensionality makes it difficult to directly use as control parameters. The fully connected layer reduces these high-dimensional features to a scalar, which serves as the predicted loudness gain value, guiding subsequent audio signal gain optimization. This step ensures that the deep learning model transforms complex temporal features into a concise and intuitive control signal, facilitating real-time adjustment of the audio output based on this signal. After processing by the second LSTM layer, the resulting second-layer hidden state data already contains global temporal features extracted from the original multi-dimensional feature data. The next goal is to map this high-dimensional hidden state data to a specific predicted loudness gain value, a process implemented by the fully connected layer (also known as a dense layer). The fully connected layer multiplies the input vector with the weight matrix and adds a bias, thereby achieving feature transformation and dimensionality reduction. Assuming the hidden state of the second LSTM network at the last time step is h2(T), the calculation process of the fully connected layer can be represented by the following formula: Where A represents the predicted loudness gain value of the output. h2(T) is the hidden state vector of the second LSTM layer at the last time step, containing global information of the entire temporal feature. W out Let be the weight matrix of the fully connected layer, with a size of 1×n, where n is the dimension of h2(T). out The bias of the fully connected layer is a scalar. It maps high-dimensional temporal feature data to a single predicted loudness gain value A. The fully connected layer fuses all hidden state features according to certain weights through a linear combination operation, outputting a corrected gain value that represents the overall loudness adjustment requirement.

[0049] Step S2: Perform loudness simulation processing on the audio signal to obtain predicted loudness, and perform adaptive filtering and gain calculation processing on the predicted loudness to obtain real-time loudness compensation value.

[0050] Meanwhile, after preprocessing, the audio signal has been divided into frames, weighted by window functions, and subjected to Fast Fourier Transform to obtain audio spectrum data. This spectrum data reveals the amplitude and phase information of each frame of the signal at various frequencies, and is key data describing the frequency composition of the signal. In order to simulate the loudness of the audio signal according to preset standard rules, it is necessary to extract the energy information of each frequency band from the spectrum data and calculate the loudness according to international standards to obtain the predicted loudness value. The main reason for this process is that loudness is an important indicator reflecting the human ear's perception of sound intensity, and the energy value obtained directly from the time-domain signal often cannot accurately reflect the auditory characteristics of the human ear. Therefore, it is necessary to convert and calculate through spectrum data. The implementation method of this application adopts the improved ITU-RBS.1770 standard, and the energy calculation and loudness simulation processing are as follows: Predicted loudness value L p (m) is an indicator reflecting the overall loudness level of the m-th frame of the audio signal within the current time period, and is an important basis for subsequent gain adjustment. First, the energy contribution of the signal at frequency k is obtained by squared the amplitude of the complex spectral value Y(m,k) of the m-th frame at frequency index k. Then, the weighting coefficients K(k) of the K-weighted filter (used to simulate the sensitivity of the human ear to different frequencies, and its value varies at different frequencies) are multiplied by the energy contribution to achieve weighted processing of the energy at different frequencies, highlighting the energy influence of frequencies more sensitive to the human ear. Next, the weighted energy is calculated within the frequency index range from k1 to k2. Finally, 10.log 10 The operation converts the total weighted energy to a logarithmic scale because the human ear's perception of loudness follows a logarithmic response. This compresses a large range of energy values ​​into a smaller numerical range, better reflecting the perceptual patterns of the human ear. Finally, a constant term of -0.691 (a correction value specified in the ITU-RBS.1770 standard, used to adjust the calculation results to a suitable loudness range) is added to obtain the predicted loudness value L for the m-th frame. p (m).

[0051] After obtaining the predicted loudness, the next goal is to compare this predicted loudness with the ambient noise characteristics to determine how much compensation is needed in different frequency bands. The purpose of this step is to clarify the relative difference between the audio signal and background noise in each frequency band, providing compensation requirement data for subsequent adaptive filtering adjustments, thereby achieving ideal loudness adjustment of the audio signal in complex noise environments. Assume the standardized ambient noise characteristic data is denoted as N(m,k), where k represents the frequency index and m represents the frame index. To obtain the compensation requirement data, the predicted loudness L can be compared... p The difference between (m) and the energy levels of environmental noise characteristic data in each frequency band is determined. The comparison process between predicted loudness and environmental noise can be expressed by the following formula: Δ(m,k)=max{0,N(m,k)-L p (m)} Where Δ(m,k) represents the compensation requirement data for the m-th frame at frequency k, i.e., the portion of the environmental noise exceeding the predicted loudness. N(m,k) represents the normalized environmental noise characteristic data for the m-th frame at frequency k. L p (m) represents the overall predicted loudness value for the m-th frame. N(m,k)-L p (m) represents the difference between ambient noise and predicted loudness. By quantifying the impact of ambient noise on the audio signal in each frequency band, when the noise energy is higher than the predicted loudness, compensation adjustment is needed for that frequency band to achieve overall loudness balance.

[0052] In actual processing, the above calculations are performed for each frequency band of each frame, resulting in a two-dimensional matrix Δ(m,k) representing the compensation requirement data. This data matrix shows the additional energy of environmental noise relative to the audio signal at different frequencies, providing a basis for subsequent filtering and gain calculations. The reason for generating the compensation requirement data is that the energy difference between the audio signal and environmental noise directly affects the human ear's perception of sound; if the environmental noise is high, it is necessary to appropriately increase the output gain of the audio signal in that frequency band in order to maintain clarity against the noisy background.

[0053] It should be understood that the compensation demand data Δ(m,k) shows the portion of environmental noise exceeding the predicted loudness in each frequency band, reflecting the required compensation intensity for each band. To achieve smooth and continuous frequency compensation, the compensation demand data must undergo adaptive filtering to obtain a smooth compensation gain curve, which will guide subsequent gain calculations. Adaptive filtering primarily employs smoothing algorithms, such as moving average filtering or exponential smoothing, to eliminate abrupt changes in local data and the influence of random noise. The adaptive filtering process can be represented by the following formula: G comp (m,k)=λΔ(m,k)+(1-λ)G comp (m-1,k) Among them, G comp (m,k) represents the compensation gain obtained in the m-th frame after adaptive filtering at frequency k. Δ(m,k) is the compensation requirement data for the k-th frequency band of the current frame. G comp (m-1,k) represents the compensation gain of the previous frame in the same frequency band. λ is a smoothing coefficient, ranging from [0,1], used to control the weight ratio between new data and historical data. By exponentially smoothing the compensation demand data, the output compensation gain curve changes smoothly over time, avoiding abrupt changes caused by instantaneous noise fluctuations, while preserving long-term trends.

[0054] Exponential smoothing filtering eliminates drastic fluctuations in compensation data caused by short-term random noise, ensuring continuity and stability of the compensation gain obtained in each frame across each frequency band. The filtered compensation gain curve not only reflects the trend between the current and previous frames but also provides smoother, more accurate input data for subsequent gain calculations using the K-weighted filtering model. Adaptive filtering plays a crucial role in data smoothing and noise suppression throughout the entire processing flow, ensuring that the compensation gain curve is not affected by local outliers and accurately reflects the actual difference between environmental noise and predicted loudness.

[0055] Finally, the smoothed compensation gain curve is processed using a pre-defined K-weighted filtering model to calculate the final real-time loudness compensation value. The K-weighted filtering model is a processing method based on a second-order IIR filter structure. Its design aims to simulate the human ear's response to different frequencies, ensuring appropriate gain adjustment within a specific frequency range. Specifically, the compensation gain curve is fused with the original audio signal in the frequency domain, and gain calculations are performed for each frequency band according to the K-weighted filter design to generate the final compensation value. The transfer function of the K-weighted filter is set as follows, typically employing a second-order IIR filter structure divided into two stages: high-pass filtering and high-frequency enhancement.

[0056] High-pass filtering stage: Among them, H HP (z) represents the transfer function of the high-pass filtering stage. a1 and a2 are design coefficients that ensure the filter achieves the required high-pass characteristics in the low-frequency range. -1 This indicates the delay unit and describes the delay effect of discrete-time signals. High-pass filtering is mainly used to eliminate the adverse effects of low-frequency noise on compensation calculations.

[0057] High-frequency enhancement phase: Among them, H HE (z) represents the transfer function of the high-frequency enhancement stage. b0b2 are the forward coefficients of the filter. c1 and c2 are the feedback coefficients, which control the high-frequency enhancement characteristics of the filter. High-frequency enhancement amplifies the signal in the high-frequency range to compensate for the human ear's sensitivity to mid-to-high frequencies.

[0058] Finally, the compensation gain curve G comp (m,k) is combined with a K-weighted filter. The gain can be calculated using the following formula: G real (m,k)=H total (z).G comp (m,k) H total (z)=H HP (z)×H HE (z) Among them, H total (z) represents the overall transfer function of the k-weighted filter, which can be obtained by cascading high-pass and high-frequency enhancement stages. G real (m,k) represents the final real-time loudness compensation value at frequency k in the m-th frame. By segmenting and adjusting the signals in different frequency bands, the output compensation value can accurately reflect the characteristics of human hearing.

[0059] By using a K-weighted filtering model to calculate the gain of the compensation gain curve, the real-time loudness compensation value was obtained for each frame and frequency band, reflecting the gain required to compensate the audio signal under the current noise background. This value will serve as a control parameter in the subsequent audio signal adjustment process, ensuring that the loudness of the audio output remains optimal under complex environments.

[0060] Step S3: Perform data fusion processing on the detected current scene information and the preset user historical adjustment data to obtain the user preference gain value.

[0061] The physical environment data in the current scene information is collected by various sensors installed on the equipment. These sensors include distance sensors, light sensors, temperature sensors, humidity sensors, etc., and their collection results constitute a set of raw data used to describe the actual physical environment in which the equipment is located. In order for this raw data to be useful in subsequent processing, feature extraction must be performed to convert the data from different types of sensors into a unified format physical environment parameter set P=[p1,p2,……,p n Specifically, the raw data collected by each sensor undergoes preprocessing (such as filtering, noise reduction, and smoothing) to ensure data stability. For example, median filtering is used to remove outliers from the distance sensor output data; low-pass filtering is used to reduce environmental interference from the light sensor data. Based on the working principle and measurement unit of each sensor, the preprocessed data is standardized and mapped to the same numerical range (e.g., [0,1]). Finally, the standardized data from each sensor are arranged in a predetermined order to form a set of physical environment parameters.

[0062] Another key part of the current scene information comes from current time information and device status data. Current time information refers to the current time, date, and time period collected by the system; device status data includes battery level, processor load, network connection status, etc. Performing time-domain feature extraction on this data allows us to extract features describing the device's current operating state and usage scenario (i.e., a time-domain status parameter set), thus supplementing the subsequent construction of scene parameters. Specifically, current time information is obtained, for example, by reading the current hour, minute, and second through the system clock, and then discretizing the time into different time periods to form a time feature. For example, different time periods can be assigned different encoding values ​​(e.g., morning is encoded as 0.2, daytime as 0.5, and nighttime as 0.8). Device status data is obtained, such as reading battery level, battery voltage, and processor utilization, which are obtained directly from the device hardware status interface. Furthermore, each data item undergoes preprocessing and normalization. All normalized time-domain feature data are then arranged in a predetermined order to form a time-domain status parameter set T=[T1,T2,……,T…]. n ].

[0063] Furthermore, before fusing the obtained physical environment parameter set and the obtained temporal state parameter set, their respective data must first be normalized to ensure that both are within a uniform numerical range. The purpose of normalization is to eliminate differences in units and value ranges between different data sources, ensuring that each feature has similar importance during subsequent data fusion. After normalization, each feature is mapped to a value between 0 and 1, ensuring that different features have a uniform scale. After normalization, the normalized physical environment parameter set and temporal state parameter set are concatenated together in a fixed order to form the scene parameter set.

[0064] It should be understood that user historical adjustment data records information about users' manual adjustments to audio signals in different time periods and scenarios. This data may include multiple dimensions such as adjustment amplitude, adjustment frequency, and duration. Because historical data contains noise or outliers, exponential weighting can smooth the data, highlighting recent adjustments and reducing the impact of older data. Exponential weighting distributes the weights of historical data in a decreasing order over time, generating a smooth historical adjustment value as the basis for subsequent scenario matching. The specific details of exponential weighting are as follows: First, clarify the meaning of each parameter. Indicates the first The smoothed historical adjustment value for the frame or the current moment. Indicates the first The adjustment range or other relevant adjustment indicators of the next adjustment operation. This is the index weighting factor, with a value range of (0,1). The larger the value, the more emphasis is placed on newer data. This represents the total number of historical adjustment data. Indicates the first The weights of the next set of data are determined by the number of newer data points. Newer data has a higher weight. Indicators are assigned to each adjustment operation. Multiply by its corresponding weight Then, sum all these weighted results to obtain the numerator; at the same time, add all the weights... Add them together to get the denominator. Finally, divide the numerator by the denominator to get the smoothed historical adjustment value. This exponential weighting process can smooth user historical adjustment data that contains noise or outliers, highlighting recent adjustment behavior, reducing the impact of old data, and ensuring that the final smoothed historical adjustment value can truly reflect the user's recent adjustment trend. It avoids interference from random noise and provides a basis for subsequent scenario matching.

[0065] After obtaining the scene parameter set and smoothed historical adjustment values, a user habit learning model is used for scene matching. The purpose of this process is to calculate the similarity between the current scene and historical adjustment behaviors, thereby providing a basis for the final generation of user preference gain values. Scene matching typically uses Euclidean distance or other similarity metrics to quantify the difference between the current scene and historical scenes, and converts this into matching weights through a certain function. First, in this embodiment, Euclidean distance is used to calculate the difference between the current scene vector and the historical scene vector. By measuring the difference between the current scene and the historical average scene, a smaller value indicates that the scenes are closer, while a larger value indicates a greater difference between the scenes.

[0066] Next, the Euclidean distance is converted into matching weights using an exponential function mapping. The weight calculation can be expressed by the following formula: Where w represents the scene matching weight, with a value ranging from 0 to 1. d is the Euclidean distance. σ is the standard deviation parameter, used to control the impact of distance variations on the weight. exp(.) represents the exponential function. The distance is mapped to the matching weight using the exponential decay function; the smaller the distance, the higher the matching weight, and vice versa.

[0067] Scene matching weights reflect the degree of similarity between the current scene and historical data, providing a quantitative basis for subsequent weighted fusion.

[0068] Different users exhibit significant differences in volume adjustment behavior across various scenarios, and relying solely on current scenario data or historical adjustment data is insufficient to accurately reflect user preferences. A weighted fusion of these two data sets achieves a balance, reflecting both the correlation between the current scenario and historical behavior while fully utilizing the statistical characteristics of long-term user adjustments. After obtaining the scenario matching weight and the smoothed historical adjustment values, these two data sets are weighted and fused to generate a user preference gain value. The "historical adjustment weight" in the user's historical adjustment data represents a comprehensive reflection of historical adjustment behavior; it is typically obtained through statistical analysis and normalization of historical adjustment data, representing the average trend of volume adjustments made by users in various past scenarios. Fusing the scenario matching weight and the historical adjustment weight ensures that the final user preference gain value reflects both the degree of matching between the current scenario and historical data, and also incorporates the user's long-term adjustment preferences. The weighted fusion process can be expressed by the following formula: A user =γ.w+(1-γ).H(m) Among them, A userThis represents the final user preference gain value. `w` is the scene matching weight. `H(m)` is the smoothed historical adjustment value, also known as the historical adjustment weight. `γ` is the weighted fusion coefficient, ranging from [0,1], used to balance the influence of current scene matching and historical adjustment data. Current scene information and historical user behavior data are fused according to set weights to generate a gain value that comprehensively reflects user preferences. The weighted fusion process ensures that the generated user preference gain value considers both the current actual scene conditions and fully utilizes the trend information of historical adjustment data, providing personalized parameters for the subsequent adaptive adjustment of the entire system's audio signals.

[0069] Step S4: Perform gain optimization processing on the audio signal based on the predicted loudness gain value, the user preference gain value, and the real-time loudness compensation value to obtain a gain-optimized audio signal.

[0070] In an adaptive audio signal adjustment system, three key parameters are the predicted loudness gain, the user preference gain, and the real-time loudness compensation. These parameters reflect the characteristics of the audio signal itself, the user's personalized historical behavior, and the environmental noise compensation requirements, respectively. To comprehensively consider these three factors in subsequent steps, they first need to be weighted and averaged to generate a preliminary comprehensive gain value. This process uses a weighted average formula to linearly combine the parameters according to preset weights. The data from the three sources are combined based on their relative importance in the actual adjustment. For example, if the predicted loudness gain has a large proportion in the overall calculation, w1 can be set higher; conversely, if the user's historical adjustment data and environmental compensation have a more significant effect, the weights of w2 and w3 can be appropriately increased (where w1 + w2 + w3 = 1). The weighted average process achieves a balance among multiple dimensions of information, ensuring that the final comprehensive gain value accurately reflects the overall adjustment requirements.

[0071] After obtaining the initial overall gain value, further smoothing and limiting are required to eliminate short-term data fluctuations and prevent the output gain from exceeding the preset range. Exponential smoothing introduces a smoothing coefficient, combining the current gain value with the smoothed value from the previous moment, resulting in smoother output value changes. Limiting ensures the output gain remains within a safe and reasonable range, preventing excessively high or low audio output due to abnormal data. Exponential smoothing can be expressed by the following formula: G smooth (n)=α.G init (n)+(1-α).G smooth (n-1) Among them, G smooth (n) represents the overall gain value after smoothing at time n. init(n) represents the initial integrated gain value at the current moment. G smooth (n-1) represents the smoothed gain value from the previous time step. α is the smoothing coefficient, ranging from (0,1), which determines the relative weight of the new value and historical values. Exponential smoothing ensures the gain value has temporal continuity and smoothness, preventing instantaneous fluctuations in single-frame data from directly affecting the final output. Smoothing coefficient The choice of α determines the balance between response speed and smoothing effect. A larger α tends to respond quickly to the latest data, but the smoothing effect is weaker; a smaller α results in better output smoothness, but a slower response speed.

[0072] After exponential smoothing, amplitude limiting is performed to ensure the smoothed gain value G. smooth (n) Does not exceed the system's preset minimum and maximum gain values. Limiting can be expressed by the following formula: G final (n)=min{max{G smooth (n),G min},G max} Among them, G final (n) represents the final gain correction parameter after clipping. G smooth (n) represents the gain value after exponential smoothing. G min and G max These are the system's preset minimum and maximum gain limits, respectively. max{G smooth (n),G min} Show G smooth (n) and G min The larger value in G. max} indicates that the above result is compared with G. max Compare the values ​​and take the smaller one. Limiting ensures the output gain correction parameter is always within [G]. min G max Within the range.

[0073] After obtaining the final gain correction parameters, the original audio signal needs to be decomposed in the frequency domain to divide the spectral data into several frequency bands. The main purpose of frequency band processing is to group the audio signal according to different frequency ranges in the frequency domain, because the human ear has different sensitivities to different frequency bands, and the loudness adjustment requirements for each frequency band are also different. Frequency band processing can apply different gain corrections to the signal of each frequency band individually, ensuring the fineness and personalization of the overall audio adjustment. Assume that the audio spectrum data obtained after preprocessing and FFT is denoted as Y(n,k), where n is the frame index and k is the frequency index. The implementation steps of frequency band processing are as follows: According to the design requirements, the entire frequency range (e.g., from 0Hz to half the sampling rate) is divided into multiple frequency bands, each corresponding to a certain frequency interval. For example, low frequency band (20Hz-200Hz), mid frequency band (200Hz-2000Hz), and high frequency band (2000Hz-20000Hz), etc. For each frame n and each frequency band j, the data of all frequency components k in that frequency band are extracted to form the audio data Y of that frequency band. j (n). By dividing continuous spectral data into several subsets with specific frequency ranges, it is easier to perform gain adjustment processing in each frequency band separately.

[0074] After obtaining audio data for each frequency band, the audio data for each frequency band is weighted and optimized according to the gain correction parameters. This applies the pre-calculated gain correction parameters to the audio data of different frequency bands to adjust the signal amplitude in the frequency domain, ensuring the output audio signal reaches a predetermined loudness level in each frequency band. Specifically, the audio data for each frequency band is multiplied by the corresponding weight of the gain correction parameters. The weight allocation can be set according to the characteristics of the frequency bands, and different frequency bands may use different ratios. The processed results are then weighted and summed or directly output as frequency domain gain adjustment data. The weight differences between frequency bands need to be considered during the adjustment process. By multiplying the gain correction parameters by the preset frequency band weight coefficients before applying them to the audio data, appropriate compensation can be ensured for each frequency band.

[0075] Finally, the weighted and optimized frequency-domain audio data is restored to the time domain, generating the final gain-optimized audio signal. After weighted optimization, the audio data of each frequency band already contains frequency-domain information adjusted according to the gain correction parameters, but actual audio playback usually requires a time-domain signal. Therefore, an inverse Fourier transform is needed to convert the adjusted spectrum data back to a time-domain waveform signal. Specifically, the weighted and optimized audio data of each frequency band within each frame is combined to generate the entire frame's frequency-domain gain adjustment data. The combination method can be to concatenate the data of each frequency band together or to perform a weighted summation, making the entire spectrum complete and continuous. An inverse Fourier transform is applied to the combined frequency-domain data to restore the frequency-domain optimized data to a time-domain signal, thereby generating the final signal waveform suitable for audio output (i.e., the gain-optimized audio signal).

[0076] This application applies to the field of audio adjustment technology. It obtains multi-dimensional feature data by jointly extracting features from environmental noise data and audio signals. Based on this multi-dimensional feature data, it predicts loudness gain to obtain a predicted loudness gain value. It then performs loudness simulation on the audio signal to obtain a predicted loudness, and applies adaptive filtering and gain calculation to the predicted loudness to obtain a real-time loudness compensation value. Finally, it fuses current scene information and historical user adjustment data to obtain a user-preferred gain value. Based on the predicted loudness gain value, the user-preferred gain value, and the real-time loudness compensation value, it optimizes the audio signal's gain to obtain a gain-optimized audio signal. Through multi-level preprocessing, multi-dimensional feature fusion, deep learning temporal prediction, real-time adaptive compensation, and user habit learning, this application can effectively suppress environmental noise interference and optimize the listening experience according to personalized user needs, thereby maintaining stable and comfortable audio output in dynamic environments.

[0077] like Figure 2 The diagram shown is a functional block diagram of an audio signal adaptive adjustment device provided in an embodiment of this application.

[0078] In some embodiments, the audio signal adaptive adjustment device 2 may include multiple functional modules composed of computer program segments. The computer programs for each program segment in the audio signal adaptive adjustment device 2 may be stored in the server's memory and executed by at least one processor to perform (see details). Figure 1 (Description) The function of the audio signal adaptive adjustment method.

[0079] In this embodiment, the audio signal adaptive adjustment device 2 can be divided into multiple functional modules according to its function. These functional modules may include: a sensor analysis module 21, a prediction compensation module 22, a preference gain module 23, and a signal adjustment module 24. The term "module" in this invention refers to a series of computer program segments that can be executed by at least one processor and perform a fixed function, stored in memory. In this embodiment, the functions of each module will be detailed in subsequent embodiments.

[0080] The sensing analysis module 21 is used to perform joint feature extraction processing on the detected environmental noise data and the received audio signal to obtain multi-dimensional feature data, and to perform loudness gain prediction processing on the multi-dimensional feature data using a two-layer LSTM network to obtain the predicted loudness gain value.

[0081] In an optional implementation, the sensing analysis module 21 is specifically used for: The detected environmental noise data and the received audio signal are subjected to frame segmentation and window function weighting to obtain weighted noise data and weighted audio data; The weighted noise data is subjected to Fast Fourier Transform and Wavelet Transform denoising processing to obtain noise spectrum data, and the weighted audio data is subjected to Fast Fourier Transform processing to obtain audio spectrum data. Power spectral density is calculated from the noise spectrum data to obtain energy spectral data, and short-time energy, zero-crossing rate, and Mel frequency cepstral coefficients are calculated from the audio spectrum data to obtain initial audio characteristics. The energy spectrum data is logarithmically transformed, and the transformed energy spectrum data and the initial audio features are normalized to obtain environmental noise features and audio features.

[0082] In an optional implementation, the sensing analysis module 21 is further configured to: The multidimensional feature data are arranged into time-series feature data according to the time order; The temporal feature data is processed by a two-layer LSTM network using a preset deep learning model to predict the loudness gain value.

[0083] The prediction compensation module 22 is used to perform loudness simulation processing on the audio signal to obtain predicted loudness, and to perform adaptive filtering and gain calculation processing on the predicted loudness to obtain a real-time loudness compensation value.

[0084] In an optional implementation, the prediction compensation module 22 is specifically used for: The audio spectrum data is processed for energy calculation and loudness simulation according to preset standard rules to obtain the predicted loudness. Based on the predicted loudness and the environmental noise characteristics, a frequency band difference compensation analysis is performed to obtain compensation requirement data between each frequency band. The compensation demand data is subjected to adaptive filtering and adjustment to obtain a compensation gain curve; The real-time loudness compensation value is obtained by calculating the gain of the compensation gain curve using a preset K-weighted filtering model.

[0085] The preference gain module 23 is used to perform data fusion processing on the detected current scene information and the preset user historical adjustment data to obtain the user preference gain value.

[0086] In an optional implementation, the preference gain module 23 is specifically used for: The sensor data is processed to extract physical environment features to obtain a set of physical environment parameters; The current time information and the device status data are subjected to time-domain feature extraction processing to obtain a time-domain status parameter set; The physical environment parameter set and the time domain state parameter set are normalized to obtain the scene parameter set. The preset historical user adjustment data is subjected to exponential weighting to obtain a smoothed historical adjustment value. The scene matching weight is obtained by performing scene matching on the scene parameter set and the smooth historical adjustment value through a preset user habit learning model. The user preference gain value is obtained by weighting and fusing the scenario matching weight and the historical adjustment weight in the user's historical adjustment data using the user habit learning model.

[0087] The signal conditioning module 24 is used to perform gain optimization processing on the audio signal based on the predicted loudness gain value, the user preference gain value, and the real-time loudness compensation value, so as to obtain a gain-optimized audio signal.

[0088] In an optional implementation, the signal conditioning module 24 is specifically used for: The predicted loudness gain value, the user preference gain value, and the real-time loudness compensation value are weighted and averaged to obtain a preliminary comprehensive gain value. The preliminary integrated gain value is subjected to exponential smoothing and limiting processing to obtain the gain correction parameter; The audio spectrum data is divided into frequencies to obtain audio data for each frequency band. The audio data is weighted and optimized according to the gain correction parameters to obtain frequency domain gain adjustment data. The frequency domain gain adjustment data is subjected to inverse Fourier transform processing to obtain the gain-optimized audio signal.

[0089] It should be understood that the various variations and specific embodiments of the methods provided in the above embodiments are also applicable to the audio signal adaptive adjustment device of this embodiment. Through the foregoing detailed description of the audio signal adaptive adjustment method, those skilled in the art can clearly understand the implementation method of the audio signal adaptive adjustment device in this embodiment. For the sake of brevity, it will not be described in detail here.

[0090] like Figure 3 The diagram shown is a structural schematic of an electronic device provided in an embodiment of this application.

[0091] In a preferred embodiment of the present invention, the electronic device 3 may include, but is not limited to, a memory 31, at least one processor 32, and at least one communication bus 33.

[0092] Those skilled in the art should understand that Figure 3 The structure of the electronic device 3 shown does not constitute a limitation of the embodiments of the present invention. The electronic device 3 may also include more or fewer other hardware or software than shown, or different component arrangements.

[0093] In some embodiments, the electronic device 3 is a device capable of automatically performing numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits, programmable gate arrays, digital processors, and embedded devices.

[0094] It should be noted that the electronic device 3 is merely an example. Other existing or future electronic products that are suitable for this application should also be included within the scope of protection of this application and are incorporated herein by reference.

[0095] In some embodiments, the memory 31 stores a computer program that, when executed by the at least one processor 32, implements all or part of the steps in the audio signal adaptive adjustment method described above. The memory 31 includes a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, disk storage, magnetic tape storage, or any other computer-readable medium capable of carrying or storing data. Further, the computer-readable storage medium may primarily include a program storage area and a data storage area, wherein the program storage area may store an operating system, at least one application program required for a function, etc.

[0096] In some embodiments, the at least one processor 32 is the control unit of the electronic device 3, connecting various components of the electronic device 3 via various interfaces and lines. It executes programs or modules stored in the memory 31 and calls data stored in the memory 31 to perform various functions and process data. For example, when the at least one processor 32 executes a computer program stored in the memory 31, it implements all or part of the steps of the audio signal adaptive adjustment method described in this application embodiment; or it implements all or part of the functions of the audio signal adaptive adjustment device. The at least one processor 32 may be composed of integrated circuits, such as a single-packaged integrated circuit or multiple integrated circuits with the same or different functions, including combinations of one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips.

[0097] In some embodiments, the at least one communication bus 33 is configured to enable communication between the memory 31 and the at least one processor 32, etc. Although not shown, the electronic device 3 may also include a power supply (e.g., a battery) to power the various components. Preferably, the power supply can be logically connected to the at least one processor 32 via a power management device, thereby enabling functions such as charging, discharging, and power consumption management. The power supply may also include one or more DC or AC power supplies, recharging devices, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components. The electronic device 3 may also include various sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be described in detail here.

[0098] The integrated unit implemented as a software functional module described above can be stored in a computer-readable storage medium. This software functional module, stored in a storage medium, includes several instructions to cause an electronic device (which may be a personal computer, electronic device, or network device, etc.) or processor to execute portions of the methods described in the various embodiments of this application.

[0099] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and other division methods may be used in actual implementation.

[0100] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0101] The above are all preferred embodiments of this application, and are not intended to limit the scope of protection of this application. Therefore, all equivalent changes made in accordance with the structure, shape and principle of this application should be covered within the scope of protection of this application.

Claims

1. An adaptive adjustment method for audio signals, characterized in that, The method includes: The detected environmental noise data and the received audio signal are subjected to joint feature extraction processing to obtain multi-dimensional feature data, and the loudness gain prediction processing of the two-layer LSTM network is performed based on the multi-dimensional feature data to obtain the predicted loudness gain value. The audio signal is subjected to loudness simulation processing to obtain predicted loudness, and the predicted loudness is subjected to adaptive filtering and gain calculation processing to obtain real-time loudness compensation value. The detected current scene information and the preset user historical adjustment data are fused together to obtain the user preference gain value; The audio signal is subjected to gain optimization processing based on the predicted loudness gain value, the user preference gain value, and the real-time loudness compensation value to obtain a gain-optimized audio signal.

2. The audio signal adaptive adjustment method according to claim 1, characterized in that, The multidimensional feature data includes audio features and environmental noise features; Joint feature extraction processing is performed on the detected environmental noise data and the received audio signal to obtain multidimensional feature data, including: The detected environmental noise data and the received audio signal are subjected to frame segmentation and window function weighting to obtain weighted noise data and weighted audio data; The weighted noise data is subjected to Fast Fourier Transform and Wavelet Transform denoising processing to obtain noise spectrum data, and the weighted audio data is subjected to Fast Fourier Transform processing to obtain audio spectrum data. Power spectral density is calculated from the noise spectrum data to obtain energy spectral data, and short-time energy, zero-crossing rate, and Mel frequency cepstral coefficients are calculated from the audio spectrum data to obtain initial audio characteristics. The energy spectrum data is logarithmically transformed, and the transformed energy spectrum data and the initial audio features are normalized to obtain environmental noise features and audio features.

3. The audio signal adaptive adjustment method according to claim 1, characterized in that, The step of performing loudness gain prediction processing using a two-layer LSTM network based on the multidimensional feature data to obtain the predicted loudness gain value includes: The multidimensional feature data are arranged into time-series feature data according to the time order; The temporal feature data is processed by a two-layer LSTM network using a preset deep learning model to predict the loudness gain value.

4. The audio signal adaptive adjustment method according to claim 2, characterized in that, The process of performing loudness simulation processing on the audio signal to obtain predicted loudness, and then performing adaptive filtering and gain calculation processing on the predicted loudness to obtain a real-time loudness compensation value includes: The audio spectrum data is processed for energy calculation and loudness simulation according to preset standard rules to obtain the predicted loudness. Based on the predicted loudness and the environmental noise characteristics, a frequency band difference compensation analysis is performed to obtain compensation requirement data between each frequency band. The compensation demand data is subjected to adaptive filtering and adjustment to obtain a compensation gain curve; The real-time loudness compensation value is obtained by calculating the gain of the compensation gain curve using a preset K-weighted filtering model.

5. The audio signal adaptive adjustment method according to claim 1, characterized in that, The current scene information includes sensor datasets, current time information, and device status data; The step of performing data fusion processing on the detected current scene information and preset user historical adjustment data to obtain user preference gain values ​​includes: The sensor data is processed to extract physical environment features to obtain a set of physical environment parameters; The current time information and the device status data are subjected to time-domain feature extraction processing to obtain a time-domain status parameter set; The physical environment parameter set and the time domain state parameter set are normalized to obtain the scene parameter set. The preset historical user adjustment data is subjected to exponential weighting to obtain a smoothed historical adjustment value. The scene matching weight is obtained by performing scene matching on the scene parameter set and the smooth historical adjustment value through a preset user habit learning model. The user preference gain value is obtained by weighting and fusing the scenario matching weight and the historical adjustment weight in the user's historical adjustment data using the user habit learning model.

6. The audio signal adaptive adjustment method according to claim 2, characterized in that, The step of performing gain optimization processing on the audio signal based on the predicted loudness gain value, the user preference gain value, and the real-time loudness compensation value to obtain a gain-optimized audio signal includes: The predicted loudness gain value, the user preference gain value, and the real-time loudness compensation value are weighted and averaged to obtain a preliminary comprehensive gain value. The preliminary integrated gain value is subjected to exponential smoothing and limiting processing to obtain the gain correction parameter; The audio spectrum data is divided into frequencies to obtain audio data for each frequency band. The audio data is weighted and optimized according to the gain correction parameters to obtain frequency domain gain adjustment data. The frequency domain gain adjustment data is subjected to inverse Fourier transform processing to obtain the gain-optimized audio signal.

7. An audio signal adaptive adjustment device, characterized in that, The device includes: The sensor analysis module is used to perform joint feature extraction processing on the detected environmental noise data and the received audio signal to obtain multi-dimensional feature data, and to perform loudness gain prediction processing on the multi-dimensional feature data using a two-layer LSTM network to obtain the predicted loudness gain value. The prediction compensation module is used to perform loudness simulation processing on the audio signal to obtain predicted loudness, and to perform adaptive filtering and gain calculation processing on the predicted loudness to obtain real-time loudness compensation value. The preference gain module is used to perform data fusion processing on the detected current scene information and the preset user historical adjustment data to obtain the user preference gain value. The signal conditioning module is used to perform gain optimization processing on the audio signal based on the predicted loudness gain value, the user preference gain value, and the real-time loudness compensation value, so as to obtain a gain-optimized audio signal.

8. An electronic device, characterized in that, The electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the audio signal adaptive adjustment method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the audio signal adaptive adjustment method according to any one of claims 1 to 6.

Images