Adaptive audio enhancement method, device, SoC chip and storage medium

Abstract

The present invention relates to the field of audio processing technology, and discloses an adaptive audio enhancement method, device, SoC chip and storage medium. The method comprises: obtaining an original audio signal, and performing multi-dimensional audio feature extraction, multi-dimensional weighted fusion, dimensionality reduction, feature sequence encoding, correlation calculation, semantic hierarchical analysis and enhancement parameter mapping on the original audio signal, and based on the mapped audio enhancement parameters, adaptively filtering and reconstructing the original audio signal to obtain a preliminary enhanced audio signal; using a preset audio quality assessment network to evaluate the preliminary enhanced audio signal, multi-objective optimization and gradient adjustment, and based on the adjusted audio enhancement parameters, performing secondary audio enhancement on the preliminary enhanced audio signal until the audio evaluation value is greater than the preset audio evaluation threshold, and generating a final enhanced audio signal. The present application integrates the multi-dimensional features and dynamic characteristics of the audio signal, and improves the audio enhancement effect on the original audio signal.

Description

Technical Field

[0001] The present invention relates to the field of audio processing technology, and in particular to an adaptive audio enhancement method, device, SoC chip and storage medium. Background Art

[0002] In the field of audio signal processing, audio enhancement is of great significance for improving sound quality and listening experience. With the development of digital audio technology, audio enhancement is increasingly used in smart home, medical diagnosis, audio and video entertainment and other fields. In order to meet the needs of audio enhancement in different scenarios, accurate and efficient audio signal processing is crucial to improving user experience and system performance. Therefore, how to adaptively adjust the relevant parameters of audio plays an important role in ensuring the effectiveness and reliability of audio processing.

[0003] Nowadays, deep learning algorithms are used to analyze audio features, neural network models are built to optimize audio quality, or attempts are made to apply traditional digital signal processing technology to the audio enhancement process to improve processing effects. However, these methods still face challenges such as insufficient feature extraction, fixed processing parameters, and limited ability to adapt to environmental changes. In addition, these enhancement methods often ignore some unique features of audio signals, such as time-varying spectrum characteristics, environmental noise effects, acoustic spatial characteristics, etc., which may have a significant impact on the enhancement effect. That is, the existing adaptive audio enhancement methods ignore the multi-dimensional feature fusion and dynamic characteristics of audio signals, resulting in poor final audio enhancement effects. Summary of the invention

[0004] The main purpose of the present invention is to solve the problem that the existing adaptive audio enhancement method ignores the multi-dimensional feature fusion and the dynamic characteristics of the audio signal, resulting in a poor final audio enhancement effect.

[0005] A first aspect of the present invention provides an adaptive audio enhancement method, which includes: obtaining an original audio signal to be enhanced, and extracting multi-dimensional audio features from the original audio signal to obtain multi-dimensional audio features; performing multi-dimensional weighted fusion and dimensionality reduction on the multi-dimensional audio features to obtain fused audio features, and performing feature sequence encoding and correlation calculation on the fused audio features to obtain a temporal dependency representation; performing semantic hierarchical parsing and enhancement parameter mapping on the temporal dependency representation to obtain audio enhancement parameters, and based on the audio enhancement parameters, adaptively filtering the original audio signal and reconstructing the corresponding sub-band audio signal to obtain a preliminary enhanced audio signal; evaluating the optimized preliminary enhanced audio signal using a preset audio quality evaluation network to obtain an audio evaluation value, and based on the audio evaluation value, performing multi-objective optimization and gradient adjustment on the audio enhancement parameters to obtain adjusted audio enhancement parameters, and based on the adjusted audio enhancement parameters, performing secondary audio enhancement on the preliminary enhanced audio signal until the audio evaluation value is greater than a preset audio evaluation threshold to generate a final enhanced audio signal.

[0006] Optionally, in a first implementation manner of the first aspect of the present invention, the multidimensional audio features include time domain audio features, frequency domain audio features and cepstrum audio features, and the extracting of multidimensional audio features from the original audio signal to obtain the multidimensional audio features includes: framing and windowing the original audio signal to obtain multiple short-time audio segments, and performing high-pass filtering and pre-emphasis on each of the short-time audio segments to obtain pre-emphasized short-time audio segments; extracting time domain features from the pre-emphasized short-time audio segments to obtain time domain audio features, and performing multi-scale decomposition on the pre-emphasized short-time audio segments to obtain multiple audio frequency band sub-signals, and performing short-time Fourier transform on the pre-emphasized short-time audio segments to obtain time-frequency feature spectra; calculating the energy distribution of each of the audio frequency band sub-signals to obtain audio frequency domain features, and performing Mel spectrum filtering on the time-frequency feature spectrum to obtain a Mel feature spectrum, and performing logarithmic compression and discrete cosine transform on the Mel feature spectrum to obtain cepstrum audio features.

[0007] Optionally, in a second implementation method of the first aspect of the present invention, the multi-dimensional weighted fusion and dimensionality reduction of the multi-dimensional audio features to obtain fused audio features include: time alignment and feature normalization of the time domain audio features, the frequency domain audio features and the cepstrum audio features to obtain normalized audio features, and performing a centered product mean operation and row-column arrangement on each pair of features in the normalized audio features to obtain a feature correlation matrix; weight calculation on the feature correlation matrix to obtain audio feature weights, and based on the audio feature weights, performing a weighted operation on the normalized audio features to obtain weighted audio features; channel dimension splicing and feature dimensionality reduction on the weighted audio features to obtain a reduced audio feature vector, and performing a nonlinear transformation on the reduced audio feature vector to obtain a fused audio feature.

[0008] Optionally, in a third implementation method of the first aspect of the present invention, the feature sequence encoding and correlation calculation of the fused audio features to obtain a temporal dependency representation includes: performing sliding window segmentation on the fused audio features to obtain multiple audio feature sequence fragments, and encoding the corresponding feature sequences in each of the audio feature sequence fragments to obtain a temporal context representation; performing similarity calculation on the temporal context representation of each sequence fragment to obtain a context similarity matrix, and based on a preset similarity threshold, performing threshold comparison and matrix element assignment on the context similarity matrix to obtain a temporal association graph; performing hierarchical segmentation and binary threshold segmentation on the temporal association graph to obtain temporal density clustering, and performing boundary point extraction and time axis mapping on the temporal density clustering to obtain temporal time segmentation points; merging and refining the temporal time segmentation points to obtain audio event boundaries, and based on the audio event boundaries, performing event-level segmentation and temporal aggregation on the temporal context representation to obtain a temporal dependency representation.

[0009] Optionally, in a fourth implementation method of the first aspect of the present invention, the semantic hierarchical parsing and enhancement parameter mapping of the timing dependency representation to obtain audio enhancement parameters includes: hierarchical clustering of the timing dependency representation to obtain a multi-scale semantic structure, and extracting corresponding local semantic features and global semantic features in the multi-scale semantic structure to obtain hierarchical semantic features; performing feature importance scoring and distribution normalization on the hierarchical semantic features to obtain standardized feature weights, and weight weighting and spatially projecting the standardized feature weights to obtain key semantic features; using a preset feature decision tree to perform feature grouping and threshold screening on the key semantic features to obtain initial enhancement parameters, and performing parameter sequence mutation detection and local correction on the initial enhancement parameters to obtain a continuous parameter curve; performing constrained optimization of multiple audio signal parameters on the continuous parameter curve to obtain a constrained parameter interval, and mapping multiple audio channel parameters to the constrained parameter interval to obtain audio enhancement parameters.

[0010] Optionally, in a fifth implementation of the first aspect of the present invention, the original audio signal is adaptively filtered and the corresponding sub-band audio signal is reconstructed based on the audio enhancement parameter to obtain a preliminary enhanced audio signal, including: frequency segmenting and time domain conversion of the audio enhancement parameter to obtain an audio filter coefficient, and based on the audio filter coefficient, the original audio signal is decomposed using a preset adaptive filter group to obtain a plurality of audio sub-band signals; dynamic signal compression and gain adjustment are performed on each of the audio sub-band signals to obtain a dynamic interval equalized audio sub-band signal, and baseband estimation is performed on the dynamic interval equalized audio sub-band signal. The method comprises the following steps: performing reverberation time estimation and cepstrum analysis on the audio harmonic structure to obtain the audio harmonic structure after cepstrum analysis, and using a preset inverse filter to dereverberate the audio harmonic structure after cepstrum analysis to obtain a dereverberated speech signal; performing pitch extraction and pitch enhancement on the dereverberated speech signal to obtain a pitch enhanced speech signal, and performing power spectrum estimation and Wiener filtering on the pitch enhanced speech signal to obtain a denoised speech signal; performing amplitude normalization and phase alignment on the denoised speech signal to obtain a synchronous speech signal, and performing synthesis filtering and gain balancing on the synchronous speech signal to obtain a preliminary enhanced audio signal.

[0011] Optionally, in a sixth implementation manner of the first aspect of the present invention, the using of a preset audio quality assessment network to evaluate the optimized preliminary enhanced audio signal to obtain an audio evaluation value, and based on the audio evaluation value, performing multi-objective optimization and gradient adjustment on the audio enhancement parameters to obtain the adjusted audio enhancement parameters, including: based on a preset audio quality assessment network, respectively calculating the spectral distortion between the preliminary enhanced audio signal and the original audio signal, performing sub-band energy statistics on the preliminary enhanced audio signal to obtain an audio loudness curve, and performing cepstral coefficient extraction and nonlinear mapping on the preliminary enhanced audio signal. The method comprises the steps of: projecting the spectral distortion, the audio loudness curve and the sound quality score value to obtain a sound quality score value, and performing weighted superposition on the spectral distortion, the audio loudness curve and the sound quality score value to obtain an audio evaluation value; based on a preset audio evaluation threshold, performing difference decomposition and nonlinear weighted mapping on the audio evaluation value in time-frequency dimensions to obtain an audio quality gain target, and performing multi-order gradient operations and feasible domain projection on the audio quality gain target to obtain a parameter adjustment direction; based on the parameter adjustment direction, performing a step search of candidate parameters on the audio enhancement parameters to obtain a candidate enhancement parameter set, and performing constrained optimization calculation on the candidate enhancement parameter set to obtain adjusted audio enhancement parameters.

[0012] The second aspect of the present invention provides an adaptive audio enhancement device, which includes: a feature extraction module, which is used to obtain an original audio signal to be enhanced, and extract multi-dimensional audio features from the original audio signal to obtain multi-dimensional audio features; a feature fusion module, which is used to perform multi-dimensional weighted fusion and dimensionality reduction on the multi-dimensional audio features to obtain fused audio features, and perform feature sequence encoding and correlation calculation on the fused audio features to obtain a temporal dependency representation; a preliminary enhancement module, which is used to perform semantic hierarchical analysis and enhancement parameter mapping on the temporal dependency representation to obtain audio enhancement parameters, and based on the audio enhancement parameters, adaptively filter the original audio signal and reconstruct the corresponding sub-band audio signal to obtain a preliminary enhanced audio signal; a secondary enhancement module, which is used to evaluate the optimized preliminary enhanced audio signal using a preset audio quality evaluation network to obtain an audio evaluation value, and based on the audio evaluation value, perform multi-objective optimization and gradient adjustment on the audio enhancement parameters to obtain adjusted audio enhancement parameters, and based on the adjusted audio enhancement parameters, perform secondary audio enhancement on the preliminary enhanced audio signal until the audio evaluation value is greater than a preset audio evaluation threshold to generate a final enhanced audio signal.

[0013] The third aspect of the present invention provides a SoC chip, comprising: a memory and at least one processor, wherein the memory stores instructions; the at least one processor calls the instructions in the memory so that the SoC chip executes each step of the above-mentioned adaptive audio enhancement method.

[0014] A fourth aspect of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores instructions, which, when executed on a computer, enable the computer to execute the various steps of the above-mentioned adaptive audio enhancement method.

[0015] The above-mentioned adaptive audio enhancement method, device, SoC chip and storage medium. In the embodiment of the present invention, the original audio signal to be enhanced is obtained, and multi-dimensional audio features are extracted from the original audio signal to obtain multi-dimensional audio features; the multi-dimensional audio features are weighted and dimensionality reduced to obtain fused audio features, and the fused audio features are encoded and correlated to obtain a temporal dependency representation; the temporal dependency representation is semantically hierarchically parsed and enhanced parameter mapped to obtain audio enhancement parameters, and based on the audio enhancement parameters, the original audio signal is adaptively filtered and the corresponding sub-band audio signal is reconstructed to obtain a preliminary enhanced audio signal; the optimized preliminary enhanced audio signal is evaluated using a preset audio quality evaluation network to obtain an audio evaluation value, and based on the audio evaluation value, the audio enhancement parameters are multi-objective optimized and gradient adjusted to obtain the adjusted audio enhancement parameters, and based on the adjusted audio enhancement parameters, the preliminary enhanced audio signal is subjected to secondary audio enhancement until the audio evaluation value is greater than the preset audio evaluation threshold, thereby generating a final enhanced audio signal. Compared with the prior art, this application extracts and fuses multi-dimensional features of the original audio signal to obtain the key feature representation of the audio signal, and then generates targeted enhancement parameters based on temporal correlation analysis and semantic parsing, and uses adaptive filtering and sub-band reconstruction technology to perform preliminary enhancement on the audio signal; then introduces an audio quality assessment network to perform audio quality assessment, and iteratively adjusts the enhancement parameters through multi-objective optimization to achieve secondary enhancement of the audio signal and generate the final enhanced audio signal. It achieves adaptive capture of the time-frequency features and semantic information of the audio signal, effectively improves the quality of the audio signal, and ensures the stability of the enhancement effect through a closed-loop design of quality assessment and parameter optimization; at the same time, it takes into account the unique characteristics of the audio signal and improves the audio enhancement effect of the original audio signal.

[0016] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention are realized and obtained by the structures particularly pointed out in the description, claims and drawings.

[0017] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 Schematic diagram of a first embodiment of an adaptive audio enhancement method according to an embodiment of the present invention;

[0019] Figure 2 FIG. 4 is a schematic diagram of an embodiment of an adaptive audio enhancement device in an embodiment of the present invention. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0021] The terms "including" and "having" and any variations thereof mentioned in the embodiments of the present invention are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device end including a series of steps or units is not limited to the listed steps or units, but may optionally include other steps or units that are not listed, or may optionally include other steps or units that are inherent to these processes, methods, products or device ends.

[0022] To facilitate understanding of this embodiment, the specific process of the embodiment of the present invention is described below. Figure 1 , a first embodiment of the adaptive audio enhancement method in the embodiment of the present invention includes:

[0023] 101. Acquire an original audio signal to be enhanced, and extract multi-dimensional audio features from the original audio signal to obtain multi-dimensional audio features;

[0024] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0025] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0026] In this embodiment, the original audio signal here is an analog signal that changes continuously over time and contains multiple audio characteristic dimensions (including time domain characteristic dimensions, frequency domain characteristic dimensions, auditory perception dimensions, spatial acoustic dimensions, and signal-to-noise ratio dimensions, etc.); the multi-dimensional audio features include time domain audio features, frequency domain audio features, and cepstrum audio features. The original audio signal is framed and windowed to obtain multiple short-time audio clips, and each short-time audio clip is pre-emphasized by high-pass filtering to obtain a pre-emphasized short-time audio clip; time domain feature extraction is performed on the pre-emphasized short-time audio clip to obtain time domain audio features, and multi-scale decomposition is performed on the pre-emphasized short-time audio clip to obtain multiple audio frequency band sub-signals, and short-time Fourier transform is performed on the pre-emphasized short-time audio clip to obtain a time-frequency feature spectrum; the energy distribution of each audio frequency band sub-signal is calculated to obtain an audio frequency domain feature, and Mel spectrum filtering is performed on the time-frequency feature spectrum to obtain a Mel feature spectrum, and the Mel feature spectrum is logarithmically compressed and discrete cosine transformed to obtain a cepstrum audio feature.

[0027] In practical applications, the original audio signal to be enhanced is obtained, and then the original audio signal is divided into frames. A frame length of 20-30ms and a frame shift of 10ms are usually used to ensure a certain overlap between adjacent frames, thereby maintaining the continuity of the signal. At the same time, a Hanning window or a Hamming window is applied to each signal frame for windowing processing to reduce spectrum leakage and obtain multiple short-time audio clips. The obtained short-time audio clips are then subjected to high-pass filtering and pre-emphasis processing. The pre-emphasis coefficient is usually set to 0.97 to enhance the energy of high-frequency components and improve the signal-to-noise ratio of the signal to obtain the pre-emphasized short-time audio clip. After obtaining the pre-emphasized short-time audio clip, audio features are extracted from three dimensions: time domain, frequency domain, and time-frequency domain. In terms of time domain feature extraction, the short-time energy, zero-crossing rate, and autocorrelation coefficient of each short-time audio clip are calculated (these features reflect the audio The time-domain energy distribution and periodic characteristics of the signal are obtained by multi-scale wavelet decomposition to obtain the time-domain audio features; at the same time, the short-time audio clip after pre-emphasis is decomposed into sub-signals in different frequency bands to capture the signal characteristics in different frequency ranges and obtain multiple audio frequency band sub-signals; in addition, the short-time audio clip after pre-emphasis needs to be short-time Fourier transformed to obtain the time-frequency feature spectrum reflecting the time-frequency characteristics of the signal; then the energy distribution of the audio frequency band sub-signals is calculated to obtain the audio frequency domain features reflecting the energy distribution of different frequency bands, and the time-frequency feature spectrum is subjected to Mel spectrum filtering to map the linear frequency axis to the Mel frequency axis reflecting the human ear perception characteristics to obtain the Mel feature spectrum; then, in order to further compress the feature dimension and enhance the discriminability of the feature, the Mel feature spectrum is logarithmically compressed and then discrete cosine transformed to obtain the inverse spectrum audio features describing the acoustic characteristics of the audio signal.

[0028] 102. Perform multi-dimensional weighted fusion and dimensionality reduction on the multi-dimensional audio features to obtain fused audio features, and perform feature sequence encoding and correlation calculation on the fused audio features to obtain a temporal dependency representation;

[0029] In this embodiment, the time domain audio features, frequency domain audio features and cepstrum audio features are time aligned and feature normalized to obtain normalized audio features, and each pair of features in the normalized audio features are multiplied and averaged after centering and arranged in row and column order to obtain a feature correlation matrix; the feature correlation matrix is ​​weighted to obtain audio feature weights, and based on the audio feature weights, the normalized audio features are weighted to obtain weighted audio features; the weighted audio features are channel dimensionally spliced ​​and feature dimensionally reduced to obtain a reduced audio feature vector, and the reduced audio feature vector is nonlinearly transformed to obtain a fused audio feature; the fused audio features are segmented by a sliding window to obtain multiple audio features Sequence fragments, and encode the corresponding feature sequences in each audio feature sequence fragment to obtain a temporal context representation; perform similarity calculation on the temporal context representation of each sequence fragment to obtain a context similarity matrix, and based on a preset similarity threshold, perform threshold comparison and matrix element assignment on the context similarity matrix to obtain a temporal association graph; perform hierarchical segmentation and binary threshold segmentation on the temporal association graph to obtain temporal density clustering, and perform boundary point extraction and time axis mapping on the temporal density clustering to obtain temporal time segmentation points; merge and refine the temporal time segmentation points to obtain audio event boundaries, and based on the audio event boundaries, perform event-level segmentation and temporal aggregation on the temporal context representation to obtain a temporal dependency representation.

[0030] In practical applications, first, the time domain audio features, frequency domain audio features and cepstrum audio features are aligned on the time axis to ensure that different features describe the audio content of the same time period. Then, the features of each dimension are normalized, and the feature values ​​are mapped to the same numerical range to eliminate the influence of dimensional differences and obtain normalized audio features. Then, each pair of features in the normalized audio features is centered, that is, the feature mean is subtracted to center it, and then the product between the features is calculated and the mean is obtained to obtain the covariance between the features. These covariances are arranged in the order of rows and columns of the features to construct a correlation matrix that reflects the correlation of the features. Then, based on the feature correlation matrix, the feature importance is used to calculate the feature correlation matrix. The feature scoring method calculates the weight of each feature, where the weight value reflects the importance of the feature to the audio signal representation, and the audio feature weight is obtained. Based on the audio feature weight, the normalized audio features are weighted calculated so that important features (audio features with large weights) play a greater role in subsequent processing, and the weighted audio features are obtained. The weighted audio features are then concatenated in the channel dimension to form a high-dimensional feature vector, and the principal component analysis and other dimensionality reduction methods are used to reduce the dimensionality of the feature vector, retaining the main feature information, and obtaining the audio feature vector after dimensionality reduction. Subsequently, the feature vector is nonlinearly transformed through nonlinear activation functions such as ReLU to enhance the expressiveness of the audio features and obtain the fused audio features. Secondly, in order to capture the temporal dependencies in the audio signal, the fused audio features are segmented by a sliding window. The setting of the window size and the sliding step size needs to balance the accuracy and computational efficiency of temporal modeling to obtain multiple audio feature sequence fragments. Then, a recurrent neural network is used to encode the feature sequence in each sequence fragment to obtain a temporal context representation containing context information. In order to analyze the correlation between features at different time positions, the cosine similarity between the temporal context representations of each sequence fragment is calculated to construct a context similarity matrix, and a suitable similarity threshold is set. The elements in the similarity matrix that are greater than the threshold are set to 1, and the elements that are less than the threshold are set to 0, to obtain a temporal correlation diagram that reflects the temporal correlation relationship. The joint graph is hierarchically segmented to obtain sub-graphs of different levels, and each layer of sub-graphs is segmented using a binary threshold to obtain temporal density clustering that reflects the clustering of temporal features. The boundary points of the density clustering are then extracted and mapped to the time axis to obtain temporal time segmentation points representing the boundaries of potential audio events. Similar temporal time segmentation points are then merged, and the segmentation points are refined based on the continuity of the audio signal to obtain more accurate audio event boundaries. These event boundaries are then used to segment the temporal context representation, and the feature representations belonging to the same event are aggregated together. The final temporal dependency representation is obtained through temporal aggregation, thereby achieving effective fusion of multi-dimensional related audio enhancement features to subsequently improve the audio enhancement effect.

[0031] 103. Perform semantic hierarchical analysis and enhancement parameter mapping on the temporal dependency representation to obtain audio enhancement parameters, and based on the audio enhancement parameters, perform adaptive filtering on the original audio signal and reconstruct the corresponding sub-band audio signal to obtain a preliminary enhanced audio signal;

[0032] In this embodiment, hierarchical clustering is performed on the temporal dependency representation to obtain a multi-scale semantic structure, and the corresponding local semantic features and global semantic features in the multi-scale semantic structure are extracted to obtain hierarchical semantic features; feature importance scoring and distribution normalization are performed on the hierarchical semantic features to obtain standardized feature weights, and the standardized feature weights are weighted and spatially projected to obtain key semantic features; a preset feature decision tree is used to perform feature grouping and threshold screening on the key semantic features to obtain initial enhancement parameters, and mutation detection and local correction of the parameter sequence are performed on the initial enhancement parameters to obtain a continuous parameter curve; constrained optimization of multiple audio signal parameters is performed on the continuous parameter curve to obtain a constrained parameter interval, and multi-audio channel parameters are mapped to the constrained parameter interval to obtain audio enhancement parameters; frequency segmentation and time domain conversion of the frequency response are performed on the audio enhancement parameters to obtain audio filter coefficients, and based on the audio filter coefficients, The original audio signal is decomposed by using a preset adaptive filter group to obtain multiple audio sub-band signals; dynamic signal compression and gain adjustment are performed on each audio sub-band signal to obtain an audio sub-band signal of dynamic interval equalization, and fundamental frequency estimation and harmonic enhancement are performed on the audio sub-band signal of dynamic interval equalization to obtain an audio harmonic structure; reverberation time estimation and cepstrum analysis are performed on the audio harmonic structure to obtain an audio harmonic structure after cepstrum analysis, and the audio harmonic structure after cepstrum analysis is dereverberated by using a preset inverse filter to obtain a dereverberated speech signal; fundamental pitch extraction and fundamental pitch enhancement are performed on the dereverberated speech signal to obtain a fundamental pitch enhanced speech signal, and power spectrum estimation and Wiener filtering are performed on the fundamental pitch enhanced speech signal to obtain a denoised speech signal; amplitude normalization and phase alignment are performed on the denoised speech signal to obtain a synchronous speech signal, and synthesis filtering and gain balancing are performed on the synchronous speech signal to obtain a preliminary enhanced audio signal.

[0033] In practical applications, first, in order to extract multi-level semantic information from temporal dependency representation, hierarchical clustering analysis is performed on the temporal dependency representation. A bottom-up aggregation method is adopted to gradually build a hierarchical structure of features according to feature similarity, and the Ward distance between features at different levels is calculated. The calculation formula of Ward distance is:

[0034] ;

[0035] in, and represents two clusters to be merged, x represents the sample points in the cluster, m, ,Respectively represent the mean vectors of the merged cluster and the cluster to be merged, ||·|| represents the Euclidean distance, and then the cluster pairs with the minimum Ward distance are iteratively calculated to merge, and a multi-scale semantic structure containing different semantic granularities is obtained, and then local fine-grained semantic features and global coarse-grained semantic features are extracted from this hierarchical structure, which capture the local details and overall semantic information of the audio signal respectively. By combining these different levels of features, a complete hierarchical semantic feature is obtained; in order to highlight the key semantic information, the importance score is calculated for each feature element in the hierarchical semantic feature, and the calculation of the score takes into account the distinguishability and stability of the feature, and then these scores are distributed and normalized so that the importance scores of different features are within a comparable scale range, and standardized feature weights are obtained, and then these weights are used to weight the features, and the weighted features are mapped to the new feature space by spatial projection, so as to obtain key semantic features that can better express the audio semantic information; based on the obtained key semantic features, a pre-designed feature decision tree is used to group them (where the pre-designed feature decision tree here is a feature decision tree that takes into account the audio signal The multiple key feature dimensions of the signal are analyzed by a hierarchical decision structure. Different layers from low to high can focus on the basic acoustic features, dynamic characteristics, environmental characteristics and specific values ​​of the audio signal for determining the enhancement parameters. Based on the above decision tree structure, the appropriate enhancement parameter combination can be adaptively selected for different types of audio signals. For example, for weak speech signals in a reverberant environment, a larger gain compensation, a stronger de-reverberation process and a moderate noise suppression can be selected; while for clear music signals, a smaller gain adjustment will be used to maintain the original dynamic characteristics), organize the features with strong correlation together, and perform feature screening by setting a suitable threshold to obtain the initial enhancement parameters, and then arrange these parameters in chronological order to form a parameter sequence. By detecting the mutation points in the sequence and performing local smoothing correction on these points, the continuity of the parameter change is ensured to obtain a continuous parameter curve; then the parameter curve is constrained and optimized according to the loudness, frequency distribution and other characteristics of the audio signal to obtain the constraint parameter interval that meets the requirements of various audio signal parameters, and then these parameter intervals are mapped to multiple audio processing channels to obtain specific parameters that can be used for audio enhancement, namely audio enhancement parameters.Secondly, after obtaining the audio enhancement parameters, the enhancement parameters are frequency segmented, the frequency response of each frequency band is designed, and the frequency response is converted into filter coefficients in the time domain. Then, these coefficients are used to construct an adaptive filter group, and the original audio signal is decomposed into multiple frequency bands to obtain multiple audio sub-band signals reflecting the characteristics of different frequency ranges. Then, dynamic range compression processing is performed on each sub-band signal, and the dynamic range of the signal is equalized by adaptively adjusting the compression ratio and threshold. The gain of the compressed signal is adjusted to ensure the balance of energy of each frequency band signal, and an audio sub-band signal with dynamic range balance is obtained; then, the fundamental frequency of the equalized sub-band signal is estimated, and the basic frequency component in the signal is identified. Then, the fundamental frequency and its harmonic components are selectively enhanced to improve the coordination of the sound and obtain the enhanced audio harmonic structure; then, considering the reverberation effect in the actual environment, the audio harmonics are adjusted. The structure estimates the reverberation time, analyzes the reverberation characteristics of the signal, and extracts the reverberation parameters through cepstrum analysis. Then, the corresponding inverse filter is designed to dereverberate the signal to obtain a clear dereverberated speech signal. Then, the fundamental tone is extracted from the dereverberated speech signal, the pitch information of the speech is identified, and the fundamental tone component is appropriately enhanced to improve the clarity of the speech to obtain a fundamental tone enhanced speech signal. Then, the noise power spectrum of the enhanced signal is estimated, and a Wiener filter is designed to suppress noise to obtain a denoised speech signal. Then, in order to ensure the quality of the processed signal, the denoised speech signal is amplitude normalized to eliminate excessive fluctuations in the signal amplitude, and the multi-channel signal is phase aligned to ensure the coherence of the signal to obtain a synchronized speech signal. Then, the signals of each frequency band are recombined through a synthesis filter, and the final gain balance adjustment is performed to obtain a preliminarily enhanced audio signal.

[0036] 104. Use a preset audio quality assessment network to evaluate the optimized preliminary enhanced audio signal to obtain an audio evaluation value, and based on the audio evaluation value, perform multi-objective optimization and gradient adjustment on the audio enhancement parameters to obtain adjusted audio enhancement parameters, and based on the adjusted audio enhancement parameters, perform secondary audio enhancement on the preliminary enhanced audio signal until the audio evaluation value is greater than a preset audio evaluation threshold, thereby generating a final enhanced audio signal.

[0037] In this embodiment, based on a preset audio quality assessment network, the spectral distortion between the preliminary enhanced audio signal and the original audio signal is calculated respectively, and the subband energy statistics of the preliminary enhanced audio signal are performed to obtain an audio loudness curve, and the cepstral coefficients of the preliminary enhanced audio signal are extracted and nonlinearly mapped to obtain a sound quality score value, and the spectral distortion, the audio loudness curve and the sound quality score value are weighted and superimposed to obtain an audio evaluation value; based on a preset audio evaluation threshold, the audio evaluation value is subjected to difference decomposition and nonlinear weighted mapping in the time-frequency dimension to obtain an audio quality gain target, and the audio quality gain target is subjected to multi-order gradient operations and feasible domain projection to obtain a parameter adjustment direction; based on the parameter adjustment direction, the audio enhancement parameter is subjected to a step search of candidate parameters to obtain a candidate enhancement parameter set, and the candidate enhancement parameter set is subjected to constrained optimization calculation to obtain the adjusted audio enhancement parameter. The preset audio quality assessment network here refers to a multi-task evaluation model based on deep learning.

[0038] In practical applications, a pre-trained audio quality assessment network is used to perform spectrum analysis on the preliminary enhanced audio signal and the original audio signal, and the short-time Fourier transform is used to calculate the spectrum diagram of the two signals. Then, the degree of spectrum distortion between the two is calculated by the logarithmic spectrum distance metric. At the same time, the preliminary enhanced audio signal is divided into sub-bands, and the energy distribution is statistically analyzed in each sub-band to obtain a loudness curve reflecting the change in audio loudness. The score value representing the sound quality is obtained by extracting Mel-frequency cepstral coefficients and mapping with a deep neural network. The spectrum distortion, loudness curve and sound quality score value are then weighted and superimposed by a preset weight coefficient to obtain a comprehensive audio evaluation value (wherein, the evaluation value reflects the quality performance of the enhanced audio in multiple dimensions). The audio evaluation value is then compared with a preset target evaluation threshold, and their differences in time and frequency dimensions are calculated. The gain target of the audio quality is obtained by nonlinear weighted mapping of the difference (wherein, this gain The goal guides the direction of subsequent parameter optimization); in order to achieve precise adjustment of the parameters, the first-order and second-order gradients of the quality gain target are calculated to obtain the optimized search direction, and this search direction is projected into the feasible domain of the parameters to ensure that the adjusted parameters meet the physical constraints; based on the determined parameter adjustment direction, an adaptive step search strategy is used to generate multiple sets of candidate enhancement parameters, which have different distributions in the feasible domain, providing a rich candidate set for the selection of the optimal parameters; constrained optimization calculations are then performed on the candidate enhancement parameter set, and the correlation and physical constraints between the parameters are considered to select the optimal parameter combination as the adjusted audio enhancement parameters, and this new set of enhancement parameters is applied to the initially enhanced audio signal for secondary enhancement processing, wherein the whole process is iterative, by continuously evaluating the enhancement effect and adjusting the parameters until the audio evaluation value exceeds the preset quality threshold, and finally outputting an enhanced audio signal that meets the quality requirements. For example, for a speech signal recorded in a noisy environment, the quality assessment network is used to obtain the initial enhancement effect score. If the spectrum distortion is high or the loudness is uneven, the spectrum smoothing factor and dynamic range compression ratio in the enhancement parameters will be adjusted accordingly. Through multiple rounds of iterative optimization, the speech signal is ultimately made clearer and more intelligible while maintaining its naturalness. This fully utilizes the multi-dimensional feature information of the audio signal and achieves high-quality audio enhancement effects through an adaptive parameter optimization strategy.

[0039] In the embodiment of the present invention, the application extracts and fuses the multi-dimensional features of the original audio signal to obtain the key feature representation of the audio signal, and then generates targeted enhancement parameters based on time-series correlation analysis and semantic parsing, and uses adaptive filtering and sub-band reconstruction technology to perform preliminary enhancement on the audio signal; then introduces an audio quality assessment network to perform audio quality assessment, and iteratively adjusts the enhancement parameters through multi-objective optimization to achieve secondary enhancement of the audio signal and generate the final enhanced audio signal. It achieves adaptive capture of the time-frequency features and semantic information of the audio signal, effectively improves the quality of the audio signal, and ensures the stability of the enhancement effect through the closed-loop design of quality assessment and parameter optimization; at the same time, it takes into account the unique characteristics of the audio signal, and improves the audio enhancement effect of the original audio signal.

[0040] The above describes the adaptive audio enhancement method in the embodiment of the present invention. The following describes the adaptive audio enhancement device in the embodiment of the present invention. Figure 2 , an embodiment of the adaptive audio enhancement device in the embodiment of the present invention includes:

[0041] The feature extraction module 201 is used to obtain the original audio signal to be enhanced, and extract multi-dimensional audio features from the original audio signal to obtain multi-dimensional audio features;

[0042] A feature fusion module 202 is used to perform multi-dimensional weighted fusion and dimensionality reduction on the multi-dimensional audio features to obtain fused audio features, and perform feature sequence encoding and correlation calculation on the fused audio features to obtain a temporal dependency representation;

[0043] A preliminary enhancement module 203 is used to perform semantic hierarchical analysis and enhancement parameter mapping on the temporal dependency representation to obtain audio enhancement parameters, and based on the audio enhancement parameters, perform adaptive filtering on the original audio signal and reconstruct the corresponding sub-band audio signal to obtain a preliminary enhanced audio signal;

[0044] The secondary enhancement module 204 is used to evaluate the optimized preliminary enhanced audio signal using a preset audio quality evaluation network to obtain an audio evaluation value, and based on the audio evaluation value, perform multi-objective optimization and gradient adjustment on the audio enhancement parameters to obtain adjusted audio enhancement parameters, and based on the adjusted audio enhancement parameters, perform secondary audio enhancement on the preliminary enhanced audio signal until the audio evaluation value is greater than a preset audio evaluation threshold, thereby generating a final enhanced audio signal.

[0045] In the embodiment of the present invention, the application extracts and fuses the multi-dimensional features of the original audio signal to obtain the key feature representation of the audio signal, and then generates targeted enhancement parameters based on time-series correlation analysis and semantic parsing, and uses adaptive filtering and sub-band reconstruction technology to perform preliminary enhancement on the audio signal; then introduces an audio quality assessment network to perform audio quality assessment, and iteratively adjusts the enhancement parameters through multi-objective optimization to achieve secondary enhancement of the audio signal and generate the final enhanced audio signal. It achieves adaptive capture of the time-frequency features and semantic information of the audio signal, effectively improves the quality of the audio signal, and ensures the stability of the enhancement effect through the closed-loop design of quality assessment and parameter optimization; at the same time, it takes into account the unique characteristics of the audio signal, and improves the audio enhancement effect of the original audio signal.

[0046] above Figure 2 The adaptive audio enhancement device in the embodiment of the present invention is described in detail from the perspective of modular functional entities, and the Soc chip in the embodiment of the present invention is described in detail from the perspective of hardware processing.

[0047] The Soc chip may have relatively large differences due to different configurations or performances, and may include one or more processors (central processing units, CPU) (for example, one or more processors) and memories, and one or more storage media for storing applications or data (for example, one or more mass storage devices). Among them, the memory and the storage medium can be short-term storage or permanent storage. The program stored in the storage medium 3 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the Soc chip. Furthermore, the processor can be configured to communicate with the storage medium and execute a series of instruction operations in the storage medium on the Soc chip to implement the steps of the above-mentioned adaptive audio enhancement method.

[0048] The Soc chip may also have one or more wired or wireless network interfaces, one or more input and output interfaces, and / or one or more operating systems, such as Windows Serve, Mac OS X, Unix, Linux, FreeBSD, etc. Those skilled in the art will appreciate that the Soc chip structure does not limit the Soc chip provided by the present invention, and may include more or fewer components than shown in the figure, or combine certain components, or arrange components differently.

[0049] The present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. Instructions are stored in the computer-readable storage medium. When the instructions are executed on a computer, the computer executes the steps of the adaptive audio enhancement method.

[0050] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described system, device, or unit can refer to the corresponding process in the aforementioned method embodiment and will not be repeated here.

[0051] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the whole or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0052] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An adaptive audio enhancement method, characterized in that: The adaptive audio enhancement method comprises: Acquire an original audio signal to be enhanced, and extract multi-dimensional audio features from the original audio signal to obtain multi-dimensional audio features; Performing multi-dimensional weighted fusion and dimensionality reduction on the multi-dimensional audio features to obtain fused audio features, and performing feature sequence encoding and correlation calculation on the fused audio features to obtain a temporal dependency representation; Performing semantic hierarchical analysis and enhancement parameter mapping on the temporal dependency representation to obtain audio enhancement parameters, and based on the audio enhancement parameters, performing adaptive filtering on the original audio signal and reconstructing the corresponding sub-band audio signal to obtain a preliminary enhanced audio signal; The optimized preliminary enhanced audio signal is evaluated by using a preset audio quality evaluation network to obtain an audio evaluation value, and based on the audio evaluation value, the audio enhancement parameters are multi-objective optimized and gradient adjusted to obtain adjusted audio enhancement parameters, and based on the adjusted audio enhancement parameters, the preliminary enhanced audio signal is subjected to secondary audio enhancement until the audio evaluation value is greater than a preset audio evaluation threshold, thereby generating a final enhanced audio signal.

2. The adaptive audio enhancement method according to claim 1, characterized in that: The multi-dimensional audio features include time domain audio features, frequency domain audio features and cepstrum audio features, and the multi-dimensional audio features are extracted from the original audio signal to obtain the multi-dimensional audio features, including: Performing signal framing and windowing on the original audio signal to obtain a plurality of short-time audio segments, and performing high-pass filtering and pre-emphasis on each of the short-time audio segments to obtain a pre-emphasized short-time audio segment; Performing time domain feature extraction on the pre-emphasized short-time audio segment to obtain time domain audio features, performing multi-scale decomposition on the pre-emphasized short-time audio segment to obtain multiple audio frequency band sub-signals, and performing short-time Fourier transform on the pre-emphasized short-time audio segment to obtain a time-frequency feature spectrum; The energy distribution of each audio frequency band sub-signal is calculated to obtain the audio frequency domain feature, and the time-frequency feature spectrum is subjected to Mel spectrum filtering to obtain a Mel feature spectrum, and the Mel feature spectrum is subjected to logarithmic compression and discrete cosine transformation to obtain a cepstrum audio feature.

3. The adaptive audio enhancement method according to claim 2, characterized in that: The performing multi-dimensional weighted fusion and dimensionality reduction on the multi-dimensional audio features to obtain fused audio features includes: Performing time alignment and feature normalization on the time domain audio features, the frequency domain audio features, and the cepstrum audio features to obtain normalized audio features, and performing a product mean operation after centering and arranging rows and columns in order on each pair of features in the normalized audio features to obtain a feature correlation matrix; Performing weight calculation on the feature correlation matrix to obtain audio feature weights, and performing weighted operation on the normalized audio features based on the audio feature weights to obtain weighted audio features; Channel dimension splicing and feature dimension reduction are performed on the weighted audio features to obtain a reduced-dimensional audio feature vector, and nonlinear transformation is performed on the reduced-dimensional audio feature vector to obtain a fused audio feature.

4. The adaptive audio enhancement method according to claim 1, characterized in that: The step of performing feature sequence encoding and correlation calculation on the fused audio features to obtain a temporal dependency representation includes: Perform sliding window segmentation on the fused audio features to obtain a plurality of audio feature sequence segments, and encode the feature sequence corresponding to each of the audio feature sequence segments to obtain a temporal context representation; Calculating the similarity of the temporal context representation of each sequence segment to obtain a context similarity matrix, and performing threshold comparison and matrix element assignment on the context similarity matrix based on a preset similarity threshold to obtain a temporal association graph; Performing hierarchical segmentation and binary threshold segmentation on the time series association graph to obtain time series density clustering, and performing boundary point extraction and time axis mapping on the time series density clustering to obtain time series time segmentation points; The temporal time segmentation points are merged and refined to obtain audio event boundaries, and based on the audio event boundaries, the temporal context representation is subjected to event-level segmentation and temporal aggregation to obtain a temporal dependency representation.

5. The adaptive audio enhancement method according to claim 1, characterized in that: The performing semantic hierarchical parsing and enhancement parameter mapping on the temporal dependency representation to obtain audio enhancement parameters includes: Performing hierarchical clustering on the temporal dependency representation to obtain a multi-scale semantic structure, and extracting corresponding local semantic features and global semantic features in the multi-scale semantic structure to obtain hierarchical semantic features; Performing feature importance scoring and distribution normalization on the hierarchical semantic features to obtain standardized feature weights, and performing weight weighting and spatial projection on the standardized feature weights to obtain key semantic features; Using a preset feature decision tree to perform feature grouping and threshold screening on the key semantic features to obtain initial enhancement parameters, and performing parameter sequence mutation detection and local correction on the initial enhancement parameters to obtain a continuous parameter curve; The continuous parameter curve is subjected to constraint optimization of multiple audio signal parameters to obtain a constraint parameter interval, and the constraint parameter interval is subjected to mapping of multiple audio channel parameters to obtain audio enhancement parameters.

6. The adaptive audio enhancement method according to claim 1, characterized in that: The step of adaptively filtering the original audio signal and reconstructing the corresponding sub-band audio signal based on the audio enhancement parameter to obtain a preliminary enhanced audio signal includes: Performing frequency segmentation and time domain conversion of the frequency response on the audio enhancement parameter to obtain an audio filter coefficient, and based on the audio filter coefficient, using a preset adaptive filter group to perform signal decomposition on the original audio signal to obtain a plurality of audio sub-band signals; Performing dynamic signal compression and gain adjustment on each of the audio sub-band signals to obtain a dynamically equalized audio sub-band signal, and performing fundamental frequency estimation and harmonic wave enhancement on the dynamically equalized audio sub-band signal to obtain an audio harmonic structure; Performing reverberation time estimation and cepstrum analysis on the audio harmonic structure to obtain the audio harmonic structure after cepstrum analysis, and using a preset inverse filter to dereverberate the audio harmonic structure after cepstrum analysis to obtain a dereverberated speech signal; Performing pitch extraction and pitch enhancement on the dereverberation speech signal to obtain a pitch-enhanced speech signal, and performing power spectrum estimation and Wiener filtering on the pitch-enhanced speech signal to obtain a denoised speech signal; The noise reduction speech signal is amplitude normalized and phase aligned to obtain a synchronous speech signal, and the synchronous speech signal is subjected to synthesis filtering and gain balancing to obtain a preliminary enhanced audio signal.

7. The adaptive audio enhancement method according to claim 1, characterized in that: The method of using a preset audio quality assessment network to evaluate the optimized preliminary enhanced audio signal to obtain an audio evaluation value, and performing multi-objective optimization and gradient adjustment on the audio enhancement parameters based on the audio evaluation value to obtain the adjusted audio enhancement parameters includes: Based on a preset audio quality assessment network, respectively calculating the spectral distortion between the preliminary enhanced audio signal and the original audio signal, performing sub-band energy statistics on the preliminary enhanced audio signal to obtain an audio loudness curve, performing cepstral coefficient extraction and nonlinear mapping on the preliminary enhanced audio signal to obtain a sound quality score value, and performing weighted superposition on the spectral distortion, the audio loudness curve and the sound quality score value to obtain an audio assessment value; Based on a preset audio evaluation threshold, performing time-frequency difference decomposition and nonlinear weighted mapping on the audio evaluation value to obtain an audio quality gain target, and performing multi-order gradient operation and feasible domain projection on the audio quality gain target to obtain a parameter adjustment direction; Based on the parameter adjustment direction, a step search of candidate parameters is performed on the audio enhancement parameters to obtain a candidate enhancement parameter set, and a constraint optimization calculation is performed on the candidate enhancement parameter set to obtain adjusted audio enhancement parameters.

8. An adaptive audio enhancement device, characterized in that: The adaptive audio enhancement device comprises: A feature extraction module, used to obtain an original audio signal to be enhanced, and extract multi-dimensional audio features from the original audio signal to obtain multi-dimensional audio features; A feature fusion module, used to perform multi-dimensional weighted fusion and dimensionality reduction on the multi-dimensional audio features to obtain fused audio features, and perform feature sequence encoding and correlation calculation on the fused audio features to obtain a temporal dependency representation; A preliminary enhancement module, configured to perform semantic hierarchical analysis and enhancement parameter mapping on the temporal dependency representation to obtain audio enhancement parameters, and based on the audio enhancement parameters, perform adaptive filtering on the original audio signal and reconstruction of the corresponding sub-band audio signal to obtain a preliminary enhanced audio signal; The secondary enhancement module is used to evaluate the optimized preliminary enhanced audio signal using a preset audio quality evaluation network to obtain an audio evaluation value, and based on the audio evaluation value, perform multi-objective optimization and gradient adjustment on the audio enhancement parameters to obtain adjusted audio enhancement parameters, and based on the adjusted audio enhancement parameters, perform secondary audio enhancement on the preliminary enhanced audio signal until the audio evaluation value is greater than a preset audio evaluation threshold, thereby generating a final enhanced audio signal.

9. A SoC chip, characterized in that: The SoC chip comprises: a memory and at least one processor, wherein instructions are stored in the memory; The at least one processor calls the instructions in the memory so that the SoC chip executes each step of the adaptive audio enhancement method as described in any one of claims 1-7.

10. A computer-readable storage medium having instructions stored thereon, characterized in that: When the instructions are executed by a processor, the steps of the adaptive audio enhancement method according to any one of claims 1 to 7 are implemented.

Images