Lightweight acoustic signal enhancement method based on adaptive time-frequency modeling, medium and device

Abstract

The application discloses a kind of based on adaptive time-frequency modeling light-weight sound signal enhancement method, medium and equipment, it is related to sound signal processing technology, based on adaptive time-frequency modeling light-weight sound signal enhancement method mainly includes: constructing sound signal noise reduction model;Utilize sound signal data training set to train sound signal noise reduction model and obtain trained sound signal noise reduction model;Utilize trained sound signal noise reduction model to carry out noise reduction to target sound signal, obtain the sound signal after noise reduction.It can realize the effective enhancement of sound signal in complex noise environment by implementing the based on adaptive time-frequency modeling light-weight sound signal enhancement method, medium and equipment provided by the application, improve the expression ability of model, environmental adaptability and signal restoration precision, reduce the cost of calculation.

Description

Technical Field

[0001] This invention relates to acoustic signal processing technology, and more specifically, to a lightweight acoustic signal enhancement method, medium, and device based on adaptive time-frequency modeling. Background Technology

[0002] Acoustic signal enhancement technology aims to recover a clearer target acoustic signal from an acoustic signal mixed with background noise, and it has important value in applications such as voice communication, speech recognition, and smart device interaction. As application scenarios become increasingly complex and non-stationary, multi-type noise environments become more common, traditional acoustic signal enhancement technologies are gradually revealing their limitations in terms of robustness and adaptability. This is especially true in scenarios such as industrial production, urban traffic, and voice monitoring, where acoustic signals are easily subject to significant interference, affecting the performance of subsequent tasks.

[0003] Traditional acoustic signal enhancement methods, such as spectral subtraction and Wiener filtering, are based on modeling the statistical characteristics of signals and noise and suppressing noise through spectral analysis. These methods are theoretically clear and computationally simple, and are suitable for some low-complexity scenarios. However, they are less adaptable to non-Gaussian and non-stationary noise, and usually ignore the more complex time dependence and frequency characteristics of acoustic signals, resulting in limited noise reduction performance.

[0004] To address the need for acoustic signal enhancement in complex environments, researchers have proposed several deep learning-based methods in recent years. These methods leverage the powerful feature extraction capabilities of deep neural networks to directly learn the mapping relationship from noisy signals to clean acoustic signals, significantly improving enhancement performance. Deep autoencoders, deep feedforward neural networks, recurrent neural networks (such as LSTM), and their variants have all achieved good results in enhancement tasks. Furthermore, generative adversarial networks (GANs) have been introduced into the field of acoustic signal enhancement, improving the realism of generated acoustic signals through adversarial learning. The Transformer architecture, due to its superior modeling ability for long-range dependencies, has also gradually become a research hotspot in recent years.

[0005] In summary, deep learning-based acoustic signal enhancement methods have become the mainstream development direction, demonstrating good robustness and performance scalability. However, as application demands further increase, achieving a better trade-off between model performance, complexity, and computational efficiency remains a key research topic.

[0006] While traditional acoustic signal enhancement methods offer advantages such as low computational cost and simple implementation, they generally rely on statistical assumptions about noise and the target acoustic signal, making them ineffective in dealing with complex and variable non-stationary noise in real-world environments. Furthermore, traditional methods typically only model the amplitude spectrum of the acoustic signal, neglecting phase information processing, which limits the reconstruction quality and results in significant shortcomings in the audibility and intelligibility of the enhanced acoustic signal.

[0007] While deep learning-based acoustic signal enhancement methods have made significant progress in performance, several challenges remain. Firstly, most existing methods rely on large-scale noisy-clear acoustic signal pairs for supervised training, but acquiring high-quality paired data is costly and difficult, impacting the flexibility of model deployment across multiple scenarios. Secondly, deep network models often have large parameter counts and high computational resource requirements, making them unsuitable for resource-constrained edge devices or real-time acoustic signal processing tasks, thus limiting their widespread application in lightweight computing platforms.

[0008] Furthermore, while some existing methods introduce attention mechanisms or complex network structures to enhance modeling capabilities, their feature modeling is mostly concentrated on the channel or spatial dimensions, with weak ability to model dynamic dependencies between the time and frequency domains. This results in unstable model performance when faced with rapidly changing spectra or high-noise scenarios. Simultaneously, current methods lack adaptive adjustment capabilities in feature weight allocation, making it difficult to dynamically adjust the region of interest for different types of noise, thus limiting the generalization and robustness of the enhancement algorithms in complex environments.

[0009] Therefore, there is an urgent need for a sound signal enhancement method with strong modeling capabilities, efficient structure, and time-frequency interactive modeling and feature adaptation capabilities, so as to improve its sound signal enhancement performance and practical application value in complex noise scenarios while ensuring the model is lightweight.

[0010] The above content is only used to help understand the technical solution of the present invention and does not represent an admission that the above content is prior art. Summary of the Invention

[0011] The purpose of this invention is to provide a lightweight acoustic signal enhancement method, medium, and device based on adaptive time-frequency modeling, which can effectively enhance acoustic signals in complex noise environments, improve the model's expressive power, environmental adaptability, and signal restoration accuracy, and reduce computational costs.

[0012] This invention provides a lightweight acoustic signal enhancement method based on adaptive time-frequency modeling, comprising the following steps: S1: Constructing an acoustic signal denoising model, the acoustic signal denoising model including a short-time Fourier transform module, an encoder, an adaptive time-frequency fusion Transformer module, a multi-path enhancement Taylor Transformer module, a decoder, and an inverse short-time Fourier transform module; S2: Training the acoustic signal denoising model using an acoustic signal data training set to obtain a trained acoustic signal denoising model; S3: Using the trained acoustic signal denoising model to denoise the target acoustic signal to obtain a denoised acoustic signal.

[0013] Furthermore, the short-time Fourier transform is used to decompose the noisy acoustic signal into an amplitude spectrum and a phase spectrum. The amplitude spectrum is then power-law compressed and concatenated with the phase spectrum to form a two-dimensional feature matrix. The encoder is used to extract multi-scale time-frequency features based on the two-dimensional feature matrix using multi-level convolution and downsampling modules, through dilated convolution and dense connections, and generate a low-resolution compact representation. The adaptive time-frequency fusion Transformer module is used to alternately process features in the time and frequency dimensions, using time-domain Transformer layers and frequency-domain Transformer layers respectively to capture long-range dependencies and key frequency band information. The multi-path enhancement Taylor Transformer module is used to fuse the output of the adaptive time-frequency fusion Transformer module to optimize feature representation. The decoder includes a parallel amplitude decoder and a phase decoder. The amplitude decoder generates noise-suppressed components through upsampling and masking to obtain the amplitude spectrum. The phase decoder independently learns phase features to preserve signal integrity and obtains the phase spectrum. The inverse short-time Fourier transform module is used to reconstruct the enhanced amplitude spectrum and phase spectrum into a time-domain signal through inverse short-time Fourier transform.

[0014] Furthermore, the encoder includes an initial convolutional block, a densely connected dilated convolutional layer, and a downsampled convolutional block connected in sequence; the initial convolutional block includes a two-dimensional convolutional layer, instance normalization, and a PReLU activation function; the downsampled convolutional block includes a two-dimensional convolutional layer, instance normalization, and a PReLU activation function.

[0015] Furthermore, the temporal Transformer layer is configured to: extract global information from the time series through an adaptive multi-head attention mechanism and apply residual connections to avoid the loss of original information; and through layer normalization processing, enter a feedforward network based on bidirectional GRU to capture the long-range dependence of the time series, so that the dynamic change pattern of the acoustic signal can be preserved.

[0016] Furthermore, the frequency domain Transformer layer is configured to: learn global spectral information through an adaptive multi-head attention mechanism, and after layer normalization processing, input it into a feedforward network based on bidirectional GRU to capture the characteristics of key frequency bands in the acoustic signal, while suppressing interference from non-target frequency band noise.

[0017] Furthermore, the adaptive multi-head attention mechanism is configured as follows: a two-dimensional convolution operation is performed on the output features of the multi-head attention mechanism to extract local contextual information, enhancing the local perception capability of the features; the convolution result generates a global feature distribution through a global average pooling layer; finally, the global features are mapped to normalized weights through a sigmoid activation function, as shown in the formula: , in, Indicates the adaptive weight value. This represents the Sigmoid function, used to restrict the weight range to (0,1); This represents the global feature distribution generated by the global average pooling layer; The generated dynamic weights are expanded to match the shape of the input features and applied element-wise to the output features of the multi-head attention mechanism, as shown in the formula: , in, This represents the output of the adaptive multi-head attention mechanism. This indicates a multi-head attention mechanism. These represent the query, key, and value, respectively. This indicates an element-wise multiplication operation.

[0018] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described lightweight acoustic signal enhancement method based on adaptive time-frequency modeling.

[0019] The present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the above-described lightweight acoustic signal enhancement method based on adaptive time-frequency modeling.

[0020] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the above-described lightweight acoustic signal enhancement method based on adaptive time-frequency modeling.

[0021] The lightweight acoustic signal enhancement method, medium, and device based on adaptive time-frequency modeling provided by this invention have the following beneficial effects: This invention constructs an acoustic signal enhancement network based on the U-Net framework. First, a short-time Fourier transform is performed on the input acoustic signal to extract the amplitude and phase spectra, which are then concatenated as feature inputs. Multi-scale features are extracted through multi-layer dilated convolution, and an adaptive time-frequency fusion Transformer module is introduced to achieve alternating modeling of the time and frequency dimensions. Subsequently, a multi-path enhancement Transformer structure is used to fuse channel and spatial attention information. A dynamic weight allocation strategy is introduced into the attention mechanism to improve the model's adaptability to complex noise environments. Finally, the acoustic signal spectrum is recovered through amplitude and phase decoders respectively, and the enhanced acoustic signal is reconstructed through inverse transform. This invention introduces the ATF-Transformer module to perform attention modeling and feature encoding in the time and frequency dimensions, effectively capturing the dynamic changes of acoustic signals in non-stationary noise environments, enhancing the model's ability to identify the continuity of acoustic signals and key frequency band information, and improving the time-frequency interactive modeling capability. This invention employs an adaptive multi-head attention module (AMHA), which can dynamically generate attention weights based on signal distribution, improve the model's adaptability to complex and unknown noise distributions, avoid over-smoothing of features and information loss, and realize a feature adaptive enhancement mechanism. While maintaining performance advantages, this invention has reasonable control over the overall parameters of the model. By combining multi-path fusion and sparse modeling strategies, it ensures the feasibility of edge deployment and real-time processing while reducing computational costs, and achieves a lightweight structural design. In summary, this invention achieves effective enhancement of acoustic signals in complex noise environments, improves the model's expressive power, environmental adaptability, and signal restoration accuracy, and reduces computational costs. It provides an efficient and scalable innovative solution for acoustic signal enhancement and intelligent audio processing, applicable to various acoustic signal enhancement scenarios such as voice communication, speech recognition preprocessing, and industrial audio analysis. Attached Figure Description

[0022] The present invention will be further described below with reference to the accompanying drawings and embodiments. In the accompanying drawings: Figure 1 This is a flowchart of the lightweight acoustic signal enhancement method based on adaptive time-frequency modeling provided by the present invention; Figure 2 This is a structural diagram of the acoustic signal noise reduction model provided by the present invention; Figure 3 This is a structural diagram of the encoder provided by the present invention; Figure 4 This is a diagram of the adaptive time-frequency fusion Transformer structure provided by the present invention; Figure 5 This is a structural diagram of the adaptive multi-head attention mechanism provided by the present invention; Figure 6 This is a structural block diagram of the computer device provided by the present invention. Detailed Implementation

[0023] To provide a clearer understanding of the technical features, objectives, and effects of the present invention, specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0024] Figure 1 A schematic diagram of the lightweight acoustic signal enhancement method based on adaptive time-frequency modeling of this embodiment is shown. In this embodiment, the lightweight acoustic signal enhancement method based on adaptive time-frequency modeling includes the following steps: S1: Construct a noise reduction model for acoustic signals; In one exemplary embodiment, the acoustic signal noise reduction model includes a short-time Fourier transform module, an encoder, an adaptive time-frequency fusion Transformer module, a multi-path enhanced Taylor Transformer module, a decoder, and an inverse short-time Fourier transform module. In an exemplary embodiment, the short-time Fourier transform is used to decompose the noisy acoustic signal into an amplitude spectrum and a phase spectrum. The amplitude spectrum is then power-law compressed and concatenated with the phase spectrum to form a two-dimensional feature matrix. The encoder is used to extract multi-scale time-frequency features based on the two-dimensional feature matrix using a multi-level convolution and downsampling module, through dilated convolution and dense connections, and generate a low-resolution compact representation. The adaptive time-frequency fusion Transformer module is used to alternately process features in the time and frequency dimensions, using time-domain Transformer layers and frequency-domain Transformer layers respectively to capture long-range dependencies and key frequency band information. The multi-path enhancement Taylor Transformer module is used to fuse the output of the adaptive time-frequency fusion Transformer module to optimize feature representation. The decoder includes a parallel amplitude decoder and a phase decoder. The amplitude decoder generates noise-suppressed components through upsampling and masking to obtain the amplitude spectrum. The phase decoder independently learns phase features to preserve signal integrity and obtains the phase spectrum. The inverse short-time Fourier transform module is used to reconstruct the enhanced amplitude spectrum and phase spectrum into a time-domain signal through inverse short-time Fourier transform. In one exemplary embodiment, the encoder includes an initial convolutional block, a densely connected dilated convolutional layer, and a downsampled convolutional block connected in sequence; the initial convolutional block includes a two-dimensional convolutional layer, instance normalization, and a PReLU activation function; the downsampled convolutional block includes a two-dimensional convolutional layer, instance normalization, and a PReLU activation function. In one exemplary embodiment, the temporal Transformer layer is configured to: extract global information from the time series through an adaptive multi-head attention mechanism and apply residual connections to avoid the loss of original information; and through layer normalization processing, enter a feedforward network based on bidirectional GRU to capture the long-range dependence of the time series, so that the dynamic change pattern of the acoustic signal can be preserved. In one exemplary embodiment, the global information in the time series is extracted using an adaptive multi-head attention mechanism, as shown in the formula: , in, This represents the output features of the temporal Transformer layer. To adapt to multi-head attention mechanisms, The input features are those of the temporal Transformer layer. In one exemplary embodiment, the frequency domain Transformer layer is configured to: learn global spectral information through an adaptive multi-head attention mechanism, and after layer normalization processing, input it into a feedforward network based on bidirectional GRU to capture the characteristics of key frequency bands in the acoustic signal, while suppressing interference from non-target frequency band noise; In one exemplary embodiment, the long-range dependency and key frequency band information are captured using time-domain Transformer layers and frequency-domain Transformer layers, respectively, as shown in the formula: , in, To adapt to multi-head attention mechanisms, This represents the output characteristics of the frequency domain Transformer layer. The input features are those of the frequency domain Transformer layer. In one exemplary embodiment, the adaptive multi-head attention mechanism includes a multi-head attention mechanism and an adaptive weight allocation module, wherein the adaptive weight allocation module includes a two-dimensional convolutional layer, a global average pooling layer, and a Sigmoid activation function. In one exemplary embodiment, the adaptive multi-head attention mechanism is configured as follows: a two-dimensional convolution operation is performed on the output features of the multi-head attention mechanism to extract local contextual information, enhancing the local perception capability of the features; the convolution result generates a global feature distribution through a global average pooling layer; finally, the global features are mapped to normalized weights through a Sigmoid activation function, as shown in the formula: , in, Indicates the adaptive weight value. This represents the Sigmoid function, used to restrict the weight range to (0,1); This represents the global feature distribution generated by the global average pooling layer; The generated dynamic weights are expanded to match the shape of the input features and applied element-wise to the output features of the multi-head attention mechanism, as shown in the formula: , in, This represents the output of the adaptive multi-head attention mechanism. This indicates a multi-head attention mechanism. These represent the query, key, and value, respectively. This represents an element-wise multiplication operation; S2: Train the acoustic signal denoising model using the acoustic signal data training set to obtain a trained acoustic signal denoising model; S3: Use the trained acoustic signal denoising model to denoise the target acoustic signal to obtain the denoised acoustic signal.

[0025] In some embodiments, the lightweight acoustic signal enhancement method based on adaptive time-frequency modeling described above can also be implemented in the following manner. In this embodiment, the lightweight acoustic signal enhancement method based on adaptive time-frequency modeling includes the following steps: It should be noted that this embodiment proposes an improved acoustic signal denoising model based on MUSE, namely ATF-MUSE. The core improvement is the introduction of an Adaptive Time-frequency Fusion Transformer (ATF-Transformer). The design focus of ATF-Transformer is to enhance the interactive modeling capability in the time and frequency domains, while improving the model's adaptability to better cope with complex noise environments in industrial applications. The ATF-MUSE model structure is as follows: Figure 2 As shown; The ATF-MUSE model is built on the U-Net framework. First, the input noisy industrial equipment acoustic signal is decomposed into an amplitude spectrum through short-time Fourier transform (STFT). and phase spectrum After power-law compression of the amplitude spectrum, it is concatenated with the phase spectrum to form a two-dimensional feature matrix. The encoder is used as input; it consists of multi-level convolution and downsampling modules, which extract multi-scale time-frequency features through dilated convolution and dense connections, and generate a low-resolution compact representation; the encoder structure is as follows. Figure 3 As shown; After the encoder, an ATF-Transformer module is introduced. The ATF-Transformer captures long-range dependencies and key frequency band information by alternately processing features in the time and frequency dimensions and using time-domain Transformer layers and frequency-domain Transformer layers respectively. Subsequently, the Multi-path Enhanced TaylorTransformer (MET-Transformer) module fuses T-MHA and CSA in parallel paths to optimize feature representation; among them, deformable embedding (DE) technology generates multi-scale tokens to enhance the model's adaptability to complex acoustic signals; The decoder adopts a dual-branch structure: the amplitude decoder generates noise-suppressed components through upsampling and masking, while the phase decoder independently learns phase features to preserve signal integrity. The specific structure is consistent with the decoder structure of MUSE. Finally, the enhanced amplitude spectrum and phase spectrum are reconstructed into a time-domain signal through inverse short-time Fourier transform (ISTFT). The model achieves efficient noise reduction in complex industrial noise scenarios through lightweight design and dynamic weight allocation mechanism. The ATF-Transformer, based on a dual-path attention structure, enhances the model's ability to interactively represent temporal and frequency features. The ATF-Transformer receives the feature representation output from the encoder, with the shape of... ,in Indicates batch size. For the number of channels, and These are time frame count and frequency resolution, respectively. To better model the dynamic interaction between time and frequency features, the ATF-Transformer employs a mechanism that alternately processes the time and frequency dimensions. First, the input features are reshaped into shape... The signal is fed into a time-domain Transformer layer to capture the temporal dependence of the acoustic signal; subsequently, the features are reconstructed into... The data is fed into a frequency domain Transformer layer to capture global characteristics and key frequency band information in the frequency dimension. After alternating processing, the features are reconstructed into their original shape. This is then used as an intermediate representation for output to subsequent modules; the specific structure is as follows: Figure 4 As shown; The processing of the temporal Transformer layer mainly focuses on the temporal dimension features, and the input features are... First, global information in the time series is extracted using an adaptive multi-head attention mechanism, and residual connections are applied to avoid the loss of original information. The calculation formula is shown in equation (1):

[0026] In the formula To adapt to multi-head attention mechanisms, By dynamically generating weights to adaptively adjust feature focus, the capabilities of traditional multi-head attention are extended; features are processed... After the layer, through layer normalization processing, it enters the feedforward network based on bidirectional GRU (Bi-GRU). The forward and backward features of the bidirectional GRU capture the long-range dependence of the time series, so that the dynamic change pattern of the sound signal can be preserved. Similarly, the frequency domain Transformer layer processes features specific to the frequency dimension; input features First, global spectral information is learned through an adaptive multi-head attention mechanism, and its calculation formula is shown in equation (2):

[0027] Subsequently, the features are processed by layer normalization and then input into a feedforward network based on bidirectional GRU; the frequency domain Transformer layer can capture the characteristics of key frequency bands in the acoustic signal while suppressing the interference of non-target frequency band noise. Through modeling in both the time domain and frequency domain, the ATF-Transformer demonstrates stronger modeling capabilities in the dynamic interaction of time and frequency features. Modeling in the time dimension can capture the continuous characteristics and temporal variation patterns of acoustic signals, while modeling in the frequency dimension extracts key frequency band information and effectively suppresses background noise. Finally, the features processed by the ATF-Transformer are reconstructed into the original shape. This is then used as an intermediate representation and output to the subsequent MET-Transformer module; To further enhance the model's adaptability to complex noisy environments, this embodiment introduces an adaptive weight allocation module based on the multi-head attention (MHA) mechanism, designing an Adaptive Multi-head Attention Module (AMHA), the specific structure of which is as follows: Figure 5 As shown; Traditional multi-head attention mechanisms extract global information about input features by calculating the correlation between queries, keys, and values. However, in industrial noise scenarios, noise types are diverse and their distribution is complex, making it difficult for fixed attention mechanisms to dynamically adapt to changes in input features. The adaptive attention module introduces a dynamic weight generation mechanism, enabling the model to flexibly adjust attention weights according to the distribution of input features, thereby improving the model's adaptability. The core improvement of the adaptive multi-head attention module lies in combining the traditional MHA mechanism with an adaptive weight generation strategy. First, the output features of MHA are subjected to 2D convolution to extract local context information, thereby enhancing the local perception capability of the features. The convolution result is used to generate a global feature distribution through a global average pooling layer. Finally, the global features are mapped to normalized weights through the sigmoid activation function, as shown in Equation (3).

[0028] In the formula This represents the Sigmoid function, used to restrict the weight range to (0,1); The generated dynamic weights are extended to a shape that matches the input features and are applied element-wise to the output features of the MHA, as shown in Equation (4);

[0029] in This represents an element-wise multiplication operation, which enhances the model's ability to focus on key regions by adaptively adjusting feature weights. The adaptive multi-head attention module enhances the model's dynamic adaptability and feature representation ability. First, through the dynamic weight generation mechanism, the model can adjust the weight distribution in real time according to the global and local characteristics of the input signal, thereby better adapting to different noise scenarios. Second, the combination of convolution and global average pooling enables features to retain both local and global information, thus effectively capturing the time-frequency characteristics of complex signals. In addition, sigmoid activation ensures the numerical stability of the generated weights and avoids gradient anomaly problems. After constructing the acoustic signal denoising model, it is trained using the acoustic signal data training set. The trained acoustic signal denoising model is then used to denoise the target acoustic signal to obtain the denoised acoustic signal.

[0030] This embodiment provides a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the steps of the aforementioned lightweight acoustic signal enhancement method based on adaptive time-frequency modeling. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk drive (HDD), or solid-state drive (SSD), etc.; the storage medium may also include combinations of the above-mentioned types of memory.

[0031] This embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the steps of the above-described lightweight acoustic signal enhancement method based on adaptive time-frequency modeling.

[0032] like Figure 6As shown, the computer device 120 may include: at least one processor 121, such as a central processing unit (CPU), at least one communication interface 123, memory 124, and at least one communication bus 122. The communication bus 122 is used to enable communication between these components. The communication interface 123 may include a display screen and a keyboard; optionally, the communication interface 123 may also include a standard wired interface or a wireless interface. The memory 124 may be high-speed random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Optionally, the memory 124 may also be at least one storage device located remotely from the aforementioned processor 121. The memory 124 stores application programs, and the processor 121 calls the program code stored in the memory 124 to execute any of the aforementioned method steps. The communication bus 122 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The communication bus 122 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 6The term 124 is represented by a single line, but this does not imply a single bus or a single type of bus. The memory 124 may include volatile memory, such as random-access memory (RAM); it may also include non-volatile memory, such as flash memory, hard disk drive (HDD), or solid-state drive (SSD); or it may include combinations of the above types of memory. The processor 121 may be a central processing unit (CPU), a network processor (NP), or a combination of a CPU and an NP. The processor 121 may further include hardware chips. These hardware chips may be application-specific integrated circuits (ASICs), programmable logic devices (PLDs), or combinations thereof. The PLD may be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof. Optionally, the memory 124 is also used to store program instructions. The processor 121 can invoke the program instructions to implement the lightweight acoustic signal enhancement method based on adaptive time-frequency modeling as described in this embodiment.

[0033] This embodiment provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the above-described lightweight acoustic signal enhancement method based on adaptive time-frequency modeling.

[0034] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims. All of these forms are within the protection scope of the present invention.

Claims

1. A lightweight acoustic signal enhancement method based on adaptive time-frequency modeling, characterized in that, Includes the following steps: S1: Construct an acoustic signal noise reduction model, which includes a short-time Fourier transform module, an encoder, an adaptive time-frequency fusion Transformer module, a multi-path enhancement Taylor Transformer module, a decoder, and an inverse short-time Fourier transform module. S2: Train the acoustic signal denoising model using the acoustic signal data training set to obtain a trained acoustic signal denoising model; S3: Use the trained acoustic signal denoising model to denoise the target acoustic signal to obtain the denoised acoustic signal; The short-time Fourier transform module decomposes the noisy acoustic signal into an amplitude spectrum and a phase spectrum. The amplitude spectrum is then power-law compressed and concatenated with the phase spectrum to form a two-dimensional feature matrix. The encoder, based on the two-dimensional feature matrix, uses a multi-level convolution and downsampling module to extract multi-scale time-frequency features through dilated convolution and dense connections, generating a low-resolution compact representation. The adaptive time-frequency fusion Transformer module alternately processes features in both time and frequency dimensions, using time-domain and frequency-domain Transformer layers to capture long-range dependencies and key frequency band information. The multi-path enhancement Taylor Transformer module fuses the output of the adaptive time-frequency fusion Transformer module to optimize feature representation. The decoder includes a parallel amplitude decoder and a phase decoder. The amplitude decoder generates a noise-suppressed component through upsampling and masking to obtain the amplitude spectrum. The phase decoder independently learns phase features to preserve signal integrity, obtaining the phase spectrum. The inverse short-time Fourier transform module reconstructs the enhanced amplitude and phase spectra into a time-domain signal using inverse short-time Fourier transform.

2. The lightweight acoustic signal enhancement method based on adaptive time-frequency modeling according to claim 1, characterized in that, The encoder includes an initial convolutional block, a densely connected dilated convolutional layer, and a downsampled convolutional block connected in sequence; the initial convolutional block includes a two-dimensional convolutional layer, instance normalization, and a PReLU activation function; the downsampled convolutional block includes a two-dimensional convolutional layer, instance normalization, and a PReLU activation function.

3. The lightweight acoustic signal enhancement method based on adaptive time-frequency modeling according to claim 1, characterized in that, The temporal Transformer layer is configured to extract global information from the time series through an adaptive multi-head attention mechanism and apply residual connections to avoid the loss of original information. Through layer normalization, the signal is fed into a bidirectional GRU-based feedforward network to capture the long-range dependence of the time series, thus preserving the dynamic change pattern of the acoustic signal.

4. The lightweight acoustic signal enhancement method based on adaptive time-frequency modeling according to claim 1, characterized in that, The frequency domain Transformer layer is configured to learn global spectral information through an adaptive multi-head attention mechanism, and after layer normalization, input it into a feedforward network based on bidirectional GRU to capture the characteristics of key frequency bands in the acoustic signal, while suppressing interference from non-target frequency band noise.

5. The lightweight acoustic signal enhancement method based on adaptive time-frequency modeling according to claim 3 or 4, characterized in that, The adaptive multi-head attention mechanism is configured as follows: Two-dimensional convolution is performed on the output features of the multi-head attention mechanism to extract local contextual information, enhancing the local perceptual ability of the features; the convolution result generates a global feature distribution through a global average pooling layer; finally, the global features are mapped to normalized weights using a Sigmoid activation function, as shown in the formula: ， in, Indicates the adaptive weight value. This represents the Sigmoid function, used to restrict the weight range to (0,1); This represents the global feature distribution generated by the global average pooling layer; The generated dynamic weights are expanded to match the shape of the input features and applied element-wise to the output features of the multi-head attention mechanism, as shown in the formula: ， in, This represents the output of the adaptive multi-head attention mechanism. This indicates a multi-head attention mechanism. These represent the query, key, and value, respectively. This indicates an element-wise multiplication operation.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the steps of the lightweight acoustic signal enhancement method based on adaptive time-frequency modeling as described in any one of claims 1-5.

7. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the lightweight acoustic signal enhancement method based on adaptive time-frequency modeling as described in any one of claims 1-5.

8. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the lightweight acoustic signal enhancement method based on adaptive time-frequency modeling as described in any one of claims 1-5.

Images