Audio signal processing device, program, and audio signal processing method

The dynamic sliding window strategy with data padding and a lightweight network structure addresses real-time audio denoising challenges, ensuring stable and high-quality audio playback despite fluctuating CPU usage.

JP7588679B2Active Publication Date: 2024-11-22THE TRUSTEES OF COLUMBIA UNIV IN THE CITY OF NEW YORK +1
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
JP2023076242
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Priority Date
2022-05-05
Filing Date
2023-05-02
Publication Date
2024-11-22
Estimated Expiration
2043-05-02

AI Technical Summary

Technical Problem

Existing real-time audio denoising techniques face challenges in maintaining real-time performance due to fluctuations in CPU occupancy, leading to long delays and choppy audio playback when using fixed-size sliding windows.

Method used

A dynamic sliding window strategy is employed, allowing the network to process data without waiting for a fixed buffer to fill, combined with data padding and a lightweight denoising network structure, ensuring consistent denoising quality and reduced delays.

Benefits of technology

The proposed method achieves comparable denoising quality to offline methods while maintaining stable and short delays, even in real-world scenarios with varying CPU availability, making it robust to background processes.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007588679000013
    Figure 0007588679000013
  • Figure 0007588679000014
    Figure 0007588679000014
  • Figure 0007588679000015
    Figure 0007588679000015
Patent Text Reader

Abstract

To provide a lightweight denoising network using a dynamic sliding window.SOLUTION: The denoising network receives, as an input, a spectrogram acquired by applying STFT based on a dynamic sliding window; and outputs a mask of a complex value corresponding to the input spectrogram through a 2D convolution layer, a unidirectional LSTM, and a complete connection layer. Then, the denoising network applies the mask to the input spectrogram to generate a denoised spectrogram; and applies an inverse STFT to obtain a denoised audio signal.SELECTED DRAWING: Figure 4
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to real-time, denoising, sliding window, neural network, data streams. [Background technology]

[0002] The following documents relate to the present invention: [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] Santiago Pascual, Antonio Bonafonte, and Joan Serra, "Segan: Speech enhancement generative adversarial network," arXiv preprint arXiv:1703.09452, 2017. [Non-Patent Document 2] Francois G Germain, Qifeng Chen, and Vladlen Koltun, "Speech denoising with deep feature losses," arXiv preprint arXiv:1806.10522, 2018. [Non-Patent Document 3] Craig Macartney and Tillman Weyde, "Improved speech enhancement with the wave-u-net," arXiv preprint arXiv:1811.11307, 2018. [Non-Patent Document 4] Google Scholar Szu-Wei Fu , Chien-Feng Liao , Yu Tsao , and Shou-De Lin , " Metricgan: Generative Adversarial Networks Based Black-Box Metric Scores Optimization for Speech Enhancement ," in International Conference on Machine Learning. 2031-2

Direct Environment 5

Outdoor Configuration6

Direct Environment 7

Non-Patent Document 11

Non-Patent Document 12

Non-Patent Document 13

Non-Patent Document 14

[0004] These and other aspects of the invention will be apparent from and elucidated with reference to the embodiment(s) described hereinafter.

[0005] [Figure 1] 13 illustrates a schematic of dynamic versus fixed magnitude sliding windows. [Diagram 2] 1 shows a schematic diagram of data padding. [Diagram 3] 1 shows a schematic comparison of real-time performance. [Figure 4] 1 illustrates a schematic network structure. [Diagram 5]1 illustrates a schematic representation of denoised audio. [Figure 6] An example of the hardware configuration of a computer 1200 functioning as the data processing device 100 is illustrated in schematic form. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0006] 1. Introduction Real-time speech denoising is an extremely demanding - perhaps the most demanding - audio processing task ever, as our world remains darkened by the coronavirus disease (COVID) pandemic and online meetings have become the "new normal" of our daily social life. Especially recently, all prior art real-time denoising techniques are based on neural networks [1, 2, 3, 4, 5, 6, 7, 8]. They explore novel network architectures to achieve reasonable denoising quality while retaining network simplicity to reduce processing time.

[0007] Unlike offline audio denoising, audio signals in a real-time setting are provided in a streaming manner. The network must process signal samples as they arrive and generate output samples with the shortest possible delay. As a result, a common strategy in nearly all real-time techniques is to use a fixed-length sliding window buffer.

[0008] This seems like a natural choice: the network waits until the sliding window buffer is filled with incoming audio samples and then denoises the data in that buffer. The resulting signal data is later fed to a real-time audio player. In this way, the network expects an input signal of fixed length L. As long as the processing time of the network is always shorter than L, the overall delay between the arrival of a signal sample and the playback time of the corresponding denoised sample is bounded, greater than L and less than 2L (see the analysis in Section 2.1).

[0009] However, in practice, it is difficult, if not impossible, to choose a buffer length L that ensures real-time performance. A large L will cause a long delay; a small L may result in a network processing time longer than L, resulting in choppy audio playback. This is because in a real computing environment, there are always other background processes (e.g., 4K video playback and gaming). The shorter L is, the more susceptible the denoising network is to the CPU occupancy of other processes. In short, the sliding window strategy, although fundamental for real-time denoising, remains elusive to careful investigation.

[0010] The inventors propose a different sliding window strategy, namely dynamic sliding window. In our approach, the length of the input buffer is not fixed. The network takes the currently buffered data regardless of its length and starts processing it without waiting. While the network is running, newly received data is accumulated in the buffer and is ready to be processed when the network finishes the current round of denoising. This sliding window strategy is conceptually simple, but is more robust to the CPU occupancy of other processes, thereby resulting in shorter and more stable delays. To show its advantages, the inventors publicly analyze the audio playback delay caused by our approach and a commonly used fixed-size sliding window. To the best of our knowledge, this is the first time that network delays under different sliding window strategies are investigated.

[0011] Many existing real-time denoising networks (e.g., [6, 7, 8]) cannot easily integrate dynamic sliding windows (see the discussion in Section 2.2). Therefore, we propose a lightweight denoising network adapted for the real-time setting: accepting streaming signals and processing them using dynamic sliding windows. By carefully padding and reusing data across the sliding window, our network does not suffer from loss in denoising quality compared to the offline non-streaming case.

[0012] We have performed extensive experiments comparing our proposed model with several prior art real-time denoising methods. The results show that our proposed model obtains comparable denoising quality in terms of all quality metrics, but with the lowest playback lag among all compared methods. Most importantly, compared to prior real-time denoising methods [6], our model is more robust in maintaining real-time performance in real-world scenarios where other background tasks, such as Zoom meetings, 4K video editing, and video games, may preempt CPU cycles.

[0013] 2. Method We start by analyzing the delay in audio playback when a fixed-size sliding window is used, which is compared to the delay when using our proposed dynamic sliding window (Section 2.1). Inspired by our analysis, we then propose a lightweight denoising network that exploits the dynamic sliding window for faster and more robust denoising (Section 2.2).

[0014] Figure 1 shows a schematic of dynamic vs. fixed size sliding windows. Different windows are indicated by different hatching. Each window is processed by the network individually. The grey areas indicate periods when CPU cycles are occupied by other processes, thus increasing the network processing time.

[0015] 2.1 Sliding Windows for Data Streams Fixed-size sliding windows. In virtually all existing network-based denoising models, the incoming audio signal is treated as contiguous, non-overlapping windows [X1, X2, …] in time. Each window is filled with a constant length L of audio samples (i.e., X i ∈R Lhosts. Network F captures the latest unprocessed window X i and outputs the noise-removed result F(X i ), and then waits until the next window X i+1 is satisfied (see (a) of FIG. 1). t k represents the processing time of the network for window X k . Although the window size is fixed, in practice, t k should be noted that it varies over time due to the CPU occupancy of other background processes. In the analysis of the present inventor, from the moment X i is received until the moment the network outputs F(X i ), the delay (or lag) d i is shown to be expressed as follows.

Equation

[0016] The derivation of Equation (1) is not trivial. Since space is limited here, the inventor skips the details of the derivation, but the inventor has made it generally available online 1 .

[0017] The inventor's analysis (1) states the following: In the ideal case where <L is always satisfied, the noise removal network runs smoothly in real time without accumulation of delay; the playback delay has an upper limit of 2L. However, in practice, the execution of the network is often affected by other background calculation processes, and its processing time t i may be longer than L. At the same time, audio samples of length t i arrive and accumulate in the buffer. To process this amount of data, the network has to i execute the number of times of

Equation

[0018] Dynamic Sliding Window. We propose to dynamically adjust the size of the sliding window. When the network is moving the data window X i Immediately after processing, the newly received data in the buffer is i , which may or may not be larger than L. Regardless of the length of the buffer, we denoise the available buffered data without waiting. Figure 1(b) illustrates the process. In this strategy, a window X i Delay for playing d i is as follows:

number

[0019] When i=1, the delay of the first window is d1=L0+t1, where L0 is the initial window size to begin the denoising process at the beginning. We again refer the reader to our online reference for the derivation of equation (2).

[0020] In this analysis, d i There are two consecutive windows X k-1 and X k It has been shown that the delay depends only on the processing time of the network. In contrast to a fixed-size sliding window (shown in (1)), there is no delay to accumulate. Therefore, our approach is more robust to fluctuations in computing power. This is a notable advantage because in a real computing environment, the computing power of the denoising is constantly changing (see Section 3.3).

[0021] 2.2 Network Structure and Data Padding Although the dynamic sliding window strategy is independent of a specific network structure, many existing real-time denoising networks [6, 7, 8, 9, 10] cannot be easily adapted to take advantage of it. Some of them require a given sliding window size before the network runs [6]. Others have focused on reducing the inference cost of the network, but it remains unclear how they handle the incoming data stream [7, 8, 9, 10].

[0022] Proposed network structure. We propose a lightweight denoising network that uses a dynamic sliding window. Our network is built on the denoising components of [5] (and is therefore much simpler than the denoising model there). The input to our network is a data window X i The spectrogram s obtained by applying the STFT to x The spectrogram s x is first processed by a 2D convolutional layer with kernel size (5,5) and dilation (1,1) in the time-frequency domain. The resulting feature maps are fed into a unidirectional LSTM

[11] with hidden size 400. Finally, three fully connected layers with hidden sizes (400, 600, 512) are applied for each time bin. Similar to other speech enhancement models [5, 12, 13], our network optimizes s x Finally, the denoised audio signal is processed by the inverse STFT of

number

number

[0023] During training, we optimize the subsequent loss function:

number

number

[0024] Figure 2 shows a schematic illustration of data padding. Two consecutive windows of size 8 and 6 are processed by two 1D convolutions with kernel size 3. In this case, we estimate the two future elements for each window (window X i 22, 23) and pad the preceding window (processing window X i We reuse two elements of the result of the first convolution from 15, 16) obtained from

[0025] Data padding. Our network (and many others) has convolutional layers, which require padding to process boundary data. To handle streaming data in a sliding window fashion, this means that we need to pad the data window so that it is not too late to process enough “future” data (e.g., window X in Fig. 2). i-1This means that we need to buffer the padding data (blocks 16 and 17 for ). Waiting for the arrival of the padding data introduces an additional delay (48 ms in our implementation). However, we can reuse the result of the convolution with the padding data of the next sliding window (see the diagram in Fig. 2). Apart from saving some computational cost, reusing the padding data is crucial for the denoising quality of the network. It ensures that our network maintains the same denoising quality as if it suddenly took in the entire signal. Perhaps surprisingly, such a guarantee is still lacking in existing real-time denoising networks (see experiments in Section 3.2).

[0026] 3. Experiment Our experiments are two-fold: we evaluate the denoising quality of our network by comparing it with prior art real-time denoising models (Section 3.2). We then demonstrate the performance advantage of a dynamic sliding window over a traditional fixed-size sliding window approach (Section 3.3).

[0027] Experimental Setup Datasets. We perform experiments on two publicly available datasets. The first one, provided by Xu et al. [5], contains clean audio selected from AVSPEECH

[12] and noise from AudioSet

[14] and DEMAND

[15] . We call this dataset the ADD dataset. In addition, we also validate the benchmark of Valentini

[16] , which contains audio clips from 28 speakers; each clip has corresponding clean and noise versions.

[0028] Evaluation metrics. To evaluate the quality of the noise reduction, we use the following widely used objective metrics: (i) STOI: Short-Time Objective Intelligibility

[17] ; (ii) PESQ: Perceptual evaluation of speech quality (we use the narrowband version)

[18] ; (iii) CSIG: MOS predictor of code distortion

[19] ; ​​(iv) CBAK: MOS predictor of background noise aggressiveness

[19] ; ​​(v) COVL: MOS predictor of overall quality

[19] ; ​​(vi) SSNR: Segmental Signal-to-Noise Ratio

[20] .

[0029] We use two metrics to evaluate the real-time performance of a network: average network processing time (D N ) and maximum audio playback delay (D A ). D N teeth

number

[0030] [Table 1] [Table 1] Denoising quality for the AAD dataset in both offline (top) and real-time (bottom) settings. All scores are averaged over the input audio at SNRs [-10, -7, -3, 0, 3, 7, 10]. Runners-up are underlined.

[0031] 3.2. Real-time audio noise reduction quality We compared our denoising model with several recently proposed real-time denoising networks, including Demucs48[6], FullSub[7], and RNN-Mod[8]. We train these models on the same aforementioned dataset to evaluate the quality of denoising in both real-time and offline settings. In the offline setting, the audio signal is provided immediately, so no sliding window is required. By comparing the denoising quality of the two settings from the same network, we hope to understand to what extent the sliding window strategy affects the quality of denoising.

[0032] For Demucs48, we use the sliding window implementation provided. FullSub and RNN-Mod do not provide streaming implementations, and we found that their denoising quality becomes unstable when dynamic sliding windows are added. Therefore, we adopt for them a fixed size sliding window of 80 ms with 16 ms padding at the past and future ends. We choose this sliding window setting because our experiments have shown that it provides the best possible real-time denoising quality while maintaining acceptable latency.

[0033] Table 1 summarizes the results of the evaluation on the ADD dataset. Our model has the best or comparable real-time denoising quality for all quality metrics. It is also worth noting that our model is the only one that ensures the same real-time denoising quality as its offline counterpart thanks to its data padding strategy. All other models experience a degradation in quality when switching from the offline setting to the real-time setting. Furthermore, the real-time denoising quality results for the Valentini benchmark are reported in Table 2.

[0034] 3.3.Real-time performance Controlled Experiments. First, we compare the D N and D A (See Table 3). These models take in input data of different lengths, and we also measure the network execution time to process a 200 millisecond signal at once, without splitting it into multiple windows. All measurements were done without extensive background processing. The results show that in a dedicated computing environment, our network is as fast as the prior art models.

[0035] [Table 2] [Table 2] Valentini noise reduction quality.

[0036] [Table 3] [Table 3] Timing comparison. D N and D. A In addition, the inventors also recorded 200 milliseconds of audio (S N ) where the numbers include the mean and standard deviation of the timing.

[0037] Next, we conduct controlled experiments to understand the real-time performance of dynamic sliding windows and fixed-size sliding windows in the presence of CPU resource variations. To that end, we create two denoising models: both use the same denoising network trained on the AAD dataset (Section 2.2). The first one uses a dynamic sliding window (referred to as D-model), while the second one uses a fixed-size sliding window (referred to as F-model). To make a fair comparison, we set the initial window size L0 of the D-model to be the same as the fixed window size of the F-model (L0=16 ms). We use the two models to denoise the same set of audio, each of which has a duration of 5 seconds. To improve the computational power variations, after 2 seconds, we intentionally delay the network processing by a factor s, where s is randomly selected from [1, 7]. This simulates CPU occupancy by background processes in a controlled way, when we ensure that the same length of delay is added to both the D-model and the F-model.

[0038] Figure 3 shows the D measured as the audio stream arrives over time. N and D. A At the beginning, both can run smoothly in real time (with a delay of N A <100 ms). After 2 seconds, the computational power starts to fluctuate, preventing some sliding windows from being processed in time. As a result, the F model regeneration lag D A This causes the audio playback to become choppy. In contrast, the D model A remains stable because it can dynamically increase the window size to catch up with the delay. This experiment confirms our theoretical analysis in equations (1) and (2).

[0039] Figure 3 shows a schematic comparison of real-time performance between dynamic and fixed sliding window approaches. We artificially randomly increase the network processing time by a factor of 1-7 after 2 seconds to simulate variations in CPU power. The curves are averaged over 100 trials with a fixed random seed.

[0040] Real-world experiments. We next examine the real-time performance of our model in real-world scenarios, where background processes may preempt CPU cycles. Here, we denoise the same set of audio while different software runs in the background, including 4K video playback, Zoom meetings, iMovie video editing, and the video game Apex. To run the Apex game, we use a Windows 10 PC with an 8-core Intel CPU (3.60GHz i7-9700K) and GPU (NVIDIA GeForce RTX 2070 SUPER); other software tests are done on a Macbook Pro (2.3GHz Inter Quad-Core i5). We choose these software because they require more CPU cycles.

[0041] [Table 4] [Table 4] Demucs48 and our real-time denoising delays when running other background software. Each cell shows the D A (D N ) The numbers were measured using 20 seconds of audio.

[0042] The inventors ran these software individually, and used their models and Demucs48, respectively, to calculate the D N and D. AWe measure the delay in the playback of Demucs48, which has its own streaming implementation and provides comparable denoising quality to ours. The results are reported in Table 4. When background processes become quite computationally intensive (e.g., iMovie and Apex), the playback delay of Demucs48 increases dramatically, whereas the delay of our model remains mild and stable. This is clear evidence of the performance advantage of our model and dynamic sliding window strategy.

[0043] 4. Conclusion We have proposed a dynamic sliding window strategy for real-time speech denoising. Through careful analysis and experiments, we have demonstrated its advantages over the widely used fixed-size sliding window strategy. Our denoising network achieves real-time denoising quality comparable to SOTA while retaining a short lag by utilizing a dynamic sliding window. Notably, it enables our model to perform robustly in real-world scenarios in the presence of other background tasks.

[0044] 6 shows a schematic example of a hardware configuration of a computer 1200 functioning as a data processing device 100, a CEP device 200, a batch processing device 300, or a selection device 400. A program installed on the computer 1200 can cause the computer 1200 to function as one or more "units" of the device according to the present embodiment, or to perform operations associated with the device or to execute one or more "units" thereof according to the present embodiment, and / or to perform a process or execute a step of a process according to the present embodiment. Such a program can be executed by causing the CPU 1212 to cause the computer 1200 to perform specific operations associated with some or all of the blocks of the flowcharts and block diagrams described herein.

[0045] The computer 1200 according to the present embodiment includes a CPU 1212, a RAM 1214, and a graphic controller 1216, which are connected to each other via a host controller 1210. The computer 1200 also includes input / output units such as a communication interface 1222, a storage device 1224, a DVD drive, and an IC card drive, which are connected to the host controller 1210 via an input / output controller 1220. The DVD drive may be a DVD-ROM drive, a DVD-RAM drive, etc. The storage device 1224 may be a hard disk drive, a solid state drive, etc. The computer 1200 also includes legacy input / output units such as a ROM 1230 and a keyboard, which are connected to the input / output controller 1220 via an input / output chip 1240.

[0046] The CPU 1212 operates according to a program stored in the ROM 1230 and the RAM 1214, thereby controlling each unit. The graphic controller 1216 acquires image data issued by the CPU 1212, which is in the RAM 1214 or a frame buffer provided in the graphic controller 1216 itself, and causes the image data to be displayed on the display device 1218.

[0047] The communication interface 1222 communicates with other electronic devices via a network. The storage device 1224 stores programs and data used by the CPU 1212 in the computer 1200. The DVD drive reads programs or data from a DVD-ROM or the like and provides the programs or data to the storage device 1224. The IC card drive reads programs and data from an IC card and / or writes programs and data to the IC card.

[0048] The ROM 1230 stores therein a boot program and the like that is executed by the computer 1200 during operation, and / or programs that depend on the hardware of the computer 1200. The input / output chip 1240 can also connect various input / output units to the input / output controller 1220 via a USB port, a parallel port, a serial port, a keyboard port, a mouse port, and the like.

[0049] The program is provided by a computer-readable storage medium such as a DVD-ROM or an IC card. The program is read from the computer-readable storage medium, installed in the storage device 1224, the RAM 1214, or the ROM 1230, which are also examples of computer-readable storage media, and executed by the CPU 1212. Information processing written in the program is read by the computer 1200, and as a result, the program and the various types of hardware resources described above cooperate with each other. The apparatus or method can be configured by implementing an operation or processing of information according to the use of the computer 1200.

[0050] For example, when communication is performed between the computer 1200 and an external device, the CPU 1212 executes a communication program loaded in the RAM 1214, and can instruct the communication interface 1222 to perform communication processing based on the processing written in the communication program. Under the control of the CPU 1212, the communication interface 1222 reads and transmits transmission data stored in a transmission buffer area provided in the RAM 1214, the storage device 1224, a DVD-ROM, or a recording medium such as an IC card, and transmits the read transmission data to a network, or writes reception data received from the network to a reception buffer area or the like provided in the recording medium.

[0051] Also, the CPU 1212 may cause all or a necessary portion of a file or database stored in an external recording medium such as the storage device 1224, a DVD drive (DVD-ROM), an IC card, etc. to be read into the RAM 1214, and perform various types of processing on the data on the RAM 1214. Then, the CPU 1212 may write the processed data back to the external recording medium.

[0052] Various types of information, such as various types of programs, data, tables, databases, etc., are stored in the recording medium and processed. The CPU 1212 can perform various types of processing on the data read from the RAM 1214 to write the results back to the RAM 1214, which processing is described throughout this disclosure and specified by the sequential instructions of the program, and includes various types of operations, information processing, conditional judgment, conditional branching, unconditional branching, information search / replacement, etc. The CPU 1212 can also search for information in a file, database, etc. in the recording medium. For example, when multiple entries each having an attribute value of a first attribute associated with an attribute value of a second attribute are stored in the recording medium, the CPU 1212 can search for an entry whose attribute value of the first attribute meets a specified condition from the multiple entries, read the attribute value of the second attribute stored in the entry, and thereby obtain the attribute value of the second attribute associated with the first attribute that meets the specified condition.

[0053] The above-described programs or software modules may be stored in a computer-readable storage medium on the computer 1200 or in the vicinity of the computer 1200. In addition, a recording medium such as a hard disk or a RAM provided in a server system connected to a dedicated communication network or the Internet can be used as a computer-readable storage medium, thereby providing the programs to the computer 1200 via the network.

[0054] The blocks of the flowcharts and block diagrams of the present embodiment may represent stages of a process where an operation is performed or a "unit" of an apparatus is responsible for performing an operation. Particular stages and "units" may be implemented by dedicated circuitry, programmable circuitry provided with computer-readable instructions stored on a computer-readable storage medium, and / or a processor provided with computer-readable instructions stored on a computer-readable storage medium. The dedicated circuitry may include digital and / or analog hardware circuitry, and may include integrated circuits (ICs) and / or discrete circuits. For example, the programmable circuitry may include reconfigurable hardware circuitry, such as logical AND, OR, XOR, NAND, NOR, and other logic operations, flip-flops, registers, and memory elements, such as field programmable gate arrays (FPGAs), programmable logic arrays (PLAs), and the like.

[0055] A computer-readable storage medium may include any tangible device capable of storing instructions that are executed by a suitable device, and as a result, a computer-readable storage medium having instructions stored thereon includes a product that includes instructions that can be executed to create means for performing the operations specified in the flowchart or block diagram. Examples of computer-readable storage media may include electronic storage media, magnetic storage media, optical storage media, electromagnetic storage media, semiconductor storage media, and the like. More specific examples of computer-readable storage media may include floppy disks, diskettes, hard disks, random access memories (RAMs), read-only memories (ROMs), erasable programmable read-only memories (EPROMs or flash memories), electrically erasable programmable read-only memories (EEPROMs), static random access memories (SRAMs), compact disk read-only memories (CD-ROMs), digital versatile disks (DVDs), Blu-ray disks, memory sticks, integrated circuit cards, and the like.

[0056] The computer readable instructions may include either assembler instructions, Instruction Set Architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk®, JAVA®, C++, etc., and conventional procedural programming languages, such as the “C” programming language or similar programming languages.

[0057] Computer readable instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, or a programmable circuit, locally or over a local area network (LAN), a wide area network (WAN) such as the Internet, etc., to cause the processor or programmable circuit of the general purpose computer, special purpose computer, or other programmable data processing apparatus to execute the computer readable instructions to generate means for performing operations specified in the flowcharts or block diagrams. Examples of processors include computer processors, processing units, microprocessors, digital signal processors, controllers, microcontrollers, etc.

[0058] Although the present invention has been described with reference to the embodiments, the technical scope of the present invention is not limited to the above-mentioned embodiments. It is clear to those skilled in the art that various modifications or improvements can be made to the above-mentioned embodiments. It is clear from the description of the claims that the embodiments with such modifications or improvements can also be included in the technical scope of the present invention.

[0059] It should be noted that the order of execution of each process, such as operations, procedures, steps, and stages, in the devices, systems, programs, and methods shown in the claims, embodiments, and drawings is not specifically stated as "before" or "prior to," and that the process may be performed in any order unless the output of a previous process is used in a subsequent process. Even if the operational flow in the claims, embodiments, and drawings is described using "first" or "next" for convenience, it does not mean that it is essential to perform the process in this order.

Claims

1. an acquisition unit configured to acquire an audio signal in a streaming manner; and A denoising unit configured to denoise said audio signal by using a dynamic sliding window with an input buffer of a non-fixed length. Equipped with 11. An audio signal processing apparatus, wherein the noise removal unit is configured to buffer a portion of the audio signal in the dynamic sliding window having a first size, then start removing noise from the portion of the audio signal and buffering a next portion of the audio signal, and then stop the buffering when the noise removal is completed.

2. A program for causing a computer to function as the audio signal processing device according to claim 1.

3. Obtaining an audio signal in a streaming manner; and denoising the audio signal obtained in the obtaining step by using a dynamic sliding window where the length of the input buffer is not fixed. Equipped with 11. A method for processing an audio signal, wherein the denoising step comprises buffering a portion of the audio signal in the dynamic sliding window having a first size, then commencing denoising the portion of the audio signal and buffering a next portion of the audio signal, and then stopping the buffering when the denoising is completed.

Citation Information

Patent Citations

  • Noise reducing method and noise reducing device

    JP1998240294A

  • Decoding apparatus and decoding method

    JP2007072334A

  • Device, method and program for suppressing noise and storage medium

    JP2009063747A

  • Feature-preserving noise removal

    US20160171727A1