Neural Network-Based Signal Processing

Abstract

A method for processing an input audio signal includes the steps of: training a first neural network system using a representation of the input audio signal to predict a bit-rate reduced representation of the processed input audio signal, the first neural network system being trained to generate a bit-rate reduced representation of a processed version of the given audio signal, the bit-rate reduced representation having a format associated with a predetermined audio encoding process; training a second neural network system using the bit-rate reduced representation to predict an augmented representation of the processed audio signal, the second neural network system being trained to generate an augmented representation of the given bit-rate reduced audio representation, the bit-rate reduced representation having a format associated with the predetermined audio encoding process; and converting the augmented representation of the processed audio signal into an output audio signal.

Description

[Technical Field]

[0001] [CROSS-REFERENCE TO RELATED APPLICATIONS] This application claims the benefit of priority to the following priority applications: U.S. Provisional Application No. 63 / 391,124, filed July 21, 2022, and European Patent Application No. 22188293.9, filed August 2, 2022, the entire contents of each of which are incorporated herein by reference.

[0002] [Technical field] The present invention relates to audio signal processing using generative models, including neural network systems. The signal processing may relate, for example, to signal enhancement or audio source separation. [Background technology]

[0003] For some time, generative models, including trained neural network systems, have been used in various audio signal processing applications. A common approach is for a neural network system to be trained using ground truth data, after which the trained model can be used to infer a processed signal. Specially designed neural network systems have been developed for specific applications, including decoding.

[0004] While some signal processing applications may consider successfully training neural networks to perform the intended signal processing, such neural network systems become impractically complex, requiring vast amounts of training data and also requiring vast computational resources during inference.

[0005] Several attempts have been made to mitigate this problem. For example, in one approach, disclosed in Jukebox: A generative model for music, Dhariwal et al., 2020, the input signal is first converted to a vector quantized representation before being fed into a generative model. The inferred signal is then synthesized into a complete representation. As a result of this approach, the generative model operates in vector quantized space, significantly reducing computational complexity. Summary of the Invention

[0006] A drawback of the above approaches is that vector quantization (or any other complexity reduction) makes a trade-off between complexity reduction and achievable quality. This trade-off is difficult to optimize. Another drawback is that vector quantization applied directly to the signal to be processed may remove some information relevant to solving the processing problem, thereby limiting the achievable performance.

[0007] The present invention aims to overcome these problems and to provide an improved approach to audio signal processing using neural networks.

[0008] According to a first aspect of the present invention, this object is achieved by a method for processing an input audio signal, the method comprising the steps of: training a first neural network system using a representation of the input audio signal to predict a bit-rate reduced representation of the processed input audio signal, the first neural network system being trained to generate a bit-rate reduced representation of a processed version of a given audio signal, the bit-rate reduced representation having a format associated with a predetermined audio encoding process; training a second neural network system using the bit-rate reduced representation to predict an augmented representation of the processed audio signal, the second neural network system being trained to generate an augmented representation of the given bit-rate reduced audio representation, the bit-rate reduced representation having a format associated with a predetermined audio encoding process; and converting the augmented representation of the processed audio signal into an output audio signal.

[0009] Therefore, the processing is performed in two stages, with a bit-rate reduced intermediate processing result, which is called the latent signal.

[0010] It is important to note that the intermediate processing results (latent signals) have a format related to a given audio coding process. This means that for a given (known) pair of global ground truth signals, i.e., the original signal and the processed (target) signal, the intermediate processing targets for the first stage can also be determined deterministically. For example, for supervised training, the training objectives for the first stage can be defined as a function of the network output and the audio-coded targets (not the uncoded targets). This process facilitates the definition of audio-coded latents that effectively separate the stages. As a result, the neural network systems for each stage can be trained (individually or jointly) using separate loss functions.

[0011] Individual training of two neural network systems has potential advantages because it may be easier to perform and the models implementing the networks may be relatively smaller. Joint training is expected to provide better overall performance, but is likely to require larger models and more training data.

[0012] During inference, a first network solves a processing task by providing intermediate representation results, while a second network provides a final processing result based on the intermediate representation. The fact that inference involves two specialized networks has a significant impact on computational complexity (compared to end-to-end systems). The use of two specialized networks facilitates decomposing the processing problem into subproblems that can be associated with respective training objectives. A single network solving a processing task in an end-to-end setting is expected to require a significantly larger number of trainable parameters and a significantly larger amount of training data.

[0013] Furthermore, the latent bitrate reduction (quantization) performed according to a given audio coding process achieves a tradeoff between bitrate reduction and distortion according to the given audio coding process. Audio coding algorithms optimize their bitrate-distortion tradeoff in a perceptually optimized manner, the details of which depend on the coding algorithm. Thus, the latent format ensures an appropriate tradeoff between the performance of the processing task performed by the first stage and the performance of the final synthesis task performed by the second stage.

[0014] According to a second aspect of the present invention, this object is achieved by a system for processing an input audio signal, the system comprising: a first neural network system trained to generate a bit-rate reduced representation of a processed version of a given audio signal, the bit-rate reduced representation having a format related to a predetermined audio encoding process, the first neural network system being adjusted by a representation of the input audio signal so as to predict a bit-rate reduced representation of the processed audio signal; a second neural network system trained to generate an augmented representation of the given bit-rate reduced audio representation, the bit-rate reduced representation having a format related to the predetermined audio encoding process, the second neural network system being adjusted by the bit-rate reduced representation predicted in the first neural network system so as to predict the augmented representation of the processed audio signal; and a processing stage for converting the augmented representation of the processed audio signal into an output audio signal. [Brief explanation of the drawings]

[0015] The present invention will now be described in detail with reference to the accompanying drawings, which show presently preferred embodiments of the invention. [Figure 1] FIG. 2 is a block diagram of a process according to an embodiment of the present invention. [Figure 2] The training of the neural network system in FIG. 1 is shown using separate training targets for the first stage (S1) and the second stage (S2). [Figure 3] 2 is an example of a more detailed implementation of the process in FIG. 1 operating in the MDCT domain. [Figure 4] A first example of an audio coding process is shown in FIG. [Figure 5] A second example of an audio coding process is shown in FIG. DETAILED DESCRIPTION OF THE INVENTION

[0016] The systems and methods disclosed in this application may be implemented as software, firmware, hardware, or a combination thereof. In a hardware implementation, the division of tasks does not necessarily correspond to the division into physical units; conversely, one physical component may have multiple functions, and one task may be performed by several physical components working together.

[0017] The computer hardware may be, for example, a server computer, a client computer, a personal computer (PC), a tablet PC, a set-top box (STB), a personal digital assistant (PDA), a mobile phone, a smartphone, a web appliance, a network router, a switch, or a bridge, or any machine capable of executing instructions (sequential or otherwise) that specify actions to be performed by that computer hardware. Furthermore, the present disclosure relates to any collection of computer hardware that individually or collectively executes instructions to perform any one or more of the concepts discussed herein.

[0018] Certain or all components may be implemented by one or more processors that accept computer-readable (also called machine-readable) code, including a set of instructions that, when executed by the one or more processors, perform at least one of the methods described herein. Any processor capable of executing a set of instructions (sequential or otherwise) that specify actions to be performed is included. Thus, one example is a typical processing system (i.e., computer hardware) that includes one or more processors. Each processor may include one or more of a CPU, a graphics processing unit, and a programmable DSP unit. The processing system may further include a memory subsystem, including a hard drive, an SSD, RAM, and / or ROM. A bus subsystem may be included for communication between components. Software may reside in the memory subsystem and / or in the processor during its execution by the computer system.

[0019] One or more processors may operate as stand-alone devices or may be connected, for example, networked, to other processors. Such a network may be built on a variety of different network protocols and may be the Internet, a wide area network (WAN), a local area network (LAN), or any combination thereof.

[0020] Software may be distributed on computer-readable media, which may include computer storage media (or non-transitory media) and communication media (or transitory media). As is well known to those skilled in the art, the term computer storage media includes both volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer-readable instructions, data structures, program modules, or other data. Computer storage media includes various forms of physical (non-transitory) storage media, such as, but not limited to, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and that can be accessed by a computer. Furthermore, communication media (transitory) typically embodies computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and is well known to those skilled in the art to include any information delivery media.

[0021] Figure 1 shows audio signal processing of an input signal y in a two-stage architecture with a latent signal associated with a finite bit rate. This architecture has two separate processing stages 10 and 11. The first stage, given a representation of the input signal y, generates a latent signal y according to the training target (S1) for the first stage (see Figure 2).

number

number

number

[0022] The second stage is a predicted bitrate-reduced representation of the processed input signal.

number

number

number

number

[0023] The neural network systems 12, 13 included in stages 10 and 11 may be designed to operate entirely in the domain of the underlying signal, for example a transform domain such as the MDCT, in which case the input signal is first transformed into this domain and the predictions of the second neural network are transformed back into the time domain (synthesized).

[0024] 2 illustrates the training of the neural network systems 12 and 13 in FIG. 1. For training (supervised training), an original audio signal y (the signal to be processed) and a target signal x (the processed signal) are obtained. The original signal may be, for example, a mixed piece of music containing piano, and the target signal may be the same piano as the separated signal. Furthermore, a target latent signal z is obtained by applying a predetermined audio coding process 14 to the target signal x (the separated piano signal) with an appropriate degree of bit-rate reduction (quantization). The target latent signal z is a bit-rate reduced representation of the target signal x. The first neural network system 12 calculates the latent signal z using a first loss function S1 on the target latent signal z.

number

number

number

[0025] A specific implementation of the system in Figure 1 will be described in more detail with reference to Figure 3. In this example, the audio codec format is in the transform domain, more specifically the MDCT domain. Both neural network systems are also designed to operate entirely in the MDCT domain. In the example shown, the MDCT lines are dynamics reduced (e.g., spectrally flattened).

[0026] Referring to FIG. 3 , the first stage 10 here includes an MDCT transform 21 for converting the input audio signal into the MDCT domain. The resulting MDCT lines are fed to an envelope estimator 23 to provide a spectral envelope. The MDCT lines are then flattened by a flattening function 22 using the spectral envelope determined by the envelope estimator 23 to reduce the signal's spectral dynamics. In some embodiments, the flattening function 22 may be implemented by estimating the spectral envelope (e.g., calculating the variance of the signal in a predetermined number of subbands) and then normalizing the MDCT coefficients in each subband according to the value of the spectral envelope for these bands. Examples of such normalization include normalization to unit variance (using envelope values with an index of 1.0) or normalization to the "pink domain" (using envelope values with an index of 0.5).

[0027] The resulting representation of the input signal (flattened MDCT lines and envelope) is input to a first neural network system 24. In the illustrated example, the first neural network system 24 is configured according to a generative setup and generates a latent signal

number

number

number

[0028] The second stage 11 now includes a second neural network system 26 operating in a generative setting, which generates a latent signal

number

number

number

number

number

number

number

number

number

number

number

[0029] If S1 is configured according to a negative log likelihood (NLL) loss (generative setting), the first stage provides a probability distribution at its output. To use this as a tuning for the second stage, sampler 25 must be used. If S1 is configured in a regression setting, the output of stage 1 is deterministic, and thus sampler 25 may be omitted. The second stage, S2, objective is always configured in a generative setting (according to NLL loss), and sampler 27 is always used.

[0030] In the process in Figure 3, the transform domain is the MDCT, and the neural network Systems 24 and 26 may be designed according to the topology discussed in PCT / US2021 / 054617, entitled "GENERAL MEDIA NEURAL NETWORK PREDICTOR AND A GENERATIVE MODEL INCLUDING SUCH A PREDICTOR," which is incorporated by reference, in which the neural network system includes a separate frequency prediction portion and a separate time prediction portion, with the output from one portion provided as input to the other.

[0031] However, the neural network system in PCT / US2021 / 054617 is trained by samples of MDCT lines to generate sample MDCT lines. Regarding the process in FIG. 3, the first neural network system 24 is trained and predicts not only the MDCT lines but also the envelopes (variance vectors) of the MDCT lines. Therefore, the topology of PCT / US2021 / 054617 needs to be modified in an appropriate manner. Regarding the second neural network system 26, the extended signal representation

number

number

number

number

[0032] For the process in FIG. 3, training the neural network system requires a target latent signal z obtained using an audio coding process operating in the MDCT domain.

[0033] FIG. 4 shows an example of an audio coding process 30 that maps an input signal to a bit-rate-reduced representation by quantization using a waveform codec. In the illustrated example, the process 30 includes an MDCT transform 31 for converting the target signal into the MDCT domain. The MDCT transform 31 is configured to provide a perceptually motivated division of MDCT lines, with lower frequency bands being narrower (i.e., denser information). The process further includes an envelope estimator 33 for determining a spectral envelope and a flattening function 32 for reducing the dynamics of the MDCT lines using the spectral envelope. Finally, two quantizers 34a, 34b are provided for quantizing the flattened MDCT lines and envelope to the desired bit rate R. The quantizers are configured to distribute the distortion caused by a given bit-rate reduction in a perceptually optimal manner (i.e., as inconspicuous as possible). Existing audio codec processes include such quantization algorithms.

[0034] FIG. 5 illustrates another example of a coding process 40 that maps an input signal to a bit-rate-reduced representation by parametric coding. Such a coding process may be based on a sinusoidal analysis algorithm (e.g., a matching pursuit algorithm), where the sinusoidal components are selected to minimize some perceptual criterion (e.g., a spectrally weighted mean squared error). Process 40 includes a parameterization function 41 that parameterizes the input signal into a parametric description with N sinusoids and one phase parameter per sinusoidal trajectory. The size of number N determines the level of quantization. The larger N, the higher the associated bit rate and the higher the reconstruction fidelity. In the illustrated example, the parametric description is extended with envelope information. The spectral envelope may include a set of envelope values associated with several frequency portions (using perceptually motivated banding). In some embodiments, the spectral envelope may be described by linear prediction coefficients (LPC).

[0035] For use in the process shown in Figure 3, the target potential z needs to be in the MDCT domain. For this reason, the coding process 40 now further includes a reconstruction block 42 for reconstructing the time-domain signal and an MDCT transform 43 for obtaining MDCT lines. In principle, the parametric representation may also be directly reconstructed into the MDCT domain, for example, by projecting a sinusoid on the MDCT basis. Similar to the coding process in Figure 4, the MDCT lines are flattened by a flattening function 44 using the spectral envelope. In block 45, the spectral envelope is obtained by mapping the envelope information in the parametric description to the MDCT lines.

[0036] Unless otherwise indicated, as will be apparent from the discussion that follows, it will be recognized that throughout this disclosure, discussions utilizing terms such as "processing," "calculating," "computing," "determining," "analyzing," etc. refer to the actions and / or processes of computer hardware or computing systems or similar electronic computing devices that manipulate and / or transform data represented as physical quantities, such as electronic quantities, into other data also represented as physical quantities.

[0037] In the foregoing description of exemplary embodiments of the present invention, various features of the invention are in some cases grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the disclosure and facilitating understanding of one or more of the various inventive aspects. However, this method of disclosure should not be interpreted as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in fewer than all features of a single, above-disclosed embodiment. Accordingly, the claims following the detailed description are expressly incorporated into this detailed description, with each claim standing on its own as a separate embodiment of the present invention. Furthermore, as will be understood by those skilled in the art, some embodiments described herein include some features but not other features included in other embodiments, and combinations of features from different embodiments are intended to form different embodiments within the scope of the present invention. For example, in the following claims, any of the claimed embodiments can be used in any combination.

[0038] Furthermore, some of the embodiments are described herein as a method or a combination of method elements that can be implemented by a processor of a computer system or by other means for performing a function. Thus, a processor using instructions for performing such a method or method elements forms a means for performing the method or method elements. It should be noted that when a method includes several elements, e.g., several steps, no ordering of such elements is implied unless specifically stated. Furthermore, elements described herein of apparatus embodiments are examples of means for performing the functions performed by the elements for implementing embodiments of the present invention. In the description provided herein, numerous specific details are set forth. However, it is understood that embodiments of the present invention may be practiced without these specific details. In other instances, well-known methods, structures, and techniques have not been shown in detail in order to avoid obscuring an understanding of this description.

[0039] Those skilled in the art will understand that the present invention is in no way limited to the preferred embodiments described above. On the contrary, many modifications and variations are possible within the scope of the appended claims. For example, domains other than MDCT may also be considered.

[0040] The present invention can be further understood from the following list of enumerated exemplary embodiments (EEE).

[0041] EEE1. A method for processing an input audio signal, comprising: training a first neural network system using the representation of the input audio signal to predict a bit-rate reduced representation of a processed input audio signal, the first neural network system being trained to generate a bit-rate reduced representation of a processed version of a given audio signal, the bit-rate reduced representation having a format associated with a predetermined audio encoding process; training a second neural network system using the bit-rate reduced representation to predict an extended representation of the processed audio signal, the second neural network system being trained to generate an extended representation of a given bit-rate reduced audio representation, the bit-rate reduced representation having a format associated with the predetermined audio encoding process; converting the augmented representation of the processed audio signal into an output audio signal; A method comprising:

[0042] EEE2. The method of EEE1, wherein the input audio signal and the output audio signal are in the time domain.

[0043] EEE3. The method of EEE1 or EEE2, wherein the extended representation has a format related to the predetermined audio encoding process.

[0044] EEE4. The method of any one of EEE1 to EEE3, wherein the representation of the input signal, the bit rate reduced representation and the enhanced representation are all in one and the same transform domain.

[0045] EEE5. The method of any one of EEE1 to EEE4, wherein the transform domain is a waveform transform domain.

[0046] EEE6. The method of any one of EEE1 to EEE5, wherein the representation of the input signal, the bit rate reduced representation and the extended representation all comprise a set of MDCT lines and associated envelope information.

[0047] EEE7. The method of any one of EEE1 to EEE6, wherein the MDCT line has reduced signal dynamics.

[0048] EEE8. The method of any one of EEE1 to EEE7, wherein said transforming step comprises increasing signal dynamics of said extended representation.

[0049] EEE9. The method of any one of EEE1 to EEE8, wherein the first neural network system is trained and operates in a generative setting.

[0050] EEE10. The method of any one of EEE1 to EEE9, wherein the second neural network system is trained and operates in a generative setting.

[0051] EEE11. The method of any one of EEE1 to EEE10, wherein the input audio signal is a distorted audio signal and the first neural network system predicts a reduced bit rate representation of a signal enhanced version of the input audio signal.

[0052] EEE12. The method of any one of EEE1 to EEE11, wherein the input audio signal is a mixed audio signal and the first neural network system predicts a reduced bitrate representation of a source-separated version of the input audio signal.

[0053] EEE13. A system for processing an input audio signal, comprising: a first neural network system trained to generate a bit-rate reduced representation of a processed version of a given audio signal, the bit-rate reduced representation having a format related to a predetermined audio encoding process, the first neural network system being tuned by a representation of the input audio signal to predict the bit-rate reduced representation of the processed audio signal; a second neural network system trained to generate an augmented representation of a given bit-rate reduced audio representation, the bit-rate reduced representation having a format related to the predetermined audio encoding process, the second neural network system being tuned by the bit-rate reduced representation predicted in the first neural network system to predict an augmented representation of the processed audio signal; a processing step for converting the augmented representation of the processed audio signal into an output audio signal; A system including:

[0054] EEE14. The system according to EEE13, wherein the input audio signal and the output audio signal are in the time domain.

[0055] EEE15. The system of EEE13 or EEE14, wherein the extended representation has a format related to the predetermined audio encoding process.

[0056] EEE16. The system of any one of EEE13 to EEE15, wherein the representation of the input signal, the bit rate reduced representation and the enhanced representation are all in one and the same transform domain.

[0057] EEE17. The system of any one of EEE13 to EEE16, wherein the transform domain is a waveform transform domain.

[0058] EEE18. The system of any one of EEE13 to EEE17, wherein the representation of the input signal, the bit rate reduced representation and the extended representation all comprise a set of MDCT lines and associated envelope information.

[0059] EEE19. The system of any one of EEE13 to EEE18, wherein the MDCT line has reduced signal dynamics.

[0060] EEE20. The system of any one of EEE13 to EEE19, wherein the converting step includes increasing signal dynamics of the extended representation.

[0061] EEE21. The system of any one of EEE13 to EEE20, wherein the first neural network system is trained and operates in a generative setting.

[0062] EEE22. The system of any one of EEE13 to EEE21, wherein the second neural network system is trained and operates in a generative setting.

[0063] EEE23. The system of any one of EEE13 to EEE22, wherein the input audio signal is a distorted audio signal and the first neural network system predicts a reduced bit rate representation of a signal enhanced version of the input audio signal.

[0064] EEE24. The system of any one of EEE13 to EEE23, wherein the input audio signal is a mixed audio signal and the first neural network system predicts a reduced bitrate representation of a source separated version of the input audio signal.

[0065] EEE25. A computer program product comprising computer program code portions configured to perform the method of any one of claims EEE1 to EEE12 when executed on a computer processor.

Claims

1. 1. A method for processing an input audio signal, comprising: training a first processing stage comprising a first neural network system using the representation of the input audio signal to generate a latent signal comprising a prediction of a reduced bit rate representation of a processed version of the input audio signal, the first neural network system being trained to generate a reduced bit rate representation of a target processed version of a given audio signal, the reduced bit rate representation having a format associated with a predetermined audio codec that is quantized to a desired bit rate; using the latent signal to train a second processing stage comprising a second neural network system to predict the processed version of the input audio signal, the second neural network system being trained to generate an augmented representation of a given bit-rate reduced audio representation of the processed version of the audio signal, the bit-rate reduced representation having a format associated with the predetermined audio codec; converting the predicted processed version of the input audio signal into an output audio signal; A method comprising:

2. The method of claim 1 , wherein the input audio signal and the output audio signal are in the time domain.

3. The method of claim 1 , wherein the extended representation has a format associated with the predetermined audio codec.

4. The method of claim 1 , wherein the representation of the input signal, the bit-rate reduced representation, and the enhanced representation are all in the same transform domain.

5. The method of claim 1 , wherein the transform domain is a waveform transform domain.

6. The method of claim 1 , wherein the representation of the input signal, the bit-rate reduced representation, and the extended representation all comprise a set of MDCT lines and associated envelope information.

7. The method of claim 6 , wherein the MDCT lines have reduced signal dynamics.

8. The method of claim 1 , wherein the converting step includes increasing signal dynamics of the extended representation.

9. The method of claim 1 , wherein the first neural network system is trained and operates in a generative setting.

10. The method of claim 1 , wherein the second neural network system is trained and operates in a generative setting.

11. The method of claim 1 , wherein the input audio signal is a distorted audio signal, and the first neural network system predicts a reduced bit rate representation of a signal-enhanced version of the input audio signal.

12. The method of claim 1 , wherein the input audio signal is a mixed audio signal, and the first neural network system predicts a reduced bitrate representation of a source-separated version of the input audio signal.

13. 1. A system for processing an input audio signal, comprising: a first processing stage including a first neural network system trained to generate a bit-rate reduced representation of a target processed version of a given audio signal, the bit-rate reduced representation having a format associated with a predetermined audio codec that is quantized to a desired bit-rate, the first neural network system being tuned by a representation of the input audio signal to generate a latent signal comprising a prediction of the bit-rate reduced representation of the processed version of the input audio signal; a second processing stage including a second neural network system trained to generate an augmented representation of a given bit-rate reduced audio representation of a processed version of an audio signal, said bit-rate reduced representation having a format associated with said predetermined audio codec, said second neural network system being trained by the latent signal predicted in said first neural network system to predict said processed version of the input audio signal; a processing stage for converting the predicted processed version of the input audio signal into an output audio signal; A system including:

14. The system of claim 13 , wherein the input audio signal and the output audio signal are in the time domain.

15. 15. The system of claim 13 or 14, wherein the extended representation has a format associated with the predetermined audio codec.

16. 15. The system of claim 13 or 14, wherein the representation of the input signal, the bit-rate reduced representation and the enhanced representation are all in the same transform domain.

17. 15. The system of claim 13 or 14, wherein the transform domain is a waveform transform domain.

18. 15. The system of claim 13 or 14, wherein the representation of the input signal, the bit-rate reduced representation and the extended representation all comprise a set of MDCT lines and associated envelope information.

19. The system of claim 18 , wherein the MDCT line has reduced signal dynamics.

20. 15. The system of claim 13 or 14, wherein the transforming step comprises increasing signal dynamics of the extended representation.

21. 15. The system of claim 13 or 14, wherein the first neural network system is trained and operates in a generative setting.

22. 15. The system of claim 13 or 14, wherein the second neural network system is trained and operates in a generative setting.

23. 15. The system of claim 13 or 14, wherein the input audio signal is a distorted audio signal, and the first neural network system predicts a reduced bit rate representation of a signal-enhanced version of the input audio signal.

24. 15. The system of claim 13 or 14, wherein the input audio signal is a mixed audio signal, and the first neural network system predicts a reduced bitrate representation of a source-separated version of the input audio signal.

25. A computer program product comprising computer program code portions adapted to perform the method of any one of claims 1 to 12 when the computer program product is executed on a computer processor.

Images