Audio decoding using a score-based diffusion model

Abstract

A method and system for decoding an audio signal, where a reconstructed audio signal in a target domain is obtained using an iterative score-based diffusion process. Each iteration calculates a diffusion variable based on a preceding value of the diffusion variable and an aggregate score, and the reconstructed signal is set to the diffusion variable after a final iteration. The aggregate score in each iteration is determined by an unconditional score related to a probability distribution of the diffusion variable, and a conditional score related to a probability distribution of the signal representation, and adding the unconditional score and the conditional score to obtain the aggregate score.

Description

AUDIO DECODING USING A SCORE-BASED DIFFUSION MODEL CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims the benefit of priority from U.S. Provisional Application No. 63 / 691,638, filed on 6 September 2024, and European Application No.24213251.2 filed on 15 November 2024, each of which is incorporated by reference herein in its entirety. TECHNICAL FIELD OF THE INVENTION

[0002] The present invention relates to a method and a system for decoding an audio signal. BACKGROUND OF THE INVENTION

[0003] Today, audio coding methods employing generative models use an approach where the generative model is trained in a supervised manner, while being presented pairs of the encoded signal and reference signal. Such a model learns to generate a probability distribution of an audio signal, which is conditioned on the coded representation of that signal. The decoding process includes random sampling from this probability distribution. The resulting codec architecture includes a classical encoder and a trained decoder in a form of a neural network, which closely resembles a traditional coding setup. GENERAL DISCLOSURE OF THE INVENTION

[0004] With a conventional approach, the neural network needs to be trained for a specific bitrate. At the same time, it is typically desired that an audio codec can provide multiple points of bitrate-quality trade-off. This is a challenge for the generative model setup, as it requires that training is performed for each different trade-off points. The model also needs to be large enough to handle these multiple operating points or, alternatively, an ensemble of models can be used with one dedicated model for each specific operation point.

[0005] It is an object of the present invention to provide an improved generative decoding approach mitigating the above challenge, and allowing multiple bitrates without performing multiple training. According to some aspects this object is achieved by a method and system for decoding an audio signal, where a reconstructed audio signal in a target domain is obtained using an iterative score-based diffusion process. Each iteration calculates a diffusion variable based on a preceding value of the diffusion variable and an aggregate score, and the reconstructed signal is set to the diffusion variable after a final iteration. The aggregate score in each iteration is determined by an unconditional score related to a probability distribution of the diffusionvariable, and a conditional score related to a probability distribution of the signal representation, and adding the unconditional score and the conditional score to obtain the aggregate score.

[0006] According to a first aspect of the invention, this object is achieved by a method for decoding an audio signal, the method comprising receiving a bitstream including a signal representation of the audio signal in a coding domain, obtaining a reconstructed audio signal in a target domain using an iterative score-based diffusion process, wherein each iteration calculates a diffusion variable based on a preceding value of the diffusion variable and an aggregate score, and setting the reconstructed signal to the diffusion variable after a final iteration. The aggregate score in each iteration is determined by, based on preceding value of the diffusion variable and an iteration specific noise distribution generating an unconditional score related to a probability distribution of the diffusion variable using a generic score model operating in the target domain, the generic noise conditioned score model being trained to generate a function of a probability distribution of a noisy audio signal given a noise distribution, based on the preceding value of the diffusion variable, the iteration specific noise distribution and the signal representation, calculating a conditional score related to a probability distribution of the signal representation, and adding the unconditional score and the conditional score to obtain the aggregate score.

[0007] According to a second aspect of the invention, this object is achieved by a system for decoding an audio signal, comprising a decoder for receiving a bitstream and decoding a signal representation of the audio signal in a coding domain and a score-based diffusion model for obtaining a reconstructed audio signal in a target domain. The score based diffusion model includes a diffusion stage configured to run an iterative score-based diffusion process where each iteration calculates a diffusion variable based on a preceding value of the diffusion variable, and an aggregate score, and to set the reconstructed signal to the diffusion variable after a final iteration, a generic score model operating in the target domain and trained to generate a function of a probability distribution of a noisy audio signal given a noise distribution, the generic score model configured to, based on preceding value of the diffusion variable and an iteration specific noise distribution, generate an unconditional score related to a probability distribution of the diffusion variable, a conditional score computational unit configured to, based on the preceding value of the diffusion variable, the iteration specific noise distribution and the signal representation, calculate a conditional score related to a probability distribution of the signal representation, wherein the aggregate score in each iteration is determined by summing the unconditional score and the conditional score.

[0008] The present disclosure describes a novel coding architecture comprising a conventional encoder and a new type of decoder including a score-based diffusion model based on a generic score model trained on generic audio signals. It is “generic” in the sense that it isnot trained with task-specific data, and it is not conditioned by any task-specific data. In other words, in the present context, the generic score model is not conditioned with encoder specific data. Other examples of audio tasks are loss concealment, high frequency reconstruction and source separation. The diffusion model computes a reconstructed audio signal in an iterative diffusion process using the generic score model.

[0009] By using a score-based diffusion model, an unconditional prior model (i.e., without any task-specific input) can be conditioned after its training with the aim of solving different inverse problems, simply described by their forward process. Diffusion posterior sampling has been proposed for image (JPEG) decoding, and is discussed in Kawar, Bahjat, et al. "JPEG artifact correction using denoising diffusion restoration models" (2022). Prior art solutions, however, rely on a placeholder solution function to the inverse decoding problem, leading to complex computation in every iteration of the diffusion process.

[0010] The unconditional score may represent a gradient (with respect to the diffusion variable) of a log of the probability distribution of the diffusion variable. Similarly, the conditional score may represent a gradient (with respect to the diffusion variable) of a log of the probability distribution of the signal representation.

[0011] The generic score model may be trained to generate a gradient (with respect to the diffusion variable) of a log of the probability distribution of the noisy signal.

[0012] As the generic score model is not conditioned on any coded representation (or any other task-specific input), it is bitrate agnostic and not bitrate specific. In other words, the same generic generative model may be used for any bitrate. This facilitates providing multiple points of bitrate-quality trade-off without requiring multiple trained models.

[0013] Further, a decoder according to the invention facilitates a trade-off between the quality of decoding and the computational complexity of the decoding. Specifically, the computational complexity can be adjusted by setting the number of iterations performed in the diffusion process.

[0014] The approach in the present disclosure can successfully be configured to handle a situation where the coding domain signal representation is multistage and oversampled, which is the case in some of the most commonly employed coding architectures, including some coding architectures based on the MDCT transform. For example, the signal representation may comprise coded samples (e.g. MDCT lines) and a parametric description of the signal representation (e.g. spectral envelope of scale factors). In this case, the conditioned score may include two separate contributions, each one related to one of the stages of the signal representation.

[0015] In some embodiments, the generic generative model operates on a specialized processing unit, while the other parts of the decoding process are executed on a general-purpose computation resource. The specialized processing unit could be Graphical Processing Unit (GPU), Tensor Processing Unit (TPU), and most importantly a neural processing unit (NPU). Many CPU architectures, such as those in mobile devices, often contain an NPU. The generic unconditioned model may then be executed on the NPU while the actual audio decoder may be executed on a different computing resource in the same device (possibly within the same CPU), and interact with the generic unconditioned model on the NPU.

[0016] As the generic generative model does not require any task-specific input, the same model may be used in many other audio related tasks, such as restoration, processing, loss concealment, etc. This provides an opportunity for economy of scale for building and training such a model. It is therefore possible that the generic model is provided externally, e.g. from a manufacturer of a CPU, where it may be implemented on a NPU of the CPU architecture. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The present invention will be described in more detail with reference to the appended drawings, showing currently preferred embodiments of the invention.

[0018] Figure 1 shows a conventional codec architecture.

[0019] Figure 2 shows a codec architecture according to an embodiment of the invention.

[0020] Figure 3 shows a first example of the score-based diffusion process in figure 2.

[0021] Figure 4 shows a second example of the score-based diffusion process in figure 2.

[0022] Figure 5 shows a flow chart illustrating the process performed by the decoder in figure 2.

[0023] Figure 6 illustrates a schematic block diagram of an example device or architecture that may be used to implement various aspects of the present invention. DETAILED DESCRIPTION OF CURRENTLY PREFERRED EMBODIMENTS

[0024] In the following description, the Modulated Discrete Cosine Transform (MDCT) is used as an example of a transform used in the encoder. This should not be considered as a limitation of the invention, which is applicable also to other coding domains, such as QMF. CODEC structure

[0025] Figure 1 provides an example of a codec architecture 1 employing Modulated Discrete Cosine Transform (MDCT) on an encoding side 2 and a decoding side 3. The coded signal representation is two-stage (MDCT lines and conditional information such as spectral envelope or scale factors). The present disclosure is not limited to this architecture.

[0026] On the encoding side 2, a signal in a target domain, in this case a time-domain audio signal x is transformed by a coding transform, in this case an MDCT transform 10 operating on a window of N samples of x to provide one frame of a coding domain signal representation X having K=N / 2 MDCT frequency line coefficients. The MDCT transform uses a stride (shift in time) of N / 2 samples, and is referred to as a “lapped” transform.

[0027] For each frame m, the K MDCT lines are banded into B nonuniform, non- overlapping frequency bands. The envelope of one band is then estimated in block 11 aswhere Xb(m) and Kb are the line coefficients and the number of MDCT lines, respectively, in band b. The envelope values are then quantized in block 12 with a 3 dB step, yielding the quantization indices ie(b,m) forming a vector ye(m). The complete transfer function of blocks 11 and 12 is referred to as ge, so that ye= ge(X). The vector yeis encoded into a bitstream using frequency differential coding and Huffman coding by encoder 15.

[0028] The envelope information ye is further used in block 13 to spectrally flatten the signal representation X. For each frame m, this flattening operation is done for band b aswhere êb(m) is the reconstructed envelope value for ie(b,m). The flattened MDCT lines sb(m) are then quantized in block 14 by a set of uniform quantizers, where a step size is selected on a per- band basis depending on the envelope information in ye(m) and a target bitrate. The result is a set of quantization indices is(b,m) forming a vector ys(m). The complete transfer function of blocks 13 and 14 is referred to as gs, so that ys = gs(X,ye). The vector ys is coded into the bitstream using frequency differential coding and Huffman coding by encoder 15.

[0029] On the decoding side 3, after an initial decoding by decoder 16 to obtain ye and ys, the decoding process for frame m starts in block 17 by reconstructing the quantized flattened MDCT lines ^^^(^) in band b from the indices is(b,m). An unflattening operation follows in block 18, also dependent on the envelope information ye(m).

[0030] The complete transfer function of blocks 17 and 18 is referred to as hs, so that ^^ =ℎ^(^^, ^^). Finally, the decoded MDCT lines ^^ are transformed back into the time domain, inthis case the time domain, using an inverse coding transform, in this case an inverse MDCT transform 19, to form a reconstructed audio signal ^^.

[0031] In principle, blocks 17 an 18 of the decoder side 3 in figure 1 tries to approximate h ≈ g−1. This approximation is acceptable when the codec is operating with a high bitrate. However, when the bitrate is too low, the encoded signal lacks so much information that the decoded signal will have many artifacts such as spectral holes and loss of energy. To remedy this, it is common to fill the gaps in some deterministic way e.g. by continuing missing tones or by coping the low-band tones into the high band, or in some stochastic way like filling the gaps with white noise shaped according to the envelope. Decoding using score-based diffusion model

[0032] Figure 2 illustrates schematically an example of a codec architecture including a conventional (legacy) encoder 2, and a decoder 21 according to an embodiment of the invention.

[0033] The decoder 21 includes a bitstream decoder 22, configured to decode a bitstream to obtain a representation y of an audio signal x. The representation y is in a coding domain used by the encoder 11, here the MDCT domain. The decoder 21 further includes a conditional score computation stage 23 and a generic score model 24. Finally, the decoder 21 includes a diffusion stage 25, configured to implement an iterative diffusion process by sampling the conditional and unconditional scores.

[0034] The generic score model 24 is noise conditioned, and trained to predict a gradient of a log of a probability distribution of a noisy audio signal given a noise distribution. The generic score model 24 may be implemented as a generative neural network architecture, based on e.g., U-nets or transformer neural networks. Alternatively, the generic score model may include an autoregressive model that outputs a probability distribution, which is then backpropagated to get the score.

[0035] It is noted that such a generic score model may simultaneously be used for other audio related task, such as source separation or bandwidth extension. The generic score model 24 can operate in a different domain than the coding domain used in the encoder 2, in this case the MDCT domain. For example, the decoder 21 may operate in the time domain. The generic score model 24 may be a versatile, general purpose generative model which is trained using large amounts of data and which is useful in many applications.

[0036] The generic score model 24 may be implemented in a specialized computer resource 26, e.g. a hardware accelerated implementation or a neural processing unit. The elements of the decoder, i.e. the bitstream decoder 22, conditional score computation stage 23 and diffusion stage 25, may be implemented in a general purpose computer resource 27.

[0037] From a functional perspective, the conditional score computation block 23, the generic score model 24, and diffusion stage 25 form a score-based diffusion model 28. In the case of a new coding domain, such as the QMF domain, or a different bitrate, only the decoder22 needs to be modified. The score-based diffusion model 28, and in particular the unconditioned score model 24, remain unchanged.

[0038] Turning to figure 3, the score-based diffusion model 28 of the decoder 21 is shown in more detail. The diffusion process 28 includes the conditional score calculating stage 23 and the general purpose generative model 24. The two score contributions are added in a summation point 31 and passed on to the diffusion stage 25, here shown as two stages, a sampling stage 32 and an iteration stage 33.

[0039] With reference to figure 3, and the flow chart in figure 5, operation of the decoder 21 will be described. First, in step S1, the bitstream decoder 22 receives the bitstream and provides the signal representation y.

[0040] In general, the solution to fill-in the information lost during the encoding process should depend on the type of signals that are to be decoded. For example, if a generative neural network is used, it should be trained on the same type of audio, such as music or speech or general audio (e.g., a mixture of speech, background noise, music, etc.), and also be conditioned by the quantized (incomplete) signal. Here, it is proposed to reconstruct audio signal x bysampling from the conditional probability distribution of x given y, p(x|y). Defining ^^ = ^ + ^^,where σ is a constant and n ∼ ^(0, I), Langevin sampling is employed, working with the log gradient of the posterior of the joint distribution:. Eq.2 As the encoder states are discrete, the last term of Eq.4 is defined by a probability rather than a density. This is indicated by using a capital letter P.

[0041] In each iteration, the sampling stage 32 calculates the diffusion variable ^^^^based on a preceding value of the diffusion variable ^^^^^^and an aggregate score including an unconditional score obtained from the generic score model 24 and a conditional score obtained from the conditional score computation unit 23. The conditional score and unconditional score may simply be added together, or they may be appropriately weighted. Eventually, after a final iteration, the sampling stage 32 outputs the diffusion variable ^^^as the reconstructed signal ^^.

[0042] The number of iterations, I, will determine the computational complexity of the diffusion process, and may be set based on implementation specific requirements. This makes the decoder versatile, and adaptable to different applications.

[0043] As shown in figure 5, each iteration starts in steps S2 and S3. In step S2 (‘generate unconditional score’), based on an iteration specific noise distribution, σi, and the preceding value of the diffusion variablethe generic score model 24 provides an unconditional score ^^^(^^|^) representing a gradient of a probability distribution of the diffusion variable ^^^^.

[0044] In parallel, in step S3 (‘calculate conditional score’), the conditional score calculatingstage 23, provides a conditional score ^^^(^|^^, ^) representing a gradient of a probabilitydistribution of the signal representation y.

[0045] The unconditional and conditional scores are added in summation point 31 to form the aggregate score, and the next iteration value of the diffusion variable ^^ is obtained by sampler 32 in step S4. In the conditional iteration step S5, processing returns to steps S2 and S3 for a next iteration, or, at the end of the diffusion process (i=I), the process outputs ^^.

[0046] In the specific example, the coding domain signal representation is a multi-stage description, here an MDCT representation including MDCT lines, ysand spectral envelope ye. Inthis case, the conditional score ^^^(^|^^, ^) is composed of two conditional scores(^^|^^, ^) and^^^(^^|^^, ^), representing gradients of probability distributions of ye and ys respectively.

[0047] This is illustrated in figure 4, showing another example of a score based diffusion model 28’. The model 28’ is similar to the model 28 in figure 3, and corresponding elements have been indicated with corresponding numbers and have the same functionalities as previously described. In the model 28’, however, the conditional score computational block 23’ comprises two blocks, 41 and 42, each configured to compute conditional scores for ye and ys respectively. Implementation details

[0048] A Langevin sampling algorithm is used to estimate ^^ = ^ + ^^ with n ∼ ^(0, I)iteratively asEq.3 where i = 1, ... , I is the iteration number, and t ∈ U(0, 1) describes the algorithmic time going from t0= 0 to tI= 1. Further, ^^^denotes the iteration specific noise and !^^and "^^are hyperparameters.

[0049] Typically, the noise steps ^^# > ^^^ > ⋯ > ^^& are chosen geometrically spaced,with ^^# ≫ ^^^ ≫ ^^& such that

[0050] There are several choices available for the selection of the hyperparameters α and In the present example they are chosen as !^^Therefore, conditionalLangevin sampling can be done for i = 1, ... , I − 1 asEq.4awhere the initial state ^^^# is a realization sampled from ^(0, ^^+# ,), and for i = I:withEq.5a where

[0051] The expression in Eq.5a represents the aggregate score mentioned above, and is formed by adding the unconditional score in Eq.5b and the conditional score in Eq.5c.

[0052] The generic score model 24 may be trained in many different ways. One example of an appropriate training approach is described in “Generative modeling by estimating gradients of the data distribution”, by Song and Ermon, 2019. The article proposes score matching as a denoising algorithm that minimizes the loss functionEq.6where ^0(^^; ^) is the score model trying to match ^^^(^^; ^).

[0053] In principle, it could be possible to train conditional score models to match^^^(^^|^; ^). However, as mentioned above, such a model becomes obsolete once the encoding y= g(x) changes, e.g. when a particular element in Figure 1 is replaced or when the coder runs at new bitrates. For this reason, it is here proposed to use a general unconditional score model^^^(^^; ^) in conjunction with well-defined formulations for ^^^(^|^^; ^).

[0054] To find the conditional score ^^^(^|^^, ^), in conditional score computational block23, it is required to find the distribution of y = g(x) given ^^, P(y|^^). This problem is intractable ingeneral, so a model of x|^^ is constructed. Since ^^ = ^ + ^^ then ^ = ^^ − ^^ and the object is tofind the distribution of ^ = 3(^^ − ^^) given ^^. A heuristic approach is to treat ^^ and n asindependent. While this independence assumption is generally not true (except at the beginning and the end of the Langevin sampling process), it has been used successfully in the past for tasks like source separation and inpainting resulting in working schemes. The approximation is referred to as theThis approximation facilitates the derivation of P(y|^^).

[0055] When the codec transform U is unitary, which is the case for the MDCT transform 10 in figure 1, the conditional scores can be computed in the transform domain and then transformed back into the time domain where the Langevin sampling operates,

[0056] Furthermore, the noisy mean model approximation still holds in the transform domain aswhere u = U(x) and 4^ = U(^^). Therefore, the conditional scores in the transform domain can becomputed in bands in a separable (block diagonal) way such thatwhere the indexing over the frame value m has been dropped for clarity, as it becomes redundant under the noisy mean model approximation. This property will be exploited to define conditional scores per band.

[0057] In the illustrated example, the signal representation ybfor band b indicates an envelope quantization intervaland sample quantization intervals ^^^. A second heuristic is introduced by treating these two types of conditions as independent, inspired by considering small σ in eq.9. This leads to a simple superposition, Eq.11 Both of these conditional scores can now be derived explicitly from the gaussian assumption in eq.9. As mentioned above with reference to figure 4, the two contributions are calculated in blocks 41 and 42, each configured to compute conditional scores for ye and ys respectively. Envelope Quantization Scores:

[0058] Assume that ^^ ^^ describes the event 5^(4^) ∈ [57(^^), 58(^^^)], where eb is definedas in Eq. 1. With :^(u^) = <^5^(u^) / ^+it follows from eq.9 thatwhere >′+@A(B) is the non-central Chi-squared distribution with κ degrees of freedom, and non- centrality parameter λ. Under this definition, the conditional score for envelope quantization is derived in block 41 aswith C@(:, B) and ^D@(:, B) being the probability density function (PDF) and survival function(complementary cumulative density function (CDF)) of >′+@E(B), respectively. Sample Quantization Scores:

[0059] Assume that ^^GF , the k-th element of ^G, describes the event 4 ∈ [4 (^ ^^ ^H 7 ^), 48(^^^)], where ubk is the k-th sample of ub. Due to eq.9, the conditional score for sample quantization is derived in block 42 aswherewith φσ and ϕσ being the PDF and CDF of ^(0, ^+), respectively.

[0060] The independence of sample events following from eq.9 leads to a fully separable (diagonal) conditional score for the band b,

[0061] Another approach to computing the conditional scores is based on the fact that thenoisy mean model in Eq. 7 belongs to the family of Gaussian models ^|^^~^(J(^^), K(^^)). Itcan be shown that the best approximation of this type is achieved by using Tweedie’s formula:J(^^) = ^^ + ^+^^^(^^; ^) and K(^^) = ^+∇^^J(^^). By choosing J(^^) according to the noisy mean model, the relationship between Tweedie’s mean and covariance is preserved. Tweedie’smean can be approximated as J(^^) = ^^ + ^+^0(^^; ^). In order to avoid second derivatives ofthe score model it is here proposed to set K(^^) ≈ N+I, where r2 is a function of σ, for example,r2= σ2or r2= σ2 / (1+ σ2). This approximation is compatible with the conditional score formulas introduced above as:Generalizations

[0062] Figure 6 shows a schematic block diagram of an example electronic device or architecture 200 (e.g., a mobile device such as a smart phone) suitable for implementing example embodiments of the present disclosure. Architecture 200 may embody resources 26 and 27 discussed in relation to figure 2.

[0063] As shown, the architecture 200 includes central processing unit (CPU) 201 which is capable of performing various processes in accordance with a program stored in, for example, read only memory (ROM) 202 or a program loaded from, for example, storage unit 208 to random access memory (RAM) 203. The CPU 201 may be, for example, an electronic processor 201, which may include one or more processor cores, and in some examples the processor 201 may be multiple processors. In RAM 203, the data required when CPU 201 performs the variousprocesses is also stored, as required. CPU 201, ROM 202 and RAM 203 are connected to one another via bus 204. Input / output (I / O) interface 205 is also connected to bus 204.

[0064] The following components are connected to I / O interface 205: input unit 206, that may include a keyboard, a mouse, or the like; output unit 207 that may include a display such as a liquid crystal display (LCD) and one or more speakers; storage unit 208 including a hard disk, or another suitable storage device; and communication unit 209 which may include a network interface card such as a network card (e.g., wired or wireless).

[0065] In some implementations, input unit 206 includes one or more microphones in different positions (depending on the host device) enabling capture of audio signals in various formats (e.g., mono, stereo, spatial, immersive, and other suitable formats).

[0066] In some implementations, output unit 207 include systems with various number of speakers. Output unit 207 (depending on the capabilities of the host device) can render audio signals in various formats (e.g., mono, stereo, immersive, binaural, and other suitable formats).

[0067] In some embodiments, communication unit 209 is configured to communicate with other devices (e.g., via a network). Drive 210 is also connected to I / O interface 205, as required. Removable medium 211, such as a magnetic disk, an optical disk, a magneto-optical disk, a flash drive or another suitable removable medium is mounted on drive 210, so that a computer program read therefrom is installed into storage unit 208, as required. A person skilled in the art would understand that although apparatus 200 is described as including the above-described components, in real applications, it is possible to add, remove, and / or replace some of these components and all these modifications or alteration all fall within the scope of the present disclosure.

[0068] In accordance with example embodiments of the present disclosure, the processes described above may be implemented as computer software programs or on a computer-readable storage medium. For example, embodiments of the present disclosure include a computer program product including a computer program tangibly embodied on a machine readable medium, the computer program including program code for performing methods. In such embodiments, the computer program may be downloaded and mounted from the network via the communication unit 209, and / or installed from the removable medium 211, as shown in figure 6.

[0069] Generally, various example embodiments of the present disclosure may be implemented in hardware or special purpose circuits (e.g., control circuitry), software, logic or any combination thereof. For example, the units discussed above can be executed by control circuitry (e.g., CPU 201 in combination with other components of figure 6), thus, the control circuitry may be performing the actions described in this disclosure. Some aspects may be implemented in hardware, while other aspects may be implemented in firmware or softwarewhich may be executed by a controller, a processor and / or other computing device(s), which may include control circuitry. While various aspects of the example embodiments of the present disclosure are illustrated and described as block diagrams, flowcharts, or using some other pictorial representation, it will be appreciated that the blocks, apparatus, systems, techniques, or methods described herein may be implemented in, as non-limiting examples, hardware, software, firmware, special purpose circuits or logic, general purpose hardware or controller or other computing devices, or some combination thereof.

[0070] Additionally, various blocks shown in the flowcharts may be viewed as method steps, and / or as operations that result from operation of computer program code, and / or as a plurality of coupled logic circuit elements constructed to carry out the associated function(s). For example, embodiments of the present disclosure include a computer program product including a computer program tangibly embodied on a machine readable medium, the computer program containing program codes configured to carry out the methods as described above.

[0071] In the context of the disclosure, a machine-readable medium may be any tangible medium that may contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine- readable signal medium or a machine-readable storage medium. A machine-readable medium may be non-transitory and may include but not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium would include an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random-access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0072] Computer program code for carrying out methods of the present disclosure may be written in any combination of one or more programming languages. These computer program codes may be provided to one or more processors of a general-purpose computer, special purpose computer, or other programmable data processing apparatus that has control circuitry, such that the program codes, when executed by one or more processors of the computer or other programmable data processing apparatus, cause the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may execute entirely on a computer, partly on the computer, as a stand-alone software package, partly on the computer and partly on a remote computer or entirely on the remote computer or server or distributed over one or more remote computers and / or servers.

[0073] Unless specifically stated otherwise, as apparent from the following discussions, it is appreciated that throughout the disclosure discussions utilizing terms such as “processing”, “computing”, “calculating”, “determining”, “analyzing” or the like, refer to the action and / or processes of a computer hardware or computing system, or similar electronic computing devices, that manipulate and / or transform data represented as physical, such as electronic, quantities into other data similarly represented as physical quantities.

[0074] It should be appreciated that in the above description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the disclosure and aiding in the understanding of one or more of the various inventive aspects. This method of disclosure, however, is not to be interpreted as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed embodiment. Thus, the claims following the Detailed Description are hereby expressly incorporated into this Detailed Description, with each claim standing on its own as a separate embodiment of this invention. Furthermore, while some embodiments described herein include some but not other features included in other embodiments, combinations of features of different embodiments are meant to be within the scope of the invention, and form different embodiments, as would be understood by those skilled in the art. For example, in the following claims, any of the claimed embodiments can be used in any combination.

[0075] Furthermore, some of the embodiments are described herein as a method or combination of elements of a method that can be implemented by a processor of a computer system or by other means of carrying out the function. Thus, a processor with instructions for carrying out such a method or element of a method forms a means for carrying out the method or element of a method. Note that when the method includes several elements, e.g., several steps, no ordering of such elements is implied, unless specifically stated. Furthermore, an element described herein of an apparatus embodiment is an example of a means for carrying out the function performed by the element for the purpose of carrying out the embodiments of the invention. In the description provided herein, numerous specific details are set forth. However, it is understood that embodiments of the invention may be practiced without these specific details. In other instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.

[0076] The person skilled in the art realizes that the present invention by no means is limited to the preferred embodiments described above. On the contrary, many modifications and variations are possible within the scope of the appended claims.

[0077] Various aspects of the present disclosure may be appreciated from the following Enumerated Example Embodiments (EEEs): EEE1. A method for decoding an audio signal, x, the method comprising: receiving a bitstream including a signal representation, y, of the audio signal, x, in a coding domain; obtaining a reconstructed audio signal, ^^, in a target domain using an iterative score-based diffusion process, wherein each iteration calculates a diffusion variable, ^^^^, based on a preceding value of the diffusion variable, ^^^^^^, and an aggregate score, and setting the reconstructed signal, ^^, to the diffusion variable, ^^^, after a final iteration, wherein the aggregate score, ^^^(^^|^; ^), in each iteration is determined by:based on preceding value of the diffusion variable,and an iteration specific noise distribution, σi, generating an unconditional score, ^^^(^^; ^), related to aprobability distribution of the diffusion variable, ^^, using a generic score model operating in the target domain, the generic noise conditioned score model being trained to generate a function of a probability distribution of a noisy audio signal given a noise distribution; based on the preceding value of the diffusion variable, ^^^^^^, the iteration specific noise distribution and the signal representation, y, calculating a conditional score, ^^^(^|^^; ^), related to a probability distribution of the signal representation, y; andadding the unconditional score,^), and the conditional score, ^^^(^|^^; ^),to obtain the aggregate score, ^^^(^^|^; ^).EEE2. The method according to EEE1, wherein the unconditional score represents a gradient of a log of the probability distribution of the diffusion variable, ^^, and the conditional score represents a gradient of a log of the probability distribution of the signal representation, y. EEE3. The method according to EEE1 or EEE2, wherein the generic score model (24) is trained to generate a gradient of a log of the probability distribution of the noisy signal. EEE4. The method according to any one of the preceding EEEs, wherein the generic score model (24) is bitrate agnostic.EEE5. The method according to any one of the preceding EEEs, wherein, in each iteration, the diffusion variable, ^^^^, is computed using Langevin sampling. EEE6. The method according to any one of the preceding EEEs, wherein the conditional score is calculated by: determining a conditioned score in the coding domain, and converting the conditional score in the coding domain to the conditional score in the target domain. EEE7. The method according to EEE6, wherein the converting is performed by means of an invertible transform corresponding to the coding transform applied on the encoder side to obtain the signal representation. EEE8. The method according to any one of the preceding EEEs, wherein the signal representation, y, is a multi-stage description including a series of samples in the coding domain and corresponding conditioning information. EEE9. The method according to EEE8, wherein the conditioned score is calculated as a sum of two scores, a first score related to the samples in the coding domain, ys, and a second score related to the conditioning information, ye. EEE10. The method according to any one of the preceding EEEs, wherein the target domain is different from the coding domain. EEE11. The method according to any one of the preceding EEEs, wherein the target domain is time domain. EEE12. The method according to any one of the preceding EEEs, wherein the coding domain is MDCT or QMF domain. EEE13. The method according to EEE8, wherein the coding domain is MDCT and wherein the multi-stage description includes a series of quantized MDCT lines and either a spectral envelope or a set of scale factors. EEE14. A system for decoding an audio signal, x, comprising: a decoder (22) for receiving a bitstream and decoding a signal representation, y, ofthe audio signal, x, in a coding domain; a score-based diffusion model (28) for obtaining a reconstructed audio signal, ^^, in a target domain, the score based diffusion model including: a diffusion stage (25) configured to run an iterative score-based diffusion process where each iteration calculates a diffusion variable, ^^^^, based on a preceding value of the diffusion variable,, and an aggregate score, and to set the reconstructed signal, ^^, to the diffusion variable, ^^^, after a final iteration, a generic score model (24) operating in the target domain and trained to generate a function of a probability distribution of a noisy audio signal given a noise distribution, the generic score model configured to, based on preceding value of the diffusion variable,and an iteration specific noise distribution, σi, generate an unconditional score, ^^^(^^; ^),related to a probability distribution of the diffusion variable, a conditional score computational unit (23) configure to, based on the preceding value of the diffusion variable,the iteration specific noise distribution and the signal representation, y, calculate a conditional score, ^^^(^|^^; ^), related to a probabilitydistribution of the signal representation, y, wherein the aggregate score, ^^^(^^|^; ^), in each iteration is determined bysumming the unconditional score, ^^^(^^; ^), and the conditional score, ^^^(^|^^; ^).EEE15. The system according to EEE14, wherein the unconditional score represents a gradient of a log of the probability distribution of the diffusion variable, ^^, and the conditional score represents a gradient of a log of the probability distribution of the signal representation, y. EEE16. The system according to EEE14 or EEE15, wherein the generic score model (24) is trained to generate a gradient of a log of the probability distribution of the noisy signal. EEE17. The system according to any one of EEE14 - EEE16, wherein the generic score model (24) is bitrate agnostic. EEE18. The system according to any one of EEE14 - EEE17, wherein the conditional score computational unit is configured to calculate the conditional score by:determining a conditioned score in the coding domain, and converting the conditional score in the coding domain to the conditional score in the target domain. EEE19. The system according to EEE18, wherein the converting is performed by means of an invertible transform corresponding to the coding transform applied on the encoder side to obtain the signal representation. EEE20. The system according to any one of EEE14 - EEE19, wherein the audio signal representation, y, is a multi-stage description including a series of samples in the coding domain and corresponding conditioning information, and wherein the conditional score computational unit (23) is configured to calculate the conditioned score as a sum of a first score related to the samples in the coding domain and a second score related to the conditioning information. EEE21. The system according to any one of EEE14 - EEE20, wherein the domain of the signal representation is MDCT or QMF domain. EEE22. The system according to EEE20, wherein the multi-stage description includes a series of quantized MDCT lines and either a spectral envelope or a set of scale factors. EEE23. The system according to any one of EEE14 - EEE22, wherein the unconditioned score model (24) is implemented on a neural processing unit, NPU, of a CPU. EEE24. The system according to any one of EEE14 - EEE23, wherein the unconditioned score model (24) is provided separately from the decoder as a general purpose component. EEE25. A computer program product comprising computer program code portions configured to perform the method according to any one of EEE1 – EEE13 when executed on a computer processor.

Claims

CLAIMS 1. A method for decoding an audio signal, x, the method comprising: receiving a bitstream including a signal representation, y, of the audio signal, x, in a coding domain; obtaining a reconstructed audio signal, ^^, in a target domain using an iterative score-based diffusion process, wherein each iteration calculates a diffusion variable, ^^^^, based on a preceding value of the diffusion variable, ^^^^^^, and an aggregate score, and setting the reconstructed signal, ^^, to the diffusion variable, ^^^, after a final iteration, wherein the aggregate score, ^^^(^^|^; ^), in each iteration is determined by:based on preceding value of the diffusion variable, ^^^^^^, and an iteration specific noise distribution, σi, generating an unconditional score, ^^^(^^; ^), related to aprobability distribution of the diffusion variable, ^^, using a generic score model operating in the target domain, the generic noise conditioned score model being trained to generate a function of a probability distribution of a noisy audio signal given a noise distribution; based on the preceding value of the diffusion variable,, the iteration specific noise distribution and the signal representation, y, calculating a conditional score, ^^^(^|^^; ^), related to a probability distribution of the signal representation, y; andadding the unconditional score,^), and the conditional score, ^^^(^|^^; ^),to obtain the aggregate score, ^^^(^^|^; ^).

2. The method according to claim 1, wherein the unconditional score represents a gradient of a log of the probability distribution of the diffusion variable, ^^, and the conditional score represents a gradient of a log of the probability distribution of the signal representation, y.

3. The method according to claim 1 or 2, wherein the generic score model (24) is trained to generate a gradient of a log of the probability distribution of the noisy signal.

4. The method according to any one of the preceding claims, wherein the generic score model (24) is bitrate agnostic.

5. The method according to any one of the preceding claims, wherein, in each iteration, the diffusion variable, ^^^^, is computed using Langevin sampling.

6. The method according to any one of the preceding claims, wherein the conditional score is calculated by: determining a conditioned score in the coding domain, and converting the conditional score in the coding domain to the conditional score in the target domain.

7. The method according to claim 6, wherein the converting is performed by means of an invertible transform corresponding to the coding transform applied on the encoder side to obtain the signal representation.

8. The method according to any one of the preceding claims, wherein the signal representation, y, is a multi-stage description including a series of samples in the coding domain and corresponding conditioning information.

9. The method according to claim 8, wherein the conditioned score is calculated as a sum of two scores, a first score related to the samples in the coding domain, ys, and a second score related to the conditioning information, ye.

10. The method according to any one of the preceding claims, wherein the target domain is different from the coding domain.

11. The method according to any one of the preceding claims, wherein the target domain is time domain.

12. The method according to any one of the preceding claims, wherein the coding domain is MDCT or QMF domain.

13. The method according to claim 8, wherein the coding domain is MDCT and wherein the multi-stage description includes a series of quantized MDCT lines and either a spectral envelope or a set of scale factors.

14. A system for decoding an audio signal, x, comprising: a decoder (22) for receiving a bitstream and decoding a signal representation, y, ofthe audio signal, x, in a coding domain; a score-based diffusion model (28) for obtaining a reconstructed audio signal, ^^, in a target domain, the score based diffusion model including: a diffusion stage (25) configured to run an iterative score-based diffusion process where each iteration calculates a diffusion variable, ^^^^, based on a preceding value of the diffusion variable, ^^^^^^, and an aggregate score, and to set the reconstructed signal, ^^, to the diffusion variable, ^^^, after a final iteration, a generic score model (24) operating in the target domain and trained to generate a function of a probability distribution of a noisy audio signal given a noise distribution, the generic score model configured to, based on preceding value of the diffusion variable,and an iteration specific noise distribution, σi, generate an unconditional score, ^^^(^^; ^),related to a probability distribution of the diffusion variable, a conditional score computational unit (23) configure to, based on the preceding value of the diffusion variable,the iteration specific noise distribution and the signal representation, y, calculate a conditional score, ^^^(^|^^; ^), related to a probabilitydistribution of the signal representation, y, wherein the aggregate score, ^^^(^^|^; ^), in each iteration is determined bysumming the unconditional score, ^^^(^^; ^), and the conditional score, ^^^(^|^^; ^).

15. The system according to claim 14, wherein the unconditional score represents a gradient of a log of the probability distribution of the diffusion variable, ^^, and the conditional score represents a gradient of a log of the probability distribution of the signal representation, y.

16. The system according to claim 14 or 15, wherein the generic score model (24) is trained to generate a gradient of a log of the probability distribution of the noisy signal.

17. The system according to any one of claims 14 - 16, wherein the generic score model (24) is bitrate agnostic.

18. The system according to any one of claims 14 - 17, wherein the conditional score computational unit is configured to calculate the conditional score by: determining a conditioned score in the coding domain, and converting the conditional score in the coding domain to the conditional score in the target domain.

19. The system according to claim 18, wherein the converting is performed by means of an invertible transform corresponding to the coding transform applied on the encoder side to obtain the signal representation.

20. The system according to any one of claims 14 - 19, wherein the audio signal representation, y, is a multi-stage description including a series of samples in the coding domain and corresponding conditioning information, and wherein the conditional score computational unit (23) is configured to calculate the conditioned score as a sum of a first score related to the samples in the coding domain and a second score related to the conditioning information.

21. The system according to any one of claims 14 - 20, wherein the domain of the signal representation is MDCT or QMF domain.

22. The system according to claim 20, wherein the multi-stage description includes a series of quantized MDCT lines and either a spectral envelope or a set of scale factors.

23. The system according to any one of claims 14 – 22, wherein the unconditioned score model (24) is implemented on a neural processing unit, NPU, of a CPU.

24. The system according to any one of claims 14 - 23, wherein the unconditioned score model (24) is provided separately from the decoder as a general purpose component.

25. A computer program product comprising computer program code portions configured to perform the method according to any one of claims 1 – 13 when executed on a computer processor.

Images