Audio super-resolution processing method and audio super-resolution model training method and device

Abstract

This disclosure provides an audio super-resolution processing method, an audio super-resolution model training method, and an apparatus. The method includes: acquiring multiple super-resolution data pairs, each pair comprising a low-resolution waveform signal and a high-resolution waveform signal representing the same audio signal; for each super-resolution data pair, establishing a solvable path from the low-resolution waveform signal to the high-resolution waveform signal using a Schrödinger bridge model; and training the Schrödinger bridge model based on the solvable path to obtain a trained Schrödinger bridge audio super-resolution model. The Schrödinger bridge audio super-resolution model trained according to this disclosure can directly optimize low-resolution audio in the waveform space of the audio signal to obtain high-resolution audio. While preserving low-frequency data, it recovers high-frequency audio details, significantly improving the efficiency of audio super-resolution processing and the fidelity of high-frequency details.

Description

Technical Field

[0001] This disclosure relates to the fields of computer technology and artificial intelligence technology, and in particular to an audio super-resolution processing method, an audio super-resolution model training method and apparatus. Background Technology

[0002] Speech signal processing has wide applications in many fields, such as game development, smart homes, and autonomous driving. Speech super-resolution technology is an important branch of speech signal processing. Through speech super-resolution processing, high-sampling-rate speech signals can be generated from low-sampling-rate speech signals, thereby improving the quality and effect of speech signals.

[0003] In related technologies, a diffusion model is used to gradually generate high-resolution audio from standard Gaussian noise, realizing the process of generating audio from noise without any prompting information. However, in this audio super-resolution processing scheme, the trajectory of generating audio based on noise without prompting information is relatively long, and it is not conducive to maintaining the low-frequency signal, thus resulting in low accuracy of the generated audio. Summary of the Invention

[0004] The embodiments of this disclosure provide an audio super-resolution processing method, an audio super-resolution model training method, and an apparatus.

[0005] According to a first aspect of the present disclosure, an audio super-resolution model training method is provided. The method includes: acquiring multiple super-resolution data pairs, each super-resolution data pair including a low-resolution waveform signal and a high-resolution waveform signal representing the same audio; for each super-resolution data pair, establishing a solvable path from the low-resolution waveform signal to the high-resolution waveform signal using a Schrödinger bridge model; and training the Schrödinger bridge model to be trained based on the solvable path to obtain a trained Schrödinger bridge audio super-resolution model.

[0006] According to a second aspect of the present disclosure, an audio super-resolution processing method is provided, the method comprising: acquiring a low-resolution signal to be processed; interpolating the low-resolution signal to be processed using a linear interpolation algorithm to obtain generating prior information; and generating a high-resolution target waveform based on the generating prior information using a trained Schrödinger bridge audio super-resolution model, wherein the trained Schrödinger bridge audio super-resolution model maintains the low-frequency information in the generating prior information unchanged in each sampling step of generating the high-resolution target waveform.

[0007] According to a third aspect of the present disclosure, an audio super-resolution model training apparatus is provided, comprising: a sample acquisition module for acquiring multiple super-resolution data pairs, each super-resolution data pair including a low-resolution waveform signal and a high-resolution waveform signal representing the same audio; a path generation module for establishing a solvable path from the low-resolution waveform signal to the high-resolution waveform signal using a Schrödinger bridge model for each super-resolution data pair; and an audio super-resolution model training module for training the Schrödinger bridge model to be trained based on the solvable path to obtain a trained Schrödinger bridge audio super-resolution model.

[0008] According to a fourth aspect of the present disclosure, an audio super-resolution processing apparatus is provided, comprising: an acquisition module for acquiring a low-resolution signal to be processed; an adjustment module for interpolating the low-resolution signal to be processed using a linear interpolation algorithm to obtain generation prior information; and a generation module for generating a high-resolution target waveform based on the generation prior information using a trained Schrödinger bridge audio super-resolution model, wherein the trained Schrödinger bridge audio super-resolution model maintains the low-frequency information in the generation prior information unchanged in each sampling step of generating the high-resolution target waveform.

[0009] According to a fifth aspect of the present disclosure, a computer-readable storage medium is provided, which stores a computer program for performing the above-described audio super-resolution processing method or audio super-resolution model training method.

[0010] According to a sixth aspect of the present disclosure, an electronic device is provided, comprising: a processor; a memory for storing processor-executable instructions; and a processor configured to read executable instructions from the memory and execute the instructions to implement the above-described audio super-resolution processing method or audio super-resolution model training method.

[0011] According to a seventh aspect of the present disclosure, a computer program product is provided, including computer program instructions, which, when executed by a processor, implement the above-described audio super-resolution processing method or audio super-resolution model training method.

[0012] Based on the embodiments of this disclosure, when training the audio super-resolution model, multiple super-resolution data pairs are first acquired. These data pairs include low-resolution waveform signals and high-resolution waveform signals representing a unified audio signal. For each super-resolution data pair, a solvable path from the low-resolution waveform signal to the high-resolution waveform signal is established using a Schrödinger bridge model. Based on this solvable path, the Schrödinger bridge model is trained to obtain the trained Schrödinger bridge audio super-resolution model. This technical solution utilizes super-resolution data pairs to train a Schrödinger bridge audio super-resolution model from low-resolution waveform signals to high-resolution waveform signals. The model has a small number of parameters and lightweight characteristics, allowing for more flexible deployment on resource-constrained hardware. It is more suitable for use in embedded and mobile devices and applicable to a wide range of practical application scenarios. Furthermore, the trained Schrödinger bridge audio super-resolution model can directly optimize low-resolution audio in the waveform space of the audio to obtain high-resolution audio. While preserving low-frequency data, it recovers high-frequency audio details, greatly improving the efficiency of audio super-resolution processing and the fidelity of high-frequency details.

[0013] Based on the embodiments of this disclosure, when audio super-resolution processing is required, a low-resolution signal to be processed is acquired; a linear interpolation algorithm is used to interpolate the low-resolution signal to be processed to obtain prior information; based on the prior information, a trained Schrödinger bridge audio super-resolution model is used to generate a high-resolution target waveform. The trained Schrödinger bridge audio super-resolution model maintains the low-frequency information in the prior information unchanged in each sampling step of generating the high-resolution target waveform. In this disclosure, the trained Schrödinger bridge audio super-resolution model maintains the low-frequency information in the prior information unchanged in each sampling step of generating the corresponding high-resolution target waveform, focusing on the generation of high-frequency information. This optimizes the high-frequency details and fidelity in audio super-resolution processing, making the high-resolution signal after super-resolution processing more natural and fluid. Furthermore, performing audio super-resolution processing directly in the waveform space avoids cascading errors and data space loss problems.

[0014] The technical solutions of this disclosure will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0015] The above and other objects, features, and advantages of this disclosure will become more apparent from the more detailed description of the embodiments thereof in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of this disclosure and form part of the specification. They are used together with the embodiments of this disclosure to explain the disclosure and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps.

[0016] Figure 1 This is the system framework diagram to which this disclosure applies.

[0017] Figure 2 This is a schematic flowchart of an audio super-resolution processing method provided in an exemplary embodiment of this disclosure.

[0018] Figure 3 This is a flowchart illustrating the training process of the Schrödinger bridge model provided in an exemplary embodiment of this disclosure.

[0019] Figure 4 This is a schematic diagram of the forward and backward processes of the Schrödinger bridge model provided in an exemplary embodiment of this disclosure.

[0020] Figure 5 This is a flowchart illustrating the training process of the Schrödinger bridge model provided in another exemplary embodiment of this disclosure.

[0021] Figure 6 This is a flowchart illustrating the training process of the Schrödinger bridge model provided in another exemplary embodiment of this disclosure.

[0022] Figure 7 This is a schematic diagram of the structure of an audio super-resolution processing apparatus provided in an exemplary embodiment of the present disclosure.

[0023] Figure 8 This is a schematic diagram of the structure of an audio super-resolution model training device provided in an exemplary embodiment of this disclosure.

[0024] Figure 9 This is a schematic diagram of the structure of an audio super-resolution model training device provided in another exemplary embodiment of this disclosure.

[0025] Figure 10 This is a structural diagram of an electronic device provided in an exemplary embodiment of this disclosure. Detailed Implementation

[0026] Hereinafter, exemplary embodiments according to the present disclosure will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of the present disclosure, and not all embodiments of the present disclosure, and it should be understood that the present disclosure is not limited to the exemplary embodiments described herein.

[0027] It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of this disclosure.

[0028] Those skilled in the art will understand that the terms "first," "second," etc., in the embodiments of this disclosure are only used to distinguish different steps, devices, or modules, and do not represent any specific technical meaning, nor do they indicate a necessary logical order between them.

[0029] It should also be understood that in the embodiments disclosed herein, "a plurality of" may refer to two or more, and "at least one" may refer to one, two or more.

[0030] It should also be understood that any component, data or structure mentioned in the embodiments of this disclosure can generally be understood as one or more unless expressly defined or given to the contrary in the context.

[0031] Furthermore, the term "and / or" in this disclosure is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this disclosure generally indicates that the preceding and following related objects have an "or" relationship.

[0032] It should also be understood that the description of the various embodiments in this disclosure emphasizes the differences between the various embodiments, and the similarities or similarities can be referred to each other. For the sake of brevity, they will not be described in detail.

[0033] At the same time, it should be understood that, for ease of description, the dimensions of the various parts shown in the accompanying drawings are not drawn according to actual scale.

[0034] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit this disclosure or its application or use.

[0035] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0036] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0037] The embodiments disclosed herein can be applied to electronic devices such as terminal devices, computer systems, and servers, and can operate together with a wide range of other general-purpose or special-purpose computing system environments or configurations. Examples of well-known terminal devices, computing systems, environments, and / or configurations suitable for use with electronic devices such as terminal devices, computer systems, and servers include, but are not limited to: personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputer systems, mainframe computer systems, and distributed cloud computing environments including any of the above systems, etc.

[0038] Electronic devices such as terminal devices, computer systems, and servers can be described in the general context of computer system executable instructions (such as program modules) executed by a computer system. Typically, program modules can include routines, programs, object programs, components, logic, data structures, etc., which perform specific tasks or implement specific abstract data types. Computer systems / servers can be implemented in distributed cloud computing environments, where tasks are executed by remote processing devices linked through communication networks. In distributed cloud computing environments, program modules can reside on local or remote computing system storage media, including storage devices.

[0039] This disclosure outlines

[0040] Current audio super-resolution processing schemes typically utilize a diffusion model to progressively generate high-resolution waveforms from standard Gaussian noise, thus generating audio from noise without any cueing information. However, the trajectory of audio generated from noise is relatively long and it is not conducive to preserving the low-frequency signal, resulting in low accuracy of the generated audio.

[0041] This disclosure obtains prior information by interpolating low-resolution signals, and then uses a trained Schrödinger bridge audio super-resolution model to generate high-resolution signals based on the prior information. It keeps the low-frequency information in the prior information unchanged and focuses on the generation of high-frequency information, thus optimizing the high-frequency details and fidelity in audio super-resolution processing, making the high-resolution signal after super-resolution processing more natural and smooth.

[0042] Exemplary System

[0043] Figure 1 An exemplary system architecture is shown for an audio super-resolution processing method or apparatus to which embodiments of the present disclosure can be applied.

[0044] like Figure 1 As shown, the system architecture may include a terminal device 101, a communication network 102, an audio input device 103, and an audio output device 104. The network 102 serves as the medium for providing a communication link between the terminal device 101 and the audio input device 103. The network 102 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0045] Users can use terminal device 101 to interact with audio input device 103 and audio output device 104 via communication network 102 to receive or send audio, etc. Various communication client applications can be installed on terminal device 101, such as applications for generating artificial intelligence (AI) content (e.g., AI videos or AI images), multimedia applications, search applications, web browser applications, shopping applications, instant messaging tools, etc.

[0046] Terminal device 101 can be any electronic device capable of deploying the Schrödinger bridge model, including but not limited to mobile terminals such as mobile phones, laptops, personal digital assistants (PDAs), portable Android devices (PADs), portable media players (PMPs), and fixed terminals such as digital televisions, desktop computers, and smart home appliances.

[0047] Audio input device 103 can be a device that provides various low-resolution audio. Audio output device 104 is a device that receives high-resolution audio.

[0048] It should be noted that the audio super-resolution processing method provided in the embodiments of this disclosure can be executed by the terminal device 101. Accordingly, the audio super-resolution processing device can be set in the audio input device 103 or in the terminal device 101.

[0049] The audio super-resolution model training method provided in the embodiments of this disclosure can be executed by the terminal device 101 or by another server, and the Schrödinger bridge model is deployed on the terminal device 101 after it is trained.

[0050] It should be understood that Figure 1 The number of terminal devices 101, network 102, audio input devices 103, and audio output devices 104 in the diagram is merely illustrative. Depending on implementation needs, any number of terminal devices 101, network 102, audio input devices 103, and audio output devices 104 can be included. For example, if audio super-resolution processing does not require remote processing, the above system architecture may exclude the network and server, including only terminal devices 101.

[0051] Exemplary methods

[0052] Figure 2 This is a schematic flowchart of an audio super-resolution processing method provided in an exemplary embodiment of this disclosure. This embodiment can be applied to electronic devices, such as... Figure 1 On mobile devices, such as Figure 2 As shown, it includes the following steps:

[0053] Step 201: Obtain the low-resolution signal to be processed.

[0054] The low-resolution signal to be processed refers to a signal with a low resolution; for example, audio with a resolution in the range of 2 kHz to 16 kHz is considered low-resolution audio. In this disclosure, the low-resolution signal to be processed is used to indicate an audio signal that requires super-resolution processing. Super-resolution processing indicates the process of upsampling the audio signal to a higher resolution, such as 48 kHz.

[0055] Step 202: Use a linear interpolation algorithm to interpolate the low-resolution signal to be processed to obtain the prior information.

[0056] In this embodiment, the sampling rate of the low-resolution signal to be processed is increased to the target sampling rate through linear interpolation (e.g., sinc interpolation), so that the number of sampling points used to generate prior information is the same as the number of sampling points used to generate the high-resolution target waveform. For example, interpolation is performed between any two sampling points of the low-resolution signal according to the paradigm of a sine wave, thereby increasing the sampling point between every two sampling points, increasing the sampling rate, and increasing the data space.

[0057] Step 203: Based on the aforementioned prior information, a high-resolution target waveform is generated using the trained Schrödinger bridge audio super-resolution model. The trained Schrödinger bridge audio super-resolution model maintains the low-frequency information in the prior information unchanged in each sampling step of generating the high-resolution target waveform.

[0058] Among them, the high-resolution target waveform is the high-resolution waveform corresponding to the generated prior information generated by the trained Schrödinger bridge audio super-resolution model based on the generated prior information, for example, an audio waveform with a resolution of 24kHz to 48kHz.

[0059] Among them, the trained Schrödinger bridge audio super-resolution model is used to indicate the model that can determine and remove the noise to be removed corresponding to each sampling step in the generated prior information based on the generated prior information, thereby obtaining the high-resolution target waveform.

[0060] In this example, the trained Schrödinger bridge audio super-resolution model maintains the low-frequency information of the prior information at each sampling step when performing audio super-resolution processing, while predicting the audio waveform of the super-resolution. This realizes a sampling path from data to data, avoiding the noise-to-data generation process of the diffusion model. It can better utilize low-resolution audio as prior information for generating super-resolution audio, improving the generation efficiency of super-resolution audio signals and the fidelity of high-frequency information.

[0061] In this disclosure, the trained Schrödinger bridge audio super-resolution model can be obtained through the forward process of the Schrödinger bridge model, and can be trained using multiple super-resolution datasets. The training process can be found in [reference needed]. Figure 3 The illustrated embodiment. The reverse process of the trained Schrödinger bridge audio super-resolution model is used to instruct the process of progressively removing noise to obtain the high-resolution target waveform.

[0062] Through steps 201 to 203 above, when audio super-resolution processing is required, a low-resolution signal to be processed is acquired; a linear interpolation algorithm is used to interpolate the low-resolution signal to be processed to obtain prior information; based on the prior information, a high-resolution target waveform is generated using a trained Schrödinger bridge audio super-resolution model. In each sampling step of generating the high-resolution target waveform, the trained Schrödinger bridge audio super-resolution model maintains the low-frequency information in the prior information unchanged. This disclosure utilizes a trained Schrödinger bridge audio super-resolution model to generate the corresponding high-resolution target waveform, maintaining the low-frequency information in the prior information unchanged in each sampling step, focusing on the generation of high-frequency information, optimizing the high-frequency details and fidelity in audio super-resolution processing, making the high-resolution signal after super-resolution processing more natural and fluid. Furthermore, performing audio super-resolution processing directly in the waveform space avoids cascading errors and data space loss problems. Furthermore, by interpolating low-resolution signals to obtain prior information, the waveform space of low-frequency data can be aligned to the waveform space of high-frequency data. This helps the model generate high-frequency detail data better while keeping low-frequency information unchanged or with minimal change, resulting in a more natural and smooth audio signal and improving the quality of the generated audio signal.

[0063] like Figure 3 As shown, the training process of the trained Schrödinger bridge audio super-resolution model may include the following steps.

[0064] Step 301: Acquire multiple super-resolution data pairs, each super-resolution data pair including a low-resolution waveform signal and a high-resolution waveform signal representing the same audio.

[0065] Among them, the low-resolution waveform signal is the waveform of low-resolution audio, for example, the waveform of audio with a resolution of 2kHz; the high-resolution waveform signal is the waveform of high-resolution audio, for example, the waveform of audio with a resolution of 48kHz.

[0066] In this embodiment, a signal processing filter can be used to copy the high-resolution waveform signal to obtain a low-resolution waveform signal corresponding to the high resolution. Therefore, the low-resolution waveform signal and the high-resolution waveform signal are audio waveforms with different resolutions representing the same audio.

[0067] In some alternative implementations, after copying the high-resolution waveform signal using a signal processing filter to obtain the corresponding low-resolution waveform signal, a scaling factor can be used to amplify both the low-resolution and high-resolution waveform signals in each super-resolution data pair, resulting in amplified low-resolution and high-resolution waveform signals. These amplified signals are then used to train the Schrödinger bridge model. In this implementation, amplifying the low-resolution and high-resolution waveform signals allows the trained Schrödinger bridge model to better perceive the differences between the two waveforms, ensuring the loss function remains sensitive and avoiding the gradient vanishing problem during training. This helps improve the model's training efficiency and stability.

[0068] Step 302: For each super-resolution data pair, a solvable path from the low-resolution waveform signal to the high-resolution waveform signal is established using the Schrödinger bridge model.

[0069] The Schrödinger bridge model uses stochastic differential equations to generate solvable paths, which are defined by an asymmetric noise scheduling strategy.

[0070] Specifically, see Figure 4 The diagram illustrates the process of obtaining the trained Schrödinger bridge audio super-resolution model by training the Schrödinger bridge model. First, a forward stochastic differential equation (SDE) is defined, as shown in equation (1). This stochastic differential equation is defined in the form of a symmetric noise scheduling strategy, as shown in equation (2).

[0071]

[0072] In equation (1), f(t)x t Here, g(t) is the drift term, g(t) is the diffusion coefficient, and w is the diffusion coefficient. t For the standard Wiener process, Ψ t (x t ) is used to represent a probability distribution.

[0073] Equation (2) is used to determine the noise in the audio waveform at each sampling step. Based on Equation (2), an asymmetric noise scheduling strategy for the Schrödinger bridge model can be implemented, such as... Figure 4 The noise curve is shown in reference numeral 31.

[0074] According to equation (1), a boundary condition "p(0) = x" can be constructed. HR p(T)=x LRThe path is " ". Based on this path, noise can be gradually added to the high-resolution waveform signal through a diffusion process of T sampling steps to obtain the final low-resolution waveform signal.

[0075] In this embodiment, the sampling step is used to indicate the discrete time unit of the simulation system evolution. It can be used to control the progress of the noise addition process of the Schrödinger bridge model. Usually, each sampling step corresponds to a specific noise. As the sampling step increases, the noise level gradually increases. After it increases to a certain noise level, the added noise will cancel out some of the noise in the audio, so that the noise in the audio gradually decreases.

[0076] In this disclosure, the trained Schrödinger bridge audio super-resolution model employs an asymmetric noise scheduling strategy to denoise the audio signal to be processed. The number of sampling steps for adding noise and the number of sampling steps for reducing noise are different, such as... Figure 4 The noise curve is shown in the figure, where the noise peak value σ max Located to the right of the noise curve, the number of sampling steps for adding noise is less than the number of sampling steps for denoising. Therefore, more sampling steps are used in the process of generating super-resolution audio data for denoising. Using this asymmetric noise scheduling strategy to denoise the audio signal can improve the super-resolution effect and the quality of the super-resolution audio.

[0077] The sampling step can be the time reference used by the Schrödinger bridge model when adding noise to the high-resolution waveform during the forward process, or it can be the time reference used by the Schrödinger bridge model when determining and removing noise from the low-resolution audio waveform during the backward process. The noise to be removed corresponding to each sampling step can refer to the noise that needs to be removed from the audio waveform. In practical applications, there can be one noise to be removed under each sampling step.

[0078] Step 303: Based on the solvable path, the Schrödinger Bridge model is trained to obtain the trained Schrödinger Bridge audio super-resolution model.

[0079] Among them, based on the solvable path, the Schrödinger bridge model loss and multi-scale auxiliary loss can be used to guide the training of the Schrödinger bridge model, and the trained Schrödinger bridge audio super-resolution model can be obtained.

[0080] In this disclosure, the Schrödinger bridge model loss is used to characterize the error between the predicted audio waveform signal and the real high-frequency waveform signal of the Schrödinger bridge model at each sampling step, and the multi-scale auxiliary loss is used to characterize the multi-scale auxiliary loss of the Schrödinger bridge model at each sampling step.

[0081] In this disclosure, the training process of the Schrödinger bridge model may include a pre-training process and a fine-tuning process: based on a solvable path, the Schrödinger bridge model is pre-trained using the Schrödinger bridge model loss to obtain a pre-trained Schrödinger bridge model; then, the pre-trained Schrödinger bridge model is fine-tuned using the Schrödinger bridge model loss and multi-scale auxiliary loss to finally obtain the trained Schrödinger bridge audio super-resolution model.

[0082] In this example, the Schrödinger bridge model loss can be obtained based on the error between the predicted audio waveform signal and the real high-frequency waveform signal at each sampling step.

[0083] Specifically, in the process of establishing a solvable path from low-resolution waveform signal to high-resolution waveform signal using the Schrödinger bridge model, for the t-th sampling step, the Schrödinger bridge model is responsible for predicting the waveform state of the predicted audio waveform signal of the current sampling step based on the above forward stochastic differential equation and the waveform state of the previous sampling step, and determining the Schrödinger bridge model loss based on the real high-frequency waveform signal, as shown in Equation (3).

[0084]

[0085] In equation (3), x t The waveform of sampling step t predicted by the Schrödinger bridge model is x. θ For the Schrödinger bridge model, x T x0 represents the low-resolution sample waveform, while x0 represents the high-resolution sample waveform.

[0086] The error between the predicted audio waveform signal and the real high-frequency waveform signal at each sampling step can be obtained through the loss function of the above equation (3), which is the Schrödinger bridge model loss. Based on the Schrödinger bridge model loss, the forward stochastic differential equation of the Schrödinger bridge model is optimized to guide the training of the Schrödinger bridge model, so that the Schrödinger bridge model has the ability to predict high-resolution waveforms at each sampling step t.

[0087] The audio super-resolution model training method provided in this disclosure first acquires multiple super-resolution data pairs during training. These data pairs include low-resolution and high-resolution waveform signals representing a unified audio signal. For each super-resolution data pair, a solvable path from the low-resolution waveform signal to the high-resolution waveform signal is established using a Schrödinger bridge model. Based on this solvable path, the Schrödinger bridge model is trained to obtain the trained Schrödinger bridge audio super-resolution model. This technical solution utilizes super-resolution data pairs to train a Schrödinger bridge audio super-resolution model from low-resolution to high-resolution waveform signals. The model has a small number of parameters and lightweight characteristics, allowing for more flexible deployment on resource-constrained hardware. It is more suitable for use in embedded and mobile devices and applicable to a wide range of practical application scenarios. Furthermore, the trained Schrödinger bridge audio super-resolution model can directly optimize low-resolution audio in the waveform space of the audio signal to obtain high-resolution audio. While preserving low-frequency data, it recovers high-frequency audio details, significantly improving audio quality. It improves the efficiency of high-frequency super-resolution processing and the fidelity of high-frequency details. In addition, since the Schrödinger Bridge model is a lightweight network structure with only about 1.7M parameters, it requires less computing resources during training and occupies less memory and storage space when running on devices. This improves the deployment flexibility of the trained Schrödinger Bridge audio super-resolution model. For example, it can be deployed more flexibly on resource-constrained hardware and is more suitable for use on embedded devices and mobile devices. Furthermore, the lightweight network structure can improve the speed of audio super-resolution processing and reduce resource requirements, making it suitable for a wide range of practical application scenarios.

[0088] In the above Figure 3 Based on the illustrated embodiment, to further improve the super-resolution performance of the trained Schrödinger bridge audio super-resolution model, additional multi-scale auxiliary loss can be incorporated during training to fine-tune the model, guiding it to better capture high-frequency features while maintaining consistency in the low-frequency components. For example... Figure 5 As shown, the process of calculating the multi-scale auxiliary loss includes the following steps:

[0089] Step 501: For the predicted audio waveform signal (i.e. the predicted high-frequency waveform signal) of each sampling step, the predicted audio waveform signal of each sampling step is converted into multiple frequency domain signals corresponding to multiple window functions using short-time Fourier transform, thereby obtaining multiple predicted frequency domain signals for each sampling step; and, the high-resolution waveform signal is converted into multiple frequency domain signals corresponding to multiple window functions using short-time Fourier transform, thereby obtaining frequency domain signals corresponding to multiple high-resolution waveform signals.

[0090] Among them, the frequency domain signal is used to describe the frequency characteristics of the audio signal, with the horizontal axis representing the frequency and the vertical axis representing the amplitude of the frequency signal.

[0091] In this disclosure, the predicted resolution waveform of each sampling step can be converted into a frequency domain signal by Short-Time Fourier Transform (STFT), thereby determining the frequency and phase of the frequency domain signal.

[0092] In practice, a time-frequency localization window function is selected. It is assumed that the analysis window function g(t) is stationary (pseudo-stationary) over a short time interval. By moving the window function, the power spectrum at different times is determined, resulting in a frequency domain signal with a resolution matching the chosen window function. If the resolution needs to be changed, the window function must be reselected.

[0093] In this embodiment, for the predicted audio waveform signal of each sampling step, the short-time Fourier transform can be used to convert the predicted audio waveform signal of each sampling step into multiple frequency domain signals of different resolutions based on multiple window functions, thereby obtaining multiple predicted frequency domain signals for each sampling step; and, using the short-time Fourier transform, the high-resolution waveform signal can be converted into multiple frequency domain signals corresponding to the multiple window functions based on multiple window functions, thereby obtaining frequency domain signals corresponding to multiple high-resolution waveform signals.

[0094] Furthermore, after obtaining the frequency domain signals corresponding to the multiple predicted frequency domain signals and multiple high-resolution waveform signals of each sampling step, step 502 can be executed to obtain the short-time Fourier transform amplitude loss, and step 503 can be executed to obtain the anti-wrapping phase loss.

[0095] Step 502: Based on the amplitude information of the predicted frequency domain signal in each sampling step and the amplitude information of the frequency domain signal corresponding to the high-resolution waveform signal, the short-time Fourier transform amplitude loss is obtained.

[0096] In this example, after obtaining the predicted frequency domain signals and the frequency domain signals corresponding to the high-resolution waveforms at multiple resolutions using different window functions, a set of short-time Fourier transform amplitude losses can be calculated based on the predicted frequency domain signals and the frequency domain signals corresponding to the high-resolution waveforms at each resolution. Then, the short-time Fourier transform amplitude loss of the Schrödinger bridge model can be obtained based on the short-time Fourier transform amplitude loss obtained at each resolution.

[0097] Step 503: Based on the phase information of the predicted frequency domain signal in each sampling step and the phase information of the frequency domain signal corresponding to the high-resolution waveform signal, anti-wrapping phase loss is obtained.

[0098] In this example, after obtaining the predicted frequency domain signals and the frequency domain signals corresponding to the high-resolution waveforms at multiple resolutions by using different window functions, a set of anti-wrapping phase losses can be calculated based on the predicted frequency domain signals and the frequency domain signals corresponding to the high-resolution waveforms at each resolution. Then, the anti-wrapping phase loss of the Schrödinger bridge model is obtained based on the anti-wrapping phase loss obtained at each resolution.

[0099] Step 504: Determine the weights of the short-time Fourier transform amplitude loss and the anti-wrapping phase loss to obtain the multi-scale auxiliary loss.

[0100] Among them, the multi-scale auxiliary loss includes short-time Fourier transform amplitude loss and anti-wrapping phase loss.

[0101] By calculating the weights of the short-time Fourier transform amplitude loss and the anti-wrapping phase loss, a multi-scale auxiliary loss can be obtained. The weights of the short-time Fourier transform amplitude loss and the anti-wrapping phase loss can be adjusted and optimized during model training to ensure that the obtained multi-scale auxiliary loss can better capture high-frequency features while maintaining consistency in the low-frequency components.

[0102] In this embodiment, the Schrödinger bridge audio super-resolution model trained using the Schrödinger bridge model loss is fine-tuned by utilizing multi-scale auxiliary losses (short-time Fourier transform amplitude loss and anti-wrapping phase loss). This fine-tuned Schrödinger bridge audio super-resolution model can significantly improve the accuracy of spectral details and phase information in the super-resolution audio generation process, enabling the super-resolution audio to achieve better results in both waveform space and frequency domain space. It effectively solves technical problems in audio super-resolution tasks such as consonant swallowing, high-low frequency mismatch, and poor low-frequency preservation, further enhancing the generation performance of the trained Schrödinger bridge audio super-resolution model and generating high-resolution audio waveforms more efficiently.

[0103] To better perceive the differences between low-resolution and high-resolution waveform signals and improve the performance of the high-frequency detail data generated by the trained Schrödinger bridge audio super-resolution model, in the above... Figure 3 and / or Figure 5 Based on the illustrated embodiment, this disclosure further performs scaling processing after acquiring the super-resolution data pair, such as... Figure 6 As shown, the audio super-resolution model training process includes the following steps.

[0104] Step 601: Acquire multiple super-resolution data pairs, each super-resolution data pair including a low-resolution waveform signal and a high-resolution waveform signal representing the same audio.

[0105] Step 602: Using a scaling factor, amplify the low-resolution waveform signal and the high-resolution waveform signal in each super-resolution data pair to obtain amplified low-resolution waveform signal and high-resolution waveform signal.

[0106] The amplified low-resolution waveform signal and the high-resolution waveform signal are used to train the Schrödinger bridge model.

[0107] In this disclosure, the same scaling factor can be used to amplify low-resolution waveform signals and high-resolution waveform signals respectively.

[0108] In this disclosure, the scaling factor can be obtained based on low-resolution waveform signals and high-resolution waveform signals, as shown in equation (4):

[0109]

[0110] In equation (4), x LR For low-resolution waveform signals, x HR Here, s is a high-resolution waveform signal, and s is a scaling factor. By multiplying the low-resolution waveform signal and the high-resolution waveform signal by the scaling factor, the statistical variance between the low-resolution waveform signal and the high-resolution waveform signal can be amplified to 1, thereby increasing the optimization space and efficiency and significantly improving the Schrödinger bridge model's ability to capture high-frequency details.

[0111] Step 603: For each super-resolution data pair, use the Schrödinger bridge model to establish a solvable path from the amplified low-resolution waveform signal to the amplified high-resolution waveform signal.

[0112] Step 604: Based on the solvable path, train the Schrödinger bridge model to obtain the trained Schrödinger bridge audio super-resolution model.

[0113] The implementation methods of steps 603 to 604 above can be found in [reference needed]. Figure 3 The descriptions of steps 302 to 303 in the illustrated embodiments will not be repeated here.

[0114] In this embodiment of the disclosure, by amplifying and processing the low-resolution waveform signal and the high-resolution waveform signal in the super-resolution data pair respectively, the Schrödinger Bridge model can better perceive the difference between the low-resolution waveform signal and the high-resolution waveform signal, ensuring that the loss function can remain sensitive, which helps to improve the training efficiency and stability of the model.

[0115] After obtaining the trained Schrödinger Bridge audio super-resolution model (Bridge SR) through the above embodiments, experiments and evaluations were conducted on the trained Schrödinger Bridge audio super-resolution model using the Voice Cloning Toolkit (VCTK). The experimental results are shown in Table 1. Table 1 also shows the experimental results of other audio super-resolution models using related technologies, such as the Audio Super-Resolution (AudioSR) model, the Non-Local Video Super-Resolution (NVSR) model based on deep learning, the Modified Discrete Cosine Transform Generative Adversarial Network (mdctGAN) model, the Universal Neural Audio Upsampling Model (NU-Wave2), and the Unified Diffusion Model (UDM+), based on the VCTK dataset.

[0116] VCTK contains speech data from 110 English speakers using different accents. Each user reads approximately 400 sentences selected from newspapers, rainbow articles, and elicitation paragraphs designed to identify the speaker's accent.

[0117] Table 1 illustrates the audio super-resolution performance of the Schrödinger bridge audio super-resolution model of this disclosure compared with other related audio super-resolution models under different SRs (sampling rates of the input waveform to be processed) tested on the VCTK dataset. Table 1

[0118]

[0119]

[0120] In Table 1 above, the log-spectral distance (LSD), log-low frequency band (LSD-LF), and log-high frequency band (LSD-HF) are all measured. Lower values ​​indicate better audio super-resolution performance. SISNR represents the signal-to-noise ratio; higher SISNR indicates better audio super-resolution performance. Input represents the actual sampling rate of the low-resolution waveform, which is then upsampled to 48kHz using audio super-resolution. The table illustrates the performance of various audio super-resolution algorithms (models) performing super-resolution (SR) on inputs of 8kHz, 12kHz, 16kHz, and 24kHz, upsampling them to 48kHz waveforms.

[0121] As can be seen from the illustration in Table 1 above, the trained Schrödinger bridge audio super-resolution model in the technical solution of this disclosure has a high audio super-resolution effect in various performance indicators.

[0122] Exemplary device

[0123] Figure 7 This is a schematic diagram of the structure of an audio super-resolution processing apparatus provided in an exemplary embodiment of this disclosure. This apparatus can be applied to electronic devices such as terminal devices. Figure 7 As shown, the device includes:

[0124] Acquisition module 71 is used to acquire the low-resolution signal to be processed;

[0125] The adjustment module 72 is used to interpolate the low-resolution signal to be processed using a linear interpolation algorithm to obtain the prior information to be generated.

[0126] The generation module 73 is used to generate a high-resolution target waveform based on the prior information and using the trained Schrödinger bridge audio super-resolution model. The trained Schrödinger bridge audio super-resolution model keeps the low-frequency information in the prior information unchanged in each sampling step of generating the high-resolution target waveform.

[0127] The number of sampling points used to generate prior information is the same as the number of sampling points used to generate the high-resolution target waveform.

[0128] Figure 8 This is a schematic diagram of an audio super-resolution model training device provided in an exemplary embodiment of the present disclosure. The device can be applied to electronic devices such as terminal devices and servers. The device includes:

[0129] The sample acquisition module 81 is used to acquire multiple super-resolution data pairs, each super-resolution data pair including a low-resolution waveform signal and a high-resolution waveform signal representing the same audio.

[0130] The path generation module 82 is used to establish a solvable path from low-resolution waveform signal to high-resolution waveform signal for each super-resolution data pair using the Schrödinger bridge model.

[0131] The audio super-resolution model training module 83 is used to train the Schrödinger bridge model to be trained based on the solvable path, so as to obtain the trained Schrödinger bridge audio super-resolution model.

[0132] Figure 9 This is a schematic diagram of the structure of an audio super-resolution model training device provided in another exemplary embodiment of this disclosure, such as... Figure 9 As shown, in Figure 8 Based on the embodiments shown, in some embodiments, the Schrödinger bridge model uses stochastic differential equations to generate solvable paths; the stochastic differential equations are defined in the form of an asymmetric noise scheduling strategy.

[0133] In some embodiments, the audio super-resolution model training module 83 is used to guide the training of the Schrödinger bridge model based on a solvable path, using the Schrödinger bridge model loss and multi-scale auxiliary loss, to obtain the trained Schrödinger bridge audio super-resolution model.

[0134] Among them, the Schrödinger bridge model loss is used to characterize the error between the predicted audio waveform signal and the real high-frequency waveform signal of the Schrödinger bridge model at each sampling step, and the multi-scale auxiliary loss is used to characterize the multi-scale auxiliary loss of the Schrödinger bridge model at each sampling step.

[0135] In some embodiments, the multi-scale auxiliary loss includes short-time Fourier transform amplitude loss and anti-wrapping phase loss;

[0136] The audio super-resolution model training module 83 includes:

[0137] The signal transformation submodule 831 is used to convert the predicted audio waveform signal of each sampling step into multiple frequency domain signals corresponding to the multiple window functions using short-time Fourier transform based on multiple window functions, thereby obtaining multiple predicted frequency domain signals for each sampling step; and to convert the high-resolution waveform signal into multiple frequency domain signals corresponding to the multiple window functions using short-time Fourier transform based on multiple window functions, thereby obtaining frequency domain signals corresponding to multiple high-resolution waveform signals.

[0138] The amplitude loss submodule 832 is used to obtain the short-time Fourier transform amplitude loss based on the amplitude information of the predicted frequency domain signal at each sampling step and the amplitude information of the frequency domain signal corresponding to the high-resolution waveform signal.

[0139] The phase loss submodule 833 is used to obtain anti-wrapping phase loss based on the phase information of the predicted frequency domain signal at each sampling step and the phase information of the frequency domain signal corresponding to the high-resolution waveform signal.

[0140] The auxiliary loss submodule 834 determines the weights of the short-time Fourier transform amplitude loss and the anti-wrapping phase loss to obtain the multi-scale auxiliary loss.

[0141] In some embodiments, the audio super-resolution model training module 83 is used to obtain the Schrödinger bridge model loss based on the error between the predicted audio waveform signal and the real high-frequency waveform signal at each sampling step.

[0142] In some embodiments, the sample acquisition module 81 is used to perform copy processing on the high-resolution waveform signal using a signal processing filter to obtain a low-resolution waveform signal corresponding to the high resolution.

[0143] In some embodiments, the sample acquisition module 81 is used to amplify the low-resolution waveform signal and the high-resolution waveform signal in each super-resolution data pair using a scaling factor to obtain amplified low-resolution waveform signal and high-resolution waveform signal, which are then used to train the Schrödinger bridge model.

[0144] It should be noted that the modules in this device can be disassembled and / or recombined, and these disassemblies and / or recombinations should be considered as equivalent solutions of this device.

[0145] The exemplary embodiments of this device correspond to the exemplary method section described above, and the relevant content can be referenced and cited interchangeably. The beneficial technical effects corresponding to the exemplary embodiments of this device can be found in the corresponding beneficial technical effects of the exemplary method section described above, and will not be repeated here.

[0146] Exemplary electronic devices

[0147] Figure 10 A structural diagram of an electronic device provided in an embodiment of this disclosure includes at least one processor 101 and a memory 102.

[0148] The processor 101 may be a central processing unit (CPU) or other form of processing unit with data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 10 to perform desired functions.

[0149] The memory 102 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 101 may execute one or more computer program instructions to implement the vehicle pose detection method and / or other desired functions of the various embodiments of this disclosure described above.

[0150] In one example, the electronic device may also include an input device 103 and an output device 104, which are interconnected via a bus system and / or other forms of connection mechanism (not shown).

[0151] The input device 103 may also include, for example, a keyboard, a mouse, a touch screen, a pickup device (such as a microphone array), etc.

[0152] The output device 104 can output various information to the outside, including, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc.

[0153] Of course, for the sake of simplicity, Figure 10 Only some of the components of the electronic device relevant to this disclosure are shown, omitting components such as buses, input / output interfaces, etc. In addition, the electronic device may include any other suitable components depending on the specific application.

[0154] Exemplary systems, computer program products, and computer-readable storage media

[0155] In addition to the methods and apparatus described above, embodiments of this disclosure may also be computer program products, including computer program instructions that, when executed by a processor, cause the processor to perform the steps in the audio super-resolution processing method and audio super-resolution model training method according to various embodiments of this disclosure as described in the "Exemplary Methods" section of this specification.

[0156] Computer program products can be written in any combination of one or more programming languages ​​to perform the operations of embodiments of this disclosure. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on a user's computing device, partially on a user's computing device, as a standalone software package, partially on a user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0157] Furthermore, embodiments of this disclosure may also be computer-readable storage media storing computer program instructions that, when executed by a processor, cause the processor to perform the steps in the audio super-resolution processing method and audio super-resolution model training method according to various embodiments of this disclosure as described in the "Exemplary Methods" section of this specification.

[0158] Computer-readable storage media may take the form of any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may, for example, include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0159] The basic principles of this disclosure have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this disclosure are merely examples and not limitations, and should not be considered as essential features of each embodiment of this disclosure. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the scope of this disclosure to the necessity of employing the aforementioned specific details for implementation.

[0160] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For system embodiments, since they largely correspond to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0161] The block diagrams of devices, apparatuses, and devices involved in this disclosure are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, and devices can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0162] The methods and apparatus of this disclosure may be implemented in many ways. For example, they may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above-described order of steps for the method is for illustrative purposes only, and the steps of the method of this disclosure are not limited to the order specifically described above, unless otherwise specifically stated. Furthermore, in some embodiments, this disclosure may also be implemented as a program recorded on a recording medium, the program including machine-readable instructions for implementing the method according to this disclosure. Thus, this disclosure also covers recording media storing programs for performing the method according to this disclosure.

[0163] It should also be noted that in the apparatus, devices, and methods of this disclosure, the components or steps can be disassembled and / or recombined. These disassemblies and / or recombinations should be considered as equivalent solutions to this disclosure.

[0164] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of this disclosure. Therefore, this disclosure is not intended to be limited to the aspects shown herein, but rather to be carried out within the widest scope consistent with the principles and novel features disclosed herein.

[0165] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this disclosure to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. A method for training an audio super-resolution model, characterized in that, include: Acquire multiple super-resolution data pairs, each of which includes a low-resolution waveform signal and a high-resolution waveform signal representing the same audio. For each super-resolution data pair, a solvable path from the low-resolution waveform signal to the high-resolution waveform signal is established using the Schrödinger bridge model; Based on the solvable path, the Schrödinger bridge model is trained to obtain the trained Schrödinger bridge audio super-resolution model.

2. The method according to claim 1, characterized in that, The Schrödinger bridge model uses stochastic differential equations to generate the solvable paths; the stochastic differential equations are defined in the form of an asymmetric noise scheduling strategy.

3. The method according to claim 1, characterized in that, The step of training the Schrödinger bridge model to be trained based on the solvable path includes: Based on the solvable path, the Schrödinger bridge model loss and multi-scale auxiliary loss are used to guide the training of the Schrödinger bridge model, resulting in the trained Schrödinger bridge audio super-resolution model. The Schrödinger bridge model loss is used to characterize the error between the predicted audio waveform signal and the real high-frequency waveform signal of the Schrödinger bridge model at each sampling step, and the multi-scale auxiliary loss is used to characterize the multi-scale auxiliary loss of the Schrödinger bridge model at each sampling step.

4. The method according to claim 3, characterized in that, The multi-scale auxiliary loss includes short-time Fourier transform amplitude loss and anti-wrapping phase loss; Calculating the multi-scale auxiliary loss includes: For the predicted audio waveform signal at each sampling step, the short-time Fourier transform is used to convert the predicted audio waveform signal at each sampling step into multiple frequency domain signals corresponding to the multiple window functions, thus obtaining multiple predicted frequency domain signals for each sampling step; and, Using short-time Fourier transform, the high-resolution waveform signal is converted into multiple frequency domain signals corresponding to multiple window functions based on multiple window functions, thus obtaining the frequency domain signals corresponding to multiple high-resolution waveform signals; Based on the amplitude information of the predicted frequency domain signal at each sampling step and the amplitude information of the frequency domain signal corresponding to the high-resolution waveform signal, the short-time Fourier transform amplitude loss is obtained. Based on the phase information of the predicted frequency domain signal at each sampling step and the phase information of the frequency domain signal corresponding to the high-resolution waveform signal, the anti-packaging phase loss is obtained. The weighted sum of the short-time Fourier transform amplitude loss and the anti-wrapping phase loss is determined to obtain the multi-scale auxiliary loss.

5. The method according to claim 3, characterized in that, The calculation of the loss of the Schrödinger bridge model includes: The Schrödinger bridge model loss is obtained based on the error between the predicted audio waveform signal and the real high-frequency waveform signal at each sampling step.

6. The method according to any one of claims 1-5, characterized in that, The acquisition of multiple super-resolution data pairs includes: The high-resolution waveform signal is copied using a signal processing filter to obtain the low-resolution waveform signal corresponding to the high resolution.

7. The method according to any one of claims 1-5, characterized in that, The acquisition of multiple super-resolution data pairs includes: Using a scaling factor, the low-resolution waveform signal and the high-resolution waveform signal in each super-resolution data pair are amplified to obtain amplified low-resolution waveform signal and high-resolution waveform signal. The amplified low-resolution waveform signal and the high-resolution waveform signal are used to train the Schrödinger bridge model.

8. An audio super-resolution processing method, characterized in that, include: Acquire the low-resolution signal to be processed; The low-resolution signal to be processed is interpolated using a linear interpolation algorithm to obtain the prior information. Based on the generated prior information, a high-resolution target waveform is generated using the trained Schrödinger bridge audio super-resolution model. The trained Schrödinger bridge audio super-resolution model maintains the low-frequency information in the generated prior information unchanged in each sampling step of generating the high-resolution target waveform.

9. The method according to claim 8, characterized in that, The number of sampling points used to generate the prior information is the same as the number of sampling points used to generate the high-resolution target waveform.

10. An audio super-resolution model training device, characterized in that, include: The sample acquisition module is used to acquire multiple super-resolution data pairs, each of which includes a low-resolution waveform signal and a high-resolution waveform signal representing the same audio. The path generation module is used to establish a solvable path from the low-resolution waveform signal to the high-resolution waveform signal for each super-resolution data pair using the Schrödinger bridge model; The audio super-resolution model training module is used to train the Schrödinger bridge model to be trained based on the solvable path, so as to obtain the trained Schrödinger bridge audio super-resolution model.

11. An audio super-resolution processing device, characterized in that, include: The acquisition module is used to acquire the low-resolution signal to be processed. The adjustment module is used to interpolate the low-resolution signal to be processed using a linear interpolation algorithm to obtain the generated prior information; The generation module is used to generate a high-resolution target waveform based on the prior information and using a trained Schrödinger bridge audio super-resolution model. The trained Schrödinger bridge audio super-resolution model maintains the low-frequency information in the prior information unchanged in each sampling step of generating the high-resolution target waveform.

12. A computer-readable storage medium storing a computer program for performing the method according to any one of claims 1-9.

13. An electronic device, the electronic device comprising: processor; Memory used to store the processor's executable instructions; The processor is configured to read the executable instructions from the memory and execute the instructions to implement the method described in any one of claims 1-9.

Images