11 results about "Audio restoration" patented technology

The embodiment of the invention provides an audio restoration method and device, and relates to the technical field of audio restoration. The method comprises the following steps: carrying out sonic boom detection on a to-be-repaired audio to obtain a sonic boom proportion of the to-be-repaired audio; if the detonation sound proportion is greater than a first threshold value, performing detonation sound restoration on the to-be-restored audio to obtain a first audio; performing voice detection on the first audio to obtain a voice ratio of the first audio; if the voice ratio is greater than a second threshold value, converting the first audio frequency into a first time-frequency domain signal, segmenting the first time-frequency domain signal into a first number of sub-band signals with non-overlapped frequency bands according to the resolution of the first audio frequency, and respectively acquiring spectrum features of the first number of sub-band signals, and performing voice separation on the first audio according to the spectrum features of the sub-band signals to obtain a second audio, and performing tone quality restoration on the second audio to obtain a restoration result of the to-be-restored audio. The embodiment of the invention is used for audio restoration.

A system and method for enhancing or restoring audio data utilizing an artificial intelligence module, and more particularly utilizing deep neural networks and generative adversarial networks. The system and method are both able to train the artificial intelligence module to provide for different format and other characteristic-specific transforms for determining how to restore audio to source quality and even beyond. The present invention includes the steps of acquiring source data, pre-processing the source data, implementing the artificial intelligence module, indexing the data, applying transforms, and optimizing the data for a particular audio modality.

The invention discloses an end-to-end speech separation algorithm based on a speech language model, and the algorithm comprises the steps: discretizing a continuous audio into a 32-order discrete codebook sequence through a residual vector quantization coder-decoder, and introducing a transcription start symbol lt through an SOT strategy; sOSgt, SOSgt; a special separator is lt; sCgt; and a termination symbol lt; eOSgt, EOSgt; splicing a multi-person voice sequence; extracting audio depth features by using a pre-trained WavLM model, and guiding an autoregression decoder to output a separated zero-order codebook sequence in combination with a cross attention mechanism; predicting a high-order codebook sequence step by step through a non-autoregression model, configuring an independent embedding layer to fuse low-order information, and introducing a task embedding mechanism to optimize modeling; based on a special separator lt; sCgt; and slicing the multi-order discrete codebook sequence, and outputting an independent voice source through an Encodec decoder. According to the method, the intelligibility of voice separation and the audio restoration quality can be effectively improved, the decoding speed is high, the subjective hearing experiment result and the downstream task performance are excellent, the scene that the number of speakers is unknown is supported, and the industrialization application prospect is wide.

A spatial audio restoration device of an embodiment includes a video feature amount calculation unit that calculates a video feature amount on the basis of video information, an audio feature amount calculation unit that calculates an audio feature amount on the basis of audio information that is a monaural sound corresponding to the video information, and a coefficient calculation unit that calculates a high-order Ambisonics coefficient on the basis of the video feature amount and the audio feature amount.

Embodiments of this application provide an audio restoration method and apparatus, and relates to the technical field of audio restoration. The method includes: performing pop detection on audio to be restored to obtain a pop proportion of the audio to be restored; performing pop restoration on the audio to be restored to obtain first audio in a case where the pop proportion is greater than a first threshold; performing speech detection on the first audio to obtain a speech proportion of the first audio; converting, in a case where the speech proportion is greater than a second threshold, the first audio into a first time-frequency domain signal, segmenting the first time-frequency domain signal into a first number of sub-band signals with non-overlapping frequency bands according to a resolution of the first audio, respectively obtaining spectrum features of the first number of sub-band signals.

The invention relates to an audio restoration model processing method, an audio restoration method and device, equipment, a storage medium and a program product, and relates to the technical field of audio and artificial intelligence. Processing the first audio sample of the first sampling frequency according to a second sampling frequency to obtain a second audio sample of the second sampling frequency, and performing distortion simulation processing on the second audio sample to obtain a third audio sample of the second sampling frequency, inputting the third audio sample into a to-be-trained frequency spectrum information restoration network in the audio restoration model to obtain first restoration frequency spectrum information, and processing the first restoration frequency spectrum information and the first frequency spectrum information of the second audio sample according to the first sampling frequency to obtain second restoration frequency spectrum information and second frequency spectrum information, and training the to-be-trained spectrum information repair network based on the second repair spectrum information and the second spectrum information. According to the invention, the audio restoration accuracy of the audio restoration model can be improved.

The application discloses a kind of binaural audio generation method and system of fusion position and audio general representation, it is characterized in that, including, S1, making video frame data set and audio data set;S2, short-time Fourier transform and calculation are carried out to audio data set, obtain corresponding complex spectrogram, amplitude spectrogram and phase spectrogram;S3, video frame data set, audio data set and its corresponding spectrogram are input into binaural audio restoration model containing relative position information extractor, audio general representation extractor, mask generation module and are trained and optimized;S4, based on the binaural audio restoration model of well-trained, carries out binaural audio restoration.The network model proposed in the present application can effectively extract the relative position information of sound source in video frame, obtain more effective audio general representation, for guiding the generation of binaural audio, to improve system performance.

This invention discloses a model training method, an audio processing method, and related apparatus. First, an audio processing model is constructed, including a pre-trained audio compression and reconstruction module and an audio generation module to be adjusted. Then, the audio compression and reconstruction module processes the original audio features of the fine-tuned sample audio to obtain predicted audio features and their pitch features. Next, the predicted audio features and their pitch features are input into the audio generation module, which outputs predicted audio. Finally, the audio generation module is adjusted based on the difference between the predicted audio and the fine-tuned sample audio to obtain the final audio processing model. Therefore, this invention, by introducing pitch features and predicted audio features for joint training, optimizes the module training logic and parameter adjustment criteria, significantly improving the sound quality and completeness of audio restoration and effectively optimizing the overall audio processing experience.

The invention relates to a model training and audio repairing method and device, electronic equipment and a storage medium, which are used for detecting and repairing an audio abnormal problem in real time. The model training method comprises the following steps: selecting a first sample which comprises a positive sample audio and a corresponding negative sample audio; extracting frequency spectrum image features and time domain features of the negative sample audio, and inputting the features into a compression network of a to-be-trained audio restoration model for dimension reduction compression to obtain first compression features; on the basis of a reconstruction network in an audio restoration model, dimension raising reconstruction is carried out on the first compression feature, and a reconstructed audio is obtained; based on the difference between the reconstructed audio and the positive sample audio, parameter adjustment is carried out on the audio restoration model; inputting the frequency domain feature of the sample audio in the second sample and the second compression feature of the sample audio extracted by the compression network into a to-be-trained audio detection model to obtain an anomaly detection result; and performing parameter adjustment on the audio detection model based on the difference between the abnormal detection result and the corresponding sample tag.

The invention discloses an audio restoration method and system based on multi-dimensional collaboration, and belongs to the technical field of audio restoration, and the method comprises the steps: carrying out the feature extraction and damage detection of an input to-be-restored audio, and obtaining a damage distribution map of the to-be-restored audio; performing noise removal, distortion correction and audio repair on the to-be-repaired audio according to the damage type based on the damage distribution map to obtain an audio repair result; and performing quality evaluation on the audio repair result through a preset repair evaluation rule, adjusting repair parameters of the audio repair result according to an evaluation result, and outputting a final repair result. Therefore, by implementing the method and the device, the problems that in the prior art, a large amount of training data is needed to realize audio restoration, and only single damage can be restored can be solved.

Provided in the embodiments of the present disclosure are an audio data processing method and apparatus, an electronic device and a storage medium. The method comprises: acquiring Mel spectrum data corresponding to initial audio data; using a band extension module to process the Mel spectrum data, so as to generate enhanced Mel spectrum data, the band extension module comprising a residual denoising diffusion model, and being used to predict a corresponding high-frequency feature on the basis of a low-frequency feature of the Mel spectrum data so as to generate the enhanced Mel spectrum data having the low-frequency feature and the high-frequency feature; and, on the basis of an audio restoration module, processing the enhanced Mel spectrum data to obtain optimized audio data. After the initial audio data is converted into the Mel spectrum data, the band extension module comprising the residual denoising diffusion model is used to extend the high-frequency feature of the Mel spectrum data, so as to form enhanced Mel spectrum data having the high-frequency feature and then, restoration is performed on the basis of the enhanced Mel spectrum data.