16 results about "Audio restoration" patented technology

A low-power communication method and apparatus for a wireless audio device, an SoC chip, and a storage medium. The low-power communication method for a wireless audio device comprises: sampling acoustic features of heterogeneous audio data from multiple target audio devices, calculating speech spectra corresponding to device characteristics, extracting multi-device acoustic features, and performing multi-frequency sub-band speech enhancement, heterogeneous segmentation and grouping of the multiple target audio devices, check value calculation, and differentiated encoding on each extracted acoustic feature sequence to obtain a multi-source encoded speech segment; dividing the multi-source encoded speech segment into transmission units, and performing cooperative signal modulation, cooperative resource allocation, and data stream transmission on each transmission unit to obtain a transmitted speech data stream; and acquiring the transmitted speech data stream transmitted with low power, and performing audio recovery on the transmitted speech data stream to obtain a target audio data stream transmitted by the multiple target audio devices, thereby reducing the communication power consumption of the multiple target audio devices and ensuring the audio transmission quality.

The invention discloses a lightweight lossless audio encoding and decoding method and system, and belongs to the technical field of audio signal processing. The method comprises the following steps: converting an original audio waveform into a compact coding expression through an encoder formed by connecting a TConv unit, a convolution unit, a down-sampling unit and a Local-Transform unit in series; a single finite scalar quantizer is adopted to discretize the coded representation, and a mixed quantization strategy is introduced to generate a single and consistent discrete token stream; and finally, a high-fidelity audio waveform is reconstructed through a decoder formed by connecting a Local-Transform unit, a TEConv unit, an up-sampling unit and an audio recovery unit in series. Through unit structure optimization and joint loss function training, indexes such as SDR, PESQ, STOI, WER and the like under multiple different code rates are all superior to those of a baseline model, model parameters are only 10.31-11.29 M, collaboration of lightweight deployment and high-fidelity reconstruction is achieved, resource-constrained equipment is adapted to downstream scenes such as voice communication, ASR and the like, and good engineering practicability is achieved.

The embodiment of the invention provides an audio restoration method and device, and relates to the technical field of audio restoration. The method comprises the following steps: carrying out sonic boom detection on a to-be-repaired audio to obtain a sonic boom proportion of the to-be-repaired audio; if the detonation sound proportion is greater than a first threshold value, performing detonation sound restoration on the to-be-restored audio to obtain a first audio; performing voice detection on the first audio to obtain a voice ratio of the first audio; if the voice ratio is greater than a second threshold value, converting the first audio frequency into a first time-frequency domain signal, segmenting the first time-frequency domain signal into a first number of sub-band signals with non-overlapped frequency bands according to the resolution of the first audio frequency, and respectively acquiring spectrum features of the first number of sub-band signals, and performing voice separation on the first audio according to the spectrum features of the sub-band signals to obtain a second audio, and performing tone quality restoration on the second audio to obtain a restoration result of the to-be-restored audio. The embodiment of the invention is used for audio restoration.

A system and method for enhancing or restoring audio data utilizing an artificial intelligence module, and more particularly utilizing deep neural networks and generative adversarial networks. The system and method are both able to train the artificial intelligence module to provide for different format and other characteristic-specific transforms for determining how to restore audio to source quality and even beyond. The present invention includes the steps of acquiring source data, pre-processing the source data, implementing the artificial intelligence module, indexing the data, applying transforms, and optimizing the data for a particular audio modality.

The invention discloses an end-to-end speech separation algorithm based on a speech language model, and the algorithm comprises the steps: discretizing a continuous audio into a 32-order discrete codebook sequence through a residual vector quantization coder-decoder, and introducing a transcription start symbol lt through an SOT strategy; sOSgt, SOSgt; a special separator is lt; sCgt; and a termination symbol lt; eOSgt, EOSgt; splicing a multi-person voice sequence; extracting audio depth features by using a pre-trained WavLM model, and guiding an autoregression decoder to output a separated zero-order codebook sequence in combination with a cross attention mechanism; predicting a high-order codebook sequence step by step through a non-autoregression model, configuring an independent embedding layer to fuse low-order information, and introducing a task embedding mechanism to optimize modeling; based on a special separator lt; sCgt; and slicing the multi-order discrete codebook sequence, and outputting an independent voice source through an Encodec decoder. According to the method, the intelligibility of voice separation and the audio restoration quality can be effectively improved, the decoding speed is high, the subjective hearing experiment result and the downstream task performance are excellent, the scene that the number of speakers is unknown is supported, and the industrialization application prospect is wide.

A spatial audio restoration device of an embodiment includes a video feature amount calculation unit that calculates a video feature amount on the basis of video information, an audio feature amount calculation unit that calculates an audio feature amount on the basis of audio information that is a monaural sound corresponding to the video information, and a coefficient calculation unit that calculates a high-order Ambisonics coefficient on the basis of the video feature amount and the audio feature amount.

The application provides an audio coding method, an electronic device and a program product, and relates to the technical field of audio processing. The method comprises the following steps: obtaining audio data to be processed, and converting the audio data to be processed into a spectrum graph; inputting the spectrum graph into an encoder of a pre-trained coding and decoding neural network model, sequentially performing local acoustic feature extraction, intra-frame dependence modeling and inter-frame dependence modeling on the spectrum graph, and obtaining an encoding vector output by the encoder; inputting the encoding vector into a residual vector quantizer of the coding and decoding neural network model, performing multi-layer quantization processing on the encoding vector, and obtaining a codebook index output by the residual vector quantizer; and inputting the codebook index into a decoder of the coding and decoding neural network model, performing decoding processing on the codebook index, and obtaining a reconstructed spectrum graph output by the decoder. The application can realize high-quality audio restoration in a communication environment with an ultra-low code rate.

Embodiments of this application provide an audio restoration method and apparatus, and relates to the technical field of audio restoration. The method includes: performing pop detection on audio to be restored to obtain a pop proportion of the audio to be restored; performing pop restoration on the audio to be restored to obtain first audio in a case where the pop proportion is greater than a first threshold; performing speech detection on the first audio to obtain a speech proportion of the first audio; converting, in a case where the speech proportion is greater than a second threshold, the first audio into a first time-frequency domain signal, segmenting the first time-frequency domain signal into a first number of sub-band signals with non-overlapping frequency bands according to a resolution of the first audio, respectively obtaining spectrum features of the first number of sub-band signals.

The application relates to the technical field of audio processing, and particularly discloses an electronic tube audio pre-stage processing system, which comprises an audio source input module, an MCU intelligent distribution module, a DSP audio processing module, a feedback suppression module, an echo and reverberation processing module, an electronic tube audio restoration module and an audio output module; in the application, the electronic tube audio restoration module utilizes the characteristics of even harmonics to give the audio a warm and soft listening feeling, obviously improves the sound performance of a traditional digital audio processor which lacks humanization due to excessive precision, a multi-frequency band dynamic compression algorithm guarantees the detail performance of the audio, and effectively prevents distortion, and the application is particularly suitable for processing of high dynamic range audio sources, such as live music recording and high-fidelity sound playing; through real-time spectrum analysis of FFT combined with an adaptive gain control algorithm, the system can dynamically eliminate audio howling and feedback noise, and the performance is particularly outstanding in a complex sound environment, and the system is provided with reverberation denoising and equalization optimization.

The invention relates to an audio restoration model processing method, an audio restoration method and device, equipment, a storage medium and a program product, and relates to the technical field of audio and artificial intelligence. Processing the first audio sample of the first sampling frequency according to a second sampling frequency to obtain a second audio sample of the second sampling frequency, and performing distortion simulation processing on the second audio sample to obtain a third audio sample of the second sampling frequency, inputting the third audio sample into a to-be-trained frequency spectrum information restoration network in the audio restoration model to obtain first restoration frequency spectrum information, and processing the first restoration frequency spectrum information and the first frequency spectrum information of the second audio sample according to the first sampling frequency to obtain second restoration frequency spectrum information and second frequency spectrum information, and training the to-be-trained spectrum information repair network based on the second repair spectrum information and the second spectrum information. According to the invention, the audio restoration accuracy of the audio restoration model can be improved.

The application discloses a kind of binaural audio generation method and system of fusion position and audio general representation, it is characterized in that, including, S1, making video frame data set and audio data set;S2, short-time Fourier transform and calculation are carried out to audio data set, obtain corresponding complex spectrogram, amplitude spectrogram and phase spectrogram;S3, video frame data set, audio data set and its corresponding spectrogram are input into binaural audio restoration model containing relative position information extractor, audio general representation extractor, mask generation module and are trained and optimized;S4, based on the binaural audio restoration model of well-trained, carries out binaural audio restoration.The network model proposed in the present application can effectively extract the relative position information of sound source in video frame, obtain more effective audio general representation, for guiding the generation of binaural audio, to improve system performance.

This invention discloses a model training method, an audio processing method, and related apparatus. First, an audio processing model is constructed, including a pre-trained audio compression and reconstruction module and an audio generation module to be adjusted. Then, the audio compression and reconstruction module processes the original audio features of the fine-tuned sample audio to obtain predicted audio features and their pitch features. Next, the predicted audio features and their pitch features are input into the audio generation module, which outputs predicted audio. Finally, the audio generation module is adjusted based on the difference between the predicted audio and the fine-tuned sample audio to obtain the final audio processing model. Therefore, this invention, by introducing pitch features and predicted audio features for joint training, optimizes the module training logic and parameter adjustment criteria, significantly improving the sound quality and completeness of audio restoration and effectively optimizing the overall audio processing experience.

The invention relates to a model training and audio repairing method and device, electronic equipment and a storage medium, which are used for detecting and repairing an audio abnormal problem in real time. The model training method comprises the following steps: selecting a first sample which comprises a positive sample audio and a corresponding negative sample audio; extracting frequency spectrum image features and time domain features of the negative sample audio, and inputting the features into a compression network of a to-be-trained audio restoration model for dimension reduction compression to obtain first compression features; on the basis of a reconstruction network in an audio restoration model, dimension raising reconstruction is carried out on the first compression feature, and a reconstructed audio is obtained; based on the difference between the reconstructed audio and the positive sample audio, parameter adjustment is carried out on the audio restoration model; inputting the frequency domain feature of the sample audio in the second sample and the second compression feature of the sample audio extracted by the compression network into a to-be-trained audio detection model to obtain an anomaly detection result; and performing parameter adjustment on the audio detection model based on the difference between the abnormal detection result and the corresponding sample tag.

The invention discloses an audio restoration method and system based on multi-dimensional collaboration, and belongs to the technical field of audio restoration, and the method comprises the steps: carrying out the feature extraction and damage detection of an input to-be-restored audio, and obtaining a damage distribution map of the to-be-restored audio; performing noise removal, distortion correction and audio repair on the to-be-repaired audio according to the damage type based on the damage distribution map to obtain an audio repair result; and performing quality evaluation on the audio repair result through a preset repair evaluation rule, adjusting repair parameters of the audio repair result according to an evaluation result, and outputting a final repair result. Therefore, by implementing the method and the device, the problems that in the prior art, a large amount of training data is needed to realize audio restoration, and only single damage can be restored can be solved.

Provided in the embodiments of the present disclosure are an audio data processing method and apparatus, an electronic device and a storage medium. The method comprises: acquiring Mel spectrum data corresponding to initial audio data; using a band extension module to process the Mel spectrum data, so as to generate enhanced Mel spectrum data, the band extension module comprising a residual denoising diffusion model, and being used to predict a corresponding high-frequency feature on the basis of a low-frequency feature of the Mel spectrum data so as to generate the enhanced Mel spectrum data having the low-frequency feature and the high-frequency feature; and, on the basis of an audio restoration module, processing the enhanced Mel spectrum data to obtain optimized audio data. After the initial audio data is converted into the Mel spectrum data, the band extension module comprising the residual denoising diffusion model is used to extend the high-frequency feature of the Mel spectrum data, so as to form enhanced Mel spectrum data having the high-frequency feature and then, restoration is performed on the basis of the enhanced Mel spectrum data.

A system and method for enhancing or restoring audio data utilizing an artificial intelligence module, and more particularly utilizing deep neural networks and generative adversarial networks. The system and method are both able to train the artificial intelligence module to provide for different format and other characteristic-specific transforms for determining how to restore audio to source quality and even beyond. The present invention includes the steps of acquiring source data, pre-processing the source data, implementing the artificial intelligence module, indexing the data, applying transforms, and optimizing the data for a particular audio modality.