34 results about "Audio restoration" patented technology

The invention relates to the technical field of wireless communication, and discloses a low-power-consumption communication method and device for wireless audio equipment, an SoC chip and a storage medium. The method comprises the following steps: sampling acoustic characteristics of heterogeneous audio data of a plurality of target audio devices, calculating a voice spectrum corresponding to device characteristics, and extracting multi-device acoustic characteristics, performing multi-frequency sub-band speech enhancement, heterogeneous segmentation and grouping of multi-target audio equipment, verification value calculation and differential coding on each extracted acoustic feature sequence to obtain a multi-source coding speech segment; performing transmission unit division and cooperative signal modulation, cooperative resource allocation and data stream transmission of each transmission unit on the multi-source coding voice segment to obtain a transmission voice data stream; and obtaining a transmission voice data stream transmitted with low power consumption and performing audio recovery on the obtained data to obtain a target audio data stream transmitted by the multi-target audio equipment. The communication power consumption of the multiple target audio devices is reduced, and the audio transmission quality is ensured.

Provided in the present disclosure are an audio restoration method, an audio restoration apparatus, a computer program product, a computer-readable storage medium and an electronic device. The method comprises: inputting damaged audio into a first-stage model to obtain a complex spectrum; and inputting the complex spectrum into a second-stage model to obtain restored audio, wherein the first-stage model comprises an encoder used for performing down-sampling on the complex spectrum, and a decoder used for performing up-sampling on the complex spectrum, and the second-stage model comprises a full-band module used for modeling the complex spectrum on all bands, and a sub-band module used for modeling the complex spectrum on a plurality of sub-bands. By means of the present disclosure, complex and diverse audio distortion problems can be addressed.

The invention discloses an audio restoration method and device and electronic equipment, and belongs to the technical field of artificial intelligence. The method comprises the following steps: performing feature extraction on a to-be-repaired first audio clip, a previous-text audio clip and a next-text audio clip of the first audio clip to obtain a first acoustic feature vector of the first audio clip, a second acoustic feature vector of the previous-text audio clip and a third acoustic feature vector of the next-text audio clip; and inputting the first acoustic feature vector, the second acoustic feature vector and the third acoustic feature vector into an audio large model, performing autoregression prediction through the audio large model to obtain a predicted discrete coding vector, and decoding the predicted discrete coding vector to obtain a repaired second audio clip.

An audio restoration method and apparatus, and a storage medium and an electronic device. The method comprises: extracting frequency-domain features from audio to be restored; using a frequency-domain restoration generator to perform restoration processing on the frequency-domain features, so as to obtain restored frequency-domain features; and using a signal conversion generator to perform time-domain conversion processing on the restored frequency-domain features, so as to obtain restored audio.

A low-power communication method and apparatus for a wireless audio device, an SoC chip, and a storage medium. The low-power communication method for a wireless audio device comprises: sampling acoustic features of heterogeneous audio data from multiple target audio devices, calculating speech spectra corresponding to device characteristics, extracting multi-device acoustic features, and performing multi-frequency sub-band speech enhancement, heterogeneous segmentation and grouping of the multiple target audio devices, check value calculation, and differentiated encoding on each extracted acoustic feature sequence to obtain a multi-source encoded speech segment; dividing the multi-source encoded speech segment into transmission units, and performing cooperative signal modulation, cooperative resource allocation, and data stream transmission on each transmission unit to obtain a transmitted speech data stream; and acquiring the transmitted speech data stream transmitted with low power, and performing audio recovery on the transmitted speech data stream to obtain a target audio data stream transmitted by the multiple target audio devices, thereby reducing the communication power consumption of the multiple target audio devices and ensuring the audio transmission quality.

The invention discloses a lightweight lossless audio encoding and decoding method and system, and belongs to the technical field of audio signal processing. The method comprises the following steps: converting an original audio waveform into a compact coding expression through an encoder formed by connecting a TConv unit, a convolution unit, a down-sampling unit and a Local-Transform unit in series; a single finite scalar quantizer is adopted to discretize the coded representation, and a mixed quantization strategy is introduced to generate a single and consistent discrete token stream; and finally, a high-fidelity audio waveform is reconstructed through a decoder formed by connecting a Local-Transform unit, a TEConv unit, an up-sampling unit and an audio recovery unit in series. Through unit structure optimization and joint loss function training, indexes such as SDR, PESQ, STOI, WER and the like under multiple different code rates are all superior to those of a baseline model, model parameters are only 10.31-11.29 M, collaboration of lightweight deployment and high-fidelity reconstruction is achieved, resource-constrained equipment is adapted to downstream scenes such as voice communication, ASR and the like, and good engineering practicability is achieved.

The embodiment of the invention provides an audio restoration method and device, and relates to the technical field of audio restoration. The method comprises the following steps: carrying out sonic boom detection on a to-be-repaired audio to obtain a sonic boom proportion of the to-be-repaired audio; if the detonation sound proportion is greater than a first threshold value, performing detonation sound restoration on the to-be-restored audio to obtain a first audio; performing voice detection on the first audio to obtain a voice ratio of the first audio; if the voice ratio is greater than a second threshold value, converting the first audio frequency into a first time-frequency domain signal, segmenting the first time-frequency domain signal into a first number of sub-band signals with non-overlapped frequency bands according to the resolution of the first audio frequency, and respectively acquiring spectrum features of the first number of sub-band signals, and performing voice separation on the first audio according to the spectrum features of the sub-band signals to obtain a second audio, and performing tone quality restoration on the second audio to obtain a restoration result of the to-be-restored audio. The embodiment of the invention is used for audio restoration.

The embodiment of the invention provides an audio restoration method and device, equipment, a storage medium and a program product, and the method comprises the steps: obtaining lossless audio data and damaged audio data, carrying out the damage simulation processing of the lossless audio data, and obtaining simulated audio data, training a teacher network model based on the lossless audio data and the simulated audio data; inputting the damaged audio data into the trained teacher network model to obtain target audio data, and training the student network model based on the damaged audio data and the target audio data; and obtaining a to-be-processed audio signal, performing frequency domain transformation processing on the to-be-processed audio signal to obtain a frequency domain signal, inputting the frequency domain signal into the trained student network model to obtain a frequency domain repair signal, and performing time domain transformation processing on the frequency domain repair signal to obtain an audio repair signal. According to the scheme, the student network model deployed at the mobile terminal is obtained based on teacher network model training, and the real-time audio repairing requirement is met.

The invention relates to a target signal enhancement method based on a generative adversarial network, which belongs to the technical field of signal processing, and comprises the following steps: S1, carrying out adversarial training on a generator and a discriminator of the generative adversarial network by using a training set, and adding an attention mechanism and an audio restoration module in the training process, obtaining a trained target signal enhancement model; s2, using the trained target signal enhancement model to carry out identification, extraction, denoising and restoration processing on the complex signal sample to obtain a target signal; and S3, carrying out model pruning and retraining on the trained target signal enhancement model so as to optimize the target signal enhancement model. According to the method, the precision and effect of target signal processing in complex signals are improved, and more real-time and reliable signal data processing capability is provided for complex signal scene application.

A system and method for enhancing or restoring audio data utilizing an artificial intelligence module, and more particularly utilizing deep neural networks and generative adversarial networks. The system and method are both able to train the artificial intelligence module to provide for different format and other characteristic-specific transforms for determining how to restore audio to source quality and even beyond. The present invention includes the steps of acquiring source data, pre-processing the source data, implementing the artificial intelligence module, indexing the data, applying transforms, and optimizing the data for a particular audio modality.

The invention relates to an audio processing method, an audio processing device and a storage medium. The audio processing method comprises the following steps: acquiring an audio to be processed, and acquiring text description information of the audio to be extracted; first feature information is acquired, phase information of the to-be-processed audio is acquired, and the first feature information is audio feature information of the to-be-processed audio; according to the text description information and the first feature information, second feature information is obtained through a diffusion model, and the second feature information is audio feature information of the to-be-extracted audio; and performing audio recovery processing on the second feature information according to the phase information, and determining the audio obtained by the audio recovery processing as the target audio. According to the invention, sound source separation is carried out through the diffusion model based on the guidance of the text information, the stability of the separation effect is ensured, the obtained audio is from the target sound source, and the interference of other sound sources does not exist.

The invention discloses an end-to-end speech separation algorithm based on a speech language model, and the algorithm comprises the steps: discretizing a continuous audio into a 32-order discrete codebook sequence through a residual vector quantization coder-decoder, and introducing a transcription start symbol lt through an SOT strategy; sOSgt, SOSgt; a special separator is lt; sCgt; and a termination symbol lt; eOSgt, EOSgt; splicing a multi-person voice sequence; extracting audio depth features by using a pre-trained WavLM model, and guiding an autoregression decoder to output a separated zero-order codebook sequence in combination with a cross attention mechanism; predicting a high-order codebook sequence step by step through a non-autoregression model, configuring an independent embedding layer to fuse low-order information, and introducing a task embedding mechanism to optimize modeling; based on a special separator lt; sCgt; and slicing the multi-order discrete codebook sequence, and outputting an independent voice source through an Encodec decoder. According to the method, the intelligibility of voice separation and the audio restoration quality can be effectively improved, the decoding speed is high, the subjective hearing experiment result and the downstream task performance are excellent, the scene that the number of speakers is unknown is supported, and the industrialization application prospect is wide.

The invention provides an audio encoding and decoding method, electronic equipment and a program product, and relates to the technical field of audio processing, and the method comprises the steps: obtaining to-be-processed audio data, and converting the to-be-processed audio data into a spectrogram; inputting the spectrogram into an encoder of a pre-trained encoding and decoding neural network model, and sequentially performing local acoustic feature extraction, intra-frame dependency modeling and inter-frame dependency modeling on the spectrogram to obtain an encoding vector output by the encoder; inputting the coding vector into a residual vector quantizer of a coding and decoding neural network model, and performing multi-layer quantization processing on the coding vector to obtain a codebook index output by the residual vector quantizer; and inputting the codebook index into a decoder of the coding and decoding neural network model, and decoding the codebook index to obtain a reconstructed spectrogram output by the decoder. According to the invention, high-quality audio restoration can be realized in an ultra-low bit rate communication environment.

A spatial audio restoration device of an embodiment includes a video feature amount calculation unit that calculates a video feature amount on the basis of video information, an audio feature amount calculation unit that calculates an audio feature amount on the basis of audio information that is a monaural sound corresponding to the video information, and a coefficient calculation unit that calculates a high-order Ambisonics coefficient on the basis of the video feature amount and the audio feature amount.

The invention relates to an audio processing method, an audio processing device, electronic equipment and a storage medium. The audio processing method comprises the following steps: acquiring an audio to be processed, and acquiring audio features of the audio to be processed; the audio features are subjected to audio separation through a separator, target audio features are obtained, the separator comprises a plurality of separation layers, and each separation layer sequentially executes a processing mode based on expansion convolution, a processing mode based on multi-scale fusion and a processing mode based on a multi-channel attention mechanism; the target audio feature comprises an audio feature corresponding to each sound source in the plurality of sound sources; and performing audio reduction on the target audio feature to obtain a target audio, the target audio comprising an audio corresponding to each sound source in the plurality of sound sources. According to the sound source separation method and device, when sound source separation is executed, a good separation effect is guaranteed, meanwhile, the requirement for equipment performance is lowered, and a sound source separation algorithm can stably run in portable mobile equipment.

The application provides an audio coding method, an electronic device and a program product, and relates to the technical field of audio processing. The method comprises the following steps: obtaining audio data to be processed, and converting the audio data to be processed into a spectrum graph; inputting the spectrum graph into an encoder of a pre-trained coding and decoding neural network model, sequentially performing local acoustic feature extraction, intra-frame dependence modeling and inter-frame dependence modeling on the spectrum graph, and obtaining an encoding vector output by the encoder; inputting the encoding vector into a residual vector quantizer of the coding and decoding neural network model, performing multi-layer quantization processing on the encoding vector, and obtaining a codebook index output by the residual vector quantizer; and inputting the codebook index into a decoder of the coding and decoding neural network model, performing decoding processing on the codebook index, and obtaining a reconstructed spectrum graph output by the decoder. The application can realize high-quality audio restoration in a communication environment with an ultra-low code rate.

Embodiments of this application provide an audio restoration method and apparatus, and relates to the technical field of audio restoration. The method includes: performing pop detection on audio to be restored to obtain a pop proportion of the audio to be restored; performing pop restoration on the audio to be restored to obtain first audio in a case where the pop proportion is greater than a first threshold; performing speech detection on the first audio to obtain a speech proportion of the first audio; converting, in a case where the speech proportion is greater than a second threshold, the first audio into a first time-frequency domain signal, segmenting the first time-frequency domain signal into a first number of sub-band signals with non-overlapping frequency bands according to a resolution of the first audio, respectively obtaining spectrum features of the first number of sub-band signals.

The application relates to the technical field of audio processing, and particularly discloses an electronic tube audio pre-stage processing system, which comprises an audio source input module, an MCU intelligent distribution module, a DSP audio processing module, a feedback suppression module, an echo and reverberation processing module, an electronic tube audio restoration module and an audio output module; in the application, the electronic tube audio restoration module utilizes the characteristics of even harmonics to give the audio a warm and soft listening feeling, obviously improves the sound performance of a traditional digital audio processor which lacks humanization due to excessive precision, a multi-frequency band dynamic compression algorithm guarantees the detail performance of the audio, and effectively prevents distortion, and the application is particularly suitable for processing of high dynamic range audio sources, such as live music recording and high-fidelity sound playing; through real-time spectrum analysis of FFT combined with an adaptive gain control algorithm, the system can dynamically eliminate audio howling and feedback noise, and the performance is particularly outstanding in a complex sound environment, and the system is provided with reverberation denoising and equalization optimization.

The invention relates to an audio restoration model processing method, an audio restoration method and device, equipment, a storage medium and a program product, and relates to the technical field of audio and artificial intelligence. Processing the first audio sample of the first sampling frequency according to a second sampling frequency to obtain a second audio sample of the second sampling frequency, and performing distortion simulation processing on the second audio sample to obtain a third audio sample of the second sampling frequency, inputting the third audio sample into a to-be-trained frequency spectrum information restoration network in the audio restoration model to obtain first restoration frequency spectrum information, and processing the first restoration frequency spectrum information and the first frequency spectrum information of the second audio sample according to the first sampling frequency to obtain second restoration frequency spectrum information and second frequency spectrum information, and training the to-be-trained spectrum information repair network based on the second repair spectrum information and the second spectrum information. According to the invention, the audio restoration accuracy of the audio restoration model can be improved.

The present invention provides an audio recovery method based on non - linear interleaving mapping, including: S1. Obtain a video to be processed, preprocess the video to be processed to generate a continuous sequence of grayscale images, where the video to be processed includes vibration images generated by an object affected by surrounding sound sources; S2. Perform non - linear interleaving mapping of pixel coordinates and image brightness for each image in the grayscale image sequence with a two - dimensional trigonometric function interleaving auxiliary function, so as to obtain a set of two - dimensional 0 - 1 matrices corresponding to the current image, and perform dimensionality reduction processing on each two - dimensional 0 - 1 matrix to generate one - dimensional data; S3. After filtering and denoising each of the one - dimensional data, the recovered audio is obtained. The present invention has better scale scalability from a global perspective, can improve the overall detail resolution, and helps to improve the recovery quality of sound details.

The invention relates to a method for managing an audio stream played by a group of audio playback devices, said group being arranged in a given location. According to the invention, the method comprises the steps of: Detecting, on at least one image, the user(s) present in the image and deducing, for each of said users, at least one characteristic piece of information regarding the user's position in said image; Determining, at least from the various characteristic pieces of information, an optimal bearing angle (βopt); Providing mixing means distributing the audio stream among the different audio playback devices in the group, a characteristic value of the optimal bearing angle so that the mixing means distribute the audio stream at least according to said value. FIGURE IN THE ABRIDGED DIAGRAM: Fig. 2

The invention provides an earphone communication method, and the method comprises the steps: responding to an audio restoration request of a mobile terminal, building a second wireless communication channel between the mobile terminal and a charging box, and building a third wireless communication channel between the charging box and an earphone assembly, the earphone box receives audio data sent by the mobile terminal through second wireless communication, and transmits the audio data to the earphone assembly through a third wireless communication channel; overlapping and repairing the audio data received by the earphone assembly through the first wireless communication channel and the audio data received by the earphone assembly through the third wireless communication channel, and decoding and playing the repaired audio data through the earphone assembly; according to the invention, when the audio data transmitted between the earphone assembly and the mobile terminal is missing, the earphone box is utilized to indirectly transmit the audio data to the earphone assembly, and the audio data and the missing audio data are superposed and repaired, so that the integrity of the audio data is guaranteed under the condition that the audio data is missing.

The present invention relates to the field of wireless communication technologies, and discloses a low-power communication method, apparatus, SoC chip and storage medium for wireless audio devices. The method includes: sampling the acoustic features of heterogeneous audio data of multiple target audio devices, calculating the voice spectra corresponding to the device characteristics, and extracting the acoustic features of multiple devices, and performing voice enhancement in multiple frequency subbands, heterogeneous segmentation and grouping of multiple target audio devices, check value calculation and differential coding on the extracted acoustic feature sequences to obtain multi-source coded voice segments; dividing the multi-source coded voice segments into transmission units and performing cooperative signal modulation, cooperative resource allocation and data stream transmission for each transmission unit to obtain a transmitted voice data stream; obtaining the transmitted voice data stream for low-power transmission and performing audio restoration on the obtained data to obtain a target audio data stream transmitted by multiple target audio devices. This application reduces the communication power consumption of multiple target audio devices and ensures the audio transmission quality.

The invention provides a sound quality repairing method, electronic equipment, a storage medium and a program product, and relates to the technical field of audio. In the method, firstly, acoustic features of first audio data are extracted, and the first audio data can be audio data obtained by processing original audio data; performing local feature extraction and tone quality restoration processing on the acoustic features of the first audio data through a diffusion model to obtain first frequency spectrum data, and sending the first frequency spectrum data to a generation model; performing audio recovery processing on the first spectrum data through the generation model to obtain second audio data; and performing voice filtering processing on the second audio data to obtain repaired third audio data. Through the method provided by the invention, the quality of repairing the audio data is improved, so that the retention rate of voice data processing can be greatly improved.

The application discloses a kind of binaural audio generation method and system of fusion position and audio general representation, it is characterized in that, including, S1, making video frame data set and audio data set;S2, short-time Fourier transform and calculation are carried out to audio data set, obtain corresponding complex spectrogram, amplitude spectrogram and phase spectrogram;S3, video frame data set, audio data set and its corresponding spectrogram are input into binaural audio restoration model containing relative position information extractor, audio general representation extractor, mask generation module and are trained and optimized;S4, based on the binaural audio restoration model of well-trained, carries out binaural audio restoration.The network model proposed in the present application can effectively extract the relative position information of sound source in video frame, obtain more effective audio general representation, for guiding the generation of binaural audio, to improve system performance.

A system and method for enhancing or restoring audio data utilizing an artificial intelligence module, and more particularly utilizing deep neural networks and generative adversarial networks. The system and method are both able to train the artificial intelligence module to provide for different format and other characteristic-specific transforms for determining how to restore audio to source quality and even beyond. The present invention includes the steps of acquiring source data, pre-processing the source data, implementing the artificial intelligence module, indexing the data, applying transforms, and optimizing the data for a particular audio modality.

The present invention discloses a music audio repair method and system based on DCT-DDPM, belonging to the fields of speech processing and deep learning, and solving the problem that the prior art can only perform unconditional modification and cannot restore the original segment information. The present invention includes: 1) obtaining music audio data; 2) transforming the audio data into the frequency domain; 3) processing to obtain a Mel spectrogram with a Mask; 4) training DCT-DDPM; 5) repairing the audio based on the trained DCT-DDPM; 6) transforming the repaired Mel spectrogram into the time domain. The present invention is used for music audio repair.

This invention discloses a model training method, an audio processing method, and related apparatus. First, an audio processing model is constructed, including a pre-trained audio compression and reconstruction module and an audio generation module to be adjusted. Then, the audio compression and reconstruction module processes the original audio features of the fine-tuned sample audio to obtain predicted audio features and their pitch features. Next, the predicted audio features and their pitch features are input into the audio generation module, which outputs predicted audio. Finally, the audio generation module is adjusted based on the difference between the predicted audio and the fine-tuned sample audio to obtain the final audio processing model. Therefore, this invention, by introducing pitch features and predicted audio features for joint training, optimizes the module training logic and parameter adjustment criteria, significantly improving the sound quality and completeness of audio restoration and effectively optimizing the overall audio processing experience.

The invention relates to a model training and audio repairing method and device, electronic equipment and a storage medium, which are used for detecting and repairing an audio abnormal problem in real time. The model training method comprises the following steps: selecting a first sample which comprises a positive sample audio and a corresponding negative sample audio; extracting frequency spectrum image features and time domain features of the negative sample audio, and inputting the features into a compression network of a to-be-trained audio restoration model for dimension reduction compression to obtain first compression features; on the basis of a reconstruction network in an audio restoration model, dimension raising reconstruction is carried out on the first compression feature, and a reconstructed audio is obtained; based on the difference between the reconstructed audio and the positive sample audio, parameter adjustment is carried out on the audio restoration model; inputting the frequency domain feature of the sample audio in the second sample and the second compression feature of the sample audio extracted by the compression network into a to-be-trained audio detection model to obtain an anomaly detection result; and performing parameter adjustment on the audio detection model based on the difference between the abnormal detection result and the corresponding sample tag.

The invention discloses an audio restoration method and system based on multi-dimensional collaboration, and belongs to the technical field of audio restoration, and the method comprises the steps: carrying out the feature extraction and damage detection of an input to-be-restored audio, and obtaining a damage distribution map of the to-be-restored audio; performing noise removal, distortion correction and audio repair on the to-be-repaired audio according to the damage type based on the damage distribution map to obtain an audio repair result; and performing quality evaluation on the audio repair result through a preset repair evaluation rule, adjusting repair parameters of the audio repair result according to an evaluation result, and outputting a final repair result. Therefore, by implementing the method and the device, the problems that in the prior art, a large amount of training data is needed to realize audio restoration, and only single damage can be restored can be solved.