661 results about "Speech enhancement" patented technology

The invention relates to a power grid monitoring bionic robot with multi-mode sensing and voice interaction functions. The acquisition unit acquires sound, images, temperature and power grid equipment surface micro-vibration information, completes data preprocessing and time and space calibration, and generates a sensing data packet. And the identification unit performs cross judgment based on the abrupt change characteristics and correlation of different types of data, identifies electrical, structural or environmental anomalies, and generates abnormal region description information. The construction unit analyzes whether a multi-factor coupling risk exists in combination with a power grid environment type and a historical rule, and generates task description information including processing suggestions and development path prediction. The analysis unit supports speech enhancement and tone extraction in a high-interference environment, and extracts a user interaction intention in combination with task description information. And the control unit jointly judges a behavior correction strategy according to the task description information and the interaction instruction information, generates a corresponding motion control instruction and voice feedback, and realizes autonomous monitoring and interaction response in a complex power grid environment.

A neural end-to-end system is provided for the face and voice preserving translation of videos. The system is a pipeline of multiple models that produces a video of the original speaker speaking in the target language with modified lip movement to match the target speech, while preserving emphases and prosody of the original speech, and voice characteristics of the original speaker. The pipeline starts with automatic speech recognition including emphasis detection, followed by the translation model. The translated text is then synthesized by a Text-to-Speech model that recreates the original emphases in the target sentence. The resulting synthetic speech is then converted back to the original speakers' voice using a voice conversion model. Finally, to synchronize the lips of the speaker with the translated audio, a generative model generates frames of adapted lip movements which are combined with the audio to produce the final output. The disclosure further describes several use-cases and configurations that apply these techniques to video conferencing, dubbing, low-bandwidth transmission, speech enhancement and assistive technology for the hearing impaired.

The invention relates to an intelligent microphone pickup and speech enhancement method, system and device, and the method comprises the steps: obtaining an audio signal collected by a dual-silicon microphone array, and carrying out the cross-correlation analysis of the audio signal, and obtaining a corresponding human voice signal correlation feature; performing sound source positioning analysis on the audio signal based on the human voice signal correlation feature to obtain a corresponding target human voice signal; performing frequency characteristic analysis on the target human voice signal, and performing segmented dynamic gain processing on the signal according to a preset frequency band range to obtain a corresponding voice enhancement signal; performing noise component adaptive filtering processing on the target human voice signal to obtain a corresponding noise suppression parameter; and inputting the speech enhancement signal and the noise suppression parameter into a preset echo cancellation model for joint optimization to obtain a corresponding output human voice signal. According to the invention, the coupling problem of multiple acoustic interferences can be effectively solved.

The invention relates to the technical field of voice processing, can be applied to business scenes of financial science and technology, medical health and the like, and discloses a voice enhancement method, device and equipment based on noise perception and a medium. Environment feature information is extracted and input into an audio enhancement model to generate an enhanced audio signal; obtaining a reference audio sample, extracting a personalized feature vector, and carrying out personalized processing on the enhanced audio signal; and collecting playing feedback data, determining a playing time domain adjustment parameter and a playing frequency domain adjustment parameter, adjusting the personalized enhanced audio signal, and generating an optimized audio signal. According to the method, dynamic adjustment is realized in combination with the feedback parameters in the playing process by fusing the environmental perception information and the personalized speaker characteristics, clear and natural optimized audio output with personalized styles can be generated in a complex environment, and the voice interaction quality and adaptability are improved.

The invention relates to the technical field of speech processing, can be applied to business scenes of medical health, financial science and technology and the like, and discloses a speech enhancement method based on multi-scale feature learning, which comprises the following steps: framing an input audio signal, extracting a Mel-frequency spectrum feature, extracting a frequency domain feature by using a multi-scale convolutional neural network, and carrying out multi-scale feature learning on the frequency domain feature; carrying out coding dimension reduction on the image; noise is suppressed through a deep residual network, enhanced audio features are generated, a non-autoregression generative model is adopted for feature conversion, and finally a generative adversarial network is used for reconstructing a target voice waveform. Voice frequency domain features are extracted through the multi-scale convolutional neural network, and the feature expression ability of different frequency bands is improved; noise suppression is carried out through a deep residual network, and the purity of the voice signals is enhanced; feature conversion is optimized through a non-autoregression generation model, and the modeling efficiency of speech enhancement is improved; the target voice waveform is reconstructed through the generative adversarial network, and the naturalness and definition of the generated voice are improved.

The invention relates to the technical field of voice processing, in particular to a mixed voice processing method, electronic equipment and a computer readable medium. The method comprises the following steps: collecting mixed voice and environment influence parameters; carrying out bionic frequency domain analysis on the mixed voice to obtain low-frequency attenuation compensation characteristic data; performing multipath effect propagation analysis on the low-frequency attenuation compensation characteristic data through the environmental influence parameters to generate channel distortion data; performing time domain-frequency domain joint deconvolution processing on the mixed voice by using the channel distortion data to generate a direct sound component and a reflected sound component; performing adversarial training based on the direct sound component and the reflected sound component to generate anti-multipath speech enhancement data; and constructing a dynamic frequency compensation filter based on preset environmental acoustic characteristics. Through the multi-stage signal processing, frequency compensation and real-time optimization technology, the output quality of the mixed voice is improved.

The invention belongs to the technical field of speech enhancement, and discloses a Schrodinger bridge-based diffusion model speech enhancement method and system, and the method comprises the steps: converting a diffusion process of a diffusion model into a solving process of a stochastic differential equation, and determining the stochastic differential equation according to the theoretical principle of a Schrodinger bridge, and the complex spectrum is directly used as the input of the diffusion model, so that the tedious work of extracting a phase spectrum and an amplitude spectrum from the complex spectrum and the memory overhead caused by inverse transformation are omitted, and meanwhile, the alignment problem between the phase spectrum and the amplitude spectrum is also avoided. According to the speech enhancement method, unique features of time sequence signals are captured through a Transformer module, multi-scale information is fused through a U-Net module, a loss function covering a time domain, a frequency domain and a time-frequency domain is matched, the difference between a prediction sample and a clean sample is gradually reduced, the nonlinear diffusion process from a noisy sample to the clean sample can be directly learned, and the speech enhancement accuracy is improved. Therefore, the structure information of more initial samples is reserved.

The invention discloses an unmanned aerial vehicle speech enhancement method and system based on harmonic features and deep learning, and the method comprises the following steps: collecting a rotor self-noise signal of an unmanned aerial vehicle, namely, an original sampling frequency, processing the original sampling frequency, obtaining a feature vector, processing the feature vector, and generating a noise dynamic change label. Combining the feature vector and the noise dynamic change label into a data set, constructing a deep learning model, inputting the data set into the deep learning model for processing, outputting a prediction result of the noise dynamic change label, extracting the collected data to obtain a noisy speech amplitude spectrum, smoothing the noisy speech amplitude spectrum, and obtaining a noise dynamic change label. And obtaining an enhanced signal, and outputting the enhanced signal. According to the method, the self-noise signal of the rotor wing of the unmanned aerial vehicle is extracted, and the noise change trend is predicted in combination with the deep learning model, so that the noise and the voice can be separated more accurately, and the voice signal-to-noise ratio is greatly improved.

The invention belongs to the technical field of speech enhancement, and particularly relates to a lightweight speech enhancement method based on a dual-path one-dimensional convolution group loop network. The method comprises the following steps: firstly, acquiring a noisy voice signal, and extracting a spectrum feature of the noisy voice signal; then, constructing a dual-path one-dimensional convolution group cycle network model which comprises an encoder module, a decoder module and a dual-path grouping cycle module located between the encoder module and the decoder module, and performing complex multiplication on the output characteristics of the decoder module and the input characteristics of the encoder module to obtain an estimated complex ratio mask; and finally, extracting the spectrum characteristics of a to-be-enhanced noisy voice signal, and inputting the spectrum characteristics of the to-be-enhanced noisy voice signal into the trained model to obtain an estimated complex ratio mask. Performing complex multiplication on the spectrum characteristics of the noisy voice signal to be enhanced and the estimated complex ratio mask to obtain the spectrum characteristics of the enhanced voice signal; and performing inverse short-time Fourier transform and overlap-add on the spectrum characteristics of the enhanced voice signal in sequence to obtain the enhanced voice signal. According to the method, the model parameter quantity and the calculation quantity are reduced, and the problems that an existing voice enhancement technology is insufficient in real-time processing performance, large in calculation resource demand and not suitable for being deployed on edge equipment are solved.

The embodiment of the invention discloses a speech enhancement method and device based on a generative adversarial network model, and electronic equipment, and can solve the problems of how to more accurately and effectively reduce the noise influence and improve the definition and quality of an audio signal. The method comprises the following steps: acquiring to-be-processed voice data; performing convolution processing on the to-be-processed voice data by generating a target generator in the adversarial network model to obtain initial voice data, the target generator comprising an expansion convolution layer and a standard convolution layer, and the target generator being obtained by co-training an adversarial loss function, a perception loss function and a reconstruction loss function; through a target discriminator in the generative adversarial network model, the initial voice data and the to-be-processed voice data are discriminated to obtain a target discrimination result, and the target discriminator comprises a time domain discriminator and a frequency domain discriminator; and when it is detected that the target discrimination result satisfies a preset requirement, determining that the initial voice data is voice data after voice enhancement of the to-be-processed voice data.

The invention relates to a hearing aid intelligent noise reduction and human voice enhancement system based on electroencephalogram signals, and belongs to the field of biomedical engineering and acoustic signal processing. The system comprises an electroencephalogram signal acquisition module, a multi-channel acoustic sensor array, an embedded neural signal processor, an adaptive beam forming module, a dynamic speech enhancement engine and a dual-mode output device, and constructs electroencephalogram-acoustics joint features by extracting an alpha / theta wave power ratio, a P300 component and auditory cortical Gamma phase synchronism. A deep network is driven to separate target voice, a wave beam direction and a frequency response curve are dynamically adjusted based on neural feedback, a closed-loop calibration unit is innovatively adopted, gain is reversely adjusted according to N1-P2 wave amplitude, heart rate variability and eye movement data are fused to optimize decisions, and when the signal-to-noise ratio is-5dB, the voice recognition rate reaches 89%, the auditory fatigue is reduced by 37%, and the decision conflict rate is smaller than 6%. The defects of attention blind area, noise separation failure and physiological adaptation of a traditional hearing aid are overcome. The system is suitable for the fields of hearing impairment rehabilitation, special communication and intelligent cabins.

A method and apparatus comprising computer code configured to cause a processor or processors to receive an audio signal obtained from a microphone, input the audio signal into a neural-network pipeline, the neural-network pipeline including a convolutional network that receives the audio signal and provides a first output of the convolutional network to an enhancer, the enhancer including a deep complex convolutional recurrent network that receives the first output along with a mel spectrogram of the audio signal and outputs a second output to at least one of a vocoder and a decoder, and control an output of an enhanced audio signal from the at least one of the vocoder and the decoder.

The invention discloses a speech enhancement method and system based on a selective state space model Mama, and the method comprises the steps: carrying out the short-time Fourier transform of a distorted speech, obtaining a time-frequency domain signal, splicing the amplitude, real part and imaginary part of the time-frequency domain signal into a signal of three channels, and enabling the signal to serve as an input signal; an input signal is expanded from three channels to a plurality of channels through a first convolution block of a generator network encoder, different resolution characteristics are extracted through an expanded dense convolution layer of the encoder, a receptive field is increased, and the frequency dimension of the signal is reduced to 1 / 2 of the original frequency dimension by using the last convolution block of the encoder so as to reduce the calculation complexity. The signals expanded by the encoder simulate forward and backward dependency relationships of voice signals under different resolutions through N TF-mamba blocks so as to capture local and global features of long-sequence voice signals, and feature enhancement is realized; and the amplitude and the phase of the enhanced signal are predicted through a complex decoder and a mask decoder to obtain enhanced speech after denoising and reverberation removal.

The invention discloses a multi-channel speech enhancement method based on multi-scale spatial information and spectrum feature fusion, which comprises the following steps: recombining different spectrum components according to spectrum features, and extracting in-channel, inter-channel and full-channel feature modes; the features are fused, and unified depth features are created; a local feature extraction module is introduced to enhance the feature weight of the current frame, and features under different scales are fused through a feature attention mechanism; a decomposition attention mechanism is introduced, and encoder and decoder outputs are fused in a plurality of decomposition spaces, so that detail features can also be used by a depth module. According to the method, spatial features and spectrum features are combined, and unified feature expression is created by using a feature fusion method. The attention module adaptively learns modes contained in the spatial features and utilizes the modes instead of fitting directional features with physical meanings, so that the method can adapt to different scenes more flexibly, and has a good application prospect.

The invention discloses a display equipment intelligent voice interaction method based on a large language model, and the method comprises the steps: constructing a set of display equipment intelligent voice interaction system suitable for a complex scene based on a large language model technical framework through integrating a multi-stage audio processing module, an intelligent voice recognition module, a semantic understanding and intention recognition module and a multi-mode interaction control module; the voice enhancement technology is adopted, high-precision voice instruction capture and noise reduction processing can be achieved in a noisy environment, and geographic space parameters and operation logic implied in user intentions are accurately analyzed in combination with a deep semantic analysis engine; through a multi-mode fusion control mechanism, real-time linkage interaction between a voice instruction and display equipment is supported, natural and smooth voice control experience is provided especially for scenes such as three-dimensional space positioning and topographic data calling in a holographic geographical environment, and the system has strong error recovery capability and good expansibility and is suitable for being popularized and applied. The method is especially suitable for a voice control interaction scene in a holographic environment.

The invention relates to the field of aviation voice signal processing, and discloses a voice recognition compensation pilot earphone hearing protection method and system, and the method comprises the following steps: collecting multi-modal data, and separating a sound source through tensor decomposition; inferring a pilot state by using a dynamic network; performing context recognition and semantic evaluation on the attention target voice; and finally, dynamically modulating the sound field based on deep reinforcement learning, enhancing the voice in a personalized manner, and outputting after noise suppression. The system comprises a multi-mode perception data acquisition module, a sound source decoupling module, a pilot state inference module, a voice processing and semantic evaluation module and a sound field modulation and output module. According to the invention, high-fidelity speech extraction is realized through multi-modal perception and tensor decomposition; evaluating priority key information in combination with attention and semantics; and deep reinforcement learning and model prediction control are adopted to dynamically optimize the sound field, so that the voice recognition accuracy and the pilot information acquisition efficiency are improved.

The invention relates to the technical field of voice recognition, in particular to an implementation method and device of a multi-channel voiceprint recognition system, and the implementation method comprises the steps of multi-channel data acquisition and synchronization, signal preprocessing and enhancement, feature extraction and fusion, model training, real-time deployment and adaptive optimization. Compared with the problems that a traditional multichannel voiceprint recognition system depends on a fixed beam forming algorithm and an independent clock synchronization module, the synchronization error is large, manual parameter adjustment is needed for noise suppression, and generalization is poor, hardware-level clock synchronization is achieved through a PTP protocol, and the accuracy of noise suppression is improved. The method combines an end-to-end neural network to automatically learn noise distribution and a sound source space position, dynamically generates a beam forming weight, can improve the voice quality in a complex noise scene without manual intervention, remarkably reduces the interference of a synchronization error on sound source positioning, and enables the precision and stability of far-field voice enhancement to reach a new level.

The invention discloses a speech enhancement method and system based on conditional stream matching and a vocoder, and the method comprises the following steps: S1, constructing a Mel spectrum extraction module which is used for converting an input noisy speech into a noisy Mel spectrum; s2, constructing a condition flow matching noise reduction module which is used for processing the noisy Mel spectrum obtained in the step S1 and outputting an enhanced Mel spectrum; and S3, constructing a neural network vocoder module which is used for restoring the enhanced Mel spectrum obtained in the step S2 into a time domain voice waveform so as to obtain an enhanced voice signal. The speech enhancement method combining conditional stream matching and the vocoder is proposed for the first time, the conditional stream matching method is innovatively introduced into the Mel-frequency spectrum domain, an end-to-end speech enhancement system is constructed, and a complete processing flow from noise speech input to high-quality speech signal output is realized.

Methods, systems, apparatuses for speech enhancement are described. A computing device may receive sound inputs and reduce non-speech portions of the sound inputs based on a machine learning model.

The invention relates to the technical field of artificial intelligence, can be applied to the fields of finance, medical treatment and the like, and provides a violation voice detection method, device, equipment and medium, and the method comprises the following steps: carrying out suspicious fragment detection on an input voice, and outputting a low-definition voice fragment; performing voice enhancement processing on the low-definition voice segment to obtain an optimized audio; converting the optimized audio into text data through voice recognition; and based on the text data, judging whether violation guide content exists or not and generating a violation analysis report. Compared with the prior art, real-time monitoring and accurate recognition of the violation guiding behaviors of the salesman are achieved, a whole-process closed loop covering detection, enhancement, conversion and judgment is formed, and it is ensured that the business process is compliant.

A method for training a speech enhancement network, performed by an electronic device, includes: acquiring a first clean speech sample and a noise sample, and mixing them to generate a noisy speech sample; performing noise reduction on the noisy speech sample based on the speech enhancement network to obtain an enhanced speech sample; framing the enhanced speech sample into a plurality of enhanced speech frames, classifying speech effectiveness of the enhanced speech frames, and generating a first effectiveness distribution based on classification results of the enhanced speech frames; and determining a noise reduction accuracy based on the enhanced speech sample and the first clean speech sample, determining a speech classification accuracy based on the first effectiveness distribution, determining a speech enhancement accuracy based on the noise reduction accuracy and the speech classification accuracy, and training the speech enhancement network based on the speech enhancement accuracy.

The invention provides a cascade mask network speech enhancement method fusing an auditory process and a related device, and relates to the technical field of speech signal processing. The method comprises the following steps: acquiring noisy voice, and extracting coarse-grained features from the noisy voice through a time domain coarse-grained mask network to generate a preliminarily enhanced voice waveform; and performing short-time Fourier transform on the preliminarily enhanced voice waveform, and generating a final enhanced voice by combining a frequency domain fine-grained mask network with embedded information of a target speaker. The effect of improving the hearing quality and definition of the expected voice in the noisy environment is achieved.

The invention discloses a speech enhancement method based on expansion causal convolution and a bidirectional gating recurrent neural network. The method comprises the following steps: framing and windowing noisy speech, and extracting spectrum features as original input features through Fourier transform; constructing a speech enhancement model structure which comprises a noise input layer, a noise multi-scale expansion causal convolution layer, a channel attention mechanism, a noise bidirectional gating recurrent neural layer and a denoising layer; training the speech enhancement model structure by using each original input feature to obtain a speech enhancement model; and obtaining corresponding de-noising spectrum features by using the speech enhancement model, and obtaining de-noised speech signals through inverse Fourier transform and windowing processing. According to the method, the neural network model is used for modeling noise features, low-complexity deep learning is realized through causal convolution and channel compression technologies, a noise part in a noisy voice signal is suppressed, the voice enhancement performance is improved, 8-bit quantitative deployment is adopted, the model volume is compressed to 2.3 MB, and the method has a good application prospect.

The invention provides a voice enhancement and high-precision recognition method and system in a complex environment, and relates to the technical field of voice processing, and the method comprises the steps: collecting a time domain signal in an off-road parking sentry box environment for preprocessing, detecting a mute segment signal in a standard time domain signal for noise power spectral density estimation, and obtaining a noise power spectral density value; a reverberation parameter is obtained by combining voice onset information and noise spatial correlation estimation, prediction is performed by using a deep neural network model, voice masking is applied to microphone array signals to perform enhancement processing, adaptive feature extraction is performed on time domain enhanced voice signals, and a voice signal is obtained. And performing high-precision recognition on the voice adaptive feature sequence based on an acoustic model and a language model, and outputting a target recognition text. The technical problems of poor voice signal quality and low recognition accuracy in a complex noise environment in the prior art are solved. The technical effects of improving the voice signal quality and the recognition accuracy and realizing clear, accurate and real-time voice interaction are achieved.

The invention relates to a classroom speech enhancement method based on convolution bidirectional Mama, and belongs to the technical field of speech signal processing of specific scenes. The method comprises the following steps: firstly, carrying out feature enhancement preprocessing of perceiving contrast stretching on a noiseless classroom audio, then inputting the noisy classroom audio into a speech enhancement model, and carrying out feature extraction on the noisy classroom audio by the model through an audio encoder and a convolution bidirectional Mama module; and then jointly decoding the voice features by using an amplitude mask decoder and a phase decoder, and finally improving the voice understandability in a classroom noise environment. According to the method, the voice signals in the complex noise classroom environment are denoised through the voice enhancement model, the requirement of a downstream voice task for further operation of classroom audio is met, and the method has great significance in development of the education field.

Embodiments of the present specification provide a speech processing method, a conference speech enhancement method, and a speech model training method. The speech processing method comprises: acquiring speech data to be processed; encoding the speech data to obtain a speech feature vector, the speech feature vector comprising feature sequences of a time domain channel and a frequency domain channel; performing time-frequency domain synchronization feature processing on the feature sequences of the time domain channel and the frequency domain channel to obtain a target speech feature vector; and, on the basis of the target speech feature vector, decoding to obtain a speech processing result. In the time domain channel and the frequency domain channel, feature processing is synchronously carried out, and the close correlation between the features on the time domain channel and the frequency domain channel is fully exploited, such that effective context understanding is achieved, a highly-accurate target speech feature vector is obtained for decoding, a highly-accurate speech processing result is obtained, and the effectiveness of speech processing is improved.

The invention relates to the technical field of sound reception, and discloses a system for multi-directional sound reception and noise reduction of a smart glasses microphone array, and the system comprises at least five microphones which are disposed at different positions of a smart glasses frame, and the first microphone is a unidirectional microphone, is located below the glasses frame, and is used for picking up the voice of a wearer; the second microphone and the third microphone are bi-directional microphones, are located right in front of the glasses frame and are used for picking up voice signals right in front of the wearer. The directivity weight of the microphone array is adjusted in real time through the dynamic optimization module, and the pickup effect of the target voice signal is enhanced preferentially. The voice enhancement module is matched to extract a target voice signal by using a Bayesian inference method, so that noise components in a mixed signal can be effectively separated, voice distortion is reduced, the definition and fidelity of the voice signal are remarkably improved, and the voice has better intelligibility in various complex noise environments.

The present application relates to the technical field of speech processing, and particularly relates to a mixed speech processing method, an electronic device and a computer readable medium. The method comprises the following steps: collecting mixed speech and environmental influence parameters; performing bionics frequency domain analysis on the mixed speech to obtain low-frequency attenuation compensation feature data; performing multipath effect propagation analysis on the low-frequency attenuation compensation feature data through the environmental influence parameters to generate channel distortion data; performing time domain-frequency domain joint deconvolution processing on the mixed speech by using the channel distortion data to generate direct sound components and reflected sound components; performing adversarial training based on the direct sound components and the reflected sound components to generate anti-multipath speech enhancement data; and constructing a dynamic frequency compensation filter based on preset environmental acoustic characteristics. The present application improves the output quality of mixed speech through multi-stage signal processing, frequency compensation and real-time optimization technology.

The invention discloses a voice separation method, electronic equipment, a storage medium and a computer program product, and relates to the technical field of signal processing, and the method comprises the steps: carrying out the beam forming of a microphone array signal through employing a preset near-field speaker direction as a voice enhancement direction, and obtaining a first near-field speaker signal; performing beam forming processing on the microphone array signal by taking a preset far-field speaker direction as a voice enhancement direction to obtain a first far-field speaker signal; inputting the first near-field speaker signal as a main signal and the first far-field speaker signal as a reference signal into a preset near-field separation model for processing to obtain a second near-field speaker signal; and inputting the first far-field speaker signal as a main signal and the first near-field speaker signal as a reference signal into a preset far-field separation model for processing to obtain a second far-field speaker signal. According to the invention, the far and near field signals are preliminarily separated, so that the computing power resource consumption of a voice separation algorithm is reduced.

The invention belongs to the technical field of artificial intelligence, and particularly relates to an audio processing method, an audio processing device, a computer readable medium, electronic equipment and a computer program product. The audio processing method comprises the following steps: performing feature extraction on an audio signal to obtain frequency domain features of a plurality of audio frames; the audio signal carries a voice signal and a noise signal; serialized modeling is carried out on the frequency domain features of the audio frames, intra-frame sequence features are obtained, and the intra-frame sequence features are used for representing the frequency sequence relation among a plurality of frequency points in the single audio frame; serialization modeling is carried out on the intra-frame sequence features of the multiple audio frames to obtain inter-frame sequence features, and the inter-frame sequence features are used for representing the time sequence relation among the multiple audio frames; and performing signal modulation on the audio signal according to the inter-frame sequence features to obtain a voice signal after noise signal removal. According to the embodiment of the invention, the speech enhancement noise reduction effect can be improved.