26 results about "Time frequency masking" patented technology

The invention discloses a feature attention and bilinear gating fused speech emotion recognition method and device, and the method comprises the following steps: 1, collecting an audio file, obtaining corresponding label information, generating audio waveform and time frequency representation data through preprocessing, and constructing an audio waveform mask and a time frequency mask to mark an effective information region; 2, constructing a dual-path feature extraction module which comprises a time-frequency feature extraction module and a pre-training acoustic feature coding module; wherein the time-frequency feature extraction module models emotion correlation through local convolution and a multi-dimensional attention mechanism, and performs global time sequence modeling based on a bidirectional gated loop network; the pre-training acoustic feature coding module extracts high-level speech representation with high expression ability for emotion distinguishing by using a pre-training model; and step 3, constructing a feature fusion module and an emotion classification module, and combining with a dual-path feature extraction module to form a speech emotion recognition model.

The invention provides a method for eliminating echo of a talkback system, which comprises the following steps of: according to separated near-end speech evaluation and a plurality of recognized speaker speech segments, respectively performing residual echo suppression, constructing binary time-frequency masking and reducing residual echo spectrum caused by dynamic change of echo power; the phase information of the original mixed signal is combined with the enhanced near-end voice power spectrum, the voice waveform of each speaker is reconstructed through inverse short-time Fourier transform, and transition between voice frames is smoothed in an overlap-add mode; whether voice exists at the near end and the far end or not is judged, if the double-talk state is detected, parameters of a nonlinear echo cancellation algorithm are adjusted, and the situation that the voice at the near end is mistakenly suppressed is avoided; and carrying out transmission path modeling on the processed near-end voice signal, and evaluating propagation characteristics of the voice signal from a speaker to a microphone, including time delay and attenuation of direct sound and early reflected sound.

The present disclosure provides an audio noise reduction method, apparatus, device, and storage medium. The method includes: First, obtaining audio data to be noise-reduced, and then using a preset real number network model to estimate the magnitude time-frequency mask of the audio data to be noise-reduced, so as to obtain a first-order enhanced magnitude spectrum corresponding to the audio data to be noise-reduced. Furthermore, using a preset complex number network model to estimate the complex time-frequency mask of the audio data to be noise-reduced, and combining the first-order enhanced magnitude spectrum and the complex time-frequency mask to determine the noise-reduced result audio data corresponding to the audio data to be noise-reduced. In the embodiments of the present disclosure, the magnitude spectrum of the audio data to be noise-reduced is enhanced by using a preset real number network model, and the magnitude spectrum and phase spectrum of the audio data to be noise-reduced are simultaneously enhanced by using a preset complex number network model. It can be seen that the embodiments of the present disclosure can implement noise reduction processing on the audio data to be noise-reduced, thereby better improving the audio quality.

The present application relates to a single-channel human-computer interaction speech separation method, device, computer equipment and storage medium. The method comprises: training a first time-frequency mask for extracting a machine speech spectrum and a second time-frequency mask for extracting a real person speech spectrum through a time-series recurrent neural network; performing speech separation processing on the target human-computer interaction speech to be separated according to the first time-frequency mask and the second time-frequency mask trained by the time-series recurrent neural network, and obtaining the machine speech and real person speech separated from the target human-computer interaction speech. The embodiment of the present application can significantly reduce the training data required for deep learning and shorten the training time without losing the separation accuracy of the monophonic human-computer interaction speech, while also improving the speech separation speed.

The application discloses a bearing composite fault diagnosis method and system, first, a bearing composite fault vibration observation signal is acquired; then, the observation signal is subjected to short-time Fourier transform and cepstrum threshold processing, two-dimensional time-frequency mask blind source separation, inverse short-time Fourier transform is performed on the obtained time-frequency domain separation signal and band-pass filtering is performed, independent time domain estimation signals are obtained; then, the estimation signals are subjected to Hilbert envelope demodulation and Fourier transform, envelope spectra of the filtered signals are obtained; finally, each order characteristic frequency band of a target fault is searched, a harmonic energy impact index is constructed, and an impact pulse value is calculated; the calculated impact pulse value is compared with a threshold value, and whether the bearing has a target type fault is diagnosed.

An audio system is provided that includes a first loudspeaker and a second loudspeaker. The first loudspeaker transmits an audio signal including audio data and a signature tone into a listening environment. The second loudspeaker includes at least one controller programmed to receive the audio signal including the audio data and the signature tone and to determine a direction of arrival (DOA) of the audio signal as received at the second loudspeaker based at least on the signature tone. The at least one controller is further programmed to perform a time frequency masking operation on the received audio signal prior to determining the DOA of the audio signal.

This application provides a device control method, a recognition model training method, and related devices. The device control method includes: acquiring speech to be processed emitted by a target user; inputting the speech to be processed into a recognition model to obtain recognition information output by the recognition model, wherein the recognition information includes control speech and a target speech channel, or the recognition information includes control speech, a target speech channel, and a time-frequency masking code, wherein the target speech channel is a speech transmission channel with a signal-to-noise ratio greater than a signal-to-noise ratio threshold, and the time-frequency masking code is used for sound source localization of the target user; and waking up or controlling the device's functions based on the recognition information. This application improves the accuracy of recognizing control information in speech.

The invention provides a voice separation method and device, electronic equipment and a storage medium, and the method comprises the steps: carrying out the first-stage sound region division of multiple groups of voice signals, obtaining the first-stage masks of a plurality of first-stage sound regions, enabling the multiple groups of voice signals to be obtained through the synchronous pickup of a plurality of microphone arrays, and enabling the first-stage sound regions to be in one-to-one correspondence with the microphone arrays; performing secondary sound area division on each group of voice signals to obtain secondary masks of a plurality of secondary sound areas under each primary sound area; based on the primary masks of the plurality of primary sound regions, the secondary masks of the plurality of secondary sound regions and the mapping relationship between the secondary sound regions and the recombination sound region, determining a time frequency mask of the recombination sound region; the recombination sound area is a dynamic sound area adapted to personnel distribution; and performing voice separation on the multiple groups of voice signals based on the time-frequency mask of the recombined voice area. According to the method, the device, the electronic equipment and the storage medium provided by the invention, dynamic self-adaptive adjustment of the sound area in the space is realized, so that the accuracy of voice interaction and the user experience are ensured.

The present application provides a method, device, terminal device, medium and product for adjusting the vocals of a song. According to an example of the present application, the method is applied to a terminal device, and the method may include: converting a first time domain signal in a song into a first time-frequency domain signal; extracting features from the first time-frequency domain signal to obtain time-frequency domain audio features; inputting the time-frequency domain audio features into a vocal extraction model to obtain a time-frequency mask output by the vocal extraction model, the time-frequency mask being used to represent the vocal signal in the first time-frequency domain signal; adjusting the first time-frequency domain signal based on the time-frequency mask to obtain a second time-frequency domain signal; converting the second time-frequency domain signal into a second time domain signal to obtain an adjusted song, and the vocal signal in the song can be separated and adjusted to achieve real-time adjustment of the vocal volume in the song on the terminal device.

The invention discloses a voice noise reduction method and device in combination with non-negative matrix factorization and time-frequency masking and a storage medium, and belongs to the technical field of Bluetooth audios, and the method comprises the steps: inputting monaural voice PCM data, and executing low-delay improved discrete cosine transform to obtain an MDCT spectral coefficient; calculating sub-band energy according to the MDCT spectrum coefficient, and constructing a noisy voice amplitude spectrum observation matrix; performing non-negative matrix factorization on the noisy voice amplitude spectrum observation matrix, and calculating an NMF amplitude spectrum gain; performing feature extraction according to the sub-band energy, and calculating an NN amplitude spectrum gain through a pre-trained deep neural network; fusing the NMF amplitude spectrum gain and the NN amplitude spectrum gain, and outputting a noise reduction gain; obtaining a noise reduction spectral coefficient according to the noise reduction gain and the MDCT spectral coefficient; and according to the noise reduction spectrum coefficient, continuing to execute the coding process, and outputting a noise reduction voice code stream. According to the invention, based on the MDCT spectral coefficient, voice noise reduction is realized by combining non-negative matrix factorization and time-frequency masking, and the tone quality is ensured.

The invention discloses a sound source separation method, device and equipment, a storage medium and a vehicle. The method comprises the following steps: acquiring a time-frequency graph corresponding to a mixed audio; frequency band division is carried out on the time-frequency diagram through a Mel filter bank, K sub-band time-frequency diagrams are obtained, an overlapping part exists between every two adjacent sub-band time-frequency diagrams, and K is a positive integer larger than or equal to 1; performing feature extraction on the K sub-band time-frequency graphs to obtain a target feature tensor; performing mask estimation on the target feature tensor by using a target sound source separation model to obtain a target time frequency mask corresponding to a target audio, the target sound source separation model being used for separating the target audio from the mixed audio; and determining the audio corresponding to the target time frequency mask in the mixed audio as the target audio. According to the sound source separation method provided by the embodiment of the invention, the accuracy of sound source separation can be ensured, and the sound quality of the target audio obtained by sound source separation is improved.

In at least one embodiment, an audio system including a first loudspeaker and a second loudspeaker and at least one controller is provided. The first loudspeaker transmits a first audio signal including a first signature tone into a listening environment. The second loudspeaker transmits a second audio signal including a second signature tone into the listening environment and receive the first audio signal including and the first signature tone. The second loudspeaker receives the second audio signal including the second signature tone after transmitting the second signature tone into the listening environment and determines an estimated distance between the first loudspeaker and the second loudspeaker based at least on the first signature tone and the second signature tone. The second loudspeaker performs a time frequency masking operation to extract a least one of the first signature tone and the second signature tone from the noisy and reverberant mixture.

The invention discloses a bearing composite fault diagnosis method and system. Firstly, a bearing composite fault vibration observation signal is obtained; then short-time Fourier transform and cepstrum threshold processing are carried out on the observation signals, two-dimensional time-frequency masking blind source separation is carried out, inverse short-time Fourier transform and band-pass filtering are carried out on obtained time-frequency domain separation signals, and independent time domain estimation signals are obtained; performing Hilbert envelope demodulation and Fourier transform on the estimated signal to obtain an envelope spectrum of the filtered signal; finally, searching to obtain each order of characteristic frequency band of the target fault, constructing a harmonic energy impact index # imgabs0 #, and calculating an impact pulse value; and comparing the calculated impact pulse value with a threshold value, and diagnosing whether the bearing has a target type of fault or not.

A method and apparatus for adjusting human voice in a song, and a terminal device, a medium and a product. The method comprises: converting a first time-domain signal in a song into a first time-frequency-domain signal (102); performing feature extraction on the first time-frequency-domain signal, so as to obtain a time-frequency-domain audio feature (104); inputting the time-frequency-domain audio feature into a human voice extraction model, so as to obtain a time-frequency mask outputted by the human voice extraction model, wherein the time-frequency mask is used for representing the strength of a human voice signal at corresponding time and frequency points in the first time-frequency-domain signal (106); on the basis of the time-frequency mask, adjusting the first time-frequency-domain signal, so as to obtain a second time-frequency-domain signal (108); and converting the second time-frequency-domain signal into a second time-domain signal, so as to obtain an adjusted song (110).

The invention discloses a transformer discharge sound signal denoising method, a transformer discharge sound signal denoising device and electronic equipment. The method comprises the following steps: acquiring a sound signal corresponding to a transformer; signal time-frequency parameters corresponding to the sound signals are determined, and the signal time-frequency parameters represent parameters of signal frequency parameters changing along with time; dividing the signal time-frequency parameter to obtain a plurality of sub time-frequency parameters; determining time frequency mask identifiers respectively corresponding to the plurality of sub time frequency parameters; and denoising the sound signal according to the signal time-frequency parameter and the time-frequency mask identifiers corresponding to the plurality of sub-time-frequency parameters to obtain a denoised signal. According to the invention, the technical problem of inaccurate partial discharge detection of the transformer caused by poor denoising effect of the discharge sound signal of the transformer in the prior art is solved.

The invention discloses an active noise reduction method and device, equipment, a medium, a program product and a vehicle. The method comprises the following steps: acquiring a reference signal transmitted from a noise source to a first position; performing short-time Fourier transform on the reference signal to obtain a reference signal matrix; inputting the reference signal matrix into a time-frequency masking model to obtain a time-frequency masking value output by the time-frequency masking model; processing the reference signal matrix based on the time-frequency masking value to obtain a noise reduction signal; and superposing the noise reduction signal and an original noise signal transmitted to the second position by the noise source to realize noise reduction at the second position. According to the embodiment of the invention, active noise reduction can be carried out on equipment such as a vehicle, the inhibition capability of an ANC system on complex noise is improved, noise reduction is carried out by adopting a mode of generating the time-frequency masking value by the time-frequency masking model, and compared with a mode of carrying out noise reduction by directly outputting a signal for driving a secondary loudspeaker through the model, the complexity of the model is reduced, and the noise reduction efficiency is improved. And a possible solution is provided for hardware landing.

The invention provides an earphone communication connection method and device. The method comprises the following steps: receiving noisy voice data; performing time-frequency decomposition and voice feature extraction on the data to obtain an amplitude spectrum of the voice data with noise, and identifying noise features of the amplitude spectrum; obtaining a similarity coefficient corresponding to the noisy voice data based on the component proportion of expected voice data and noise data in the noisy voice data; adjusting the time-frequency mask based on the similarity coefficient to obtain a target mask; and obtaining enhanced voice data based on the target masking. A similarity coefficient is introduced, the energy proportion of noisy voice and noise and the energy proportion of the noisy voice and target voice are adjusted from the angle of human ear perception, and the masking value of each time frequency unit is adjusted, so that the trained target masking is closer to the masking value of pure voice under the low signal-to-noise ratio, and the training accuracy is improved. Interference noise is removed, meanwhile, the problem of voice distortion caused by excessive noise removal in ideal ratio masking is avoided, and more target voice components are effectively reserved.

The present application provides a local sound reinforcement method, device, terminal equipment, medium and product. According to an example of the present application, the method may include: extracting features from a received first time-frequency domain signal to obtain a first time-frequency domain feature; extracting features from a second time-frequency domain signal to be played to obtain a second time-frequency domain feature; inputting the first time-frequency domain feature and the second time-frequency domain feature into a human voice extraction model to obtain a time-frequency mask output by the human voice extraction model, the time-frequency mask being used to represent the human voice time-frequency domain signal in the first time-frequency domain signal; based on the time-frequency mask, suppressing the acoustic factors in the first time-frequency domain signal to obtain a human voice time-frequency domain signal; converting the human voice time-frequency domain signal into a human voice time-domain signal, and locally amplifying the human voice time-domain signal, which can simultaneously suppress howling, environmental noise, room reverberation and echo residue in the first time-frequency domain signal, thereby improving the clarity and audibility of local sound reinforcement.

The invention discloses an opportunity signal self-positioning method based on a time-frequency multi-scale mask pre-training large model, belongs to the technical field of opportunity signal self-positioning, and solves the problem of insufficient positioning precision caused by limited label data of an existing positioning task. The method comprises the following steps: acquiring time-frequency representation of wavelet packet transform domain sequences of a plurality of signal segments corresponding to each opportunity signal sample; constructing a pre-training set and a labeled data set; training a pre-training neural network based on the random-time frequency mask by using the pre-training set; cutting the pre-trained neural network, adding an average pooling layer and a multi-layer perceptron, and constructing a positioning model; training and verifying the positioning model by using a labeled data set to obtain a successfully verified positioning model; opportunity signals of a point to be positioned are collected in real time, time-frequency representation of the wavelet packet transform domain sequences of the corresponding multiple signal segments is obtained, and coordinates of the point to be positioned are obtained through prediction after processing of the verified positioning model.

A local sound amplification method and apparatus, a terminal device, a medium, and a product. The method may comprise: performing feature extraction on a first time-frequency domain signal to be played back, and obtaining a first time-frequency domain feature; performing feature extraction on a second time-frequency domain signal to be played back after the first time-frequency domain signal, and obtaining a second time-frequency domain feature; inputting the first time-frequency domain feature and the second time-frequency domain feature into a human voice extraction model, and obtaining a time-frequency mask outputted by the human voice extraction model, the time-frequency mask being set to represent the intensity of a human voice time-frequency domain signal at corresponding time and frequency points in the first time-frequency domain signal; on the basis of the time-frequency mask, suppressing acoustic factors in the first time-frequency domain signal, and obtaining a human voice time-frequency domain signal corresponding to the first time-frequency domain signal; and converting the human voice time-frequency domain signal corresponding to the first time-frequency domain signal into a human voice time-domain signal, and performing local sound amplification on the human voice time-domain signal.

The embodiment of the invention discloses a voice processing method and device, electronic equipment and a storage medium, and the method comprises the steps: sampling a reference position in a target voice region, determining a target phase difference when a voice signal from the reference position is collected, carrying out the signal alignment of the collected to-be-processed voice signal based on the target phase difference, obtaining a sensor signal pair, and carrying out the recognition of the to-be-processed voice signal. Determining an observation signal difference according to the difference of the sensor signal pair in the time-frequency domain, determining an observation phase difference of the to-be-processed voice signal on the acoustic sensor pair, and performing time-frequency masking prediction based on the to-be-processed voice signal, the observation signal difference and the observation phase difference to obtain sound area masking information, and the target voice signal of the target voice region is extracted from the to-be-processed voice signal based on the voice region masking information, so that the target voice signal is extracted by introducing the observation signal difference on the basis of the observation phase difference, the amount of information carried by the spatial features is enriched, and the accuracy of voice processing is improved. The accuracy can be effectively improved when the target voice signal corresponding to the target voice area is extracted.

The invention discloses a voice signal denoising method and device, electronic equipment and a program product, and relates to the technical field of artificial intelligence, and the denoising method comprises the steps: collecting voice data of a target user based on a preset collection strategy, and carrying out the denoising of the voice data through employing a preset back diffusion model, and obtaining initial voice data; inputting the initial voice data into a preset dual-path recursive network, and outputting a voice feature sequence; generating a time frequency mask for the speech feature sequence, and determining a target speech spectrum based on the time frequency mask; and based on the target voice spectrum, performing voice waveform reconstruction by adopting inverse short-time Fourier transform to obtain a target voice signal. According to the invention, the technical problem of low speech recognition accuracy under multi-noise and multi-speaker interference in the prior art is solved.

The present application belongs to the technical field of audio processing, and particularly relates to an audio signal processing method and system, comprising obtaining an environmental audio signal, and pre-processing the environmental audio signal based on a two-path parallel front-end processing architecture to obtain a pre-processed audio signal; performing multi-dimensional feature extraction and fusion on the pre-processed audio signal to obtain fused features; matching the fused features based on a light-weight time-frequency fusion network to obtain the type of the environmental audio signal; performing time-frequency mask separation on the pre-processed audio signal according to the type of the environmental audio signal to obtain independent time-domain audio signals; performing speech recognition and keyword detection on the independent time-domain audio signals to obtain a speech recognition result; and performing a transparent or noise reduction operation on the independent time-domain audio signals based on a dynamic mode decision engine and the speech recognition result to obtain a transparent signal or a noise reduction signal. By using the present application, personalized noise reduction or transparent strategies can be performed for different types of noise.

The present application provides a method for canceling echo in an intercom system, comprising: performing residual echo suppression on each of the identified multiple speaker speech segments based on a separated near-end speech evaluation, constructing a binary time-frequency mask, and reducing the residual echo spectrum caused by dynamic changes in echo power; utilizing the phase information of the original mixed signal in combination with the enhanced near-end speech power spectrum to reconstruct the speech waveform of each speaker through an inverse short-time Fourier transform, and employing an overlap-add method to smooth the transition between speech frames; determining whether speech is present simultaneously at the near-end and far-end ends, and if a double-talk state is detected, adjusting the parameters of a nonlinear echo cancellation algorithm to prevent erroneous suppression of the near-end speech; and performing transmission path modeling on the processed near-end speech signal to evaluate the propagation characteristics of the speech signal from the speaker to the microphone, including the time delay and attenuation of the direct sound and early reflections.

The application discloses a deep learning speech enhancement method based on amplitude-phase mixed feature intersection; according to the collected noisy speech signal, an enhanced mixed intersection feature is obtained; according to the collected clean speech signal and the corresponding noisy speech signal, a label intersection compressed complex mask used for training of an amplitude-phase noise reduction network APNSN is calculated; the enhanced mixed intersection feature is input into the trained APNSN network to obtain an estimated intersection compressed complex mask; according to the estimated intersection compressed complex mask and the spectrum of the noisy speech signal, a time domain reconstructed signal is obtained; compared with a single feature method, such as amplitude spectrum mapping and time-frequency masking based on amplitude spectrum features, the method can further improve speech quality and intelligibility under the condition of the same model size; under a relatively small model, the method can obtain speech quality and intelligibility comparable to the single feature method.

Noise reduction device, comprehensive: a target noise vector selection unit (103) for selecting from pre-acquired control vectors indicating the arrival direction of noises with respect to a sensor arrangement (200) comprising two or more acoustic sensors (200a, 200b, 200c), of a target noise control vector indicating an arrival direction of a target noise; an interference noise vector selection unit (104) for selecting, from the pre-acquired control vectors, an interference noise control vector specifying an arrival direction of interference noise other than the target noise; and a signal processing unit (105) for acquiring, based on two or more observation signals received from the sensor arrangement, the target noise control vector selected by the target noise vector selection unit and the interference noise control vector selected by the interference noise vector selection unit, a signal obtained by eliminating the interference noise from the observation signals, wherein the signal processing unit acquires a signal by time-frequency masking using a mask to block a time-frequency spectrum of the interference noise, which is obtained by eliminating the interference noise from the observation signals, and where, in time-frequency masking, a control vector for each time frequency is estimated from the two or more observation signals, and a similarity is calculated between a control vector of the estimated observation signal and the target noise control vector and the interference noise control vector, and if the control vector, which exhibits the maximum calculated similarity, is the target noise control vector, a time-frequency spectrum of the observation signal is passed through, and if the control vector exhibiting the maximum calculated similarity is not the target noise control vector, a time-frequency spectrum of the observation signal is blocked, where the mask is given by B ( ω , τ ) = { 1 ( ‖ a trg ( ω ) − a ^ ( ω , τ ) ‖ < ‖ a dst ( ω ) − a ^ ( ω , τ ) ‖ ) 0 (otherwise) where B(ω, τ) denotes the mask, ω represents a discrete frequency, τ represents a discrete frame number, a trg(ω) denotes the target noise control vector, â(ω, τ) denotes the estimated control vector of the observation signal, and a dst (ω) denotes the interference noise control vector.