47 results about "Sound source separation" patented technology

The embodiment of the invention provides a sound source separation method and device, and relates to the technical field of data processing. The method comprises the following steps: converting a to-be-separated audio signal from a time domain signal into a time-frequency domain signal; frequency band segmentation is carried out on the time-frequency domain signal, so that the time-frequency domain signal is segmented into a plurality of sub-band signals, and frequency bands of the plurality of sub-band signals are not overlapped; respectively acquiring spectrum characteristics of the plurality of sub-band signals; acquiring a frequency spectrum mask of at least one sound source of the audio signal to be separated according to the frequency spectrum characteristics of the plurality of sub-band signals; and acquiring an audio signal of the at least one sound source according to the spectrum mask of the at least one sound source and the time-frequency domain signal. The embodiment of the invention is used for improving the robustness of a sound source separation algorithm.

The invention discloses a generative audio anonymization reconstruction method, device, equipment and program product based on sound source separation and semantic preservation, and relates to the technical field of voice privacy protection and audio signal processing. The method comprises the following steps: acquiring an original audio, and carrying out sound source separation on the original audio to obtain at least one speaker sound track and an environment background sound track; performing authentication on each speaker sound track, and determining an authentication result of each speaker sound track; wherein the authentication result of the speaker sound track comprises an authorized person sound track and an unauthorized person sound track; carrying out anonymization processing on the unauthorized human voice track to obtain an anonymized human voice track; and carrying out re-synthesis on the anonymized human voice track and the environment background audio track to obtain an anonymized scene audio track. According to the technical scheme provided by the embodiment of the invention, an audio stream breakage phenomenon can be avoided, the scene continuity of the audio is improved, and the intelligibility and the overall quality of the audio are further improved.

The application provides a recording processing method and related devices. The method can include: an electronic device can perform sound source positioning based on the sound collected by a microphone, obtain the position of a target sound source and the number of sound sources in the recording environment, and then perform sound source separation on the sound collected by the microphone according to the position of the target sound source and the number of sound sources in the recording environment to obtain the sound corresponding to the target sound source, i.e., a target audio signal. The electronic device can also determine the signal-to-noise ratio and display the current sound pickup quality to the user. This method can monitor and display the sound pickup quality to the user in real time, so that the user can adjust in time when the sound pickup quality is poor, thereby obtaining high-quality audio and improving the user experience.

A sound data acquisition unit 110 acquires recording data obtained by recording environmental sound. A sound source separation unit 112 separates the recording data into environmental sound data and sound effect data. A first generation unit 116 generates an environmental sound file including the separated environmental sound data. A second generation unit 118 generates a sound effect file including the separated sound effect data.

This disclosure relates to a speech enhancement network training, speech enhancement method, apparatus, and electronic device. The method includes acquiring sample speech information, a clean speech signal, an original impulse response signal, a first device frequency response, and target sound source distribution data corresponding to target sound source characteristics; performing sound source separation processing on the original impulse response signal to obtain an original direct source response signal and an original reflected source response signal; performing sound source characteristic alignment processing based on the target sound source distribution data, the original direct source response signal, and the original reflected source response signal to obtain a target impulse response signal; generating target enhanced speech information corresponding to the sample speech information based on the target impulse response signal, the clean speech signal, and the first device frequency response; and training a preset neural network for speech enhancement based on the sample speech information and the target enhanced speech information to obtain a target speech enhancement network corresponding to the target sound source characteristics. Utilizing embodiments of this disclosure can improve speech enhancement effects.

This invention provides a method, medium, and system for acoustic early warning of thermal runaway in lithium-ion batteries based on feature reconstruction, belonging to the field of lithium-ion battery technology. The invention constructs a positive sample set by collecting safety valve opening sounds through multi-condition thermal runaway experiments, and expands the samples using data augmentation. Multi-resolution Mel spectra are extracted from the audio signals. These Mel spectra are then input into a complex-domain phase-aware separation model for joint estimation of complex-domain amplitude masking and phase residuals. Physical prior corrections are applied to the reconstruction results using a sound source separation algorithm based on wave equation time-frequency inverse scattering and a low-rank sparse time-frequency matrix decomposition algorithm based on random matrix theory. The three corrected signals are weighted and fused to obtain a corrected Mel spectra, which are finally input into a temporal convolutional network to classify and identify the safety valve opening sounds and output a thermal runaway early warning signal. This invention solves the technical problem of insufficient accuracy in thermal runaway early warning caused by acoustic feature reconstruction distortion in complex noise environments.

The application provides an audio processing method, a training method of a sound source separation model and an electronic device. The audio processing method comprises: fully extracting audio features of to-be-processed audio data by using a two-dimensional convolution-based feature extraction network, a frequency band interaction layer and a frequency point interaction layer of a sound source separation model, and then performing sound source separation based on the fully extracted audio features. The scheme adds the frequency band interaction layer and the frequency point interaction layer on the basis of the two-dimensional convolution-based feature extraction network, extracts frequency band interaction features between multiple sub-frequency bands and global frequency point interaction features in each sub-frequency band, so that the advantages of small calculation amount of the two-dimensional convolution-based feature extraction network are retained, and the defects of insufficient extraction capability of the two-dimensional convolution-based feature extraction network for global interaction features are made up, so that the entire sound source separation process can achieve the effect of relatively high separation precision and relatively short processing delay.

The invention discloses an audio control method and system, a storage medium, electronic equipment and a vehicle. The audio control method comprises the step of controlling an audio playing effect of at least one sound source in fused audio according to a command action of a user. According to the invention, the audio data of the plurality of sound sources can be separated from the fused audio, and the audio data of the plurality of sound sources can be played, so that the playing of the fused audio is realized. Through sound source separation, the position of each sound source and the audio playing effect can be flexibly adjusted, so that a user can feel sound from different directions and different sound sources, the sense of space and immersion of audio playing are enhanced, and the immersive listening experience is realized. Besides, according to the embodiment of the invention, the command action of the user is identified, the audio playing effect of at least one sound source is controlled according to the command action, and the command process of the user is fused during audio playing, so that the user can feel that the user becomes a music commander, the interactivity and entertainment are enhanced, and the personalized sound listening experience is realized.

The invention provides a data processing method and device, equipment, a medium and a product, and the method comprises the steps: obtaining multi-modal data collected in a business scene, and the multi-modal data comprises audio data and visual data; the audio data comprises multi-sound-source audio signals obtained by carrying out audio signal collection on N objects in the service scene, the visual data comprises visual signals of the N objects synchronously collected in the process of collecting the multi-sound-source audio signals, and N is an integer greater than 1; based on the audio data and the visual data, sound source separation is carried out on the multi-sound-source audio signals in a multi-modal fusion processing mode, an audio separation result is obtained, and the audio separation result comprises the audio signal of each object; and performing optimization processing on the audio signal of each object in the audio separation result to obtain an optimized audio signal of each object. The audio signals and the visual signals are fused, sound source separation is carried out on the multi-sound-source audio signals, and the accuracy of audio separation is improved.

The invention discloses a rotating sound source separation method based on fast Bayesian inference, and the method comprises the steps: dividing a plane where a sound source is located into uniform or non-uniform equivalent source grid points, enabling each grid point to represent an equivalent sound source so as to describe the sound field energy contribution of a rotating sound source and a static sound source, and dividing scanning grid points which coincide with the equivalent source grid points; for mixed sound signals collected by a microphone array, scanning grid point output is calculated by adopting a traditional beam forming algorithm and a modal composition beam forming method considering the Doppler effect of a rotating sound source, key features of two kinds of beam forming output are extracted based on a convolution kernel, and preliminary acceleration is achieved by calculating an approximate transfer matrix through convolution.

The embodiment of the invention discloses an audio processing method, device and system based on multi-mode noise reduction and sound source separation. The method comprises the following steps: acquiring an original audio signal acquired by an array consisting of at least two microphones; performing beam forming processing on the original audio signal to obtain an initial audio signal; dividing the residual noise in the initial audio signal into steady-state noise and unsteady-state noise, and respectively suppressing the noise by adopting a corresponding noise reduction strategy to obtain a denoised first audio signal; performing sound source separation on the first audio signal to obtain multiple paths of separated second audio signals of different sound sources; automatically adjusting a compression ratio and a compression range according to audio characteristics of the second audio signal to compress the second audio signal to obtain compressed audio data; and transmitting the audio data to the pre-binding device based on a preset transmission protocol. According to the embodiment of the invention, the technical problem of how to improve the audio quality and meet the low-delay transmission of the real-time transcription requirement is solved.

The invention provides a zero sample singing sound conversion method and system, and the method comprises the following steps: carrying out the fundamental frequency disturbance and wet sound simulation of an input audio, and generating enhanced training data; obtaining speaker embedding by using a singing enhanced tone and style extractor; and based on enhanced training data and speaker embedding, harmonic waves and noise excitation are generated in combination with a neural source filter, and a target singing waveform is synthesized. The problems of environmental noise, sound source separation residue and unnatural singing style expression in a real scene are solved.

The application discloses an audio playing control method, comprising: acquiring first audio data; acquiring an environmental noise decibel value; performing sound source separation on the first audio data to obtain a plurality of second audio data with different frequency structures; adjusting the volume of the plurality of second audio data according to the environmental noise decibel value; performing audio mixing on the second audio data after volume adjustment; and adjusting the playing volume of the second audio data after audio mixing according to the environmental noise decibel value. The audio playing control method provided by the application can realize sound source separation of playing audio data, and adjust the volume of, for example, target sound source data and environmental sound source data according to the environmental noise condition, thereby improving the automatic adjustment capability of the system. The volume of sound of different sound sources is adjusted, so that sound containing substantial content such as a human voice sound source is highlighted in a noisy environment, and all kinds of sound sources are balanced in a quiet environment, giving people a sense of stability and improving the auditory experience.

The invention is suitable for the technical field of artificial intelligence, and provides a voice recognition method based on multi-sound-source separation, and the method comprises the steps: obtaining voice data in a multi-sound-source environment; performing time-frequency feature extraction processing on the voice data to obtain time-frequency spectrum features; performing feature focusing processing on the time-frequency spectrum feature to obtain a target sound source focusing feature; based on the target sound source focusing feature, performing multi-sound source separation on the voice data, and determining target voice data; and performing voice recognition and correction processing on the target voice data, and outputting a voice recognition result corresponding to the target voice data. The method solves the problem that the existing voice recognition method cannot effectively distinguish the space and spectrum characteristics of different sound sources under the condition of multi-sound-source interference, so that the voice recognition accuracy is low.

The application discloses an audio processing method, device and equipment and a storage medium, relates to the technical field of audio processing, and discloses an audio processing method, which comprises the following steps: inputting a stereo audio signal into a preset sound source separation model to obtain a plurality of independent audio objects; obtaining original spatial information of each audio object from the stereo audio signal, and obtaining music structure information of the stereo audio signal; generating target spatial parameters of the audio objects in a three-dimensional immersive sound field based on the original spatial information and the music structure information, wherein the target spatial parameters comprise position coordinates and / or movement trajectories; and rendering each audio object based on the target spatial parameters to obtain a multi-channel immersive audio signal. The application can improve the generation quality of the multi-channel immersive audio signal.

The invention relates to an industrial noise source positioning and separating method and device, equipment and a storage medium, and the method comprises the steps: collecting a multi-channel audio signal, and carrying out the preprocessing and local feature extraction of the multi-channel audio signal, and obtaining a local feature vector packet; carrying out sound source localization based on the local feature vector packet by adopting a hybrid localization strategy to generate space position information of a sound source, and calculating localization confidence; sound source separation is carried out on the multi-channel audio signals, and separated audio signals and sound source separation quality confidence coefficients of all independent sound sources are obtained; collaborative management is carried out on the output of each agent through the arbitration agent, and confidence fusion and conflict judgment are carried out; and when the confidence coefficient is lower than a first preset threshold value, triggering an artificial verification process or scheduling a mobile detection unit to carry out auxiliary measurement so as to determine target sound source positioning and separation data, and storing the target sound source positioning and separation data in a vector database. The method has the advantages of high-precision positioning and reliable separation of the industrial noise source, and can adapt to the change of a complex acoustic environment.

A sound source separation device according to an embodiment of the present disclosure includes a plurality of microphones, a matrix unit, and an output unit. The plurality of microphones may receive a plurality of microphone input signals transmitted from a plurality of sound sources. The matrix unit generates an objective function according to an estimated source vector and an estimated noise vector estimated based on the plurality of microphone input signals, and replace a first term and a second term included in the objective function using a log-likelihood function to estimate a demixing matrix. The output unit provides output vectors calculated based on the microphone input signals and the demixing matrix.

The invention discloses a space rendering method and device, audio equipment and a computer readable storage medium, and relates to the technical field of audio equipment, and the method comprises the steps: carrying out the sound source separation processing of a first audio signal, and obtaining a first human sound audio signal and a first background sound audio signal; performing spatial rendering on the first human voice audio signal based on the first playback configuration to obtain a second human voice audio signal, and performing spatial rendering on the first background sound audio signal based on the second playback configuration to obtain a second background sound audio signal, the spatial perceptibility corresponding to the first playback configuration is higher than the spatial perceptibility corresponding to the second playback configuration; performing sound mixing processing on the second human voice audio signal and the second background voice audio signal to obtain a second audio signal; and performing crosstalk elimination on the second audio signal based on the target transfer function to obtain a target audio signal. According to the method, the stability and the focusing sense of human voice can be considered while the space rendering effect is realized.

The embodiment of the invention discloses a method, device and equipment for training a sound source separation model. In the embodiments of the present invention, a plurality of sound signals of a wearer and / or a non-wearer are obtained, the plurality of sound signals are generated by recording a playing signal by an intelligent device in different scenes and different paths, and the playing signal is a sound signal emitted by the wearer and / or the non-wearer; determining a plurality of room impulse response (RIR) functions according to the plurality of sound signals; generating a plurality of basic training data according to the plurality of RIR functions; training a basic separation model according to the multiple basic training data; acquiring a plurality of real scene data of a wearer and a non-wearer; and performing fine tuning on the basic separation model according to the multiple pieces of real scene data to generate a target separation model. Through the method, the target separation model is generated, the voice signal separation effect is effectively improved, and the user experience is improved.

This invention relates to the field of audio signal processing technology, and discloses a method and system for dynamic audio control optimization based on convolutional neural networks. The method includes: separating the mixed audio signals using a source separation convolutional neural network to obtain independent estimated spectra for each source category; extracting energy demand prediction vectors and transient feature descriptors for each source category using a spectral residual convolutional neural network; calculating the driving characteristic matching score between each source category and each speaker unit; performing propagation calculations based on the room transfer function and constructing an arrival power prediction matrix; calculating the masking threshold matrix and perceived loudness estimate for each listener position; solving for the optimal power allocation coefficient through a joint optimization objective function; extracting position-specific perception sensitivity coefficients using an auditory masking perception convolutional neural network; generating adaptive dynamic compression gain values ​​and finally outputting the driving signals for each speaker unit.

For example, an effective voice quality conversion process is performed.An information processing apparatus includes: a voice quality conversion unit that performs sound source separation of a vocal signal and an accompaniment signal from a mixed sound signal and performs voice quality conversion using a result of the sound source separation.

A virtual bass enhancing device for enhancing a virtual bass of an input audio signal includes a demixer, configured to extract at least one audio channel from the input audio signal, wherein the audio channel corresponds to an acoustic source, or to a group of acoustic sources, of the input audio signal, at least one virtual bass enhancing unit configured to generate overtones for enhancing a bass perception of the audio channel, and at least one adder configured to add the overtones to the input audio signal so as to generate an enhanced audio signal.

The application provides a song detection model training method, a song detection method, equipment and a medium, and belongs to the technical field of artificial intelligence. The method comprises the following steps: obtaining original singing data of a singing object; performing sound source separation on the original singing data based on a sound source separation model to obtain sample human voice spectrograms; performing audio fusion on the sample human voice spectrograms and reference audio to obtain sample audio data, wherein the reference audio is audio data that does not contain lyrics; performing segmentation processing on the sample audio data to obtain sample audio segments; inputting the sample audio segments into a neural network model to perform song detection, thereby obtaining a sample song time sequence of the sample audio data, wherein the sample song time sequence contains a time period in which lyrics exist in the sample audio data; and optimizing a loss function of the neural network model based on the sample song time sequence to train the neural network model, thereby obtaining a song detection model. The application can improve the accuracy of the song detection model in detecting songs.

A signal processing apparatus is provided, comprising: a sound source separation unit that applies sound source separation processing to a mixed sound signal that combines signals from multiple sound sources; and a bandwidth expansion unit that applies bandwidth expansion processing to each sound source-separated signal separated by the sound source separation unit.

The present disclosure provides a sound source separation method and device, and relates to the technical field of intelligent vehicles, which comprises the following steps: performing sound source separation on a plurality of sound collection signals in a previous observation window according to a separation matrix corresponding to a previous round of sound source separation to obtain a plurality of separation estimation signals corresponding to the previous round of sound source separation, and obtaining a speech existence probability corresponding to the separation estimation signals; updating the separation matrix corresponding to the previous round of sound source separation according to the speech existence probability; and performing sound source separation on a plurality of sound collection signals in a current observation window according to the updated separation matrix to obtain a plurality of separation estimation signals corresponding to the current round of sound source separation. The speech existence probability of the separation estimation signals obtained based on the previous round of sound source separation is used to update the separation matrix, so that the speech existence information of the sound collection signals in the previous observation window is added when the separation matrix is updated, and the separation coefficients of various sound sources are adjusted accordingly, thereby enhancing the separation effect of the sound sources.

The application discloses a multi-sound source separation system and method based on a generative adversarial network, which comprises a sound signal acquisition and processing all-in-one machine, a server platform, and a multi-sound source separation client. The sound signal acquisition and processing all-in-one machine is used for collecting original audio data emitted by multiple sound sources and processing the original audio data to determine the number of sound sources. The server platform determines whether to process the original audio data according to the determination result of the number of sound sources. If the number of sound sources is one, the original audio data is not processed. If the number of sound sources is multiple, the original audio data is subjected to cross-correlation to determine the position of each sound source, and then the multiple sound source signals are separated through adaptive beamforming. The speech signals after beamforming are optimized by using a generative adversarial network, and the output image data of the generative adversarial network is restored to audio data. The multi-sound source separation client acquires the separation result of the server platform on the multiple sound source signals. The application can effectively separate the speech signals of multiple sound sources, and the speech signals can obtain a higher speech recognition accuracy and a lower word error rate after being recognized.

The invention relates to the technical field of sound detection, in particular to an unmanned aerial vehicle sound detection and early warning method and system. The sensor network layout is pre-optimized based on digital twinning and sound wave simulation, efficient coverage and redundant backup are achieved, deployment blind areas are reduced, the system detection basic capacity is improved, signal preprocessing and event detection are carried out at the node end, data lightweight and intelligent packaging are achieved, transmission and cloud burden is relieved, and the system reliability is improved. The real-time performance and robustness of the system are enhanced, cloud multi-node data fusion, sound source separation and deep learning feature comparison are utilized, unmanned aerial vehicle and non-unmanned aerial vehicle sound sources are effectively distinguished, the recognition precision is improved, false alarms are reduced, digital twin space information and multi-node synchronization signals are combined, unmanned aerial vehicle three-dimensional geodetic coordinate calculation and continuous trajectory generation are achieved, and the method has the advantages of being high in practicability and easy to popularize. Accurate tracking and behavior prediction are supported, dynamic threat assessment is performed based on a flight path and a behavior mode, and graded early warning and early response are realized.

The objective is to provide an information processing device and program that can efficiently generate time-saving content. [Solution] The information processing device of this embodiment is an information processing device that receives and processes content data, and comprises: an acquisition unit that stores audio data acquired from the content data in a storage area; an inference unit that performs sound source separation processing using the audio data stored in the storage area and outputs sound source data; and a generation unit that generates time-saving content with a shortened playback time of the content data based on the output sound source data.

The application relates to the technical field of medical informatics, and particularly relates to an AI voice recognition-based psychiatric electronic medical record generation device. A voice processing module collects doctor-patient conversation audio signals, generates a time-stamped text sequence through sound source separation and voice recognition. A semantic analysis module performs context modeling, entity recognition and relationship extraction on the text sequence, and outputs clinical entities and semantic relationships. A knowledge graph module performs graph reasoning based on psychiatric clinical knowledge data, and generates a diagnosis hypothesis. A template adaptation module selects a medical record template and fills in fields according to the analysis result and the diagnosis hypothesis, and generates a preliminary medical record document. A quality control module performs multistage checking on the document, and feeds back the checking result to each module to realize parameter optimization. Through data flow conversion and closed-loop feedback mechanism among the modules, the device realizes automatic conversion of unstructured doctor-patient conversation into standardized medical records, and effectively improves the accuracy and integrity of psychiatric medical record recording.

This application relates to a method, system, device, and medium for separating overlapping speech in video conferencing based on AI noise suppression, belonging to the field of online video conferencing service technology. The method for separating overlapping speech in video conferencing includes: acquiring and cleaning the mixed audio stream of participants, segmenting and converting it into a short-time frame spectrogram; extracting the voiceprint features from the short-time frame spectrogram to obtain a general acoustic feature vector and a participant representation vector; separating and mapping the general acoustic feature vector according to a preset sound source separation network to generate a sound source speech mask and record potential sound source labels; aggregating the participant representation vector and the sound source speech mask according to the potential sound source labels to obtain a participant speech mask; reconstructing the short-time frame spectrogram based on the participant speech mask, and adaptively applying gain and noise suppression to obtain the participant speech stream; using a deep learning model to distinguish speech, noise, and multiple sound sources, achieving the separation of overlapping speech and improving speech intelligibility in noisy environments.