119 results about "Sound source separation" patented technology

The invention discloses a sound acquisition and processing system based on cooperation of multiple microphone arrays. The system comprises a sound acquisition module, a multi-channel signal preprocessing module, a sound signal feature extraction module, an abnormal sound recognition module, a sound source positioning module and an alarm module. A multi-channel mixed data signal is collected through a circularly-arranged multi-microphone array formed by a plurality of microphones, after echo cancellation, wave beam domain noise reduction and multi-sound-source separation, a single-sound-source feature vector is extracted, according to the single-sound-source feature vector, abnormal sound including explosion, screaming or glass breakage is recognized through a BiLSTM and an attention mechanism model, and the abnormal sound is recognized through an attention mechanism model. And the GCC-PHAT and MDS-MUSIC algorithms are combined to position abnormal sound production, and alarm information is generated. According to the invention, accurate identification, positioning and alarm of the abnormal sound can be realized, and the real-time performance, the accuracy and the multi-target processing capability of abnormal sound monitoring in a complex environment can be obviously improved.

The invention discloses an electronic commerce product live broadcast selling voice monitoring method and system, and relates to the technical field of voice monitoring, and the method comprises the steps: carrying out the sound source separation of a live broadcast audio stream, extracting an anchor sound track, and generating a structured semantic unit; constructing a knowledge graph, calculating cosine similarity between entities in the semantic unit and nodes of the knowledge graph, updating the knowledge graph, and calculating illegal path propagation probability; dynamically adjusting a time window according to the entity density of the semantic units, aggregating the semantic units in the window to generate event objects, detecting associated event pairs, and combining to form an event chain; according to the method, the comprehensive risk value of the current live broadcast is calculated through the maximum violation path propagation probability, the total duration and the audience report frequency of the event chain, the risk level is divided according to the comprehensive risk value, and the hierarchical response measure is executed, so that the problems of difficulty in hidden violation verbal skill identification and weak cross-time violation element association in e-commerce live broadcast selling are solved, and the user experience is improved. And the accuracy and integrity of illegal behavior monitoring are obviously improved.

The invention discloses a sound source separation method and system for a microphone array to pick up voice signals. The method comprises the following steps: picking up voice signals sent by a target reactor sound source and an interference sound source in a space by using the microphone array; performing segmentation and Fourier transform on the voice signal to obtain a time-frequency domain signal; calculating the time difference of arrival and the phase difference between each pair of microphones, and obtaining the direction estimation result of each sound source in the space by combining the geometric arrangement information of the microphone array; generating a time-frequency mask, applying the time-frequency mask to the time-frequency domain signal, and separating the time-frequency domain signal of each sound source; performing inverse short-time Fourier transform and fragment splicing to obtain a corresponding continuous time domain voice signal; and outputting to different audio output channels. According to the invention, the specific sound source signal generated by the reactor can be effectively separated, the operation state of the reactor can be accurately analyzed, the abnormal condition can be timely found, and the reliability and precision of monitoring the reactor can be further improved.

The invention relates to the technical field of voice signals, in particular to a two-step mixed sound source separation and de-reverberation method, which comprises the following steps of: firstly, carrying out separation network training by taking different types of signals as training targets in various separation networks for separating attention and the like; the invention provides an improved de-reverberation method based on a time convolution network-weight prediction error, multiple improvement strategies such as taking a scale invariance signal-to-noise interference ratio as a network loss function, adopting a transposition mechanism for an input signal and a mechanism for additionally adding a residual value to a network unit are used, and finally, the de-reverberation method based on the time convolution network-weight prediction error is obtained. And cascading the separation attention network with the best separation noise reduction effect with the time convolution-transpose-residual error-WPE network. According to the invention, the problem that the separation effect of the mixed audio signal with reverberation and noise signals is not good under the actual sound field condition is solved, compared with a single one-step or two-step existing separation, de-mixing and noise reduction network, the method is significantly improved, and the quality of the separated voice can be improved for later recognition and discrimination.

The invention relates to the technical field of intelligent operation and maintenance of a power system, in particular to an intelligent early warning method and system for preventing external damage of a power transmission line. The method comprises the following steps: acquiring acoustic data of a power transmission line through an acoustic sensor; performing voiceprint feature sparse reconstruction according to the transmission line acoustic data to obtain voiceprint feature data; performing sparse causal graph matching on the voiceprint feature data to obtain sound source behavior causal graph data; performing double-domain attention cross recognition according to the sound source behavior causal atlas data to obtain sound source separation recognition data; and performing scene map generation on the sound source separation identification data to obtain sound source scene map data. According to the invention, by constructing the nested causal chain graph and fusing the propagation weight mechanism, the development evolution and multi-stage trigger relationship of the external damage event of the power transmission line can be accurately captured. Compared with a traditional single-point detection method, the method has higher chain reasoning ability and early risk identification ability, and the response foresight and causal interpretability of an early warning system are effectively improved.

The invention provides an out-of-vehicle voice control method and device, a storage medium and electronic equipment, and relates to the technical field of artificial intelligence, and the method comprises the steps: firstly obtaining audio signals collected by a microphone array on a vehicle for different areas outside the vehicle; the audio signals are preprocessed; performing sound source positioning and sound source separation according to the preprocessed audio signal to obtain a multi-channel audio signal; then performing wake-up word recognition based on the multi-channel audio signal, and determining a target channel where the wake-up word is located; and executing a corresponding vehicle voice control command according to the audio signal corresponding to the target channel. According to the method and the device, the audio signals are acquired through the microphone arrays in different areas around the vehicle, and the multichannel audio signals are obtained through preprocessing, sound source positioning and separation, so that accurate wake-up word recognition and voice control command execution are realized, outdoor complex environmental noise can be effectively shielded, and the accuracy of out-of-vehicle voice control is greatly improved.

The invention relates to an audio processing method and device, computer equipment, a storage medium and a program product. The method comprises the following steps: acquiring an original audio which is acquired by a microphone array and comprises a multi-channel audio; according to the spectrum feature of each channel audio included in the original audio and the physical spacing of each microphone in the microphone array, determining direction features corresponding to the original audio in a plurality of preset virtual sound source directions; inputting the frequency spectrum features of the channel audio and the direction features corresponding to the directions of the pseudo sound sources into a pre-trained sound source separation model; the pre-trained sound source separation model is used for carrying out filtering operation on the spectrum features of the channel audios according to the direction features corresponding to the virtual sound source directions to obtain filtered spectrum features of the channel audios, and the filtered spectrum features of the channel audios are utilized to generate and output a target spectrum. By adopting the method, the accuracy of sound enhancement and separation in a complex acoustic scene can be improved.

The invention discloses a multi-voice separation method based on a lightweight dual-path Transform network, and the method comprises the steps: collecting audio multi-voice data, and carrying out the preprocessing of the data, and forming a data set; the method comprises the following steps: constructing a dual-path Transform network model DPTNet, and introducing a recurrent neural network to optimize the dual-path Transform network model DPTNet; and training the dual-path network model DPTNet, and performing engineering deployment based on the trained model. The method is beneficial to obtaining higher-quality audio fingerprint recognition capability, sound source separation capability and voice enhancement function, can be used for tracking and positioning the position of a sound source, helps positioning and tracking related applications, can be expanded to the medical field, can be used for heart sound segmentation, namely, recognition of specific signals of the heart, and can be applied to the field of medical science. The method helps to diagnose cardiovascular and other medical problems, and has technical innovation and practical application value.

The embodiment of the invention provides a sound source separation method and device, and relates to the technical field of data processing. The method comprises the following steps: converting a to-be-separated audio signal from a time domain signal into a time-frequency domain signal; frequency band segmentation is carried out on the time-frequency domain signal, so that the time-frequency domain signal is segmented into a plurality of sub-band signals, and frequency bands of the plurality of sub-band signals are not overlapped; respectively acquiring spectrum characteristics of the plurality of sub-band signals; acquiring a frequency spectrum mask of at least one sound source of the audio signal to be separated according to the frequency spectrum characteristics of the plurality of sub-band signals; and acquiring an audio signal of the at least one sound source according to the spectrum mask of the at least one sound source and the time-frequency domain signal. The embodiment of the invention is used for improving the robustness of a sound source separation algorithm.

The invention provides a robust audio watermarking method with sound source separation resistance. The robust audio watermarking method comprises the steps of preprocessing an input audio, and embedding and extracting watermarks based on a reversible neural network. According to the method, a structure based on a reversible neural network is adopted, the watermark information and the audio spectrum are deeply fused, the resistance of the watermark to sound source separation, compression and other signal processing attacks is effectively improved by introducing a disturbance enhancement training mechanism, and compared with a traditional time domain or frequency domain embedding method, the robustness is higher; according to the method, watermark embedding is carried out on an audio amplitude spectrum, phase information of an original audio is fully reserved, and high-quality audio reconstruction is realized in combination with a perception fidelity strategy and a window compensation mechanism; according to the method, the symmetric reversible neural network structure is designed, it is ensured that information in the embedding and extracting processes is symmetric and can be restored, watermark recovery can be completed without accessing the original audio, and the good blind detection capability is achieved.

Real-time source separation is desirable for its ability to clearly transmit desired audio signals but is computationally costly when applied across large areas. Thus, a tradeoff between sound source separation quality and system performance is presented. For that reason, this disclosure of area-based sound source separating techniques resolves this tradeoff. Concepts herein include an area-based sound source separating machine learning architecture defines subareas and redefines those subareas as necessary to capture desired audio signal(s). By redefining the subarea, less noise or unwanted audio signals are received / processed, significantly reducing computational resources while increasing throughput of the desired audio signal(s).

The invention relates to a digital conference voice processing method, system and device and a storage medium, and the method comprises the following steps: carrying out the pickup of a conference voice, obtaining a mixed voice signal, carrying out the framing sampling, and forming a voice sampling sequence; carrying out sound source direction estimation based on the sequence to obtain multi-sound-source position information, and carrying out beam forming and spatial filtering on the voice sampling sequence according to the multi-sound-source position information to obtain a sound source separation signal; performing voice segment segmentation on the signal to obtain a voice segment sequence, extracting voiceprint features of each segment, and generating a speaker feature mark; and performing time sequence recombination on the voice fragment sequence by using the mark, constructing a speaking time sequence table, selectively outputting the voice fragment sequence according to the table, and finally generating clear and ordered conference voice. The technical problems that due to the fact that a traditional voice processing method lacks effective space-voiceprint joint constraint, voice separation is not thorough, identities of speakers are confused, and the speaking time sequence is disordered are solved.

The present invention provides a sound source localization method based on a polyhedral microphone array, comprising: S1: collecting multi-channel data collected by multiple microphones forming a polyhedral structure; S2: iteratively processing the multi-channel data using an independent component analysis algorithm to obtain noise-reduced target sound source data; and S3: determining the orientation of the target sound source based on the attenuation of the target sound source by each microphone. The advantages of the present invention are that by collecting sound data using a polyhedral microphone array, the microphone array receives signals that form a non-singular matrix, thereby enabling the extraction of the sound source using an independent component analysis algorithm, and then determining its orientation relative to the microphone array based on the attenuation of the sound source. This further achieves sound source localization based on sound source separation, and can meet the needs of scenarios such as abnormal sound source localization and fault detection.

The invention discloses a generative audio anonymization reconstruction method, device, equipment and program product based on sound source separation and semantic preservation, and relates to the technical field of voice privacy protection and audio signal processing. The method comprises the following steps: acquiring an original audio, and carrying out sound source separation on the original audio to obtain at least one speaker sound track and an environment background sound track; performing authentication on each speaker sound track, and determining an authentication result of each speaker sound track; wherein the authentication result of the speaker sound track comprises an authorized person sound track and an unauthorized person sound track; carrying out anonymization processing on the unauthorized human voice track to obtain an anonymized human voice track; and carrying out re-synthesis on the anonymized human voice track and the environment background audio track to obtain an anonymized scene audio track. According to the technical scheme provided by the embodiment of the invention, an audio stream breakage phenomenon can be avoided, the scene continuity of the audio is improved, and the intelligibility and the overall quality of the audio are further improved.

The application provides a recording processing method and related devices. The method can include: an electronic device can perform sound source positioning based on the sound collected by a microphone, obtain the position of a target sound source and the number of sound sources in the recording environment, and then perform sound source separation on the sound collected by the microphone according to the position of the target sound source and the number of sound sources in the recording environment to obtain the sound corresponding to the target sound source, i.e., a target audio signal. The electronic device can also determine the signal-to-noise ratio and display the current sound pickup quality to the user. This method can monitor and display the sound pickup quality to the user in real time, so that the user can adjust in time when the sound pickup quality is poor, thereby obtaining high-quality audio and improving the user experience.

Disclosed are a system and method for automating the design of a sound source separation deep learning model. A method of automating a design of a sound source separation deep learning model, which is performed by a design automation system, may include automatically searching for a combination of hyper parameters of a separation model constructed in a sound source separation deep learning model by using a neural architecture search (NAS) algorithm and reconstructing the sound source separation deep learning model based on the retrieved combination of the hyper parameters of the separation model.

A sound data acquisition unit 110 acquires recording data obtained by recording environmental sound. A sound source separation unit 112 separates the recording data into environmental sound data and sound effect data. A first generation unit 116 generates an environmental sound file including the separated environmental sound data. A second generation unit 118 generates a sound effect file including the separated sound effect data.

This disclosure relates to a speech enhancement network training, speech enhancement method, apparatus, and electronic device. The method includes acquiring sample speech information, a clean speech signal, an original impulse response signal, a first device frequency response, and target sound source distribution data corresponding to target sound source characteristics; performing sound source separation processing on the original impulse response signal to obtain an original direct source response signal and an original reflected source response signal; performing sound source characteristic alignment processing based on the target sound source distribution data, the original direct source response signal, and the original reflected source response signal to obtain a target impulse response signal; generating target enhanced speech information corresponding to the sample speech information based on the target impulse response signal, the clean speech signal, and the first device frequency response; and training a preset neural network for speech enhancement based on the sample speech information and the target enhanced speech information to obtain a target speech enhancement network corresponding to the target sound source characteristics. Utilizing embodiments of this disclosure can improve speech enhancement effects.

This invention provides a method, medium, and system for acoustic early warning of thermal runaway in lithium-ion batteries based on feature reconstruction, belonging to the field of lithium-ion battery technology. The invention constructs a positive sample set by collecting safety valve opening sounds through multi-condition thermal runaway experiments, and expands the samples using data augmentation. Multi-resolution Mel spectra are extracted from the audio signals. These Mel spectra are then input into a complex-domain phase-aware separation model for joint estimation of complex-domain amplitude masking and phase residuals. Physical prior corrections are applied to the reconstruction results using a sound source separation algorithm based on wave equation time-frequency inverse scattering and a low-rank sparse time-frequency matrix decomposition algorithm based on random matrix theory. The three corrected signals are weighted and fused to obtain a corrected Mel spectra, which are finally input into a temporal convolutional network to classify and identify the safety valve opening sounds and output a thermal runaway early warning signal. This invention solves the technical problem of insufficient accuracy in thermal runaway early warning caused by acoustic feature reconstruction distortion in complex noise environments.

The application provides an audio processing method, a training method of a sound source separation model and an electronic device. The audio processing method comprises: fully extracting audio features of to-be-processed audio data by using a two-dimensional convolution-based feature extraction network, a frequency band interaction layer and a frequency point interaction layer of a sound source separation model, and then performing sound source separation based on the fully extracted audio features. The scheme adds the frequency band interaction layer and the frequency point interaction layer on the basis of the two-dimensional convolution-based feature extraction network, extracts frequency band interaction features between multiple sub-frequency bands and global frequency point interaction features in each sub-frequency band, so that the advantages of small calculation amount of the two-dimensional convolution-based feature extraction network are retained, and the defects of insufficient extraction capability of the two-dimensional convolution-based feature extraction network for global interaction features are made up, so that the entire sound source separation process can achieve the effect of relatively high separation precision and relatively short processing delay.

The present technology relates to signal processing apparatus and method which can perform audio reproduction with a realistic feeling. A signal processing apparatus includes a sound source separation unit that extracts, from an input audio signal including a plurality of sound source signals, one or a plurality of the sound source signals by sound source separation; a position information generation unit that generates position information of the extracted sound source signal on the basis of a result of the sound source separation; and an output unit that outputs the extracted sound source signal and the position information as data of an audio object. The present technology can be applied to a signal processing apparatus.

The invention discloses an audio control method and system, a storage medium, electronic equipment and a vehicle. The audio control method comprises the step of controlling an audio playing effect of at least one sound source in fused audio according to a command action of a user. According to the invention, the audio data of the plurality of sound sources can be separated from the fused audio, and the audio data of the plurality of sound sources can be played, so that the playing of the fused audio is realized. Through sound source separation, the position of each sound source and the audio playing effect can be flexibly adjusted, so that a user can feel sound from different directions and different sound sources, the sense of space and immersion of audio playing are enhanced, and the immersive listening experience is realized. Besides, according to the embodiment of the invention, the command action of the user is identified, the audio playing effect of at least one sound source is controlled according to the command action, and the command process of the user is fused during audio playing, so that the user can feel that the user becomes a music commander, the interactivity and entertainment are enhanced, and the personalized sound listening experience is realized.

The embodiment of the invention provides a voice control screen display method and device, computer equipment and a storage medium, and is used for the technical field of display control. The voice control screen display method comprises the steps of performing audio separation and voiceprint comparison on mixed audio data to determine a target sounding direction, performing sound source tracking collection according to the target sounding direction, and filtering collected audio information to obtain an audio data stream, semantic recognition and sentiment analysis are carried out on the audio data stream to obtain a semantic command set and a demonstration sentiment label, and a screen display adjustment instruction is generated according to the semantic command set and the demonstration sentiment label through an instruction format. According to the technical scheme, through sound source separation and voiceprint comparison, sound source tracking, semantic recognition, sentiment analysis and instruction mapping, screen display can be controlled in real time only by an authorized user in a complex noisy environment, so that the false triggering rate is remarkably reduced, and the command recognition accuracy and the consistency of system interaction experience are improved.

A method of providing a sound source edit function performed by an electronic device, and the electronic device supporting the same, are provided. The method includes receiving an original sound source including a plurality of sound source elements, selecting a test sound source corresponding to the original sound source from among a plurality of test sound sources that are pre-stored, the multiple test sound sources each being matched to at least one of a plurality of sound source separators according to a performance indicator, selecting at least one sound source separator, which corresponds to separation of the test sound source, from among the plurality of sound source separators, extracting the plurality of sound source elements by separating the original sound source by using the selected at least one sound source separator, and storing an edited sound source including the extracted plurality of sound source elements.

A sound source separation system is configured to perform blind source separation of individual sound source signals by an independent component analysis (ICA) method from a plurality of mixed signals in which two or more sound source signals are mixed. The sound source separation system includes an n number of microphones, where n≥2; a virtual microphone signal generator configured to generate, from output signals of the n number of the microphones, an m number of the virtual microphone signals that are output signals of the m number of virtual unidirectional microphones having directivities in different directions, where m>n; and an ICA processor configured to separate an L number of the sound source signals that are signals of the L number of different sound sources, by the ICA method, from the m number of the virtual microphone signals that are the plurality of the mixed signals, where L>n and L≤m.

The invention provides a data processing method and device, equipment, a medium and a product, and the method comprises the steps: obtaining multi-modal data collected in a business scene, and the multi-modal data comprises audio data and visual data; the audio data comprises multi-sound-source audio signals obtained by carrying out audio signal collection on N objects in the service scene, the visual data comprises visual signals of the N objects synchronously collected in the process of collecting the multi-sound-source audio signals, and N is an integer greater than 1; based on the audio data and the visual data, sound source separation is carried out on the multi-sound-source audio signals in a multi-modal fusion processing mode, an audio separation result is obtained, and the audio separation result comprises the audio signal of each object; and performing optimization processing on the audio signal of each object in the audio separation result to obtain an optimized audio signal of each object. The audio signals and the visual signals are fused, sound source separation is carried out on the multi-sound-source audio signals, and the accuracy of audio separation is improved.

The invention discloses a rotating sound source separation method based on fast Bayesian inference, and the method comprises the steps: dividing a plane where a sound source is located into uniform or non-uniform equivalent source grid points, enabling each grid point to represent an equivalent sound source so as to describe the sound field energy contribution of a rotating sound source and a static sound source, and dividing scanning grid points which coincide with the equivalent source grid points; for mixed sound signals collected by a microphone array, scanning grid point output is calculated by adopting a traditional beam forming algorithm and a modal composition beam forming method considering the Doppler effect of a rotating sound source, key features of two kinds of beam forming output are extracted based on a convolution kernel, and preliminary acceleration is achieved by calculating an approximate transfer matrix through convolution.

Aspects of the subject technology relate to systems, methods, and computer readable media for estimating acoustic spectra. Acoustic data can be received at a hydrophone array from a first acoustic source and a second acoustic source in a downhole environment. An initial noise spatial correlation matrix estimation can be generated based on the acoustic data. The initial noise spatial correlation matrix estimation can be applied to a beamformer to generate a first source spectra estimation for the first acoustic source and the second acoustic source. A revised noise spatial correlation matrix estimation can be generated based on the first source spectra estimation. The revised noise spatial correlation matrix estimation can be applied to the beamformer to generate a second source spectra estimation for the first acoustic source and the second acoustic source in the downhole environment based on the first source spectra estimation.

The embodiment of the invention discloses an audio processing method, device and system based on multi-mode noise reduction and sound source separation. The method comprises the following steps: acquiring an original audio signal acquired by an array consisting of at least two microphones; performing beam forming processing on the original audio signal to obtain an initial audio signal; dividing the residual noise in the initial audio signal into steady-state noise and unsteady-state noise, and respectively suppressing the noise by adopting a corresponding noise reduction strategy to obtain a denoised first audio signal; performing sound source separation on the first audio signal to obtain multiple paths of separated second audio signals of different sound sources; automatically adjusting a compression ratio and a compression range according to audio characteristics of the second audio signal to compress the second audio signal to obtain compressed audio data; and transmitting the audio data to the pre-binding device based on a preset transmission protocol. According to the embodiment of the invention, the technical problem of how to improve the audio quality and meet the low-delay transmission of the real-time transcription requirement is solved.

The invention relates to a three-dimensional point cloud fused power transformation main equipment sound source separation method, which comprises the following steps of: acquiring high-density point cloud data, preprocessing the point cloud data, and performing equipment segmentation; extracting sound signal features; fusion features are obtained; inputting the fusion features into a sound source separation network based on Transform, and carrying out sound signal separation to obtain separated sound signals; and inputting the separated sound signals into a sound source separation network based on Transform to carry out iteration for multiple times to optimize a separation result. The invention provides a novel multi-sound-source separation method, and the method can remarkably improve the sound source positioning and recognition precision through the combination of the three-dimensional point cloud and sound signals. Compared with a traditional sound separation method, the method has the advantages that a sound source can be positioned more accurately by means of the three-dimensional point cloud data, noise interference is reduced, the reliability of substation equipment fault diagnosis is improved, and the method has wide application prospects and particularly has important significance in the aspects of automatic monitoring and maintenance of a power system.