236 results about "Voice activity detection" patented technology

The invention provides an intelligent sound box voice processing method and system based on artificial intelligence, and the method comprises the steps: obtaining audio data and mouth shape video data, carrying out the processing of the audio data and the mouth shape video data, and carrying out the multi-modal feature fusion, and obtaining a fusion feature; performing bimodal voice activity detection on the fusion features to obtain effective voice data; performing context sensing recognition of audio and video fusion on the effective voice data to obtain a first text; constructing a user feature model, and performing semantic understanding on the text based on the model to obtain an understanding result; performing intention recognition and slot filling based on the understanding result to obtain user intention and key information; generating a response strategy in combination with the user intention, the key information and the environment perception data; generating response voice according to the response strategy; and monitoring feedback information of the user to the response voice in real time, and updating the user feature model and the response strategy evaluation model based on feedback. According to the scheme, the voice can be recognized more accurately, and the safety and robustness of the system are enhanced.

The invention relates to the technical field of voice processing, can be applied to business scenes of financial science and technology, medical health and the like, and discloses a voice intention recognition method, device and equipment and a medium, and the method comprises the steps: obtaining a to-be-processed voice signal, carrying out the voice activity detection processing of the voice signal, dividing the voice signal into a plurality of voice segments, analyzing semantic contents of the plurality of voice segments, determining semantic correlation information of each voice segment, analyzing sound source attributes of the plurality of voice segments, determining sound field type information of each voice segment, screening out a target voice segment from the plurality of voice segments according to the semantic correlation information and the sound field type information, and executing intention recognition processing based on the target voice segment to generate an intention recognition result. According to the invention, through a dual analysis mechanism of semantic correlation information and sound field type information, effective screening of voice segments is realized before voice recognition, non-target voice or interference segments are effectively prevented from being sent to an intention recognition model, and the accuracy of a recognition result is improved.

The invention relates to the technical field of voice recognition, in particular to a multi-channel voice conversion and synchronous transmission system applied to simultaneous interpretation. Multi-language voice signals are collected through a multi-microphone array, background noise is dynamically eliminated by adopting a noise suppression technology combining spectral subtraction and deep learning, and a dynamic information source verification symbol is generated to ensure data synchronization integrity; a blind source separation technology is combined with time-frequency analysis and time delay estimation to realize multi-language signal separation and synchronization, and a time sequence is dynamically adjusted through voice activity detection; an end-to-end ASR-NMT-TTS model is constructed to realize voice real-time translation and synthesis, and low-delay transmission is carried out based on a 5G network; the real-time monitoring module is adopted to dynamically adjust the output delay and the signal-to-noise ratio, and the translation delay and the synchronization precision are optimized in combination with user feedback. According to the method, dynamic noise reduction, multi-source synchronization and a self-adaptive feedback mechanism are integrated, the problems of distortion and delay of multi-language simultaneous transmission in a complex noise environment are solved, and the obvious technical synergistic effect and practicability are achieved.

The invention discloses a zero-configuration adaptive speaker recognition method and system, and relates to the technical field of voice signal processing. The method comprises the following steps: receiving an audio stream, carrying out voice activity detection, obtaining a single-person voice segment, extracting a voiceprint embedding vector of the single-person voice segment, and carrying out online clustering to generate a speaker identity pool; and the identity pool is updated by calculating the multi-dimensional fusion similarity between the remaining single-person voice segments and the voice model, and temporary identity tags corresponding to the single-person voice segments are output. According to the speaker recognition method provided by the invention, a voiceprint template does not need to be registered in advance, and the flexibility and real-time performance of the system are improved.

A system for detecting speech from reverberant signals is disclosed. The system is programmed to receive spectral temporal amplitude data in the modulation frequency domain. The system is programmed to then enhance the spectral temporal amplitude data by reducing reverberation and other noise as well as smoothing based on certain properties of the spectral temporal spectrogram associated with the spectral temporal amplitude data. Next, the system is programmed to compute various features related to the presence of speech based on the enhanced spectral temporal amplitude data and other data in the modulation frequency domain or in the (acoustic) frequency domain. The system is programmed to then determine an extent of speech present in the audio data corresponding to the received spectral temporal amplitude data based on the various features. The system can be programmed to transmit the extent of speech present to an output device.

Disclosed in the present invention are an automatic delay calibration method for a smart speaker system, a device, and a storage medium. The method comprises: initializing a smart speaker system and performing environmental testing to preliminarily configure system parameters; exciting a target sound channel and playing a target test signal; locating a first test signal on the basis of a voice activity detection (VAD) algorithm, determining the start and end of the first test signal, and identifying a time difference of the first test signal to obtain a delay of main channels; locating a time of arrival of a second test signal on the basis of a fast peak search algorithm, and identifying a delay of a subwoofer; and performing delay calibration on the subwoofer and the main channels, and dynamically adjusting delay calibration parameters of the channels on the basis of a delay calibration amount, thereby achieving accurate time alignment between the subwoofer and the main channels. The present invention solves the problem of low-frequency signal delays that are difficult to process in conventional technologies, and particularly makes a breakthrough progress in the coordination of a subwoofer and other sound channels.

Techniques and apparatuses are described for performing internal-noise-source filtering for active acoustic sensing. With internal-noise-source filtering (202), interference caused by a hearable (102) performing other operations (e.g., rendering audio content (204), performing active noise cancellation (206), or operating in accordance with a transparency mode (214)) can be attenuated within a received ultrasound signal to improve sensitivity and accuracy for audioplethysmography. This performance improvement enables audioplethysmography to be performed while the hearable (102) performs these other operations. Furthermore, it improves the ability of audioplethysmography to be used for voice processing, which can include voice activity detection, speech recognition, and / or conversation detection.

The invention discloses a speech recognition method, a speech recognition device and a speech recognition system. The speech recognition method provided by the embodiment of the invention comprises the following steps: acquiring audio stream data; in the process of acquiring the audio stream data, acquiring a multi-modal endpoint signal corresponding to the audio stream data; segmenting the audio stream data based on the multi-modal endpoint signal and a voice activity detection model to obtain at least one audio segment; performing streaming speech recognition on the at least one audio segment based on an offline speech recognition model to obtain a recognition text; the recognized text is streamed. Therefore, the delay of outputting the recognition text is reduced, the occupation of hardware resources is reduced, and the method can be suitable for a real-time interaction scene and a scene with a relatively high recognition precision requirement.

The invention provides a real-time translation method and interaction system based on streaming voice segmentation and semantic verification. The method comprises the following steps: receiving a conference audio stream, identifying effective audio frames based on dual-channel voice activity detection, and pressing the effective audio frames into a dynamic buffer area; according to a preset dynamic segmentation strategy, outputting an initial voice segment from the dynamic buffer area for voice recognition, and obtaining a corresponding initial text segment; performing multi-stage reliability verification on the initial text fragment, and dynamically correcting or complementing the initial text fragment according to a verification result to obtain a reliable recognition result; and after the reliable identification result is obtained, triggering an asynchronous parallel translation task. According to the method, a semantic correction mechanism cooperating with the adaptive truncation depth is designed, so that the semantic fragmentation problem of the long-sequence audio during streaming truncation is solved, and the translation accuracy of a complex word order language is greatly improved while low delay is ensured.

Techniques and devices for performing internal noise source filtering for active acoustic sensing are described. With internal noise source filtering (202), interference caused by the hearing-worn device (102) performing other operations (e.g., rendering audio content (204), performing active noise cancellation (206), or operating according to a pass-through mode (214)) may be attenuated within the received ultrasound signal to improve the sensitivity and accuracy of audio volumetric tracing. This performance improvement enables audio plethysmography to be performed when the audible wearable device (102) performs these other operations. Further, it improves the ability of audio plethysmography for speech processing, which may include speech activity detection, speech recognition, and / or dialog detection.

The invention provides a low-delay real-time voice transcription method and system, and the method comprises the following steps: A, carrying out the coding compression of original audio data collected by a client, transmitting the obtained audio coding data to a server through a real-time transmission protocol, introducing an error correction and packet loss retransmission mechanism, and improving the transmission reliability; b, detecting voice in the audio data by a voice activity detection algorithm at the server side, dividing the audio data judged as the voice into a plurality of independent parts, and adding a time sequence identifier for each part; and C, performing real-time transcription on the segmented audio data according to a time sequence to generate corresponding text content.

The invention relates to a voice activity detection method and device, computer equipment and a storage medium. The method comprises the following steps: acquiring audio sampling data, and extracting acoustic features from the audio sampling data; inputting the acoustic features into a convolutional neural network to obtain a first intermediate feature vector; inputting the first intermediate feature vector into a graph convolutional neural network for spatial relationship construction to obtain a second intermediate feature vector containing spatial relationship information among a plurality of feature channels; inputting the second intermediate feature vector into a multi-head attention mechanism network, and performing multi-head attention calculation based on rotation position coding on the second intermediate feature vector through the multi-head attention mechanism network to obtain a third intermediate feature vector; and performing voice activity detection according to the third intermediate feature vector. By adopting the method, the accuracy of voice activity detection can be improved.

A method for reducing voice noise, including: acquiring an algorithm activity detection result corresponding to a current audio frame to be processed by detecting the current audio frame using a predetermined voice activity detection algorithm; acquiring a target activity detection result corresponding to the current audio frame by merging a model activity detection result corresponding to a previous audio frame and the algorithm activity detection result corresponding to the current audio frame, wherein the model activity detection result is outputted by a predetermined voice noise reduction network model; acquiring an initial noise reduction audio frame by performing, based on the target activity detection result, noise estimation and noise elimination on the current audio frame; and outputting a target noise reduction audio frame and a model activity detection result corresponding to the current audio frame by inputting the initial noise reduction audio frame into the predetermined voice noise reduction network model.

A voice activity detection method includes a voice pickup module, a processing module coupled to the voice pickup module and a prompt module coupled to the processing module. The processing module is used to perform a voice activity detection method, including: receiving voice information by the voice pickup module; acquiring a volume value of the voice information by the processing module; determining by the processing module whether the volume value is less than or equal to a first volume threshold; when it is determined that the volume value is less than or equal to the first volume threshold, generating by the prompt module a first prompt message for increasing the volume value; and when it is determined that the volume value is more than the first volume threshold, generating by the prompt module a second prompt message that indicates a volume value standard has been satisfied.

The invention provides an air traffic control dialogue type speech recognition method and device based on multi-task learning. Cooperative optimization of voice segmentation and content recognition is realized by constructing a joint learning model of a voice feature representation learning module and a voice activity detection module. The method comprises the following steps: firstly, designing a segmented shielding comparison pre-training strategy, and constructing a supervision signal through a product quantization codebook, so that a speech feature representation learning module learns fine-grained acoustic features in unlabeled data; secondly, a double-branch joint training mode fine tuning model is designed, a main branch optimizes a voice recognition task through connection time sequence classification loss, an auxiliary branch improves voice segment boundary detection precision through binary cross entropy loss, and meanwhile, a dynamic task weight adjustment strategy is combined to balance a multi-target optimization direction; and finally, innovatively introducing a dynamic local window attention mechanism, and focusing context modeling of an effective voice segment according to a real-time segmentation index.

Techniques for automatically de-identifying sensitive information in audio conversations by combining un-transcribed voice activity detection (VAD) with large language model (LLM) analysis are disclosed. An audio de-identification system processes speech-to-text transcriptions while identifying segments where automatic speech recognition (ASR) failed to transcribe spoken content. These un-transcribed segments are represented as placeholders in prompts sent to an LLM, which analyzes the surrounding textual context to determine if sensitive information (such as PII or PHI) was likely spoken during these gaps. When sensitive content is identified, the system modifies the corresponding audio segments through an audio identification tactic. This approach addresses the technical challenge of incomplete de-identification in automated audio processing by leveraging LLMs' contextual understanding to detect sensitive information in segments that traditional ASR systems miss, particularly in scenarios involving poor audio quality or diverse accents. The result is a more comprehensive and reliable audio de-identification system.

The present disclosure relates to the technical field of artificial intelligence, and provides a method for training an audio processing model, an apparatus for training an audio processing model, a computer storage medium, and an electronic device. The method for training an audio processing model comprises: acquiring a training sample set; using a first sample set to pre-train a first branch network of an audio processing model to be trained, so as to obtain a pre-trained first branch network, and using a second sample set to pre-train a second branch network of said audio processing model, so as to obtain a pre-trained second branch network; and using the training sample set to jointly train the pre-trained first branch network and the pre-trained second branch network, so as to obtain a trained audio processing model, wherein the first branch network is used for executing echo cancellation and speech enhancement tasks, and the second branch network is used for executing a voice activity detection task. In the present disclosure, multiple audio processing tasks can be executed by means of one model, thereby reducing system power.

The invention provides a conference summary generation method and device, and the method comprises the steps: obtaining an audio and video file of a target conference, and the audio and video file comprises an audio file and / or a video file; performing voice activity detection processing, text conversion processing and speaker recognition processing on the audio and video file to obtain a target dialogue text; constructing prompt words based on the application scene of the target conference and the target dialogue text; and inputting the cue word into a conference summary generation large model, and generating a conference summary corresponding to the target conference. According to the invention, the problem of low conference summary generation efficiency is solved, and more time is saved; the generation result of the conference summary generation large model is optimized by adopting a prompt word engineering mode, so that the accuracy of generating the conference summary can be further improved; the conference summary is generated by adopting the conference summary generation large model, compared with a traditional neural language programming technology, feature engineering construction is omitted, and the performance is better.

Disclosed are a method, system, and apparatus of a real-time multilingual transcription system and method. In one embodiment, a method includes continuously capturing an audio data and segment it into short segments; implementing a pre-trained enterprise-grade voice activity detection (“VAD”) system on each of the short segmental and filtering out non-speech segments to reduce computational waste, focusing resources on relevant audio data and minimizing latency. If speech is detected, a particular short segment is added to a processing queue. If speech is not detected, declining to add the particular segment to the processing queue, thereby reducing unnecessary processing.

The invention discloses a live broadcast subtitle generation method and a related device, and relates to the field of live broadcast, and the method comprises the steps: obtaining live broadcast audio stream data, carrying out the streaming voice recognition to obtain a streaming subtitle text, segmenting the live broadcast audio stream data based on the audio short-time energy and the voice activity detection confidence, and obtaining a plurality of audio blocks, and performing offline voice recognition to obtain an offline subtitle text, performing timestamp alignment on the streaming subtitle text and the offline subtitle text to obtain a target streaming subtitle text and a target offline subtitle text, and correcting the target streaming subtitle text by using the target offline subtitle text to obtain live subtitles. According to the invention, the streaming voice recognition technology is utilized to preliminarily recognize the live audio stream data, and then the offline subtitle text is obtained by the offline voice recognition to correct the streaming voice recognition result, so that the real-time response of the streaming voice recognition and the collaborative output of the offline high-precision corrected voice recognition result are realized. And the balance between the real-time performance and the accuracy of the live subtitles is achieved.

The invention discloses a cough sound recognition method based on a PSO-GBDT-LR model, and belongs to the technical field of signal recognition. The method includes acquiring audio signals; de-noising the audio signal by using a Berouti spectral subtraction method to obtain a de-noised audio signal; the audio event detection VAD is used for segmenting the part, where the sound appears, of the audio; 7-dimensional time domain features are extracted from each segmented audio sample; performing short-time Fourier transform (STFT) on each segmented audio sample, and extracting two-dimensional frequency domain features from a frequency spectrum; combining the extracted 7-dimensional time domain features and the extracted 2-dimensional frequency domain features to form a 9-dimensional feature vector combination; and marking the feature vector of the cough audio sample as a class 1, and marking the feature vector of the non-cough audio sample as a class 0. According to the method, the problems of excessive noise features and abnormal features of the sound in the cough sound recognition process are solved, the features of the cough and non-cough sound can be accurately distinguished, and the generalization ability is high.

A wearable multifunction device or earpiece or a pair of earpieces includes one or more processors, at least one microphone coupled to the one or more processors, a biometric sensor coupled to the one or more processors, and a memory coupled to the one or more processors, the memory having computer instructions causing the one or more processors to perform the operations of sensing a remaining battery life and based on the sensing, prioritizing one or more of the functions of always on recording, biometric measuring, biometric recording, sound pressure level measuring, voice activity detection, key word detection, key word analysis, personal audio assistant functions, transmission of data to a tethered phone, transmission of data to a server, transmission of data to a cloud device.

The invention discloses an end-cloud collaborative three-level decision-making full duplex voice dialogue dynamic management and control method, and relates to the technical field of voice interaction control, and the method comprises the steps: an end side collects microphone audio, obtains loudspeaker reference audio, judges a broadcast state, and switches a voice activity detection strategy; end-side framing detection is carried out, voice segment events are output through a three-state state machine, characteristics such as duration, relative volume and broadcast overlapping degree are extracted, and candidate interruption events are generated in combination with an automatic voice recognition intermediate text; the end side coarse screening reports a gray area event to the cloud side semantic research and judgment, and pre-suppression, buffer foresight contraction and block buffer are executed during waiting; and the cloud side issues an instruction containing a period of validity, a deduplication key and an interruption type, and the execution side stops generating, synthesizing and playing and empties buffer so as to play a completion receipt, submit a dialogue history and roll back unplayed content, thereby reducing false triggering and missed triggering, shortening interruption delay, reducing residual broadcast and inhibiting context drift.

A voice activity detection method and system and a voice enhancement method and system are provided. A voice presence probability of a target voice signal present in microphone signals may be determined by calculating a linear correlation between a signal subspace where the microphone signals are located and a target subspace where the target voice signal is located. The voice enhancement method and system may be used to calculate filter coefficients based on the voice presence probability, so as to perform voice enhancement on the microphone signals. The calculation accuracy of the voice presence probability is improved, and the voice enhancement effect is also improved.

The invention provides an AI glasses automatic shooting method and system based on voice control, and relates to the technical field of intelligent wearable equipment, accurate interaction is realized through a multi-channel directional microphone array and an end-cloud collaborative voice recognition engine, the microphone array optimizes the pickup angle based on the wearing position characteristics of a user, and the user experience is improved. In combination with real-time voice activity detection, environmental noise is filtered out, it is ensured that a clear voice instruction can still be captured in a noisy environment, an end-cloud cooperation mode operates a lightweight model locally to guarantee the off-line response speed, a cloud large model is called when a network is available to improve the complex instruction analysis capability, and a composite statement containing parameter adjustment can be recognized; and an operation intention and parameters are automatically bound through a natural language processing engine, so that one-step execution of the instruction is realized.

A method performed by a first device, which includes performing an audio call with a second device by transmitting a microphone signal as an uplink signal and receiving a downlink signal for driving a first speaker and while performing the audio call, performing a joint media playback session in which both devices independently stream a piece of media content for synchronous playback such that both devices receive an audio signal of the piece of media content for driving respective speakers at the same time, determining that a voice activity detection (VAD) signal indicates that the downlink signal includes speech, in response to determining that the VAD signal indicates that the downlink signal includes speech, processing the audio signal of the piece of media content by applying a scalar gain, and driving the first speaker with a mix of the downlink signal and the audio signal.

The invention discloses a voice interaction control method and device, a safety helmet, a storage medium and a program product, and relates to the technical field of voice interaction. According to the intelligent safety helmet, a throat vibration sensor and a lip movement detection camera are integrated on the intelligent safety helmet, a throat vibration signal collected by the throat vibration sensor and a lip movement signal collected by the lip movement detection camera are obtained respectively, whether a wearer generates throat vibration or not is determined based on the throat vibration signal, and whether the wearer generates throat vibration or not is determined based on the throat vibration signal. And determining whether the wearer generates lip vibration or not based on the lip movement signal, and performing voice activity detection on the wearer when the wearer generates throat vibration and lip vibration. Therefore, according to the added throat vibration and lip vibration detection, the accuracy of the voice activity detection function integrated on the intelligent safety helmet is improved, and the probability of semantic misjudgment is reduced.

The invention relates to the field of teaching interaction quality evaluation, in particular to a teaching interaction quality evaluation method and system based on a large language model. The method comprises the following steps: audio transcription: converting classroom audio into an original transcription text through voice activity detection, speaker classification, automatic voice recognition and punctuation recovery; transcriptional refining: performing context-based text error correction on the original transcriptional text by using a large language model in combination with a preschool education field knowledge base to generate a refined transcriptional text; a quality evaluation step: based on a preschool education quality evaluation scale, using few sample example guidance and thinking chain reasoning for each scoring point, judging whether a voice segment conforming to the scoring point exists in the refined transcriptional text, performing binary scoring, and determining whether the voice segment conforms to the scoring point; and generating an interactive quality evaluation report containing the standard-reaching rate of each evaluation dimension, teaching bright spot analysis and staged teaching optimization suggestions. The evaluation efficiency is remarkably improved. The method is suitable for teaching interaction quality evaluation.

The invention relates to the technical field of robots, and particularly discloses a desktop robot microphone array sound source positioning system and method. Comprising a main control chip, acquisition microphones and a power supply module, an ESP32 main control board is connected with a circular array formed by the four acquisition microphones through double I2S bus pins, the array diameter of the four acquisition microphones is 50 mm, the included angle between every two adjacent acquisition microphones is 90 degrees, the acquisition microphones are grounded through GND lines, and the power supply module is connected with the ESP32 main control board. And the ESP32 main control board provides 3.3 V voltage for the acquisition microphone through a power supply line. A four-microphone circular array with the diameter of 50 mm and ESP32 double I2S synchronous acquisition are adopted, 360-degree full horizontal positioning is achieved, the hardware size is small, the miniaturization requirement of a desktop accompanying robot is met, a VAD voice activity detection module is introduced, algorithm operation is triggered only when human voice is detected, power consumption in a dormant state is small, and the endurance of the robot is prolonged.

The invention discloses an audio speaker recognition method and system, a storage medium and electronic equipment, and belongs to the field of audio processing and artificial intelligence, and the method comprises the steps: carrying out the preprocessing of an input audio, extracting the sound channel information of the input audio, and judging whether the audio type is a single sound channel or a double sound channel; for the dual-channel audio, judging whether the audio type is a pseudo dual-channel or a true dual-channel by comparing at least two acoustic characteristic parameters of a left channel and a right channel; selecting a processing strategy according to the audio type; sequentially performing noise reduction preprocessing, voice activity detection, speaker segmentation, content recognition and punctuation recovery on the selected sound channel; performing speaker role marking on an audio content recognition result by adopting a large language model; and combining the content fragments of the same speaker role to generate structured output. Through multidimensional acoustic feature analysis and cooperation of the deep learning model and the large language model, the recognition accuracy and the automation degree are improved, and the method can adapt to various complex audio scenes.