801 results about "Audio segment" patented technology

A method for early diagnosis of Parkinson's disease based on multimodal deep learning is provided. Audio-visual data of a to-be-diagnosed subject while performing a speech task is acquired. The audio-visual data are preprocessed to extract a plurality of audio segments and a plurality of video segments. A face image sequence is extracted from each of the plurality of video segments. A Mel-spectrogram of each of the plurality of audio segments is calculated. The face image sequence and the Mel-spectrogram are input into a multimodal deep learning model to output a classification result for Parkinson's disease early diagnosis of the to-be-diagnosed subject. A system for early diagnosis of Parkinson's disease based on multimodal deep learning is also provided.

The invention discloses an audio stream adaptive decryption method, system and device and a storage medium. The method comprises the following steps: acquiring an MPEG-DASH list file and analyzing the MPEG-DASH list file; the standard DRM interface is initialized; performing feature extraction on the identified audio segmentation information in the audio segmentation mode, and dynamically adjusting a segmentation request and processing logic according to the extracted features to perform adaptive segmentation downloading and preprocessing; extracting the encryption parameter to identify an encryption mark of the target audio stream platform, and selecting a corresponding decryption algorithm according to the encryption mark to decrypt the current audio segment; and performing audio frame reconstruction on the decrypted data to form an updated audio stream, caching the updated audio stream to a preset cache region based on a self-adaptive buffer strategy, and outputting the final audio stream to ensure the continuity of the final audio stream and realize efficient parallel processing of decryption and playing, so that the decryption efficiency is improved, and the user experience is improved. And particularly, in a high-concurrency scene, the user experience is improved.

The invention relates to the technical field of audio transmission systems, and provides a noise interference compensation method applied to an audio transmission system, and the method comprises the steps: firstly obtaining a noise-containing audio signal set of a target audio transmission channel, the noise-containing audio signal set comprises a plurality of noise-containing audio segments and pure audio segments corresponding to the noise-containing audio segments; then noise interference feature extraction is carried out on the noise frequency signal set to obtain a time domain noise feature and a frequency domain noise feature of each noise frequency segment, and then dynamic feature fusion is carried out on the time domain noise feature and the frequency domain noise feature based on a preset noise compensation model to generate a noise interference feature set; the method comprises the following steps: acquiring a noise interference feature set, training a noise compensation model according to a mapping relation between the noise interference feature set and a corresponding pure audio clip, generating a noise compensation parameter set, finally loading the noise compensation parameter set to a real-time processing module, and performing noise interference compensation operation on a noise-containing audio signal transmitted in real time. Therefore, the influence of noise on audio signals is effectively reduced, and the audio transmission quality is improved.

Methods and systems for generating training data for training a contrastive language-audio machine-learning model. A plurality of audio segments are retrieved from a speech emotion recognition (SER) database along with metadata associated with the audio segments. The metadata of each audio segment includes an emotion class. Words or terms associated with emotions are retrieved from a lexicon. A large language model (LLM) is executed on (i) the classes of emotion associated with the audio segments and (ii) the words or terms from the lexicon. This generates a plurality of text captions associated with emotion, which are stored in a caption pool. For each audio segment retrieved from the SER database, that audio segment is paired with one or more of the text captions from the caption pool that were generated based on the emotion class associated with that audio segment. This yields audio-text pairs for training a contrastive learning model.

The invention provides a voice recognition method and device, electronic equipment and a computer readable storage medium, and the method comprises the steps: carrying out the voice activity detection of to-be-processed voice data, and obtaining a plurality of voice segments; determining the number of speakers in each voice segment; under the condition that the number of speakers in the voice segment is multiple, performing voice separation on the voice segment to obtain multiple audio segments; performing voice recognition on the plurality of audio segments to obtain a voice recognition result of each audio segment; performing feature extraction on the plurality of audio segments to obtain a voiceprint feature vector of each audio segment; the speakers are clustered based on the voiceprint feature vectors of the multiple audio segments, a clustering result is obtained, and the clustering result comprises the audio segment corresponding to each speaker; and obtaining a voice recognition result of each speaker in the to-be-processed voice data based on the clustering result and the voice recognition result of each audio segment. According to the embodiment of the invention, multi-channel multi-speaker speech recognition can be realized.

In various examples, a technique for audio-driven facial animation with adaptive speech includes determining that a rate of speech associated with an audio segment exceeds a threshold. The technique also includes based at least on the rate of speech exceeding the threshold, upsampling a first set of features associated with the audio segment into a second set of features that is different in size than the first set of features. The technique further includes generating, using one or more machine learning models and based at least on at least a subset of the second set of features, a facial animation output corresponding to the audio segment.

The invention relates to the field of voice recognition and analysis, in particular to a voice interaction recognition and analysis method and system based on a Bluetooth headset. The method comprises the following steps: identifying environment audio information of a current wearer based on a Bluetooth earphone; carrying out noise space distribution modeling on the environment audio information of the current wearer, and constructing an environment noise real-time distribution field; carrying out noise distribution change trend analysis on the environment noise real-time distribution field, and carrying out dynamic audio receiving adaptation calculation to obtain an optimal audio adaptation receiving parameter; carrying out real-time interactive audio perception on the wearer according to the optimal audio adaptive receiving parameters, carrying out sound quality detail enhancement processing, and constructing a sound quality detail enhanced audio signal; and performing deep audio semantic analysis and instant semantic ambiguity analysis on the sound quality detail enhanced audio signal, and marking an audio segment of semantic ambiguity. According to the invention, accurate and efficient voice interaction recognition analysis is realized.

The invention is a divisional application of 202311870114.7, and discloses a speech synthesis method, which comprises the following steps: dividing a target text into a plurality of target text segments according to a text sequence; inputting the first target text segment in the plurality of target text segments into a speech synthesis model to obtain a first target audio segment; splicing the first target text segment and the first target audio segment to obtain a text-audio alternating sequence; acquiring a next target text segment, and adding the next target text segment to the text-audio alternating sequence; inputting the text-audio alternating sequence into a speech synthesis model to obtain a next target audio segment of the next target text segment, and adding the next target audio segment into the text-audio alternating sequence until a final target audio segment corresponding to the final target text segment is obtained; and splicing the target audio segments to generate a target audio. According to the invention, text understanding can be carried out by using contexts of texts and audios, and rhythms, melodies and emotions in a scene can be captured.

The invention discloses a video editing method and device, electronic equipment and a nonvolatile storage medium. The method comprises the following steps: acquiring an audio data stream of a target video, and segmenting the audio data stream into a plurality of audio clips; determining a classification result corresponding to the audio clip, and determining a time period corresponding to the audio clip as a candidate ball hitting time period when the classification result is that the audio clip contains the ball hitting sound; acquiring a video frame corresponding to the candidate ball hitting time period in the target video, and judging that the candidate ball hitting time period is a real ball hitting time period under the condition that the visual feature of the video frame is matched with a preset ball hitting rule; and determining an editing time point according to the real ball hitting time period, and editing the target video according to the editing time point to obtain a ball hitting round video clip. The technical problem that the accuracy of ball hitting detection segment detection is low due to the fact that ball hitting detection is conducted only through sound and is easily interfered by environmental noise in the prior art is solved.

The invention provides an audio-to-text conversion method and device, electronic equipment and a storage medium, and belongs to the technical field of voice recognition, and the method comprises the steps: carrying out the feature extraction of a target audio, and obtaining an audio feature used for representing the content of the target voice; performing segmentation processing based on the time sequence of the audio features to obtain a plurality of audio segments; generating a plurality of text segments based on the plurality of audio segments; and adding corresponding punctuation marks to the plurality of text segments based on the semantic model. According to the audio-to-text conversion method and device, the electronic equipment and the storage medium provided by the invention, high-precision text output can be realized.

The embodiment of the invention provides a speech translation method and device, equipment and a storage medium. The method comprises the following steps: in response to a first audio clip in a source language, converting the first audio clip into first audio embedding; in response to determining that the at least one audio clip is obtained before the first audio clip and / or at least one translation clip in a target language corresponding to the at least one audio clip, constructing a first model input based on the first audio feature embedding, at least one audio embedding corresponding to the at least one audio clip, and at least one translation embedding corresponding to the at least one translation clip; and using the trained machine learning model to generate a first translation segment in the target language corresponding to the first audio segment based on the first model input.

The present disclosure provides a speech synthesis model training method. The method can comprise: acquiring initial training data; selecting a plurality of first text segments, second text segments, and third text segments from coherent text, and acquiring first audio segments, second audio segments, and third audio segments from coherent audio; detecting whether the plurality of audio segments and text segments meet a stitching condition; on the basis of the text sequence, stitching, by alternatively arranging text and audio, the plurality of text segments and audio segments meeting the stitching condition to generate combined training data, so as to generate a combined training data set; on the basis of the combined training data set, training the initial speech generation model to obtain a trained speech synthesis model, wherein during the training, prosodic, tonal, and / or emotional features from the second text segments and / or the third text segments are extracted. Further disclosed is a speech synthesis method. The present disclosure allows for use of the context of the text and audio to realize text comprehension to capture prosody, tonality, and emotion in the context.

The invention discloses a conference voice translation method and device and electronic equipment, the method is applied to the field of artificial intelligence, and the method comprises the steps: collecting original voice information in a conference through target equipment, and extracting an audio clip with sound activity from the original voice information through a voice activity detection model; performing voice recognition on the effective voice information in the audio clip with the sound activity to obtain first text information; and determining a to-be-translated target language type of the target user, and translating the first text information through the target translation model to obtain target text information. Through the conference recording method and device, the problem that the translated conference record is inaccurate due to the fact that traditional conference recording mainly depends on manual recording and translation and is easily influenced by subjective factors of a recorder in the prior art is solved.

The invention is a divisional application of 202311870114.7, and discloses a speech synthesis method, which comprises the following steps: dividing a target text into a plurality of target text segments according to a text sequence; inputting a final target text segment in the plurality of target text segments into the speech synthesis model to obtain a final target audio segment; splicing the final target text segment and the final target audio segment to obtain a text-audio alternating sequence; obtaining a previous target text segment, and adding the previous target text segment to the text-audio alternating sequence; inputting the text-audio alternating sequence into a speech synthesis model to obtain a previous target audio segment of a previous target text segment, and adding the previous target audio segment to the tail of the text-audio alternating sequence until a first target audio segment corresponding to the first target text segment is obtained; and splicing the target audio segments to generate a target audio. According to the invention, text understanding can be carried out by using contexts of texts and audios, and rhythms, melodies and emotions in a scene can be captured.

The invention relates to a conference summary summarization method based on an AI large model, and belongs to the technical field of voice processing, and the technical scheme specifically comprises the following steps: obtaining audio information, recognizing all spokesmen involved in the audio information by using a preset spokesman separation algorithm, and splitting the audio information into a plurality of audio segments according to the spokesmen; analyzing and processing each audio segment by using a preset voice transcription algorithm to generate a transcription text; and based on a preset large language model, extracting and summarizing the transcriptional text, and generating and outputting summarized contents for describing the audio information, so that a user can know the summarized contents. The method has the effect of optimizing the information definition and the text readability of the transcribed text.

Techniques for performing automatic speech recognition (ASR) processing are described. The ASR processing may involve use of a segmenter and a decoder. The segmenter may be configured to identify audio segments, from audio data containing speech, based on word boundaries. The decoder may be configured to generate multiple word hypotheses for individual audio segments. A fixed-size representation of an audio segment may be generated. The described ASR processing techniques may be computationally less expensive than at least some other systems. Also, the described ASR processing techniques may maintain a larger number of word predictions per audio segment as compared to at least some other systems.

Video processing using artificial intelligence is disclosed. An embedded interface including products for sale is accessed. A user requests an interaction based on one of the products. The embedded interface initiates a video segment including a synthetic human in response to the user request. The user submits a question or comment. The interface collects the user input and converts it into a dataset readable by a large language model (LLM). The LLM generates a response to the user request. The response is used to generate an audio stream. The audio stream includes simulated human speech errors and pauses. The audio stream is segmented and a video clip is synthesized for each audio segment. The video clips are assembled into a new video segment which is presented to the user. Additional user interactions are collected and new video segments are generated in response.

The embodiment of the invention discloses a target voice regulation and control method, device and equipment based on intelligent glasses, belongs to the technical field of intelligent wearing, and solves the problem that in the prior art, a user is difficult to obtain a voice signal meeting the own requirement due to the fact that standardization processing is generally performed on a voice signal. Comprising the following steps: acquiring audio information of a speaker in a current scene, and dividing the audio information into a plurality of audio clips; acquiring portrait information in the current scene, and associating the portrait information with the corresponding audio clip to determine a target speaker; determining corresponding sound preference information of the user in the current scene based on a mapping relation between the noise level and intelligent glasses use preference data; analyzing the sound of the target speaker, responding to a dynamic sound adjustment strategy based on the obtained sound characteristics, and optimizing the sound of the target speaker; and based on the sound preference information, performing frequency band adjustment on the optimized sound through a power amplifier arranged on the intelligent glasses to realize target voice regulation and control.

The invention is suitable for the technical field of intelligent voice processing, and provides a language classification model training method, a language recognition method and device and intelligent equipment. The language classification model training method comprises the following steps: acquiring a first acoustic feature, wherein the first acoustic feature refers to an acoustic feature of an audio clip; the acoustic features of the audio clips are sequentially input into N layers of encoders of an initial language classification model, first encoding features output by the N layers of encoders respectively are obtained, and N is an integer larger than 1; inputting each first coding feature into a fusion module of the initial language classification model to obtain a first fusion feature; and based on the first fusion feature and a language recognition module of the initial language classification model, training the initial language classification model to obtain a target language classification model, the target language classification model being used for recognizing a language category of the audio data. According to the invention, the accuracy and robustness of language recognition can be improved.

The invention discloses an audio processing method and device, electronic equipment and a computer readable medium, and relates to the technical field of audios, and the method comprises the steps: obtaining a target data segment of a to-be-processed audio, the target data segment comprising continuous audio data of a first number of frames; inputting the complex frequency spectrum features corresponding to the target data segment into a track division music extraction model to obtain the complex frequency spectrum features of the target audio track as first frequency spectrum features; and obtaining a sub-audio segment corresponding to the target audio track based on the first spectrum feature. Therefore, the complexity of processing the data each time is reduced, the time delay is reduced, the complex spectral features of the audio are used as input, the amplitude and the phase are predicted at the same time, and unnecessary time delay and computing power consumption are avoided.

Methods and apparatus for generating a synchronized video-audio pair. According to an example embodiment, a method of generating a synchronized video-audio pair includes: applying one or more text inputs to a generative model, the one or more text inputs including a speech text, the generative model including a neural network; with the generative model, converting the speech text into an audio segment for the synchronized video-audio pair; and with the generative model, generating a video segment for the synchronized video-audio pair, the video segment including a talking head having lip movements corresponding to the speech text and in synchronization with the audio segment.

The invention discloses a speech recognition method, a speech recognition device and a speech recognition system. The speech recognition method provided by the embodiment of the invention comprises the following steps: acquiring audio stream data; in the process of acquiring the audio stream data, acquiring a multi-modal endpoint signal corresponding to the audio stream data; segmenting the audio stream data based on the multi-modal endpoint signal and a voice activity detection model to obtain at least one audio segment; performing streaming speech recognition on the at least one audio segment based on an offline speech recognition model to obtain a recognition text; the recognized text is streamed. Therefore, the delay of outputting the recognition text is reduced, the occupation of hardware resources is reduced, and the method can be suitable for a real-time interaction scene and a scene with a relatively high recognition precision requirement.

The invention relates to a voice interaction large model family health assistant dialogue method and device, equipment and a medium. The method comprises the following steps: carrying out fragmentation processing according to an original voice stream of a user to generate an audio fragment with a medical mark, and carrying out voice recognition and entity extraction on the audio fragment to generate a dynamic entity map; generating an evidence-based decision prompt based on the map, inputting the prompt into a preset medical big model for processing, and outputting a result containing an essential symptom list; and according to the symptom matching degree of the necessary symptom list and the dynamic entity map, generating a diagnosis report or a question-asking list, if the diagnosis report is output, performing medical rule chain verification operation on the diagnosis report to generate a quality control report, and based on the question-asking list or the quality control report, generating a synthetic voice stream. According to the method, through medical intention directional screening, map entity analysis, large model diagnosis, voice synthesis and the like, the voice recognition accuracy, the diagnosis suggestion reliability and the inquiry interaction efficiency of the family health assistant in the medical scene are improved.

The invention discloses a speaker role separation method, related equipment and a computer program product, and the method comprises the steps: determining a transcription result of target audio data, the transcription result comprising a transcription text and speaker turning point information; and carrying out time alignment on the transliteration text and the target audio data, and segmenting to obtain a single-person audio clip. The voiceprint information is extracted by sliding the window in each single-person audio clip, and the single-person audio clip only contains the content of a single speaker, so that the window length can be increased without worrying about the risk that multiple persons speak in the window, and the voiceprint accuracy can be improved through the larger window length. All sliding window audios are clustered according to voiceprint information in a global range, text fragments corresponding to the sliding window audios of the same cluster are endowed with the same speaker identity, text fragments corresponding to the sliding window audios of different clusters are endowed with different speaker identities, the accuracy of speaker role separation on a transliteration text level can be improved, and the user experience is improved. The method is especially suitable for long audio scenes.

The invention discloses a bipartite graph matching-based speaker recognition method and electronic equipment, and belongs to the technical field of audio recognition. The method comprises the following steps: acquiring an audio stream, and segmenting the audio stream into continuous target duration audio clips in real time; determining a voiceprint embedding vector corresponding to each audio clip of the target duration; determining an embedding matrix based on the voiceprint embedding vector; determining a target voiceprint feature corresponding to each target duration audio clip according to a bipartite graph method; calculating the similarity between the voiceprint embedding vectors; and grouping the voiceprint embedding vectors based on the similarity between the voiceprint embedding vectors, and clustering the voiceprint embedding vectors corresponding to the same speaker to obtain a clustering result and generate a target recognition report. According to the invention, the calculation complexity can be reduced and the identification precision can be improved.

In some embodiments, an exemplary inventive system for improving computer speed and accuracy of automatic speech transcription includes at least components of: a computer processor configured to perform: generating a recognition model specification for a plurality of distinct speech-to-text transcription engines; where each distinct speech-to-text transcription engine corresponds to a respective distinct speech recognition model; receiving at least one audio recording representing a speech of a person; segmenting the audio recording into a plurality of audio segments; determining a respective distinct speech-to-text transcription engine to transcribe a respective audio segment; receiving, from the respective transcription engine, a hypothesis for the respective audio segment; accepting the hypothesis to remove a need to submit the respective audio segment to another distinct speech-to-text transcription engine, resulting in the improved computer speed and the accuracy of automatic speech transcription and generating a transcript of the audio recording from respective accepted hypotheses for the plurality of audio segments.

The invention discloses an audio processing method and device, a storage medium and an electronic device, and relates to the technical field of data processing, and the audio processing method comprises the steps: obtaining voice feature data and semantic feature data of original audio data; the original audio data is audio data extracted from the target audio and video data; generating segmentation position data according to the voice feature data and the semantic feature data; and generating audio paragraph data and paragraph visualization data according to the segmentation position data, and providing the paragraph visualization data to a user interaction interface. According to the technical scheme of the embodiment of the invention, the utilization rate of the audio information can be improved, the data processing cost in an application scene is remarkably reduced, and the data application efficiency and scene adaptability are improved.

Embodiments of the present disclosure relate to a method and apparatus for synthesizing a singing voice, an electronic device and a program product. The method comprises obtaining a musical score file with breath-taking identifiers, and segmenting the musical score file into a plurality of musical score segments based on the breath-taking identifiers. The method further comprises generating a plurality of audio segments corresponding to the plurality of musical score segments, and synthesizing a singing voice corresponding to the musical score file based on the plurality of audio segments. In an embodiment of the present disclosure, the musical score file is segmented into a plurality of musical score segments according to the breath-taking identifier, a plurality of audio segments corresponding to the plurality of musical score segments are generated, and finally, a singing voice corresponding to the musical score file is synthesized based on the plurality of audio segments.

The present invention relates to the field of computer vision technology, and discloses a virtual digital human lip synchronization optimization method, device, equipment and storage medium. The virtual digital human lip synchronization optimization method includes: obtaining the target audio segment to be output by the virtual digital human at the next moment; judging whether the target audio segment belongs to the audio type to be processed; if the target audio segment belongs to the audio type to be processed, then based on a preset lip synchronization optimization strategy, generating a 3D human face mouth shape parameter frame sequence corresponding to the target audio segment; based on the 3D human face mouth shape parameter frame sequence, generating a corresponding 3D human face mouth shape image frame sequence and rendering it into the virtual digital human. The present invention can adapt to various audio types and improve the fluency and naturalness of the virtual digital human's mouth shape under different audio types.