775 results about "Automatic speech" patented technology

A system includes a processor; a memory; an electronic network; an application including computing instructions configured to be executed by the processor; and a multilingual dialogue server accessible by the application, comprising a language detection model, and an automatic speech recognition model; wherein the computing instructions of the app, when executed by the one or more processors, are configured to cause the one or more processors to: receive input; analyze the input using the trained model; select a speech model; analyze the input; generate metadata; and transmit the metadata. A method includes receiving input; analyzing the input; selecting a speech model; analyzing the input; generating metadata; and transmitting the metadata. A non-transitory computer readable medium containing program instructions that when executed, cause a computer to: receive input; analyze the input; select a trained speech model; analyze the input; generate metadata; and transmit the metadata.

A neural end-to-end system is provided for the face and voice preserving translation of videos. The system is a pipeline of multiple models that produces a video of the original speaker speaking in the target language with modified lip movement to match the target speech, while preserving emphases and prosody of the original speech, and voice characteristics of the original speaker. The pipeline starts with automatic speech recognition including emphasis detection, followed by the translation model. The translated text is then synthesized by a Text-to-Speech model that recreates the original emphases in the target sentence. The resulting synthetic speech is then converted back to the original speakers' voice using a voice conversion model. Finally, to synchronize the lips of the speaker with the translated audio, a generative model generates frames of adapted lip movements which are combined with the audio to produce the final output. The disclosure further describes several use-cases and configurations that apply these techniques to video conferencing, dubbing, low-bandwidth transmission, speech enhancement and assistive technology for the hearing impaired.

One example method includes receiving, during a virtual conference from a client device, a request to generate a real-time summary of the virtual conference, a plurality of client devices participating in the virtual conference and exchanging audio and video streams; receiving, during the virtual conference, a plurality of utterances generated by automatic speech recognition (“ASR”) of the audio streams; generating a group of consecutive utterances from the plurality of utterances; determining a segment based on the group of consecutive utterances; generating, using a large language model (“LLM”), a segment summary based on the segment; and providing the segment summary to the client device.

A task-oriented dialog system determines an endpoint in a user utterance by receiving incremental portions of a user utterance that is provided in real time during a task-oriented communication session between a user and a virtual agent (VA). The task-oriented dialog system recognizes words in the incremental portions using an automated speech recognition (ASR) model and generates semantic information for the incremental portions of the utterance by applying a natural language processing (NLP) model to the recognized words. An acoustic-prosodic signature of the incremental portions of the utterance is generated using an acoustic-prosodic model. The task-oriented dialog system can generate a feature vector that represents the incrementally recognized words, the semantic information, the acoustic-prosodic signature, and corresponding confidence scores of the model outputs. A model is applied to the feature vector to identify a likely endpoint in the user utterance.

Techniques for personalized batch and streaming speech-to-text transcription of audio reduce the error rate of automatic speech recognition (ASR) systems in transcribing rare and out-of-vocabulary words. The techniques achieve personalization of connectionist temporal classification (CT) models by using adaptive boosting to perform biasing at the level of sub-words. In addition to boosting, the techniques encompass a phone alignment network to bias sub-word predictions towards rare long-tail words and out-of-vocabulary words. A technical benefit of the techniques is that the accuracy of speech-to-text transcription of rare and out-of-vocabulary words in a custom vocabulary by automatic speech recognition (ASR) system can be improved without having to train the ASR system on the custom vocabulary. Instead, the techniques allow the same ASR system trained on a base vocabulary to realize the accuracy improvements for different custom vocabularies spanning different domains.

The invention relates to the field of voice signal processing, and discloses a multi-level labeling method and device in a dialogue scene. Through the endpoint detection and automatic speech recognition method of the dialogue audio data, noise and mute in real speech can be removed, and effective speech segments can be obtained. And carrying out clustering analysis based on the effective voice segments, obtaining target voice segments and text data of the corresponding speakers, and carrying out emotion feature extraction through multi-modal emotion analysis. And finally, realizing automatic annotation of the inquiry strategy or intention understanding of the corresponding speaker by utilizing a large language model. According to the method, the recorded conversation voice can be subjected to multi-level labeling, an efficient and accurate information management and analysis tool is provided for a conversation scene, and the arrangement efficiency of conversation records can be greatly improved. The annotation information can be used for training and improving an artificial intelligence system, so that the artificial intelligence system plays a greater role in future psychological consultation service, and the overall quality and efficiency of psychological health service are improved.

The invention discloses a voice recognition model optimization method and device based on reinforcement learning, equipment and a medium, and relates to the technical field of voice recognition, and the method comprises the steps: determining a current optimization strategy of a current time step; randomly selecting one piece of enhanced speech training data and corresponding real text training data from the data training set as target training data, and inputting the target training data into the current automatic speech recognition model for prediction to obtain a plurality of prediction texts output by the current automatic speech recognition model; determining a text matching degree and a semantic similarity between each prediction text and real text training data to obtain a group of target reward values, updating an optimization strategy at the current moment based on the target reward values, and skipping to execute the step of determining the current optimization strategy of the current time step to obtain a target reward value; and outputting the corresponding optimized automatic speech recognition model until the current automatic speech recognition model meets the optimization ending condition. And reasonable optimization of the automatic speech recognition model is realized.

The technology relates to enhancing speech emotion recognition models with methods that enable the use of unlabeled data by inferring weak emotion labels. This is done by pre-trained large language models through weakly-supervised learning. For inferring weak labels constrained to a taxonomy, a textual entailment approach selects an emotion label with the highest entailment score for a speech transcript extracted via automatic speech recognition. The system may employ a method that generates, by one or more processors, a text transcript for a snippet of input speech, and then applies the text transcript to a pre-trained language model. The system can generate, using the pre-trained language model according to an engineered prompt and a predetermined taxonomy, a textual entailment from the text transcript. Based on this, the system may generate, by the one or more processors using the textual entailment, a predicted emotion corresponding to the input speech.

Techniques for processing with respect to a user input as contextual information is available are described. A system generates a first task prediction using first context data that is available when a user input is received. The system generates a second task prediction (e.g., updated first task prediction) when second context data is received, and then further generates a third task prediction when third context data is received. Example first context data may include device type information, time information, location, etc. Example second context data may include automatic speech recognition (ASR) data. Example third context data may include natural language understanding (NLU) data. Using the third task prediction, the system generates an output responsive to the user input.

The invention relates to the technical field of artificial intelligence and the field of financial science and technology, and discloses a method for adjusting the playing speed of voice content in a video, and the method comprises the steps: obtaining a video file containing to-be-adjusted voice content, carrying out the voice recognition of an audio track of the video file, and converting the audio data of the voice content into text data; dividing the text data into a plurality of sentences or phrases; identifying keywords in each sentence or phrase, performing emotion content classification on the sentences or phrases containing the keywords, and determining the importance level of each sentence or phrase; according to the emotion content and the importance level, adjusting the playing speed of the corresponding voice content, and generating adjusted audio data; aligning the adjusted audio data with the video image to generate synchronous audio and video data; and a final video is generated. Through automatic voice recognition, sentiment analysis and playing speed adjustment, the steps of manual participation are reduced, and the video editing efficiency is greatly improved.

Disclosed herein are systems and methods for processing speech signals in mixed reality applications. A method may include receiving an audio signal; determining, via first processors, whether the audio signal comprises a voice onset event; in accordance with a determination that the audio signal comprises the voice onset event: waking a second one or more processors; determining, via the second processors, that the audio signal comprises a predetermined trigger signal; in accordance with a determination that the audio signal comprises the predetermined trigger signal: waking third processors; performing, via the third processors, automatic speech recognition based on the audio signal; and in accordance with a determination that the audio signal does not comprise the predetermined trigger signal: forgoing waking the third processors; and in accordance with a determination that the audio signal does not comprise the voice onset event: forgoing waking the second processors.

Automatic captioning pipelines and methods for automatically annotating video data with subtitles, which can be obtained using automatic speech recognition (ASR). An automatic captioning pipeline with inputs of multimodal data scales up the dataset of high-quality video-caption pairs. The automatic captioning pipeline generates video-caption pairs by establishing and using a large video-language dataset along with an automatic captioning approach leveraging multimodal inputs, such as textual video description, subtitles, and individual video frames.

Systems and methods are provided for enhancing the speech modality in a large language model (LLM) and for retaining in-context learning capabilities without overfitting to trained tasks. Systems obtain a first set of training data comprising tuples of a sample of speech combined with synthetically generated pairings of speech comprehension test questions and answers that correspond to the sample of speech and obtain a second set of training data comprising pairings of automatic speech recognition data. Systems generate and align a first set of encodings of the first set of training data and a second set of encodings of the second set of training data. Systems train the LLM on a greater amount of the first set of training data than the second set of training data and use the trained LLM to perform a natural language processing task.

A speech-processing system may determine potential endpoints in a user's speech. Such endpoint prediction may include determining a potential endpoint in a stream of audio data, and may additionally including determining an endpoint score representing a likelihood that the potential endpoint represents an end of speech representing a complete user input. When the potential endpoint has been determined, the system may publish a transcript of speech that preceded the potential endpoint, and send it to downstream components. The system may continue to transcribe audio data and determine additional potential endpoints while the downstream components process the transcript. The downstream components may determine whether the transcript is complete; e.g., represents the entirety of the user input. Final endpoint determinations may be made based on the results of the downstream processing including automatic speech recognition, natural language understanding, etc.

Disclosed are systems and techniques for training machine learning models. The techniques include generating, using a first automatic speech recognition (ASR) model, a first text output based on a vector representation of a first speech data and generating, using a second ASR model, a second text output, wherein the second ASR model adds noise to a vector representation of the first text output to obtain a noisy vector representation of the first text output and is trained to remove the noise from the noisy vector representation of the first text output. The techniques include calculating a first loss of the second ASR model based at least on a comparison between the second text output and the first text output and modifying learnable parameters of the second ASR model to improve an accuracy of the second ASR model.

Techniques for ASR processing using language model (LM)-generated context are described. A LM is prompted to generate words that are relevant for / may be included in a future user input. The prompt to the LM can include words from user interaction history, dialog history, dialog topic, user preferences, etc. The information included in the prompt may focus on rare or unique words rather than words that the ASR model is already confident in recognizing. The techniques can be plugged into an existing / pretrained ASR model and can be used with any existing / pretrained LM, thus saving resources needed to implement and maintain the components.

A method includes obtaining a plurality of training samples that each include a respective speech utterance and a respective textual utterance representing a transcription of the respective speech utterance. The method also includes fine-tuning, using quantization and sparsity aware training with native integer operations, a pre-trained automatic speech recognition (ASR) model on the plurality of training samples. Here, the pre-trained ASR model includes a plurality of weights and the fine-tuning includes pruning one or more weights of the plurality of weights using a sparsity mask and quantizing each weight of the plurality of weights based on an integer with a fixed-bit width. The method also includes providing the fine-tuned ASR model to a user device.

Transcript generation as part of automatic speech recognition may be guided using section types. Audio data is received for transcription. An initial transcript of the audio data may be generated and evaluated to determine a section type for the audio data. The section type may then be used to focus generation of a second version of the transcript on one speaker over another speaker.

Systems and methods for processing calls to determine if the call is potentially fraudulent or unwanted. The system extracts a speech signal from an audio signal associated with a call. The system identifies audio characteristics based on analysis of the audio signal. The system generates textual transcript of the audio signal based on automatic speech recognition of the speech signal, which is used to assign text categories for the call based on an automated multi-label textual classification of the textual transcript. The system assigns audio categories for the call based on automated multi-label acoustic classification of the speech signal. The system generates an output label for the call based on a combined analysis of the text categories, the audio categories, and the audio characteristics. The language spoken during the call may be detected and used to generate the textual transcript and to assign the text categories and the audio categories.

The invention relates to the technical field of voice recognition, and discloses a voice instruction recognition method based on a BERT pre-training language model, and the method comprises the steps: receiving a to-be-recognized voice, and converting the to-be-recognized voice into a text sequence through an automatic voice recognition model; fine-tuning the BERT pre-training language model, and performing text classification on the text sequence by using the fine-tuned BERT pre-training language model to obtain an instruction category corresponding to the to-be-recognized voice; performing model compression on the fine-tuned BERT pre-training language model by utilizing knowledge distillation to obtain a sequence classification model; according to the method, the situation that the audio data is more difficult to obtain than the text data is considered, the BERT is selected to pre-train the language model, the classification accuracy is improved, the knowledge distillation is utilized to perform model compression, the instruction category label of the to-be-recognized voice is recognized, and the voice recognition accuracy is improved. And data support is provided for man-machine interaction.

An automatic speech recognition (ASR) computing system and methodology are provided to predict a textual representation of received input speech data. A context acoustic biasing (CAB) engine of the ASR computing system receives historical textual content and an ontology data structure. The CAB engine matches key terms identified in the historical textual content with concepts present in the ontology data structure to generate a contextual term list data structure comprising the concept terms related to concepts matching the key terms. The CAB engine generates acoustic representations of the concept terms in the contextual term list data structure and inputs them to an ASR computer model of the ASR computing system which processes an input speech signal to generate a predicted textual representation of the input speech signal. The predicted textual representation is biased towards the acoustic representations of the concept terms in the contextual term list data structure.

Systems and computer-implemented methods for determining an accuracy of automatically transcribed pathological speech comprise recording speech from a person to obtain an original speech recording; combining a perturbation with the original speech recording to obtain a perturbed speech recording; performing automatic speech recognition, ASR, on the original speech recording to obtain a first transcript; performing automatic speech recognition on the perturbed speech recording to obtain a second transcript; comparing the first transcript with the second transcript to quantify a mismatch between the first transcript and the second transcript.

The invention discloses a dialogue record question and answer method and system based on a thinking chain and an intelligent agent reflection mechanism. The method comprises the following steps: transcribing a voice call into a text through an automatic voice recognition (ASR) technology, and extracting a retrieval condition and a question to be answered from user input based on a thinking chain cue word; and then, generating an SQL statement by using a natural language, retrieving a target dialogue record from a local database, and optimizing a retrieval result through an agent reflection mechanism so as to ensure the query accuracy. Finally, answers are generated in combination with a retrieval enhancement generation (RAG) technology, and high-quality intelligent question and answer services are provided for the user. According to the invention, the voice technology and the language model are deeply coordinated, so that efficient and accurate intelligent question and answer service can be provided for users in scenes such as enterprise management, customer service and intelligent cabins.

An AI-based personality and behavior analytic and assessment system includes an input interface configured for receiving input data containing audio speech, an automatic speech recognition module configured to receive the input data, recognize the spoken words in the input data, and generate an output representing the spoken words attributed to the individual, the output including speaking moments and audio slices, a machine learning text-based feature generation pipeline configured to receive the speaking moments and generate a numerical text-based feature set, a machine learning audio-based feature generation pipeline configured to receive the audio slices and generate a numerical audio-based feature set, a machine learning inference processor configured to receive at least one of the numerical text-based feature set and the numerical audio-based feature set, develop inferences from the feature sets, and generate scores representing the probabilities for a number of personality and behavioral traits of the individual.

A method includes obtaining an automatic speech recognition (ASR) model pre-trained on an initial training dataset, creating a set of canary speech utterances, and speeding up each canary speech utterance in the set of canary speech utterances. The operations also include fine-tuning the ASR model on the set of sped-up canary speech utterances and measuring un-intended memorization of the fine-tuned ASR model based on speech recognition results performed by the fine-tuned ASR model on the sped-up canary speech utterances.

The invention relates to an AI-based psychological counseling method, and aims to provide personalized psychological counseling service through an intelligent technology. The method comprises the following steps: firstly, acquiring voice data of a user, and converting the voice data into text data through an automatic voice recognition technology; then, voice and text data of the user are analyzed, and user feature information including gender, age, emotional features and the like is extracted; and based on the analysis results, generating a cue word template for guiding the target psychological counseling model to generate personalized text psychological counseling suggestions meeting user requirements. Through automatic and intelligent analysis processes, the real-time performance and pertinence of psychological consultation are remarkably improved, and more accurate psychological health support can be provided for users.

The invention relates to the field of data processing, in particular to a data vectorization label processing method and device based on search self-feedback, and the method comprises the steps: extracting text clues through meta-file attribute extraction, optical character recognition, automatic voice recognition or subtitle analysis technologies; generating a content abstract by using the language model and the text clues; performing analysis processing on the content abstract by using an NER model based on a BERT architecture, and extracting features to obtain a tag set; processing the label set through a FastText algorithm, and performing dimension reduction to obtain candidate label vectors; matching the candidate tag vector and the basic vector through a tag system, and determining a search result corresponding to the candidate tag vector based on a search result set of the basic vector; and collecting user behavior data, calculating a user interest weight, and simulating and marking error cases by using the generative adversarial network to optimize the label system. In this way, for heterogeneous data, the accuracy rate of label marking can be greatly improved.

Systems and methods are provided for generating a pseudo-labeled training dataset by at least one of: (1) extracting a set of intermediate outputs from an automatic speech recognition model based on applying the automatic speech recognition model to the set of unlabeled speech data, clustering the set of intermediate outputs into different clusters, and generating a first set of pseudo-labels comprising cluster assignments associated with the different clusters and which correspond to the unlabeled speech data, or (2) generating a set of decoded word sequences for the unlabeled speech data by applying the automatic speech recognition model to the set of unlabeled speech data, and generating a second set of pseudo-labels associated with the unlabeled speech data by applying the automatic speech recognition model to both (i) the set of decoded word sequences and (ii) the set of unlabeled speech data.

The embodiment of the invention provides a marketing short video generation method and device based on a large language model, and the method comprises the steps: carrying out the matching degree analysis of a hot title and product data based on the large language model, and outputting a hot topic which is most matched with a product according to the sequence of the matching degree from high to low; searching a hot video related to the most matched hot topic in a video platform, and extracting an oral copy in the hot video based on an automatic voice recognition technology; generating a creation copywriting by using a large language model based on the oral copywriting, the hot topics and the product data; and performing similarity matching on the embedded vector corresponding to the created copywriting and the semantic vector of the video slice in the database, and sorting and generating a mixed video meeting the duration requirement according to the similarity.

One example method includes receiving an audio stream comprising speech; generating, by automatic speech recognition (“ASR”) software, a plurality of hypotheses, each hypothesis comprising a transcription of a first portion of the speech; rescoring, using a first trained language model, each hypothesis of the plurality of hypotheses; and responsive to a first hypothesis not satisfying a threshold, generating and outputting, using a trained large language model (“LLM”), a final transcription based on the plurality of hypotheses.