615 results about "Automatic speech" patented technology

A neural end-to-end system is provided for the face and voice preserving translation of videos. The system is a pipeline of multiple models that produces a video of the original speaker speaking in the target language with modified lip movement to match the target speech, while preserving emphases and prosody of the original speech, and voice characteristics of the original speaker. The pipeline starts with automatic speech recognition including emphasis detection, followed by the translation model. The translated text is then synthesized by a Text-to-Speech model that recreates the original emphases in the target sentence. The resulting synthetic speech is then converted back to the original speakers' voice using a voice conversion model. Finally, to synchronize the lips of the speaker with the translated audio, a generative model generates frames of adapted lip movements which are combined with the audio to produce the final output. The disclosure further describes several use-cases and configurations that apply these techniques to video conferencing, dubbing, low-bandwidth transmission, speech enhancement and assistive technology for the hearing impaired.

One example method includes receiving, during a virtual conference from a client device, a request to generate a real-time summary of the virtual conference, a plurality of client devices participating in the virtual conference and exchanging audio and video streams; receiving, during the virtual conference, a plurality of utterances generated by automatic speech recognition (“ASR”) of the audio streams; generating a group of consecutive utterances from the plurality of utterances; determining a segment based on the group of consecutive utterances; generating, using a large language model (“LLM”), a segment summary based on the segment; and providing the segment summary to the client device.

The invention relates to the field of voice signal processing, and discloses a multi-level labeling method and device in a dialogue scene. Through the endpoint detection and automatic speech recognition method of the dialogue audio data, noise and mute in real speech can be removed, and effective speech segments can be obtained. And carrying out clustering analysis based on the effective voice segments, obtaining target voice segments and text data of the corresponding speakers, and carrying out emotion feature extraction through multi-modal emotion analysis. And finally, realizing automatic annotation of the inquiry strategy or intention understanding of the corresponding speaker by utilizing a large language model. According to the method, the recorded conversation voice can be subjected to multi-level labeling, an efficient and accurate information management and analysis tool is provided for a conversation scene, and the arrangement efficiency of conversation records can be greatly improved. The annotation information can be used for training and improving an artificial intelligence system, so that the artificial intelligence system plays a greater role in future psychological consultation service, and the overall quality and efficiency of psychological health service are improved.

The invention discloses a voice recognition model optimization method and device based on reinforcement learning, equipment and a medium, and relates to the technical field of voice recognition, and the method comprises the steps: determining a current optimization strategy of a current time step; randomly selecting one piece of enhanced speech training data and corresponding real text training data from the data training set as target training data, and inputting the target training data into the current automatic speech recognition model for prediction to obtain a plurality of prediction texts output by the current automatic speech recognition model; determining a text matching degree and a semantic similarity between each prediction text and real text training data to obtain a group of target reward values, updating an optimization strategy at the current moment based on the target reward values, and skipping to execute the step of determining the current optimization strategy of the current time step to obtain a target reward value; and outputting the corresponding optimized automatic speech recognition model until the current automatic speech recognition model meets the optimization ending condition. And reasonable optimization of the automatic speech recognition model is realized.

Techniques for processing with respect to a user input as contextual information is available are described. A system generates a first task prediction using first context data that is available when a user input is received. The system generates a second task prediction (e.g., updated first task prediction) when second context data is received, and then further generates a third task prediction when third context data is received. Example first context data may include device type information, time information, location, etc. Example second context data may include automatic speech recognition (ASR) data. Example third context data may include natural language understanding (NLU) data. Using the third task prediction, the system generates an output responsive to the user input.

Automatic captioning pipelines and methods for automatically annotating video data with subtitles, which can be obtained using automatic speech recognition (ASR). An automatic captioning pipeline with inputs of multimodal data scales up the dataset of high-quality video-caption pairs. The automatic captioning pipeline generates video-caption pairs by establishing and using a large video-language dataset along with an automatic captioning approach leveraging multimodal inputs, such as textual video description, subtitles, and individual video frames.

Systems and methods are provided for enhancing the speech modality in a large language model (LLM) and for retaining in-context learning capabilities without overfitting to trained tasks. Systems obtain a first set of training data comprising tuples of a sample of speech combined with synthetically generated pairings of speech comprehension test questions and answers that correspond to the sample of speech and obtain a second set of training data comprising pairings of automatic speech recognition data. Systems generate and align a first set of encodings of the first set of training data and a second set of encodings of the second set of training data. Systems train the LLM on a greater amount of the first set of training data than the second set of training data and use the trained LLM to perform a natural language processing task.

Techniques for ASR processing using language model (LM)-generated context are described. A LM is prompted to generate words that are relevant for / may be included in a future user input. The prompt to the LM can include words from user interaction history, dialog history, dialog topic, user preferences, etc. The information included in the prompt may focus on rare or unique words rather than words that the ASR model is already confident in recognizing. The techniques can be plugged into an existing / pretrained ASR model and can be used with any existing / pretrained LM, thus saving resources needed to implement and maintain the components.

An automatic speech recognition (ASR) computing system and methodology are provided to predict a textual representation of received input speech data. A context acoustic biasing (CAB) engine of the ASR computing system receives historical textual content and an ontology data structure. The CAB engine matches key terms identified in the historical textual content with concepts present in the ontology data structure to generate a contextual term list data structure comprising the concept terms related to concepts matching the key terms. The CAB engine generates acoustic representations of the concept terms in the contextual term list data structure and inputs them to an ASR computer model of the ASR computing system which processes an input speech signal to generate a predicted textual representation of the input speech signal. The predicted textual representation is biased towards the acoustic representations of the concept terms in the contextual term list data structure.

Systems and computer-implemented methods for determining an accuracy of automatically transcribed pathological speech comprise recording speech from a person to obtain an original speech recording; combining a perturbation with the original speech recording to obtain a perturbed speech recording; performing automatic speech recognition, ASR, on the original speech recording to obtain a first transcript; performing automatic speech recognition on the perturbed speech recording to obtain a second transcript; comparing the first transcript with the second transcript to quantify a mismatch between the first transcript and the second transcript.

The invention discloses a dialogue record question and answer method and system based on a thinking chain and an intelligent agent reflection mechanism. The method comprises the following steps: transcribing a voice call into a text through an automatic voice recognition (ASR) technology, and extracting a retrieval condition and a question to be answered from user input based on a thinking chain cue word; and then, generating an SQL statement by using a natural language, retrieving a target dialogue record from a local database, and optimizing a retrieval result through an agent reflection mechanism so as to ensure the query accuracy. Finally, answers are generated in combination with a retrieval enhancement generation (RAG) technology, and high-quality intelligent question and answer services are provided for the user. According to the invention, the voice technology and the language model are deeply coordinated, so that efficient and accurate intelligent question and answer service can be provided for users in scenes such as enterprise management, customer service and intelligent cabins.

A method includes obtaining an automatic speech recognition (ASR) model pre-trained on an initial training dataset, creating a set of canary speech utterances, and speeding up each canary speech utterance in the set of canary speech utterances. The operations also include fine-tuning the ASR model on the set of sped-up canary speech utterances and measuring un-intended memorization of the fine-tuned ASR model based on speech recognition results performed by the fine-tuned ASR model on the sped-up canary speech utterances.

The invention relates to the field of data processing, in particular to a data vectorization label processing method and device based on search self-feedback, and the method comprises the steps: extracting text clues through meta-file attribute extraction, optical character recognition, automatic voice recognition or subtitle analysis technologies; generating a content abstract by using the language model and the text clues; performing analysis processing on the content abstract by using an NER model based on a BERT architecture, and extracting features to obtain a tag set; processing the label set through a FastText algorithm, and performing dimension reduction to obtain candidate label vectors; matching the candidate tag vector and the basic vector through a tag system, and determining a search result corresponding to the candidate tag vector based on a search result set of the basic vector; and collecting user behavior data, calculating a user interest weight, and simulating and marking error cases by using the generative adversarial network to optimize the label system. In this way, for heterogeneous data, the accuracy rate of label marking can be greatly improved.

Systems and methods are provided for generating a pseudo-labeled training dataset by at least one of: (1) extracting a set of intermediate outputs from an automatic speech recognition model based on applying the automatic speech recognition model to the set of unlabeled speech data, clustering the set of intermediate outputs into different clusters, and generating a first set of pseudo-labels comprising cluster assignments associated with the different clusters and which correspond to the unlabeled speech data, or (2) generating a set of decoded word sequences for the unlabeled speech data by applying the automatic speech recognition model to the set of unlabeled speech data, and generating a second set of pseudo-labels associated with the unlabeled speech data by applying the automatic speech recognition model to both (i) the set of decoded word sequences and (ii) the set of unlabeled speech data.

The embodiment of the invention provides a marketing short video generation method and device based on a large language model, and the method comprises the steps: carrying out the matching degree analysis of a hot title and product data based on the large language model, and outputting a hot topic which is most matched with a product according to the sequence of the matching degree from high to low; searching a hot video related to the most matched hot topic in a video platform, and extracting an oral copy in the hot video based on an automatic voice recognition technology; generating a creation copywriting by using a large language model based on the oral copywriting, the hot topics and the product data; and performing similarity matching on the embedded vector corresponding to the created copywriting and the semantic vector of the video slice in the database, and sorting and generating a mixed video meeting the duration requirement according to the similarity.

Techniques for performing automatic speech recognition (ASR) processing are described. The ASR processing may involve use of a segmenter and a decoder. The segmenter may be configured to identify audio segments, from audio data containing speech, based on word boundaries. The decoder may be configured to generate multiple word hypotheses for individual audio segments. A fixed-size representation of an audio segment may be generated. The described ASR processing techniques may be computationally less expensive than at least some other systems. Also, the described ASR processing techniques may maintain a larger number of word predictions per audio segment as compared to at least some other systems.

Techniques for speech recognition models using modulation spectrum are disclosed herein. A modulation spectrum is generated from time series data output of an encoder layer of a speech recognition model and used as input into a decoder layer of the speech recognition model to improve accuracy of the model such as for recognizing subword units. The modulation spectrum is determined by applying a convolution filter to the output of the encoder layer of the speech recognition model. The time series data and / or the modulation spectrum can be normalized. A rectified linear unit activation function can be applied to the output of the convolution filter. The output of the encoder layer may be residually connected to the output of the rectified linear unit activation function prior to being input into the decoder layer.

This disclosure relates generally to systems, methods, and computer readable media for providing improved insights and annotations to enhance recorded audio, video, and / or written transcriptions of testimony. For example, in some embodiments, a method is disclosed for correlating non-verbal cues recognized from an audio and / or video recording of testimony to the corresponding testimony transcript locations. In other embodiments, a method is disclosed for providing testimony-specific artificial intelligence-based insights and annotations to a testimony transcript, e.g., based on the use of machine learning, natural language processing, and / or other techniques. In still other embodiments, a method is disclosed for providing smart citations to a testimony transcript, e.g., which track the location of semantic constructs within the transcript over the course of various modifications being made to the transcript. In yet other embodiments, a method is disclosed for providing intelligent speaker identification-related insights and annotations to an audio recording of a testimony transcript.

The invention belongs to the technical field of semantic understanding order establishment systems, and discloses a fragmented semantic understanding order establishment system and method based on a knowledge graph. The method comprises the following steps: collecting multi-source data of the same user in the same preset time window to obtain a multi-modal data set with a timestamp; performing automatic voice recognition and screenshot character recognition on the multi-modal data set to obtain text segments; standardizing the text fragments to obtain a text fragment set with timestamps; identifying entities in the text fragment set, and merging the identified entities to obtain a merged entity set; positioning and merging nodes corresponding to entities in the entity set in a pre-constructed domain knowledge graph, and generating a temporary knowledge sub-graph in combination with a timestamp sequence and a context relationship in the text fragment set; according to the invention, the manual information comparison cost of operation and maintenance personnel is reduced, and the fault diagnosis and solution efficiency is improved.

Systems and methods for providing interaction recordings with removed personally identifiable information (PII) are disclosed, the systems and methods involving: extracting, from an interaction recording, using an automatic speech recognition engine, a timestamped recording transcript; identifying, using an artificial intelligence (AI) engine, time periods of the timestamped recording transcript which disclose PII; and removing, from the interaction recording, data present during the time periods which disclose PII, to produce a secure interaction recording.

Techniques for generating a natural language prompt to further a goal of a dialog, are described. During a dialog, the system receives one or more user inputs including a user question, a user response to the question, and a request to generate a further question following the response. The system determines ASR output data corresponding to the user inputs, and determines dialog history data of the dialog. Using the ASR output data and the dialog history data, the system determines a category and an explanation of relevance corresponding to the category. Using the ASR output data, the dialog history, the category, and the explanation, the system determines the further question to be output to the user.

The invention relates to the technical field of artificial intelligence education, in particular to a Russian spoken language practice personalized recommendation method and system based on artificial intelligence, and the method comprises the steps: 1, outputting a phoneme sequence with a timestamp through Russian automatic voice recognition; 2, collecting an exercise interruption position and repeated read-after behavior data; 3, generating a dynamic learner portrait; 4, mapping high-frequency errors in the learner portrait into abnormal path weights of map nodes; 5, a lattice tail error option and a non-matching body verb interference item are injected; 6, when the voice fluency attenuation of the learner exceeds a dynamic threshold value, the sentence complexity is reduced; and 7, calculating an error rate descent gradient based on the exercise completion data, and dynamically adjusting the abnormal path weight of the knowledge graph. Through audio stream analysis and syntax tree construction, the system can accurately identify errors of the learner in grammar, pronunciation and other aspects, and the learning efficiency is improved.

Disclosed are apparatuses, systems, and techniques that leverage one or more artificial intelligence models for efficient automatic speech recognition (ASR) of speech in a diacritized language. The techniques include processing, using an ASR model, audio frame(s) encoding a speech in the diacritized language to generate, for a transcription token (TT) of the speech, likelihoods that the TT corresponds to various vocabulary tokens that include both non-diacritized and diacritized tokens of the language, and generating, using the likelihoods, a transcription of the speech.

The invention relates to a speech recognition technology, discloses a speech recognition model training method and device, equipment and a medium, and aims to improve the speech recognition rate in a noise environment. The method comprises the following steps: firstly, training an automatic speech recognition network to obtain a pre-trained network; a noise reduction module is introduced in front of an output classification layer, the noise reduction module is trained by taking embedded features output by the pre-trained automatic speech recognition network as a reference, and parameters of the pre-trained automatic speech recognition network are fixed during training to form an initial speech recognition model; and finally, noise reduction module parameters are fixed, the pre-trained automatic speech recognition network is retrained, and a final model is obtained. Through staged training and a parameter fixing strategy, training target conflicts among modules are avoided, and the training stability and the convergence speed are improved; the noise reduction module focuses on feature denoising required by recognition, is high in adaptability, has the characteristics of light weight and low delay, and can be widely applied to low-resource real-time scenes such as embedded equipment and edge computing.

Disclosed is an dynamic speech recognition system and associated methods for improving speech recognition accuracy by biasing, tuning, and / or otherwise adjusting a speech recognition model to account or compensate for different speech characteristics of individual speakers and / or different environmental factors that have different effects on the characteristics of the audio recorded from each speaker. The system receives an audio stream, identifies a speaker in the audio stream, selects a first vector that is generated by a first machine learning model and that encodes speech characteristics of the speaker, selects a second vector that is generated by a second machine learning model and that encodes audio characteristics that affect a capture of the audio stream, and adjusts a third machine learning model based on the first vector and the second vector. The system uses the third machine learning model after it is adjusted to convert speech into text.

A translation device provides real-time speech-to-text conversion between different languages while maintaining original audio. The device includes an input port for receiving audio in a first language, an output port for displaying text in a second language, and a processing unit for translation. The processing unit incorporates an automatic speech recognition engine and large language model to enable accurate real-time translation. The technology is implemented in two primary configurations: a plug-and-play unit for multimedia applications and a portable handheld unit for field deployment. The plug-and-play version connects to entertainment systems through HDMI interfaces and allows users to control translation settings via smartphone or remote. The portable version includes a built-in display, microphone, and battery power for field operations.

Modern automatic speech recognition (ASR) systems can utilize artificial intelligence (AI) models to service ASR requests. The number and scale of AI models used in a modern ASR system can be substantial. The process of configuring and reconfiguring hardware to execute various AI models corresponding to a substantial number of ASR requests can be time consuming and inefficient. Among other features, the described technology utilizes batching of ASR requests, splitting of the ASR requests, and / or parallel processing to efficiently use hardware tasked with executing AI models corresponding to ASR requests. In one embodiment, the compute graphs of ASR tasks are used to batch the ASR requests. The corresponding AI models of each batch can be loaded into hardware, and batches can be processed in parallel. In some embodiments, the ASR requests are split, batched, and processed in parallel.

Systems and methods for surfacing natural language queries from one or more transcripts. An example method may include converting received audio to text, through automated speech recognition, to form a transcript of the audio, wherein the transcript includes text of the audio and identifications of speakers associated with portions of the text corresponding to utterances from the respective speakers; generating input signals based on at least the transcript; executing at least one of one or more heuristics or a trained machine-learning (ML) model, using the generated input signals as an input, to generate at least one of a suggested natural language query for searching the transcript or a key moment within the received audio; and causing at least one of the suggested natural language query or the key moment to be surfaced on one or more remote devices.

A method for training a model includes obtaining a plurality of training samples. Each respective training sample of the plurality of training samples includes a respective speech utterance and a respective textual utterance representing a transcription of the respective speech utterance. The method includes training, using quantization aware training with native integer operations, an automatic speech recognition (ASR) model on the plurality of training samples. The method also includes quantizing the trained ASR model to an integer target fixed-bit width. The quantized trained ASR model includes a plurality of weights. Each weight of the plurality of weights includes an integer with the target fixed-bit width. The method includes providing the quantized trained ASR model to a user device.

A computing system may receive natural language audio input data. The computing system may automatically generate structured text data based on the natural language audio input data. The computing system may receive text input from an automatic speech recognizer or an input device providing for entry of typed characters. A machine learning model executing on the computing system may identify, based on the structured text data, one or more changes to be made to the text input. The machine learning model may include one or more of a statistical language model and a large language model. The machine learning model may generate, based on the one or more changes to be made to the text input, one or more proposed changes to the text input. The computing system may output the one or more proposed changes to the text input.