1828 results about "Utterance" patented technology

Various embodiments of the present disclosure relate to operator assistance based on extracting natural language characters from one or more sensed objects. For instance, particular embodiments may generate a natural language utterance based on extracting natural language text in a nearby traffic sign. In an illustrative example, particular embodiments may detect, via object detection and within image data, one or more regions of the image data depicting the traffic sign. Particular embodiments can then extract one or more first natural language characters represented in the traffic sign based at least on performing optical character recognition within the one or more regions of the image data in response to detecting the one or more regions of the image data depicting the traffic sign.

The disclosure herein describes using a deep learning model to identify topic segments of a communication transcript. A communication transcript including a set of utterances is obtained. The set of utterances is divided into a plurality of utterance windows, wherein each utterance window of the plurality of utterance windows includes a different subset of utterances of the set of utterances, and wherein each utterance of the set of utterances is included in at least one utterance window of the plurality of utterance windows. For each utterance window of the plurality of utterance windows, each utterance in the utterance window is classified as a topic boundary or a non-boundary using a deep learning model. Topic segments of the communication transcript are identified based on utterances of the set of utterances that are classified as topic boundaries. A communication transcript summary is generated using the communication transcript and the identified topic segments.

Systems, methods, and computer program products are disclosed for generating a common record based on differing source inputs. Generating a common record based on differing source inputs may include receiving via at least one sensor first non-audible signals indicative of verbalization of an individual; interpreting first words of the individual at least in part using the first non-audible signals received from the at least one sensor; receiving second signals generated by a source other than the individual; performing speech recognition on the second signals to interpret second words from the source; and using the first words and the second words to generate a record.

One example method includes receiving, during a virtual conference from a client device, a request to generate a real-time summary of the virtual conference, a plurality of client devices participating in the virtual conference and exchanging audio and video streams; receiving, during the virtual conference, a plurality of utterances generated by automatic speech recognition (“ASR”) of the audio streams; generating a group of consecutive utterances from the plurality of utterances; determining a segment based on the group of consecutive utterances; generating, using a large language model (“LLM”), a segment summary based on the segment; and providing the segment summary to the client device.

A task-oriented dialog system determines an endpoint in a user utterance by receiving incremental portions of a user utterance that is provided in real time during a task-oriented communication session between a user and a virtual agent (VA). The task-oriented dialog system recognizes words in the incremental portions using an automated speech recognition (ASR) model and generates semantic information for the incremental portions of the utterance by applying a natural language processing (NLP) model to the recognized words. An acoustic-prosodic signature of the incremental portions of the utterance is generated using an acoustic-prosodic model. The task-oriented dialog system can generate a feature vector that represents the incrementally recognized words, the semantic information, the acoustic-prosodic signature, and corresponding confidence scores of the model outputs. A model is applied to the feature vector to identify a likely endpoint in the user utterance.

Automatic recognition of non-verbal messages in speech, and in particular to detection or analysis of prosodic multilayered analysis of intonation units such as prosodic unit prototypes and their multi-labeled variations, may form a hierarchical classification for the analysis of non¬ verbal information or cues in speech. A speech captured by a microphone is fed to a weakly- supervised deep learning acoustic model for speech recognition and transcription, that may be based on encoder-decoder Transformer architecture, such as Whisper by OpenAI. The model is trained to output multiple words form the text in the captured speech, to identify Intonation Units (IUs) that include one or more words, and associate non-verbal labels to each of the IUs. The labels may indicate a prototype, a discourse function (such as a conversation action), an emotion, an emphasis, or an attitude, as well as a genre of a part of, or whole of, the entire captured speech.

Techniques for augmenting training data include accessing training data comprising a plurality of training examples comprising a first training example comprising a first natural language utterance and a first logical form for the first natural language utterance. A second natural language utterance is generated by adding or replacing one or more values in the first natural language utterance. A logical form for the second natural language utterance is generated. A second training example is generated, comprising the second natural language utterance and the logical form for the second natural language utterance. The training data is augmented by adding the second training example to the plurality of training examples to generate an augmented training data set. A machine learning model is trained to generate logical forms for utterances using the augmented training data set.

Techniques for logical text passage generation and retrieval for retrieval-augmented generation. The techniques involve processing markup language documents to generate logical text passages and their corresponding embeddings. These embeddings are indexed for efficient retrieval. Upon receiving a user utterance, a user query is formed and transformed into an embedding to query the index. Relevant text passages are identified and used to prompt a large language model (LLM), which generates a completion. This completion is then sent as a response to the user. The process effectively bridges user queries with relevant information through advanced embedding and natural language processing techniques, enabling accurate and contextually appropriate interactions within a user-agent dialogue framework.

An organization-specific cybersecurity dialogue system (“system”) intelligently generates responses to user utterances for cybersecurity by employing documentation and playbook agents for an associated cybersecurity organization. The system extracts an intent and an intent category from a user utterance, extracts relevant entities from the intent and uses them to determine relevant user telemetry data, and matches the intent, intent category, and user telemetry data to documents of the cybersecurity organization. The system then determines whether to generate a response based on matching document, matching playbook agents, or to prompt a user to create a ticket according to logic applied to the intent, intent category, user telemetry data, and matching documents and generates a corresponding response with large language models.

Techniques are disclosed herein for transforming natural language conversations into a visual output. In one aspect, a computer-implement method includes generating an input string by concatenating a natural language utterance with a schema representation comprising a set of entities for visualization actions, generating, by a first encoder of a machine learning model, one or more embeddings of the input string, encoding, by a second encoder of the machine learning model, relations between elements in the schema representation and words in the natural language utterance based on the one or more embeddings, generating, by a grammar-based decoder of the machine learning model and based on the encoded relations and the one or more embeddings, an intermediate logical form that represents at least the query, the one or more visualization actions, or the combination thereof, and generating, based on the intermediate logical form, a command for a computing system.

The invention discloses an accurate international communication digital human real-time dialogue method fused with a multi-modal technology. The method comprises the following steps: S1, constructing a digital human image and tone; s2, propagation content generation and problem guidance; s3, semantic analysis and intention clarification based on the real-time voice dialogue; s4, geographic preference modeling and path planning; s5, cross-context propagation content generated based on retrieval enhancement is generated; s6, visually displaying the propagation content; and S7, carrying out digital human-driven multi-language propagation content real-time output and feedback closed-loop optimization. Through accurate utterance expression analysis, accurate international propagation problem recommendation is provided, cross-context propagation content generation based on semantic understanding is realized, digital people with voice features and visual images are constructed, real-time dialogue interaction of users is realized, and user experience is improved. The system can carry out geographic modeling according to the region where the accurate problem is located, language preference and propagation object culture characteristics, and differential propagation path planning is achieved.

Example solutions perform natural language query processing on hybrid utterances. A precise segment is identified, within the hybrid utterance, and processed with a symbolic AI interpreter configured to generate a first interpretation. The precise segment is replaced, within the hybrid utterance, with a placeholder term thereby resulting in a vague utterance. The vague utterance is processed with a statistical AI interpreter configured to generate a second interpretation. The first interpretation is merged with the second interpretation using the hybrid utterance as a template for the merger and using the placeholder term as the location for the first interpretation within the second interpretation. A complete interpretation is generated and transmitted to a query generator.

Approaches are generally described for corrupted speech detection in voice-based computer interfaces. First input data including first audio data representing a user utterance may be received. First data representing the first audio data may be generated using a first encoder. First text data representing a transcription of the user utterance may be generated. Second data representing the first text data may be generated using a second encoder different from the first encoder. Third data may be generated by combining the first data and the second data. The third data may be sent to a classifier network trained to predict a relevant corruption state for speech processing inputs. The classifier network may determine that the first input data corresponds to a first corruption state.

The invention discloses an ASR identification enhancement method based on audio and video recording. According to the method, the accuracy and compliance of voice recognition in the financial service interaction process are improved by fusing the audio and environment feature information in the banking business double-recording scene. The method comprises the following steps: firstly, constructing an acoustic model for a bank outlet environment, and extracting audio features and interaction scene information of conversation between a client and a worker; and then, designing a vocabulary recognition module special for the financial field, dynamically adjusting language model parameters according to professional term libraries and utterance modes of different business types, and effectively coping with key links such as financial product introduction, risk prompt and customer confirmation. Compared with a traditional ASR system, the voice recognition accuracy in the banking business handling process is remarkably improved, particularly, key term recognition and important information extraction are prominent, and more reliable technical support is provided for financial service standardized management and double-recording quality inspection.

Disclosed are systems and methods including software processes executed by a server that detect audio-based synthetic speech (“deepfakes”) in a call conversation. The server applies an NLP engine to transcribe call audio and analyze the text for anomalous patterns to detect synthetic speech. Additionally or alternatively, the server executes a voice “liveness” detection system for detecting machine speech, such as synthetic speech or replayed speech. The system performs phrase repetition detection, background change detection, and passive voice liveness detection in call audio signals to detect liveness of a speech utterance. An automated model update module allows the liveness detection model to adapt to new types of presentation attacks, based on the human provided feedback.

Systems and methods to cause an intent discovery system to identify new user intents without additional training. The system may comprise of two neural networks. The first neural network generates a prompt tailored to a particular domain (e.g., travel), and may include known intents pertinent to the domain selected examples from a training dataset to provide context to the prompt. The second neural network may use this prompt to identify intents from new utterances in the prompt. The identified intents that are not in the list of known intents are then used to update the database.

A system for contextual analysis and metadata database generation for user-specific speech patterns is disclosed. The system accesses a speech signal of a user and identifies the user based on the voice print associated with the user. The system splits the speech signal into a first set of audio frames, where each audio frame comprises an utterance of one or more words. The system determines a context associated with each word. In response, the system detects a context change between a first text and a second text. The system generates a contextually split set of frames by splitting the speech signal into a second set of audio frames according to the detected context changes.

Techniques are disclosed herein for improving model robustness on operators and triggering keywords in natural language to a meaning representation language system. The techniques include augmenting an original set of training data for a target robustness bucket by leveraging a combination of two training data generation techniques: (1) modification of existing training examples and (2) synthetic template-based example generation. The resulting set of augmented data examples from the two training data generation techniques are appended to the original set of training data to generate an augmented training data set and the augmented training data set is used to train a machine learning model to generate logical forms for utterances.

Provided herein are systems and methods for providing instructions for speech based on speech classifications of verbal communications from users. Ae computing system can generate speech characteristics for a first verbal communication using a first audio sample from a user. The computing system can determine, from a plurality of speech classifications, a first speech classification for the first verbal communication based on the speech characteristics. The computing system can select, from a plurality of actions, an action comprising modifying one or more of the speech characteristics to define an utterance for the user based on the first speech classification. The computing system can provide an instruction presenting a message to prompt the user to perform the utterance defined by the action selected from the plurality of actions. The efficacy of the medication that the user is taking to address the condition may be increased.

In one embodiment, a method includes receiving training utterances associated with a domain, receiving ontology labels for the domain, wherein the ontology labels comprise one or more of an intent or a slot, generating an inventory for the domain, wherein the inventory comprises at least a respective index and respective span for each intent or slot, wherein the respective span comprises a respective descriptive label associated with the intent or slot, and wherein the respective descriptive label comprises a natural-language description of the intent or slot, generating frames for training utterances based on the training utterances and the inventory by a natural-language understanding (NLU) model, wherein each frame comprises a structural representation of the respective training utterance, wherein the structural representation is generated based on a comparison between the corresponding training utterance and the inventory, and updating the NLU model based on the frames.

The invention discloses audio-driven facial animation using machine learning. Systems and methods of the present disclosure include animating an avatar or proxy according to input audio and one or more selected or determined emotions and / or styles. For example, a deep neural network may be trained to output motion or deformation information of a character representing that the character spokes an utterance contained in the audio input. The character may have different facial components or regions (e.g., head, skin, eyes, tongue) that are modeled individually, such that the network may output motion or deformation information for each of these different facial components. During training, the network may train associated decoders using weighted feature vectors using converter-based audio encoders with locked parameters. The network output may be provided to a renderer to generate a mood-accurate audio-driven facial animation.

The present disclosure relates to a data augmentation system and method that uses a large pre-trained encoder language model to generate new, useful intent samples from existing intent samples without fine-tuning. In certain embodiments, for a given class (intent), a limited number of sample utterances of a seed intent classification dataset may be concatenated and provided as input to the encoder language model, which may generate new sample utterances for the given class (intent). Additionally, when the augmented dataset is used to fine-tune an encoder language model of an intent classifier, this technique improves the performance of the intent classifier.

A method includes obtaining a plurality of training samples that each include a respective speech utterance and a respective textual utterance representing a transcription of the respective speech utterance. The method also includes fine-tuning, using quantization and sparsity aware training with native integer operations, a pre-trained automatic speech recognition (ASR) model on the plurality of training samples. Here, the pre-trained ASR model includes a plurality of weights and the fine-tuning includes pruning one or more weights of the plurality of weights using a sparsity mask and quantizing each weight of the plurality of weights based on an integer with a fixed-bit width. The method also includes providing the fine-tuned ASR model to a user device.

Methods and apparatuses are described for dialog control flow in information retrieval applications. A server establishes a chat-based communication session between an information retrieval application and a client device. The server determines a user intent from utterances received from a user of the client device and initiates a first dialog workflow associated with the user intent. The server invokes NLP services using the utterances to determine a comprehension score for the user intent and identifies a first one of the NLP services to continue the first dialog workflow when the comprehension score is at or above a threshold value, including generating a response to the utterances using the first NLP service. The server delegates the communication session to a second dialog workflow when the comprehension score is below the threshold value, including invoking a generalized language processing service associated with the second dialog workflow using the user intent to generate a response to the utterances. The server transmits the generated response to the client device as part of the chat-based communication session.

The present disclosure relates to manufacturing training data by leveraging an automated pipeline that manufactures visualization training datasets to train a machine learning model to convert a natural language utterance into meaning representation language logical form that includes one or more visualization actions. Aspects are directed towards accessing an original training dataset, a visualization query dataset, an incremental visualization dataset, a manipulation visualization dataset, or any combination thereof. One or more visualization training datasets are generated by: (i) modifying examples in the original training dataset, the visualization query dataset, or both to include visualization actions, (ii) generating examples, using the incremental visualization dataset, the manipulation visualization dataset, or both, that include visualization actions, or (iii) both (i) and (ii). An augmented training dataset is generated by adding the one or more visualization training datasets to the original training dataset and then used to train the machine learning model.

A system and method for scalable generations of synthetic <logical form, utterance> pairs for training a semantic parser is disclosed. A semantic parser is trained on pairs of <logical form, utterance>. An ontology graph is constructed and derived from a plurality of enterprise documents and provides relationships among the concepts or classes of an organization. One or more paths are traversed among a plurality of source and destination node pairs, facilitating comprehensive semantic representation. Attributed query subgraphs are generated of source nodes, destination nodes, and hidden nodes. Each path is recognized among a variety of possible paths between source and destination nodes. Each path in the ontology query subgraph is validated by considering a plurality of predicates and a knowledge graph generates a natural language utterance. The utterances are refined and rephrased using a large language model, enhancing their coherence and linguistic quality.

Methods and systems are presented for providing multi-tiered cache system that works with an artificial intelligence (AI)-based conversation system for facilitating a conversation with users and processing transactions for the users. The multi-tiered cache system includes multiple tiers of cache modules that use different structures for caching and / or querying data. As a new utterance is received, the cache system uses each of the cache modules in sequence to determine whether a cache hit occurs. If a cache miss occurs at a first cache module, the cache system determines if a cache hit occurs at a second cache module. When a response is obtained from one of the cache modules and / or the AI model, the cache system updates the cache modules using the response.

A system and method of real-time feedback confirmation to solicit a virtual assistant response from an evolving semantic state of at least a portion of an utterance. A user accesses a virtual assistant on an electronic device having the system and / or method configured to capture a command, a question, and / or a fulfillment request from audio such as, the speech emitted from the speaking user. The speech may be intercepted by a speech engine configured to transcribe the speech into text that is matched with the fragment pattern's regular expression to generate a fragment and / or the speech may be processed with a machine learning model to identify fragments. The fragments are identified by a domain handler configured to update a data structure of the current semantic state of the utterance in real-time on an interface of an electronic device.

A system for emotionally enhancing dialogue includes a computing platform having processing hardware and a system memory storing a software code including a predictive model. The processing hardware is configured to execute the software code to receive dialogue data identifying an utterance for use by a digital character in a conversation, analyze, using the dialogue data, an emotionality of the utterance at multiple structural levels of the utterance, and supplement the utterance with one or more emotional attributions, using the predictive model and the emotionality of the utterance at the multiple structural levels, to provide one or more candidate emotionally enhanced utterance(s). The processing hardware further executes the software code to perform an audio validation of the candidate emotionally enhanced utterance(s) to provide a validated emotionally enhanced utterance, and output an emotionally attributed dialogue data providing the validated emotionally enhanced utterance for use by the digital character in the conversation.

Methods, systems, and apparatus, including computer programs encoded on computer storage media for speech recognition. One method includes obtaining an input acoustic sequence, the input acoustic sequence representing one or more utterances; processing the input acoustic sequence using a speech recognition model to generate a transcription of the input acoustic sequence, wherein the speech recognition model comprises a domain-specific language model; and providing the generated transcription of the input acoustic sequence as input to a domain-specific predictive model to generate structured text content that is derived from the transcription of the input acoustic sequence.