Patents
Literature
Patsnap Eureka AI that helps you search prior art, draft patents, and assess FTO risks, powered by patent and scientific literature data.

10196results about "Speech synthesis" patented technology

Real-time contextually aware artificial intelligence (AI) assistant system and a method for providing a contextualized response to a user using ai

An artificial intelligence (AI) assistant system and a method for providing a contextualized response to a user using AI are disclosed. The system comprises an audio input device for receiving voice input, an audio output device for providing output, a processor, a wireless communication device, a contextual memory unit for storing conversational context data on a sliding window basis, and a non-volatile system memory unit. The processor executes instructions to receive voice input, determine user identification, update conversational context data with user identification and a tokenized representation of the voice input, process the voice input using a transformer-based language model to generate a response, update the conversational context data with a tokenized representation of the generated response, and output the response via the audio output device. The method comprises receiving voice input, determining user identification, updating conversational context data, processing voice input, and generating and outputting a conversational response.
Owner:VASYLYEV SERGIY

System that conducts dialogs using artificial intelligence

An artificial intelligence system that conducts dialogs with one or more participants to achieve an educational outcome, such as teaching material with a Socratic question-and-answer method, reviewing material to reinforce ideas or to prepare a participant for future tests or events, or evaluating a participant's knowledge or communication skills. The system may use an artificial intelligence engine that can understand and generate natural language, such as a large-language model like ChatGPT. Text-to-speech and speech-to-text converters may be coupled to the AI engine to enable the system to communicate directly with participants using spoken language. The system may provide an interface for entry of contextual information for a dialog, such as background information on topics for discussion. It may provide an interface for moderation of a dialog, for example by a teacher, lawyer, thesis advisor, etc. System may evaluate the performance of the participants at the end of a dialog.
Owner:SOCRATIC METRIC LLC

Customization of a neural-network persona

A computer system that customizes an output provided by a pretrained neural network is described. During operation, the computer system may receive a prompt (or input) associated with a first individual. Then, the computer system may access stored information (in memory) associated with a persona associated with a second individual. For example, the second individual may be different from the first individual. However, in some embodiments, the second individual may be the same as the first individual. Note that the stored information may include memories associated with one or more different sessions of interacting with the pretrained neural network than a current session. Moreover, the computer system may modify the prompt based at least in part on the persona associated with the second individual. Next, the computer system may generate, using the pretrained neural network, the output based at least in part on the modified prompt.
Owner:1MIND AI INC

Natural language generation

PendingUS20250104693A1Biological modelsSpeech synthesis
Techniques for using a language model (e.g., a large language model (LLM)) to generate a natural language response to a user input and prosody information (e.g., voice characteristics associated with a synthetic voice to output the natural language response to the user) are described. The prosody information may correspond to a natural language (e.g., text or tokenized) description, a spectrogram, and / or a latent representation of the voice characteristic(s) associated with the natural language response. In some embodiments, the natural language response and the prosody information may be generated by different portions of layers of the language model. In such embodiments, the output of the layer(s) of the language model configured to generate the natural language response may be provided to the layer(s) of the language model configured to generate the prosody information and the output may be used to generate the prosody information, and vice versa.
Owner:AMAZON TECH INC

Server, display device, and digital human processing method

PCT designated stage expiredWO2025066217A1Semantic analysisCharacter and pattern recognition
Embodiments of the present disclosure provide a server, a display device, and a digital human processing method. The method comprises: receiving voice data inputted by a user and sent by a display device; on the basis of the voice data, determining a broadcast voice; extracting voice features of the broadcast voice; on the basis of the voice features, determining mouth shape parameters; determining emotion parameters and acquiring user image data; on the basis of the user image data, the emotion parameters, and the mouth shape parameters, generating digital human image data; and sending the broadcast voice and the digital human image data to the display device, so that the display device plays the broadcast voice and, on the basis of the digital human image data, displays a digital human image. According to the embodiments of the present disclosure, expression parameters and mouth shape parameters are determined on the basis of voice data inputted by a user, and are combined to generate a digital human image that has better facial expressions, thus achieving emotion customization and control.
Owner:HISENSE VISUAL TECH CO LTD

Machine operation assistance using language model-augmented perception

Various embodiments of the present disclosure relate to operator assistance based on extracting natural language characters from one or more sensed objects. For instance, particular embodiments may generate a natural language utterance based on extracting natural language text in a nearby traffic sign. In an illustrative example, particular embodiments may detect, via object detection and within image data, one or more regions of the image data depicting the traffic sign. Particular embodiments can then extract one or more first natural language characters represented in the traffic sign based at least on performing optical character recognition within the one or more regions of the image data in response to detecting the one or more regions of the image data depicting the traffic sign.
Owner:NVIDIA CORP

Using a wearable to interpret facial skin micromovements

Systems, methods, and non-transitory computer readable media including instructions for interpreting facial skin micromovements are disclosed. An example head mountable system includes a housing configured to be worn on a head of a wearer. The head mountable system also includes at least one detector integrated with the housing and configured to receive light reflections from a facial region of the head and to output associated reflection signals. The head mountable system also includes at least one microphone associated with the housing and configured to capture sounds produced by the wearer and to output associated audio signals. The head mountable system also includes and at least one processor in the housing, configured to use both the reflection signals and the audio signals to generate output that corresponds with words articulated by the wearer.
Owner:APPLE INC

Natural language generation

Techniques for generating a prompt for a language model to determine an action responsive to a user input, are described. In some embodiments, the system receives a user input, determines one or more application programming interfaces (APIs) configured to perform actions that are relevant to the user input and exemplars representing examples of using the APIs with respect to user inputs similar to the current user input. The system further determines device states of devices that are determined to be related to the user input and also determines other contextual information (e.g., weather information, time of day, geographic location, etc.). The system generates a prompt including the user input, the APIs, the exemplars, the device states, and the other contextual information. A language model processes the prompt to determine an action responsive to the user input and the system causes performance of the action.
Owner:AMAZON TECH INC

Semi-supervised dialect emotion speech synthesis system based on hybrid experts

The invention relates to dialect speech synthesis, in particular to a semi-supervised dialect emotion speech synthesis system based on hybrid experts, which comprises a text analysis module for preprocessing an input dialect text and generating a text representation vector through feature extraction and feature fusion; the mixed expert module is used for acquiring dialect acoustic features, rhythm features, emotional features and general acoustic features; the dynamic routing module is used for realizing intelligent cooperation among experts through a task-aware soft routing algorithm; the semi-supervised learning module is used for training supervised learning by using the marked dialect emotion voice data and training self-supervised learning by using the unmarked dialect emotion voice data; the acoustic parameter generation module is used for integrating the output of each expert to generate a complete acoustic parameter set; the neural vocoder is used for converting the acoustic parameter set into final dialect emotion voice; the method can effectively overcome the defect that the dialect emotional voice is difficult to accurately synthesize under the condition of lack of sample resources.
Owner:ANHUI XINGHONGYE INTELLIGENT TECH CO LTD

Script editor for routine creation

A method include receiving a natural language prompt from a user comprising a command to generate a code script for an automated assistant to perform a routine. The routine includes multiple discrete actions specified by the natural language prompt. The method further includes processing, by a pre-trained large language model (LLM), the natural language prompt to generate the code script as an LLM output, and processing the code script to determine the code script is incomplete, thereby rendering the code script unsuitable for the automated assistant to fulfill performance of the routine. Based on determining the code script is incomplete, the method includes issuing a user prompt soliciting the user to provide additional information needed to complete the code script and receiving user input of the additional information needed to complete the code script. The method includes supplementing the code script with the additional information to render completed code script.
Owner:GOOGLE LLC

system

Provide a system. [Solution] A system including: a means for utilizing generative AI to provide information that serves as an alternative to vision; a means for accessing the AI ​​via a smart device or headset; a means for acquiring environmental information in real time; a means for recognizing surrounding objects and people; and a means for conveying information in the form of voice, audio messages, haptic feedback, etc.
Owner:SOFTBANK GROUP CORP

Virtual conversational companion

Techniques for rendering visual content, in response to one or more utterances, are described. A device receives one or more utterances that define a parameter(s) for desired output content. A system (or the device) identifies natural language data corresponding to the desired content, and uses natural language generation processes to update the natural language data based on the parameter(s). The system (or the device) then generates an image based on the updated natural language data. The system (or the device) also generates video data of an avatar. The device displays the image and the avatar, and synchronizes movements of the avatar with output of synthesized speech of the updated natural language data. The device may also display subtitles of the updated natural language data, and cause a word of the subtitles to be emphasized when synthesized speech of the word is being output.
Owner:AMAZON TECH INC

Using continuous gestures for selectively processing facial movements

Systems, methods, and computer program products are disclosed for selectively employing a processing mode based on a continuous gesture. Selectively employing a processing mode may include detecting an existence of a mode-selection gesture by an individual. Upon detecting the existence of the mode-selection gesture, a first mode for processing facial micromovements of the individual may be continuously implemented while continuously detecting the mode-selection gesture. Following the continuous detection of the mode-selection gesture, a cessation of the mode-selection gesture may be detected. In response, implementation of the first mode may cease, and implementation of a second mode different from the first mode may be initiated for processing facial micromovements of the individual.
Owner:APPLE INC

Emphasizing portions of synthesized speech

A neural text-to-speech system may be configured to emphasize words. Applying emphasis where appropriate enables the TTS system to better reproduce prosodic characteristics of human speech. Emphasis may make the resulting synthesized speech more understandable and engaging than synthesized speech lacking emphasis. Emphasis may be manually annotated to, and / or predicted from, a source text (e.g., a book). In some implementations, the system may use a generative model such as a variational autoencoder to generate word acoustic embeddings indicating how emphasis is to be reflected in the synthesized speech. A phoneme encoder of the TTS system may process phonemes to generate phoneme embeddings. A decoder may process the word acoustic embeddings and the phoneme embeddings to generate spectrogram data representing the synthesized speech.
Owner:AMAZON TECH INC

Image analysis for personal interaction

Systems, methods and non-transitory computer readable media for image analysis for personal interaction are provided. Audio data may be received. The audio data may include an input from an entity in a natural language. The input may include at least a first part and a second part. Further, image data may be received. The image data may depict a particular movement. The particular movement may be a movement of a particular portion of a particular body. The particular movement and the first part may be concurrent. The particular body may be associated with the entity. Further, a conversational artificial intelligence model may be used to analyze the audio data and the image data to generate a response in the natural language to the input. The response may be based on the input and the particular movement. Further, the generated response may be provided to the entity.
Owner:ZASS RON +1

Using facial movements to generate a conversational record

Systems, methods, and computer program products are disclosed for generating a common record based on differing source inputs. Generating a common record based on differing source inputs may include receiving via at least one sensor first non-audible signals indicative of verbalization of an individual; interpreting first words of the individual at least in part using the first non-audible signals received from the at least one sensor; receiving second signals generated by a source other than the individual; performing speech recognition on the second signals to interpret second words from the source; and using the first words and the second words to generate a record.
Owner:APPLE INC

Robot anthropomorphic interaction method based on multi-modal emotion recognition and customized portrait generation

The invention discloses a robot anthropomorphic interaction method based on multi-modal emotion recognition and customized portrait generation. The method comprises the following steps: S1, dynamically fusing multi-modal emotions; the method comprises the following steps: S1, synchronously acquiring voice, visual and text signals through a multi-source heterogeneous sensor, capturing a user voice stream by a high-fidelity microphone array, and extracting acoustic characteristics such as intonation and speed, S2, performing cross-modal reasoning; s3, synchronously generating contents; step S4: style migration; step S5, anthropomorphic voice and expression generation; according to the method, man-machine interaction emotion is analyzed and generated by utilizing a large language model and multi-modal information fusion, the singleness of interaction emotion and the deficiency of emotional sharing ability are avoided, a strong emotion interaction characteristic is achieved, the image of the robot is obtained through a generative technology and can be migrated to any image, the limitation that a specific image is independently made is broken through, and the interaction effect of the robot is improved. The advantage that one robot can be suitable for different scenes is achieved.
Owner:JIANGSU YUNMU ZHIZAO TECH CO LTD

Low-Latency Conversational Large Language Models

A method includes receiving a transcription of an utterance, processing, using a first model, the transcription to generate a first text segment that represents an initial portion of a response to the utterance, processing, using a TTS system, the first text segment to generate a first synthesized speech representation, and providing, for audible output, the first synthesized speech representation. The method also includes providing, to a second model different from the first model, the transcription and the first text segment, the second model comprising an LLM configured to process the transcription and the first text segment to generate a second text segment that represents a remaining portion of the response to the utterance. The method further includes obtaining a second synthesized speech representation generated from the second text segment, and providing, for audible output by the user device, the second synthesized speech representation.
Owner:GOOGLE LLC

Artificial intelligence device for a digitial avatar with 3D interation capabilities and control method thereof

A method for controlling an artificial intelligence (AI) device for implementing a digital avatar can include receiving an audio signal corresponding to a user query, converting, by a speech-to-text neural network model, the audio signal into a text query, inputting the text query into a large language gesture instruction model to generate high level movement instructions, inputting the text query and the high level movement instructions into an information retrieval model to generate a text response including at least one sentence and digital avatar control information, and inputting the text response into a text-to-speech neural network model to generate an audio response. Also, the method can include inputting the audio response into an audio-to-facial animation model and an audio-to-conversational gesture model to generate updated digital avatar control information including gesture information, and outputting the audio response and the updated digital avatar control information for controlling the digital avatar.
Owner:LG ELECTRONICS INC

Natural language generation

Techniques for using a model to generate a response to a user input, where the response is associated with a personality determined to be relevant to the user input, are described. The system receives a user input and context data associated with the user input. Using the user input data and / or the context data, the system determines a personality (e.g., including a personality type and / or personality characteristics) relevant to the user input. The system generates a prompt instructing a model to generate a response to the user input that corresponds to the personality. The model processes the prompt to generate a response to the user input that corresponds to the personality. In some embodiments, the model generates a request for another component of the system to generate information responsive to the user input. The model may transform the responsive information into the personality-associated response.
Owner:AMAZON TECH INC

earphones

The present disclosure provides an earphone comprising a housing assembly, a speaker, and an ear hook assembly. The housing assembly includes a first accommodating cavity and a connecting portion that is provided with a socket extending along a first direction. The first accommodating cavity and a first channel and a second channel included in the socket are sequentially connected. A first abutment surface is disposed at transition between the first and second channels. The first abutment surface is inclined to the first direction. The ear hook assembly includes a first connecting end that includes a plug portion. The plug portion is inserted into the socket and includes a second abutment surface and an annular surface. The second abutment surface is inclined to the first direction and faces the first abutment surface. The annular surface extends from an inner edge of the second abutment surface toward the first abutment surface.
Owner:SHENZHEN SHOKZ CO LTD

Voice generation method and device, equipment and medium

The invention relates to the technical field of artificial intelligence, can be applied to business scenes of medical health, financial science and technology and the like, and discloses a voice generation method which comprises the following steps: constructing a multi-language voice synthesis model, obtaining plain text data and paired voice text data, and constructing an expansion vocabulary; updating a language perception embedding layer and model parameters, and converting an input text into a mark sequence; and the encoder extracts context semantic features, extracts pronunciation rule features, and the decoder fuses the features to generate an acoustic feature sequence, and converts the acoustic feature sequence into target voice data. According to the invention, the multi-language speech synthesis model is combined with the language perception embedding layer, so that the speech generation capability of a low-resource language is improved; the text conversion accuracy is improved by expanding the vocabulary, the target language learning ability is enhanced by unsupervised training, the low data environment adaptability is optimized by supervised training, and the speech naturalness and fluency are improved by feature fusion.
Owner:PING AN TECH (SHENZHEN) CO LTD

Voice generation method and device based on multi-modal fusion, equipment and medium

The invention relates to the technical field of artificial intelligence, can be applied to business scenes of medical health, financial science and technology, cultural transmission and the like, and discloses a voice generation method based on multi-modal fusion, which comprises the following steps: collecting audio data to extract timbre features, and training a field feature timbre generation model; analyzing text semantic recognition emotion information, adjusting speech synthesis parameters, combining personalized information to construct a parameter mapping table, fusing to generate a synthesis control parameter sequence, aligning the synthesis control parameter sequence with character labels, visual elements and background music data, driving a domain feature timbre generation model, and generating synthesis data of synchronous speech, text, vision and music. Domain timbres are generated through timbre feature training, speech expression is optimized by combining semantic analysis and emotion recognition, user requirements are matched based on personalized information, and time alignment is performed by fusing text, vision and music data, so that the synthesized speech has domain features, emotion adaptability and personalization, and the speech quality is improved. And the voice immersion and the information transmission capability are improved.
Owner:PING AN TECH (SHENZHEN) CO LTD

Voiceprint recognition method, graphical interface, and electronic device

This application discloses a voiceprint recognition method, a graphical interface, and an electronic device. In the voiceprint recognition method, a voiceprint model is preset in the electronic device, and then the electronic device trains and updates the preset voiceprint model based on a voiceprint feature extracted from a voice of a registered user to obtain an exclusive voiceprint model belonging to the registered user. Finally, the electronic device uses the exclusive voiceprint model to generate a registered user representation based on the voiceprint feature of the voice of the registered user, and uses the registered user representation as a reference standard to realize voiceprint recognition on a voice of a speaker. Since the exclusive voiceprint model is trained based on voiceprint features of personal voices of the registered user, the registered user representation generated can accurately express voiceprint features of the user, thereby improving accuracy of voiceprint recognition.
Owner:HONOR DEVICE CO LTD

Low-resource language adaptive speech recognition method based on AdLoRA Plus

The invention belongs to the technical field of speech recognition. The invention provides a low-resource language self-adaptive speech recognition method based on AdLoRA Plus. The low-resource language self-adaptive speech recognition method comprises the following steps of: obtaining a speech According to the embodiment of the invention, the AdLoRA Plus fine tuning technology is introduced, so that the speech recognition precision of the Whisper model is remarkably improved in a low-resource language environment. Through a low-rank adapter layer and dynamic learning rate adjustment, efficient parameter updating is realized under limited training data. According to the method, a strategy of freezing basic parameters is adopted, only an AdLoRA Plus adapter layer is trained, calculation overhead and training time are remarkably reduced, pre-training knowledge is kept, and overfitting is prevented. The method has wide adaptability and expansibility, and is ensured to be outstanding in practical application. The voice recognition precision, the reasoning efficiency and the training efficiency are effectively improved, multiple challenges in application in small languages and specific fields in the prior art are solved, and the method has important practical application value.
Owner:XIAN UNIV OF TECH

Rich-Media Document Auxiliary Generation Apparatus

Disclosed in the present disclosure is a rich-media document auxiliary generation apparatus. The apparatus comprises a material extraction module, a theme sorting module, a semantic retrieval module, a structured data text generation module, an illustration recommendation module and a video composition module. The present disclosure uses intelligent means to assist a user to efficiently generate a high-quality rich-media composite document, thereby quickly and accurately describing a theme event in an all-round way.
Owner:10TH RES INST OF CETC

Electrical audio signal processing systems and devices

According to an aspect of the present invention, there is provided an electrical audio signal processing system and device, comprising: a computer graphics processing and selective visual display system with a screen; an eye tracking device; a processor; one or more computer memory devices; wherein the processor is arranged for operations comprising: measuring the user's eye movements to ascertain the specific word on which the user is fixated, by the eye tracking device; modifying the display at the user's current fixation point; applying a delay between the presentation of successive graphic elements based on the user's calculated rate to accommodate the user's required time; and presenting elements to the user at a rate based upon the user's required time.
Owner:DECHARMS RICHARD CHRISTOPHER

Face-translator: end-to-end system for speech-translated lip-synchronized and voice preserving video generation

A neural end-to-end system is provided for the face and voice preserving translation of videos. The system is a pipeline of multiple models that produces a video of the original speaker speaking in the target language with modified lip movement to match the target speech, while preserving emphases and prosody of the original speech, and voice characteristics of the original speaker. The pipeline starts with automatic speech recognition including emphasis detection, followed by the translation model. The translated text is then synthesized by a Text-to-Speech model that recreates the original emphases in the target sentence. The resulting synthetic speech is then converted back to the original speakers' voice using a voice conversion model. Finally, to synchronize the lips of the speaker with the translated audio, a generative model generates frames of adapted lip movements which are combined with the audio to produce the final output. The disclosure further describes several use-cases and configurations that apply these techniques to video conferencing, dubbing, low-bandwidth transmission, speech enhancement and assistive technology for the hearing impaired.
Owner:WAIBEL ALEXANDER