539 results about "Spoken language" patented technology

The present disclosure relates to systems and methods for enhancing the interaction between users and automated agents, such as digital assistants, by employing Large Language Models (LLMs) to infer the intent of spoken language. The invention involves continuously monitoring ambient audio, converting speech to text, and utilizing LLMs to determine whether spoken language is intended for the automated agent. A structured prompt, including the converted text and specific instructions, is sent to the LLM, which is fine-tuned to process domain-specific prompts. The LLM provides a structured output in a standardized format, indicating the user's intent. The system may involve multiple prompts to perform separate tasks, such as identifying intent and generating additional context-specific data. This approach facilitates a more natural and intuitive user experience by eliminating the need for wake words and allowing seamless conversational interaction with virtual assistants across various platforms and devices.

A method includes receiving a natural language query from a user that solicits a response from an assistant large language model (LLM), receiving a prompt composition including an instruction parameter that specifies a task for the assistant LLM to respond to user queries concisely, structuring a conciseness prompt by concatenating the prompt composition to the natural language query, and processing, using the assistant LLM, the conciseness prompt to generate a concise response to the natural language query. The method also includes providing, for output from a user device, the concise response to the natural language query.

Disclosed herein are methods and systems for a training a model for real-time patient diagnosis. A system may include a computer configured to receive audio data and video data of a clinical encounter, the audio data comprising spoken words by an entity and the video data depicting the entity; retrieve clinical data regarding the entity; execute a model using the words of the audio data and the retrieved clinical data regarding the entity as input, the execution causing the model to output a plurality of clinical diagnoses for the entity; concurrently render the corresponding video data and audio data and the plurality of clinical diagnoses via a computing device associated with a user; and store an indication of a selected clinical diagnosis of the plurality of clinical diagnoses responsive to receiving a selection of the clinical diagnosis at the computing device.

A speech module is joint trained with a knowledge module by transforming a first knowledge graph into an acoustic knowledge graph. The knowledge module is trained on the acoustic knowledge graph. Then, the knowledge module is integrated with the speech module to generate an integrated knowledge-speech module. In some instances, the speech module included in the integrated knowledge-speech module is aligned with a language module to generate an optimized speech model configured to leverage acoustic information and acoustic-based knowledge information, along with language information.

An integrated development environment (IDE) leverages a generative AI model to generate industrial control code in accordance with specified functional requirements, which can be provided to the industrial IDE system as intuitive natural language spoken or written text. The industrial IDE can also analyze written code in response to natural language prompts submitted against the code, generate answers to user-submitted questions about the code, and offer recommendations for improving the code in response to specific questions or requests submitted by the user.

A method includes obtaining labeled training data including a plurality of spoken terms spoken during a conversation. For each respective spoken term, the method includes generating a corresponding sequence of intermediate audio encodings from a corresponding sequence of acoustic frames, generating a corresponding sequence of final audio encodings from the corresponding sequence of intermediate audio encodings, generating a corresponding speech recognition result, and generating a respective speaker token representing a predicted identity of a speaker for each corresponding speech recognition result. The method also includes training the joint speech recognition and speaker diarization model jointly based on a first loss derived from the generated speech recognition results and the corresponding transcriptions and a second loss derived from the generated speaker tokens and the corresponding speaker labels.

The invention belongs to the technical field of intelligent voice processing, and particularly relates to a spoken language pronunciation training and correcting system based on intelligent equipment, which acquires a user rhythm feature set including pitch change rate, accent intensity, pause duration and intonation contour by acquiring spoken language audio data sent by a user for a target text, and corrects the spoken language pronunciation training and correcting system. The method comprises the following steps: acquiring spoken language audio data, converting the spoken language audio data into a phoneme sequence aligned with target text time, generating a rhythm deviation degree report according to comparison evaluation of a user rhythm feature set and the phoneme sequence, an execution intonation mode, accent distribution, speech stream sound change and speech speed rhythm, and determining the rhythm deviation degree according to a rhythm problem type in the report in combination with user historical learning data. And forming and outputting a correction scheme including a text prompt, a targeted minimum contrast training unit and a listen-and-read simulation task, solving the problem of weak capability of correcting hyper-phoneme in a second language spoken language of an adult, and improving the authentic and fluency of the spoken language, thereby realizing efficient personalized learning.

The invention discloses a spoken language practice scene-oriented education large model evaluation method, which comprises the following steps of: obtaining voice data and a corresponding standard text, interacting with a test model through the voice data, and obtaining a recognition text, audio data and an answer text provided by the test model; the standard text and the recognition text are compared and evaluated, and the voice recognition accuracy is obtained; carrying out pronunciation accuracy, fluency and rhythm evaluation on the audio data based on multi-modal feature fusion to obtain an audio score; performing grammar accuracy evaluation on the answer text based on a dynamic punishment mechanism and the constructed prompt large model, and performing question degree evaluation on the answer text based on the constructed prompt large model to obtain an answer text score; and comprehensively summarizing the voice recognition accuracy, the audio score and the answer text score to obtain a final evaluation result.

Disclosed are apparatuses, systems, and techniques that leverage one or more language models (LMs)—such as large language models (LLMs—for efficient multi-speaker speech recognition. The techniques include processing, using a speaker diarization model, an audio feature to generate a first association of the audio feature with one or more prospective speakers, the audio feature being representative of one or more spoken words. The techniques further include providing, to an LM, a first prompt requesting the LM to identify a second association of the one or more spoken words with the one or more prospective speakers and receiving, from the LM, a first response identifying the second association of the one or more spoken words with the one or more prospective speakers. The techniques further include determining, using the first association and the second association, one or more speakers that produced the one or more spoken words.

The invention relates to the technical field of speech recognition and artificial intelligence, in particular to an automatic evaluation and correction system for spoken Chinese pronunciation based on AI speech recognition, which comprises a speech input module, a speech preprocessing module, a pronunciation analysis module, a pronunciation scoring module and a pronunciation correction module. Wherein the voice input module is used for receiving a Chinese spoken pronunciation signal of a user; the voice preprocessing module is used for preprocessing the voice data; the pronunciation analysis module is used for analyzing pronunciation of the user in real time by using an AI voice recognition technology; the pronunciation scoring module is used for calculating a score value of pronunciation of the user; and the pronunciation correction module is used for providing a targeted pronunciation correction scheme for the user. According to the invention, through the multi-dimensional pronunciation evaluation and personalized correction scheme based on the AI speech recognition technology, the pronunciation deviation of the user can be accurately recognized, real-time feedback can be provided, and the pronunciation correction accuracy and efficiency can be effectively improved.

The invention discloses a multi-modal emotion calculation enabling adaptive interactive language learning system and a matching method thereof, and belongs to the technical field of artificial intelligence, and the system comprises a multi-modal emotion real-time perception and quantification module MHAE-Net, a personalized adaptive dialogue and intervention strategy module ADIS-RL, and a virtual dialogue partner interactive interface module EC-NLG. The multi-modal emotion real-time perception and quantification module MHAE-Net is composed of a voice signal acquisition and processing unit, a voice emotion analysis unit, a text emotion analysis unit, a facial expression emotion analysis unit, a physiological signal emotion analysis unit and a multi-modal emotion fusion and decision-making unit. The interactive strategy is dynamically adjusted, the oral anxiety of the language is effectively relieved, the oral confidence is improved, the system adopts the multi-mode emotion calculation technology, the accuracy and robustness of emotion recognition are improved, the targeted interactive strategy is designed for different emotion states, and personalized emotion support and language practice are provided for learners.

The invention discloses a spoken English man-machine conversation interaction training method and system, and relates to the technical field of spoken English training, spoken English audio signals of a user are monitored in real time through a microphone, and the frequency spectrum information of the audio signals is refined into each frame of spectrogram and cepstrum coefficient feature vectors by using Fourier transform and feature extraction technologies; therefore, the voice signal characteristics of the user can be completely captured; three core dimensions of semantic similarity, grammatical structure similarity and fluency are incorporated into an evaluation model, a comprehensive oral English training evaluation system is formed, a text sequence is generated by analyzing a time sequence dependency relationship through a convolutional neural network, and the oral English training evaluation model of a deep learning technology is combined, so that the oral English training evaluation accuracy is improved. According to the method, the expressions of the user in the aspects of semantics, grammar and fluency can be accurately measured, the oral training level of the user can be comprehensively judged by the system by comparing the comprehensive evaluation coefficient with the preset qualified threshold value, and a personalized learning guide is provided according to a result.

The invention relates to a table question and answer method and device, computer equipment, a computer readable storage medium and a computer program product. The method comprises the following steps: acquiring a form file and question data uploaded by a target user; analyzing table information in the table file to obtain a candidate label set; based on the question data and the candidate tag set, determining a spoken language understanding result corresponding to the question data, wherein the spoken language understanding result comprises an intention and a keyword corresponding to the question data; performing reverse verification on the spoken language understanding result according to the question data to obtain a verification result; under the condition that the verification result indicates that the spoken language understanding result passes verification, a cue word is constructed according to table information, question data and the spoken language understanding result in the table file; calling the large language model according to the cue word for question-answering to obtain a question-answering result. By adopting the method, a large-data-volume table can be processed, and meanwhile, the accuracy and the stability of large-model table question answering can be improved.

The invention relates to the technical field of visual positioning, and discloses a structured visual positioning method, system and device based on intention recognition, and the method comprises the steps: obtaining a to-be-positioned image and a current voice instruction of a user; inputting the current voice instruction into a pre-trained intention recognition model to perform semantic structuring processing so as to construct a semantic topological graph; wherein the semantic topological graph is used for describing target structured data corresponding to the current voice instruction; reasoning based on the semantic topological graph to obtain a semantic reference corresponding to the current voice instruction; and performing feature alignment on the semantic reference and the to-be-positioned image so as to position a target object matched with the semantic reference in the to-be-positioned image. According to the invention, through semantic structuring processing on the voice instruction, accurate reasoning can be carried out on the semantics containing the spoken language instruction, so that the robustness of a visual positioning system is greatly improved, and the visual positioning effect in a daily dialogue scene is improved.

To effectively resolve the issue of inappropriate handovers during communication sessions, methods, systems, and machine-readable mediums which utilize speech features captured by microphones in the original wireless peripheral device and / or the wireless peripheral device to which the communication session is to be handed over to determine if a handover should proceed or should be reversed. Speech features refer to the composite attributes of spoken language that encompass both acoustic and linguistic features. Acoustic features characterize the sound properties of speech and include, but are not limited to, timbre, pitch, intonation, speaking rate, articulation, prosody, melody, spectral features, formant frequencies, and the like. Linguistic features pertain to the actual content conveyed, comprising words, phrases, syntax, and semantics. This includes the analysis of words, phrases, syntax, and semantics to understand the context and continuity of the conversation.

The invention relates to the technical field of artificial intelligence education, in particular to a Russian spoken language practice personalized recommendation method and system based on artificial intelligence, and the method comprises the steps: 1, outputting a phoneme sequence with a timestamp through Russian automatic voice recognition; 2, collecting an exercise interruption position and repeated read-after behavior data; 3, generating a dynamic learner portrait; 4, mapping high-frequency errors in the learner portrait into abnormal path weights of map nodes; 5, a lattice tail error option and a non-matching body verb interference item are injected; 6, when the voice fluency attenuation of the learner exceeds a dynamic threshold value, the sentence complexity is reduced; and 7, calculating an error rate descent gradient based on the exercise completion data, and dynamically adjusting the abnormal path weight of the knowledge graph. Through audio stream analysis and syntax tree construction, the system can accurately identify errors of the learner in grammar, pronunciation and other aspects, and the learning efficiency is improved.

The invention discloses a spoken language understanding joint model method based on label attention and a window mechanism. The spoken language understanding joint model method is used for improving the effects of intention recognition and slot filling. The method comprises the following steps: firstly, performing semantic coding on a statement input by a user by adopting a self-attention mechanism and Bi-LSTM coding to generate basic semantic representation; then, dynamically adjusting attention distribution of each lexical element through a tag attention mechanism, extracting sentence-level intention and slot tag semantics, and constructing an overall semantic context, so as to form an intention tag attention module and a slot tag attention module; the intention module preliminarily predicts the intention of a statement by using a block-level sliding window, and the slot module preliminarily predicts slot position information by means of a slot classifier. And then, through a graph convolution layer, carrying out adaptive fusion on the preliminarily predicted intention and slot position information, and realizing information interaction between nodes to obtain an updated label embedding representation. And finally, decoding the embedded representation through a classifier module, and generating a final intention and slot position recognition result.

At least one exemplary embodiment is directed to a method and device for voice operated control with learning. The method can include measuring a first sound received from a first microphone, measuring a second sound received from a second microphone, detecting a spoken voice based on an analysis of measurements taken at the first and second microphone, learning from the analysis when the user is speaking and a speaking level in noisy environments, training a decision unit from the learning to be robust to a detection of the spoken voice in the noisy environments, mixing the first sound and the second sound to produce a mixed signal, and controlling the production of the mixed signal based on the learning of one or more aspects of the spoken voice and ambient sounds in the noisy environments.

Disclosed are apparatuses, systems, and techniques that may use machine learning for implementing automatic speech recognition (ASR) facilitated with a search for target words. The techniques include applying an ASR model to audio data to generate an ASR output representative of a likelihood that the audio data comprises one or more spoken speech units (SUs), generating, using the ASR output, a first score characterizing a likelihood that the audio data comprises a first word, wherein the first word comprises a dictionary word, generating, using the ASR output, a second score characterizing a likelihood that the audio data comprises a second word, wherein the second word comprises a word of a plurality of target words, wherein the plurality of target words is identified based at least on a context of the audio data, and predicting, using the first score and the second score, a spoken word associated with the audio data.

A method for operating a voice trigger is provided. In some implementations, the method is performed at an electronic device including one or more processors and memory storing instructions for execution by the one or more processors. The method includes receiving a sound input. The sound input may correspond to a spoken word or phrase, or a portion thereof. The method includes determining whether at least a portion of the sound input corresponds to a predetermined type of sound, such as a human voice. The method includes, upon a determination that at least a portion of the sound input corresponds to the predetermined type, determining whether the sound input includes predetermined content, such as a predetermined trigger word or phrase. The method also includes, upon a determination that the sound input includes the predetermined content, initiating a speech-based service, such as a voice-based digital assistant.

Methods and apparatus to improve user experience on computing devices are disclosed. An example computing device includes a microphone to capture audio corresponding to spoken words. The example computing device further includes a speech analyzer to: detect a keyword prompt from among the spoken words, the keyword prompt to precede a query statement of a user of the computing device; and identify topics associated with a subset of the spoken words, the subset of the spoken words captured by the microphone before the keyword prompt. The example computing device also includes a communications interface to, in response to detection of the keyword prompt, transmit information indicative of the query statement and ones of the identified topics to a remote server.

In accordance with at least one aspect of this disclosure, an artificial intelligence driven bi-directional medical assistant is provided. The assistant comprises an input module configured to recognize, in real time, spoken language and convert the spoken language to a computer readable form to generate a patient embedding. The computer readable form includes, in certain embodiments, a mathematical vector associated with the patient embedding, and the spoken language includes a conversation between a clinician and a patient during a patient visit.

The invention discloses a correction system, method and device for spoken language vocal training, and relates to the technical field of speech recognition, and the system comprises an environment data analysis module which is used for collecting environment noise parameters in real time, preliminarily judging whether a training environment meets the requirements or not according to the environment noise parameters, and optimizing the training environment; the multi-mode environment adaptability module is used for collecting optimized environment noise parameters and adjusting voice judgment weights and visual judgment weights according to the optimized environment noise parameters; and the spoken language accuracy judgment module is used for collecting pronunciation data and facial image data of a pronunciator, performing pronunciation accuracy evaluation in combination with a voice judgment weight and a visual judgment weight, and providing targeted correction suggestions according to an evaluation result, so that the problem of evaluation deviation caused by failure of a single mode in the prior art is solved, and the accuracy of the pronunciation is improved. The comprehensive evaluation of the pronunciation accuracy is realized, the overall environmental adaptability is improved, and the pronunciation evaluation accuracy is enhanced.

One or more embodiments include: receiving a first frame of acoustic input and one or more prior textual tokens associated with a prior frame of the acoustic input, wherein the prior textual token represents one or more spoken word included in the acoustic input; generating a multi-dimensional embedding associated with the prior textual token, wherein each dimension of the embedding represents a different characteristic of the prior textual token, and at least one dimension of the embedding represents pronunciation information associated with the prior textual token; and generating a textual token associated with the first frame based at least on an encoded representation of the first frame and the multi-dimensional embedding associated with the prior textual token.

A method for operating a voice trigger is provided. In some implementations, the method is performed at an electronic device including one or more processors and memory storing instructions for execution by the one or more processors. The method includes receiving a sound input. The sound input may correspond to a spoken word or phrase, or a portion thereof. The method includes determining whether at least a portion of the sound input corresponds to a predetermined type of sound, such as a human voice. The method includes, upon a determination that at least a portion of the sound input corresponds to the predetermined type, determining whether the sound input includes predetermined content, such as a predetermined trigger word or phrase. The method also includes, upon a determination that the sound input includes the predetermined content, initiating a speech-based service, such as a voice-based digital assistant.

The invention relates to the technical field of teaching equipment, and discloses an online teaching auxiliary device for oral language teaching of teachers, which comprises a fixed support column used for the structure and installation of all structures, a fixed support column matched with a fixed disc I for fixing a scissor-type support mechanism I, a fixed support column matched with a fixed disc II for fixing a scissor-type support mechanism II, and a fixed support column matched with a fixed disc II for fixing a scissor-type support mechanism II, the two-way transmission mechanism and the operation motor are located at the top of the fixed supporting column and matched for automatic unfolding of the first shear type supporting mechanism, and the first shear type supporting mechanism, the shear type linkage mechanism and the second shear type supporting mechanism are synchronously unfolded through the linkage column. Students can practice and cope with various situations in a safe environment, so that the adaptive capacity and decision-making skills of the students are improved, the system covers multiple aspects of oral teaching, comprehensive education content is provided from reception, reservation to dispute handling, it is ensured that the students can comprehensively understand and master various aspects of oral teaching, and the teaching efficiency is improved. And visualization and multi-sensory learning are realized.

The invention provides a foreign language slang teaching device with a fuzzy recognition function and a use method, and relates to a foreign language teaching device. Comprising a mounting base, a learning machine body mounted on the mounting base and a supporting mechanism for supporting the mounting base, and one end of the learning machine body is hinged to the top of the mounting base through a pin shaft. According to the device, by driving the earphones in the storage box, the headphones can transmit speech during oral training to the speech recognition module in the learning machine body through the microphones on the earphones for recognition, and sliding of the sliding frame on the mounting base is adjusted through the adjusting mechanism; the learning machine body is pushed to turn over under the action of the connecting rod, the observation and use angle of the touch screen display is adjusted, students can observe whether their mouth shapes are correct or not through the arranged screen when doing oral language practice, the students are further assisted in oral language learning, and the teaching effect is improved.

Devices, systems, and techniques are disclosed that implement the training and deployment of automatic transcription-based translation systems using language models. The techniques include: processing, using a first speech-to-text (S2T) model, an initial input that includes spoken language in a first language to generate a transcription of the spoken language; and processing, using a second S2T model, a second input to generate a translation of the spoken language into a second language. The second input includes at least a representation of the spoken language and the transcription of the spoken language.

An estimation method includes: obtaining a first voice feature group of a plurality of persons who speak a first language; obtaining a second voice feature group of a plurality of persons who speak a second language; obtaining a voice feature of a subject; correcting the voice feature of the subject according to a relationship between the first voice feature group and the second voice feature group; estimating, from the voice feature of the subject that has been corrected, an oral function or a cognitive function of the subject by using an estimation process for an oral function or a cognitive function based on the second language; and outputting a result of estimation of the oral function or the cognitive function of the subject.