450 results about "Spoken language" patented technology

The present disclosure relates to systems and methods for enhancing the interaction between users and automated agents, such as digital assistants, by employing Large Language Models (LLMs) to infer the intent of spoken language. The invention involves continuously monitoring ambient audio, converting speech to text, and utilizing LLMs to determine whether spoken language is intended for the automated agent. A structured prompt, including the converted text and specific instructions, is sent to the LLM, which is fine-tuned to process domain-specific prompts. The LLM provides a structured output in a standardized format, indicating the user's intent. The system may involve multiple prompts to perform separate tasks, such as identifying intent and generating additional context-specific data. This approach facilitates a more natural and intuitive user experience by eliminating the need for wake words and allowing seamless conversational interaction with virtual assistants across various platforms and devices.

A method includes receiving a natural language query from a user that solicits a response from an assistant large language model (LLM), receiving a prompt composition including an instruction parameter that specifies a task for the assistant LLM to respond to user queries concisely, structuring a conciseness prompt by concatenating the prompt composition to the natural language query, and processing, using the assistant LLM, the conciseness prompt to generate a concise response to the natural language query. The method also includes providing, for output from a user device, the concise response to the natural language query.

An integrated development environment (IDE) leverages a generative AI model to generate industrial control code in accordance with specified functional requirements, which can be provided to the industrial IDE system as intuitive natural language spoken or written text. The industrial IDE can also analyze written code in response to natural language prompts submitted against the code, generate answers to user-submitted questions about the code, and offer recommendations for improving the code in response to specific questions or requests submitted by the user.

A method includes obtaining labeled training data including a plurality of spoken terms spoken during a conversation. For each respective spoken term, the method includes generating a corresponding sequence of intermediate audio encodings from a corresponding sequence of acoustic frames, generating a corresponding sequence of final audio encodings from the corresponding sequence of intermediate audio encodings, generating a corresponding speech recognition result, and generating a respective speaker token representing a predicted identity of a speaker for each corresponding speech recognition result. The method also includes training the joint speech recognition and speaker diarization model jointly based on a first loss derived from the generated speech recognition results and the corresponding transcriptions and a second loss derived from the generated speaker tokens and the corresponding speaker labels.

The invention belongs to the technical field of intelligent voice processing, and particularly relates to a spoken language pronunciation training and correcting system based on intelligent equipment, which acquires a user rhythm feature set including pitch change rate, accent intensity, pause duration and intonation contour by acquiring spoken language audio data sent by a user for a target text, and corrects the spoken language pronunciation training and correcting system. The method comprises the following steps: acquiring spoken language audio data, converting the spoken language audio data into a phoneme sequence aligned with target text time, generating a rhythm deviation degree report according to comparison evaluation of a user rhythm feature set and the phoneme sequence, an execution intonation mode, accent distribution, speech stream sound change and speech speed rhythm, and determining the rhythm deviation degree according to a rhythm problem type in the report in combination with user historical learning data. And forming and outputting a correction scheme including a text prompt, a targeted minimum contrast training unit and a listen-and-read simulation task, solving the problem of weak capability of correcting hyper-phoneme in a second language spoken language of an adult, and improving the authentic and fluency of the spoken language, thereby realizing efficient personalized learning.

The invention discloses a spoken language practice scene-oriented education large model evaluation method, which comprises the following steps of: obtaining voice data and a corresponding standard text, interacting with a test model through the voice data, and obtaining a recognition text, audio data and an answer text provided by the test model; the standard text and the recognition text are compared and evaluated, and the voice recognition accuracy is obtained; carrying out pronunciation accuracy, fluency and rhythm evaluation on the audio data based on multi-modal feature fusion to obtain an audio score; performing grammar accuracy evaluation on the answer text based on a dynamic punishment mechanism and the constructed prompt large model, and performing question degree evaluation on the answer text based on the constructed prompt large model to obtain an answer text score; and comprehensively summarizing the voice recognition accuracy, the audio score and the answer text score to obtain a final evaluation result.

The invention relates to the technical field of speech recognition and artificial intelligence, in particular to an automatic evaluation and correction system for spoken Chinese pronunciation based on AI speech recognition, which comprises a speech input module, a speech preprocessing module, a pronunciation analysis module, a pronunciation scoring module and a pronunciation correction module. Wherein the voice input module is used for receiving a Chinese spoken pronunciation signal of a user; the voice preprocessing module is used for preprocessing the voice data; the pronunciation analysis module is used for analyzing pronunciation of the user in real time by using an AI voice recognition technology; the pronunciation scoring module is used for calculating a score value of pronunciation of the user; and the pronunciation correction module is used for providing a targeted pronunciation correction scheme for the user. According to the invention, through the multi-dimensional pronunciation evaluation and personalized correction scheme based on the AI speech recognition technology, the pronunciation deviation of the user can be accurately recognized, real-time feedback can be provided, and the pronunciation correction accuracy and efficiency can be effectively improved.

The invention discloses a multi-modal emotion calculation enabling adaptive interactive language learning system and a matching method thereof, and belongs to the technical field of artificial intelligence, and the system comprises a multi-modal emotion real-time perception and quantification module MHAE-Net, a personalized adaptive dialogue and intervention strategy module ADIS-RL, and a virtual dialogue partner interactive interface module EC-NLG. The multi-modal emotion real-time perception and quantification module MHAE-Net is composed of a voice signal acquisition and processing unit, a voice emotion analysis unit, a text emotion analysis unit, a facial expression emotion analysis unit, a physiological signal emotion analysis unit and a multi-modal emotion fusion and decision-making unit. The interactive strategy is dynamically adjusted, the oral anxiety of the language is effectively relieved, the oral confidence is improved, the system adopts the multi-mode emotion calculation technology, the accuracy and robustness of emotion recognition are improved, the targeted interactive strategy is designed for different emotion states, and personalized emotion support and language practice are provided for learners.

The invention relates to the technical field of visual positioning, and discloses a structured visual positioning method, system and device based on intention recognition, and the method comprises the steps: obtaining a to-be-positioned image and a current voice instruction of a user; inputting the current voice instruction into a pre-trained intention recognition model to perform semantic structuring processing so as to construct a semantic topological graph; wherein the semantic topological graph is used for describing target structured data corresponding to the current voice instruction; reasoning based on the semantic topological graph to obtain a semantic reference corresponding to the current voice instruction; and performing feature alignment on the semantic reference and the to-be-positioned image so as to position a target object matched with the semantic reference in the to-be-positioned image. According to the invention, through semantic structuring processing on the voice instruction, accurate reasoning can be carried out on the semantics containing the spoken language instruction, so that the robustness of a visual positioning system is greatly improved, and the visual positioning effect in a daily dialogue scene is improved.

To effectively resolve the issue of inappropriate handovers during communication sessions, methods, systems, and machine-readable mediums which utilize speech features captured by microphones in the original wireless peripheral device and / or the wireless peripheral device to which the communication session is to be handed over to determine if a handover should proceed or should be reversed. Speech features refer to the composite attributes of spoken language that encompass both acoustic and linguistic features. Acoustic features characterize the sound properties of speech and include, but are not limited to, timbre, pitch, intonation, speaking rate, articulation, prosody, melody, spectral features, formant frequencies, and the like. Linguistic features pertain to the actual content conveyed, comprising words, phrases, syntax, and semantics. This includes the analysis of words, phrases, syntax, and semantics to understand the context and continuity of the conversation.

The invention relates to the technical field of artificial intelligence education, in particular to a Russian spoken language practice personalized recommendation method and system based on artificial intelligence, and the method comprises the steps: 1, outputting a phoneme sequence with a timestamp through Russian automatic voice recognition; 2, collecting an exercise interruption position and repeated read-after behavior data; 3, generating a dynamic learner portrait; 4, mapping high-frequency errors in the learner portrait into abnormal path weights of map nodes; 5, a lattice tail error option and a non-matching body verb interference item are injected; 6, when the voice fluency attenuation of the learner exceeds a dynamic threshold value, the sentence complexity is reduced; and 7, calculating an error rate descent gradient based on the exercise completion data, and dynamically adjusting the abnormal path weight of the knowledge graph. Through audio stream analysis and syntax tree construction, the system can accurately identify errors of the learner in grammar, pronunciation and other aspects, and the learning efficiency is improved.

The invention discloses a spoken language understanding joint model method based on label attention and a window mechanism. The spoken language understanding joint model method is used for improving the effects of intention recognition and slot filling. The method comprises the following steps: firstly, performing semantic coding on a statement input by a user by adopting a self-attention mechanism and Bi-LSTM coding to generate basic semantic representation; then, dynamically adjusting attention distribution of each lexical element through a tag attention mechanism, extracting sentence-level intention and slot tag semantics, and constructing an overall semantic context, so as to form an intention tag attention module and a slot tag attention module; the intention module preliminarily predicts the intention of a statement by using a block-level sliding window, and the slot module preliminarily predicts slot position information by means of a slot classifier. And then, through a graph convolution layer, carrying out adaptive fusion on the preliminarily predicted intention and slot position information, and realizing information interaction between nodes to obtain an updated label embedding representation. And finally, decoding the embedded representation through a classifier module, and generating a final intention and slot position recognition result.

Disclosed are apparatuses, systems, and techniques that may use machine learning for implementing automatic speech recognition (ASR) facilitated with a search for target words. The techniques include applying an ASR model to audio data to generate an ASR output representative of a likelihood that the audio data comprises one or more spoken speech units (SUs), generating, using the ASR output, a first score characterizing a likelihood that the audio data comprises a first word, wherein the first word comprises a dictionary word, generating, using the ASR output, a second score characterizing a likelihood that the audio data comprises a second word, wherein the second word comprises a word of a plurality of target words, wherein the plurality of target words is identified based at least on a context of the audio data, and predicting, using the first score and the second score, a spoken word associated with the audio data.

A method for operating a voice trigger is provided. In some implementations, the method is performed at an electronic device including one or more processors and memory storing instructions for execution by the one or more processors. The method includes receiving a sound input. The sound input may correspond to a spoken word or phrase, or a portion thereof. The method includes determining whether at least a portion of the sound input corresponds to a predetermined type of sound, such as a human voice. The method includes, upon a determination that at least a portion of the sound input corresponds to the predetermined type, determining whether the sound input includes predetermined content, such as a predetermined trigger word or phrase. The method also includes, upon a determination that the sound input includes the predetermined content, initiating a speech-based service, such as a voice-based digital assistant.

Methods and apparatus to improve user experience on computing devices are disclosed. An example computing device includes a microphone to capture audio corresponding to spoken words. The example computing device further includes a speech analyzer to: detect a keyword prompt from among the spoken words, the keyword prompt to precede a query statement of a user of the computing device; and identify topics associated with a subset of the spoken words, the subset of the spoken words captured by the microphone before the keyword prompt. The example computing device also includes a communications interface to, in response to detection of the keyword prompt, transmit information indicative of the query statement and ones of the identified topics to a remote server.

In accordance with at least one aspect of this disclosure, an artificial intelligence driven bi-directional medical assistant is provided. The assistant comprises an input module configured to recognize, in real time, spoken language and convert the spoken language to a computer readable form to generate a patient embedding. The computer readable form includes, in certain embodiments, a mathematical vector associated with the patient embedding, and the spoken language includes a conversation between a clinician and a patient during a patient visit.

The invention discloses a correction system, method and device for spoken language vocal training, and relates to the technical field of speech recognition, and the system comprises an environment data analysis module which is used for collecting environment noise parameters in real time, preliminarily judging whether a training environment meets the requirements or not according to the environment noise parameters, and optimizing the training environment; the multi-mode environment adaptability module is used for collecting optimized environment noise parameters and adjusting voice judgment weights and visual judgment weights according to the optimized environment noise parameters; and the spoken language accuracy judgment module is used for collecting pronunciation data and facial image data of a pronunciator, performing pronunciation accuracy evaluation in combination with a voice judgment weight and a visual judgment weight, and providing targeted correction suggestions according to an evaluation result, so that the problem of evaluation deviation caused by failure of a single mode in the prior art is solved, and the accuracy of the pronunciation is improved. The comprehensive evaluation of the pronunciation accuracy is realized, the overall environmental adaptability is improved, and the pronunciation evaluation accuracy is enhanced.

A method for operating a voice trigger is provided. In some implementations, the method is performed at an electronic device including one or more processors and memory storing instructions for execution by the one or more processors. The method includes receiving a sound input. The sound input may correspond to a spoken word or phrase, or a portion thereof. The method includes determining whether at least a portion of the sound input corresponds to a predetermined type of sound, such as a human voice. The method includes, upon a determination that at least a portion of the sound input corresponds to the predetermined type, determining whether the sound input includes predetermined content, such as a predetermined trigger word or phrase. The method also includes, upon a determination that the sound input includes the predetermined content, initiating a speech-based service, such as a voice-based digital assistant.

The invention provides a foreign language slang teaching device with a fuzzy recognition function and a use method, and relates to a foreign language teaching device. Comprising a mounting base, a learning machine body mounted on the mounting base and a supporting mechanism for supporting the mounting base, and one end of the learning machine body is hinged to the top of the mounting base through a pin shaft. According to the device, by driving the earphones in the storage box, the headphones can transmit speech during oral training to the speech recognition module in the learning machine body through the microphones on the earphones for recognition, and sliding of the sliding frame on the mounting base is adjusted through the adjusting mechanism; the learning machine body is pushed to turn over under the action of the connecting rod, the observation and use angle of the touch screen display is adjusted, students can observe whether their mouth shapes are correct or not through the arranged screen when doing oral language practice, the students are further assisted in oral language learning, and the teaching effect is improved.

Devices, systems, and techniques are disclosed that implement the training and deployment of automatic transcription-based translation systems using language models. The techniques include: processing, using a first speech-to-text (S2T) model, an initial input that includes spoken language in a first language to generate a transcription of the spoken language; and processing, using a second S2T model, a second input to generate a translation of the spoken language into a second language. The second input includes at least a representation of the spoken language and the transcription of the spoken language.

An estimation method includes: obtaining a first voice feature group of a plurality of persons who speak a first language; obtaining a second voice feature group of a plurality of persons who speak a second language; obtaining a voice feature of a subject; correcting the voice feature of the subject according to a relationship between the first voice feature group and the second voice feature group; estimating, from the voice feature of the subject that has been corrected, an oral function or a cognitive function of the subject by using an estimation process for an oral function or a cognitive function based on the second language; and outputting a result of estimation of the oral function or the cognitive function of the subject.

A computer-implemented method, a computer program product, and a computer system for conversational collaboration in a metaverse environment. A computer uses natural language processing to understand a question of a user about an object in the metaverse environment. A computer constructs an aggregated response to the question, based on one or more responses in the metaverse environment. A computer identifies authentic sources and unauthentic sources of the aggregated response. A computer deploys first avatars for the authentic sources and second avatars for the unauthentic sources. A computer causes the first avatars to perform a spoken conversation based on the authentic sources, with a first type of tone and body language. A computer causes the second avatars to perform a spoken conversation based on the unauthentic sources, with a second type of tone and body language.

The application discloses a multi-intent spoken language understanding method based on syntax analysis, which comprises the following steps: firstly, obtaining an intent feature matrix and a slot feature matrix according to a user input sentence, and constructing a multi-level intent feature from the intent feature matrix; secondly, obtaining initial intent labels and initial slot prediction labels by using an intent decoding module and a slot decoding module respectively on the intent feature matrix and the slot feature matrix; then, inputting the multi-level intent feature and the initial slot prediction labels into a slot-intent interaction module to obtain an enhanced intent feature matrix, and inputting the slot feature matrix, the multi-level intent feature and the initial intent labels into an intent-slot interaction module to obtain an enhanced slot feature matrix; finally, inputting the enhanced intent feature matrix and the slot feature matrix into the intent decoding module and the slot decoding module respectively to obtain intent labels and slot sequence labels of the spoken language understanding task. The application improves the accuracy of intent recognition and slot sequence labeling and the accuracy of multi-intent recognition.

The invention provides a spoken language pronunciation scoring method and system and a spoken language pronunciation scoring all-in-one machine, and relates to the technical field of spoken language evaluation, and the method comprises the steps: firstly obtaining a scoring task text, a to-be-scored voice, and an evaluation text corresponding to the to-be-scored voice; and then extracting voice features of the to-be-scored voice, and inputting the scoring task text, the voice features and the evaluation text into the evaluation model to obtain an evaluation score for the to-be-scored voice output by the evaluation model. According to the method, multi-modal input is provided for the evaluation model, the to-be-scored voice can be evaluated without participation of a phoneme recognition module, the evaluation score is further obtained, the scoring process is simple, the calculation overhead can be reduced, and the pronunciation scoring efficiency can be improved. And moreover, by utilizing a single model, the accuracy of the evaluation score can be improved. The method can be widely applied to spoken language evaluation services such as learning machines and teaching and examination platforms.

The invention discloses a spoken language evaluation method and device based on deep learning and a medium, and relates to the technical field of spoken language evaluation, and the method comprises the steps: collecting spoken language audio signals, carrying out the acoustic feature extraction of the spoken language audio signals through Mel-frequency cepstrum coefficient transformation, and generating an acoustic feature vector sequence; constructing a deep learning pronunciation diagnosis model, inputting the acoustic feature vector sequence into the deep learning pronunciation diagnosis model, calculating a multi-dimensional distance between each voice segment in the acoustic feature vector sequence and the phoneme prototype in a measurement space, and generating a pronunciation diagnosis result; converting the acoustic feature vector sequence into a text sequence through a speech recognition conversion method; and constructing a deep learning role analysis model, and inputting the text sequence into the deep learning role analysis model to generate a semantic role graph. According to the method, the acoustic deviation between the quantized speech segment and the standard phoneme in the measurement space is calculated through the phoneme prototype distance, and accurate space-time positioning and quantitative guidance of the pronunciation defect are realized.

A method includes receiving a training example that includes audio data representing a spoken utterance and a ground truth transcription. For each word in the spoken utterance, the method also includes inserting a placeholder symbol before the respective word identifying a respective ground truth alignment for a beginning and an end of the respective word, determining a beginning word piece and an ending word piece, and generating a first constrained alignment for the beginning word piece and a second constrained alignment for the ending word piece. The first constrained alignment is aligned with the ground truth alignment for the beginning of the respective word and the second constrained alignment is aligned with the ground truth alignment for the ending of the respective word. The method also includes constraining an attention head of a second pass decoder by applying the first and second constrained alignments.

An integrated development environment (IDE) for designing, programming, and configuring aspects of an industrial automation system uses a generative artificial intelligence (AI) model and associated neural networks to generate portions of an industrial automation project in accordance with functional requirements provided to the industrial IDE system in intuitive formats, such as spoken or written plain language text. The system uses generative AI to translate plain language requests or functional specifications into industrial control code, human-machine interface (HMI) applications, device configuration settings, or other aspects of an industrial control project.

The invention discloses a text understanding model training method and device, equipment and a storage medium. Comprising the steps of obtaining a sample text and a sample label set; inputting the plurality of sample intention tags and the plurality of sample slot tags into a preset text understanding model for semantic relationship coding processing to obtain an intention tag relationship representation and a slot tag relationship representation; performing intention decoding processing on the sample text representation and the intention label relation representation based on a preset text understanding model to obtain a predicted intention result, and performing slot decoding processing on the sample text representation and the slot label relation representation to obtain a predicted slot result; determining target loss according to the prediction intention result, the target intention label, the prediction slot position result and the target slot position label; and performing iterative training on a preset text understanding model based on the target loss to obtain a trained text understanding model. The method can improve the prediction accuracy of intention prediction and slot prediction, and can be used for various scenes such as artificial intelligence, spoken language understanding, task-based dialogues and the like.

The invention discloses a calibration data set acquisition method, a spoken language test scoring method and a related device. The method comprises the following steps: acquiring at least one abnormal voice data; wherein the abnormal voice data is first answering voice data with an abnormal sound event, and the first answering voice data is obtained by performing voice answering on the oral test by an answerer; obtaining a target calibration data set by using the at least one abnormal voice data and the initial calibration data set; wherein the target calibration data set is used for training the oral test scoring system. In this way, the coverage range of the calibration data set can be wider.