155 results about "Speech interaction" patented technology

The present invention relates to the field of voice interaction analysis technology, and in particular to a voice interaction analysis method and system for an intelligent voice robot. The method comprises the following steps: improving the clarity of user interaction voice through multi-scale noise decomposition and filtering enhancement, performing granular segmentation and long-distance semantic analysis on clear voice, generating a semantic structure, performing multi-level semantic recognition based on the structure, extracting specific meanings and making preliminary responses, generating personalized semantic graphs through independent graph structure mapping, predicting user interaction dialogues, building an advance response engine based on dialogue templates, preparing to respond to user needs, embedding the engine into an intelligent voice robot, and realizing efficient voice interaction analysis. The present invention realizes a more accurate and efficient voice interaction analysis method for an intelligent voice robot.

The invention discloses a voice interaction task execution method and device based on a large model, equipment and a medium, and relates to the field of artificial intelligence, and the method comprises the steps: carrying out the voice recognition of voice features, obtaining a text character sequence, and carrying out the optimization of the text character sequence through a preset large language model, and obtaining a target text character sequence; performing entity recognition on the target text character sequence by using a preset large language model to obtain an entity recognition result, determining a relationship type among entities in the target text character sequence according to the entity recognition result, and constructing a knowledge graph according to the relationship type among the entities; generating an initial triple based on the knowledge graph, and optimizing the initial triple by using a preset large language model to obtain a target triple; and fusing the target triple with the initial knowledge graph, and executing a voice interaction task in the target voice interaction scene based on the updated knowledge graph. According to the method and the device, the efficiency and the accuracy of extracting the structured knowledge from the Chinese speech are improved.

An interactive method based on a large model, a training method, and an intelligent agent, which relate to fields of artificial intelligence, speech recognition, speech interaction, deep learning, large models, and application scenarios of knowledge search, autonomous driving, intelligent customer service, intelligent speech control, smart e-commerce, AI healthcare. The interactive method includes: acquiring a request speech; performing a speech recognition on the request speech to obtain a speech recognition feature representing a request semantics; and processing the speech recognition feature using the large model to obtain a response text, where the response text includes response words arranged in sequence, a target response word among the response words is determined by processing the speech recognition feature and an associated response word feature using an attention fusion layer of the large model, and the associated response word feature is related to an associated response word arranged before the target response word.

The invention provides a voice replication and low-delay streaming voice synthesis method and system based on an ultra-short sample, relates to the technical field of artificial intelligence, and is suitable for intelligent interaction, outbound service and multi-mode communication scenes. Deep personalized customization of intelligent voice interaction and real-time generation of ultra-low delay are realized, and bidirectional streaming interaction of a system level is supported, so that the fluency and response speed of dialogues are improved; an ultra-short sample sound duplicating module specially designed for processing ultra-short audio samples and a speech synthesis engine with ultra-low delay and bidirectional streaming output capability are integrated, and the capabilities of the ultra-short sample sound duplicating module and the speech synthesis engine are applied to a real-time and interactive intelligent speech interaction process to form a complete and efficient solution. The industrial pain point is directly solved in a targeted manner, and the method has important commercial application value.

A method for processing cross-modal question answering based on large model, an apparatus, and a storage medium are suggested, which relates to the field of artificial intelligence technologies such as speech interaction processing, large models, machine learning and natural language processing. The specific implementation includes: performing an activity detection on a target speech input by a user; in response to detecting a pause in the inputting of the target speech, obtaining a first text corresponding to a first input speech before the moment of the pause in the target speech; performing a text response processing using a pre-trained speech question answering processing system based on the first text and the first input speech.

The present disclosure relates to the technical field of speech processing. Provided are a dialog processing method and apparatus, and an electronic device, a computer-readable storage medium and a computer program product. The method comprises: when user speech of a user is input during the playing of robot speech, stopping the playing of the robot speech, and inputting the user speech and the robot speech into a resuming model; when it is determined by means of the resuming model that the user has no intent to really interrupt the robot speech, acquiring timestamp information corresponding to the robot speech when the user speech was input, and on the basis of the timestamp information, determining a playing-resumed node corresponding to the robot speech, e.g., setting the playing-resumed node at a preset pause position which corresponds to the robot speech when the user speech was input; and playing the robot speech from the playing-resumed node. By means of the present disclosure, the intent of a user can be determined, the situation in which speech is accidentally interrupted by the user is avoided, and the smoothness and continuity of speech resuming are ensured, thereby improving the speech interaction effect.

A speech interaction method includes receiving, by a server, a first play message, where the first play message includes an identifier of first audio content corresponding to a first non-speech instruction. The server determines a first intent and first slot information that correspond to the first non-speech instruction. In response to the first play message, the server instructs a playback device to play the first audio content. The server receives a first speech instruction input by a user into the playback device, where a second intent or second slot information or both in the first speech instruction are incomplete. The server determines, based on the first intent and the first slot information, the second intent and the second slot information that correspond to the first speech instruction, and the server, based on the second intent and the second slot information, instructs the playback device to play second audio content.

A speech interaction method, a speech interaction system and a storage medium. The method includes: receiving interactive speech input by a user; determining an emotional tag corresponding to the interactive speech based on the interactive speech and interactive text corresponding to the interactive speech; determining, based on the emotional tag, a response text corresponding to the interactive text, and a first prosodic feature and a second prosodic feature corresponding to the response text. The first prosodic feature is used for characterizing the whole sentence prosodic feature of the response text, and the second prosodic feature is used for characterizing a local prosodic feature of each character in the response text; and generating and outputting a response speech corresponding to the interactive speech based on the response text, the first prosodic feature and the second prosodic feature.

The invention discloses a multi-modal voice interaction large model training method and system based on voice acoustic feature regulation and control, terminal equipment and a medium, and relates to the technical field of multi-modal voice interaction.The method comprises the steps that a text token of a text training sample is obtained, a corresponding voice token is constructed, and pre-training data used for converting the text token into the voice token is obtained; in combination with the multi-modal input sample and the pre-training data, constructing fine-tuning training data for speech understanding and dialogue generation; constructing and pre-training a basic model by using the pre-training data; a multi-modal voice interaction large model is constructed based on a pre-training basic model, and fine tuning data is used for training, so that voice acoustic features can be regulated and controlled based on multi-modal input, and voice is output. According to the method, through alignment and staged training of the text token and the voice token, fine regulation and control of voice acoustic characteristics are realized, long voice continuity and interaction naturalness are improved, and a model is efficiently endowed with voice interaction capability of controllable timbre and emotion.

The invention relates to a voice recognition engine switching method and device, computer equipment, a storage medium and a computer program product. The method comprises the following steps: recognizing a to-be-recognized audio stream by adopting a current voice recognition engine, and inputting the audio features of each audio frame in the to-be-recognized audio stream into a language recognition model to obtain the probability that each audio window belongs to a preset language; determining the number of target audio windows in a first preset number of continuous audio windows; when the number of the target audio windows is greater than or equal to a second preset number, determining that the to-be-recognized audio stream belongs to a preset language, and determining a voice recognition engine corresponding to the preset language as a target voice recognition engine; and when the target voice recognition engine is inconsistent with the current voice recognition engine, switching the current voice recognition engine to the target voice recognition engine, and recognizing the to-be-recognized audio stream through the target voice recognition engine. By adopting the method, the speech recognition effect of the multilingual speech interaction scene can be improved.

The invention belongs to the technical field of natural language processing, and discloses an AI voice rate adjustment method and system for an AI platform, and the method comprises the steps: firstly obtaining historical data, including user behavior data and user feedback data; performing feature extraction on the user behavior data to obtain corresponding behavior features; screening out data subsets with excellent and good user feedback from historical data, and extracting corresponding behavior features to form a standard feature group; standard behavior characteristics are obtained based on the calculation; extracting real-time behavior characteristics in a real-time dialogue process of the AI and the user; calculating a characteristic deviation value between the real-time behavior characteristics and the standard behavior characteristics; and finally, adjusting the voice rate of the A I according to the real-time voice rate and the characteristic deviation value. The AI voice rate is automatically adjusted by analyzing the interaction habit of the user, so that the cumbersome process that the user needs to manually set is avoided, and the naturalness of AI voice interaction and the user satisfaction are improved.

The invention discloses a VR large-space active noise reduction and directional speech enhancement system, particularly relates to the technical field of speech enhancement, and aims to obtain a user attention direction by fusing multi-mode sensor data in user wearing equipment, guide a sound source positioning module to quickly lock a speech interaction main direction in a multi-sound-source interference environment, and improve the speech enhancement efficiency. Non-target speakers are further filtered through voiceprint feature recognition, it is ensured that enhanced voices have individual consistency, through deep anti-reverberation modeling, reflection residual sounds generated by multi-path propagation are effectively stripped, semantic fuzziness caused by pseudo sound source aliasing is avoided, a voice enhancement and active noise reduction module corrects the frequency spectrum phase while enhancing the amplitude spectrum, and the noise reduction effect is improved. The definition and naturalness of output voice are remarkably improved, the enhanced focusing performance of the system is monitored in real time by introducing a target locking drift coefficient and a beam pointing deviation coefficient, and adaptive reconstruction is triggered based on a closed-loop correction index.

The invention discloses a microphone voice recognition system and method based on multi-mode audio-visual fusion, and belongs to the technical field of artificial intelligence and voice interaction. Firstly, an audio module collects voice signals through a microphone, voice is converted into texts by means of a cloud voice recognition API, and words are further mapped into 300-dimensional semantic vectors through Word2Vec. A visual module extracts lip movement and log-Mel frequency spectrum features, after being subjected to Dlib detection and normalization processing, lip images are sent into a 3D CNN and a dense space-time CNN to extract space-time features, key areas are highlighted with the assistance of a space attention mechanism, and finally sequence visual features are extracted through bidirectional GRU. Meanwhile, a log-Mel spectrogram is generated from the audio signal, and the perception characteristic is enhanced through Mel filtering and logarithm processing. The audio word vector, the lip movement feature and the log-Mel feature are spliced into a multi-modal fusion vector, the multi-modal fusion vector is sent to a CTC decoder, and a text is predicted through Beam Search decoding. An Adam optimizer and a small-batch training strategy are used in the training process, and the model performance and generalization ability are improved.

The invention provides a Transform-based voice correction system and method, and the system comprises a voice preprocessing module which is used for carrying out the noise suppression and feature extraction of an input southern Fujian dialect voice signal, carrying out the noise suppression through a frequency spectrum subtraction method, and extracting 40-dimensional MFCC features; the voice splicing module is used for solving the problem of voice input fragmentation; the dialect speech recognition module adopts an improved Transform encoder structure; and the character correction module adopts a character level Transform language model. According to the invention, more accurate and smooth voice interaction experience is provided for the user in the dialect area in southern Fujian. Meanwhile, the technical framework has good expandability, can adapt to other dialects and language variants, and provides important support for industrial application of the dialect voice technology.

The invention discloses an airborne flight safety enhancement system and method based on multi-modal interaction, and relates to the field of crossing of avionics systems and artificial intelligence, and the method comprises the steps that an environment sensing module generates structured environment information; the airborne equipment monitoring module performs real-time health state analysis; the large language model module comprises a computing platform sub-module, a language model sub-module, a knowledge base sub-module, a voice recognition sub-module and a voice synthesis sub-module; and the output module is used for displaying the decision support information generated by the large language model module on a cockpit display device. The method has the advantages that through deep cooperation of multi-mode perception fusion, distillation large language model intelligent decision, natural voice interaction and airborne health monitoring, on the premise of ensuring airworthiness safety, the flight safety is remarkably improved, the pilot load is reduced, and the decision efficiency is optimized.

The invention provides an emotional voice interaction method and device based on a humanoid robot, and relates to the field of robot interaction. The method comprises the following steps: acquiring voice data of a target user through a humanoid robot, and extracting acoustic features and semantic features in the voice data; inputting the acoustic features and the semantic features into a cross-modal fusion emotion classification model, and outputting an emotion recognition result corresponding to the target user; determining an emotion recognition misjudgment risk, and correcting an emotion recognition result based on the emotion recognition misjudgment risk; inputting the corrected emotion recognition result and the semantic feature into an emotion voice understanding interaction model, and outputting an emotion voice interaction result; and controlling the humanoid robot to execute a corresponding multi-modal interaction response based on the emotion voice interaction result. The problem that the voice interaction experience is greatly reduced due to the fact that the voice interaction content output by an existing humanoid robot is difficult to fit the real emotional state of the elderly user in multiple aspects such as text expression and voice emotion is solved.

The invention relates to a shared space multi-user silent voice interaction method and device and a medium, and the method comprises the following steps: taking a frequency modulation continuous wave signal as an audio carrier and a sensing medium at the same time, directionally broadcasting an audio in a shared space, and capturing a silent voice signal in a multi-user scene through lip motion reflection; performing feature extraction and separation on the silent voice signals to obtain silent voice features of each user; on the basis of silent voice features, a lip movement analysis model driven by an integrated sliding window is adopted, silent voice content of a user is recognized in real time, and real-time interaction is achieved. Compared with the prior art, multi-user silent voices can be accurately recognized in a complex environment, and silent interaction in a privacy sensitive scene is achieved.

An interactive system may be implemented in part by an audio device located within a user environment, which may accept speech commands from a user and may also interact with the user by means of generated speech. In order to improve performance of the interactive system, a user may use a separate device, such as a personal computer or mobile device, to access a graphical user interface that lists details of historical speech interactions. The graphical user interface may be configured to allow the user to provide feedback and / or corrections regarding the details of specific interactions.

Embodiments of the present application disclose a method and device for echo cancellation and noise reduction. In the embodiments of the present application, a first near-end speech and a first far-end speech are obtained, and after pre-processing, delay detection and short-time Fourier transform, a fourth near-end speech and a fourth far-end speech are generated, and linear echo cancellation is performed to generate an error signal and a linear echo signal of the linear echo cancellation; the fourth near-end speech, the fourth far-end speech, the error signal and the linear echo signal of the linear echo cancellation are input into a two-stage neural network multi-task fusion model to be trained to determine a joint loss function; the two-stage neural network multi-task fusion model is generated according to the joint loss function, and then model compression is performed to generate a target two-stage neural network multi-task fusion model, which is used to realize echo cancellation and noise reduction. Through the above method, acoustic echo cancellation and noise reduction are realized under the condition of extremely low signal-to-echo ratio, and the speech interaction and communication performance of a terminal device are improved.

This application relates to a semantic entity recognition method and a semantic entity recognition device, a speech interaction system, a vehicle, and a computer-readable storage medium. The semantic entity recognition method includes: preprocessing speech instruction information to obtain a core-mentioned word contained within the speech instruction information; and searching a scenario-semantically directed structure graph based on the core-mentioned word to obtain a target entity object that matches the core-mentioned word, wherein the scenario-semantically directed structure graph is constructed from a scenario entity object in a current scenario and semantically associated words of different granularities of the scenario entity object.

System (100) for integrating ethical artificial intelligence for smart home systems using voice assistants, wherein the system (100) comprises: a speech interaction module (104) that is used to receive and is configured to process voice-based user commands; an analysis module (106) configured to interpret the processed speech commands based on context parameters; Ethics assessment modules (108-1) configured to make automated decisions regarding bias, Assess transparency and data protection; a smart home control module (108-2) configured to perform automated actions based on ethically evaluated decisions; and a monitoring module (110) configured to monitor system behavior and ensure ongoing compliance with predefined ethical standards.

Embodiments of the present disclosure provide a method, a device, and a medium for speech interaction. The method includes: obtaining query information of a query text corresponding to user speech using a generative language model; obtaining, based on the query information and a token sequence, a first representation sequence of the token sequence using the generative language model, wherein the token sequence is output in a streaming manner by a large language model based on the query text; encoding the token sequence using a language encoding model to obtain a second representation sequence; and combining the first representation sequence and the second representation sequence to generate an answer speech for the user speech.

The invention discloses a voice interaction method and system based on artificial intelligence, and relates to the field of voice interaction. By analyzing the environment in which the voice interaction system is located, the volume is intelligently adjusted to avoid the uncomfortable experience caused by too high or too low sound; by analyzing the speech speed of new instructions and the similarity between the new instructions and the input corresponding instructions, the instruction analysis index is obtained, and different strategies are adopted for different instruction analysis indexes, so that the instructions can be reasonably handled, thereby improving the intelligence level of the voice interaction system; by analyzing the user's age, region, gender, occupation and user behavior data, the recommended value of each voice package is obtained, so that the appropriate voice package can be intelligently recommended to the user, thereby increasing the user's satisfaction and stickiness.

The invention relates to a teaching speech interaction-oriented education metadata processing method and device, and a storage medium, and the method comprises the steps: carrying out the slicing and sentence segmentation of classroom teacher and student speech corpus data, and carrying out the sampling and aggregation to form a plurality of sentence blocks; based on a preset labeling instruction and the knowledge graph, performing word-level labeling of teaching behaviors and knowledge points on the sentence blocks by utilizing an intelligent agent, and generating a behavior container through majority voting; based on a preset discrimination instruction, the labeled sentence tag and the behavior definition of the tag, performing anti-fact verification by utilizing an intelligent agent, judging whether the sentence tag is related to the behavior definition or not, and if so, storing the sentence tag as a final labeling result in a preset graph database; and if not, triggering an error correction process, generating a new label by the intelligent agent based on the correction instruction, and selecting the label with the highest category overlapping degree as a result by matching with the positive example set and sorting similarity. And the correction result is fed back to the analysis process, so that iterative optimization update of the labeling judgment and correction module is realized.

A speech recognition method and an electronic device. The method comprises: an electronic device acquiring audio which is input by a user (S801); on the basis of the audio, the electronic device detecting a speech environment when the user inputs the audio, and determining a speech interaction mode corresponding to the audio (S802); and the electronic device performing speech recognition on the audio on the basis of a speech recognition scheme corresponding to the speech interaction mode, so as to obtain a speech recognition result (S803). By detecting the environment in which the user inputs the audio, the electronic device can determine that the user is in a noisy, quiet, or another speech environment, and can then determine the speech interaction mode. The electronic device can execute corresponding speech recognition on the acquired audio by means of the speech recognition scheme corresponding to the speech interaction mode, thereby improving the accuracy of speech recognition.

A speech interaction device comprising a processor for controlling a speech interaction of a user, the processor configured to: recognize interaction contents of the user before interruption of the speech interaction; extract, on a basis of the interaction contents, a keyword in the speech interaction before the interruption as a pre-interruption keyword; to generate, on a basis of the extracted pre-interruption keyword, an icon image representing the interaction contents before the interruption; and generate a control command to display the generated icon image. The processor is configured to update, after the icon image is displayed, the icon image displayed to the user on a basis of a topic subject in which the user may have an interest and / or a driving load of the user who drives a vehicle during the interruption of the speech interaction.

The application provides a speech recognition method, which comprises: obtaining feature data of audio to be recognized; inputting the feature data into an acoustic model to obtain a time sequence label matrix corresponding to the feature data; decoding the time sequence label matrix through a first language model to obtain a plurality of decoding paths and corresponding probability scores thereof, and determining the decoding paths with the top N probability scores as N first decoding results, wherein N is a positive integer; determining a corresponding target intent field based on the N first decoding results, a previous round of speech interaction field and a current scene field; determining a corresponding second language model based on the target intent field, re-computing probability values of the decoding paths with the top N probability scores through the second language model, and generating second decoding results; and determining a speech recognition result of the audio to be recognized based on the second decoding results. The application improves the recognition efficiency while ensuring the speech recognition accuracy.

In an infotainment system of a vehicle, a mass storage unit stores map data and context data. The map data comprises geographic information about a geographic area around the vehicle and the context data comprises location-dependent context information about the geographic area for use when engaging in speech interaction with an occupant of the vehicle. The context data includes subsets of context data, each of which is pertinent to a different geographic area. A memory manager copies these subsets of context data into a dynamic memory based on the vehicle's movement.

This invention discloses a speech interruption decision-making method and system based on multi-granularity semantic integrity prediction, belonging to the field of speech interaction technology. The method includes: extracting multi-granularity semantic features and combining them into a current multi-granularity semantic feature vector; calculating the offset of the current multi-granularity semantic feature vector with a standard semantic pause model to obtain a semantic offset feature vector; performing similarity matching and weighted voting between the semantic offset feature vector and a historical interruption decision case library to obtain the current predicted interruption confidence; and generating an interruption command when the current predicted interruption confidence exceeds a dynamic threshold. This invention solves the problems of existing technologies where speech interruption decisions rely on simple energy thresholds or semantic integrity probabilities, lack utilization of user expression habits and contextual experience, and have low decision accuracy by introducing a standard semantic pause model and a historical case analogy reasoning mechanism, thus achieving more intelligent and accurate speech interruption judgment.

The present invention provides a multi-modal speech interaction method, an electronic device and a storage medium based on a large model, including: performing speech recognition on speech data in a first language input by a user to obtain a first language text; performing language translation on the first language text to translate it into a second language text; performing semantic analysis on the second language text to obtain a semantic analysis result; constructing a multi-modal knowledge graph based on a preset large model for the semantic analysis result to obtain an enhanced semantic understanding result; wherein the multi-modal knowledge graph integrates multi-modal information related to the second language text; generating a dialogue strategy for the enhanced semantic understanding result; and performing speech synthesis based on the dialogue strategy to generate reply speech data expressed in the first language. In the present invention, multi-language interaction is realized, and at the same time, various modal information is integrated.