323 results about "Spoken dialog" patented technology

The invention discloses a method and system for context semantic extraction and intention recognition in voice dialogue, and the method comprises the following steps: S1, obtaining a voice input signal, converting the voice input signal into text data, and carrying out the preprocessing; s2, constructing a dialogue context cache, and updating the cache content; s3, constructing a sparse hybrid expert Transform network, and carrying out semantic coding on the sparse hybrid expert Transform network; s4, calculating a semantic dependency relationship between dialogue statements by adopting an ordered graph neural network; s5, calculating intention probability distribution, and adjusting the weight of the sparse hybrid expert Transform network and the weight of the ordered graph neural network; and S6, updating a dialogue state, and generating a system response. According to the method, the sparse hybrid expert Transform network and the ordered graph neural network are combined, voice dialogue context semantic extraction and intention recognition are achieved, and the method has the advantages of being accurate in semantic understanding, high in calculation efficiency and high in context sensing capacity.

The invention provides an LLM technology-based real-time voice conversation method and system applied to an operation and maintenance platform, and relates to the technical field of data processing, the method comprises the following steps: receiving user voice input, carrying out noise reduction preprocessing, and converting a voice signal into text data; semantic analysis is conducted on the text data through an LLMAgent module, the intention of a user instruction is recognized, and a task execution plan is generated based on the intention; decomposing the task execution plan into a basic operation sequence which can be executed by a Unity platform, and determining the priority of task execution; executing the basic operation sequence in a Unity environment, synchronizing an operation state in real time and feeding back an execution result; user interaction data and system performance data are collected, and LLM model parameters and decision strategies are optimized. The technical problems of poor voice interaction effect, large response delay, low accuracy and the like in the existing Unity platform are solved.

The invention discloses a voice interaction optimization method and system based on a multi-modal large model, and relates to the technical field of artificial intelligence and voice interaction, and the method comprises the steps: carrying out the voice recognition in response to the real-time voice of a user, and obtaining voice text information; according to the voice text information, combining the voice waveform of the real-time voice of the user as the input of a multi-modal recognition model, so as to judge whether the voice dialogue is interrupted and recognize the interruption intention, and obtaining a voice interruption result; obtaining a new intention of the user according to a voice interruption result, and dynamically adjusting a system response strategy to realize interaction optimization; the accuracy and comprehensiveness of interruption detection are improved through multi-modal fusion, so that the interruption intention is accurately recognized to respond to the user intention in real time, the system dialogue interaction efficiency and reliability are remarkably improved, and the defects that existing voice interruption detection is not high in detection accuracy, dynamic response of user interruption behaviors cannot be achieved, and user experience is poor are overcome. Therefore, the problems of low interaction efficiency and poor reliability of the voice dialogue system are solved.

The invention discloses an accurate international communication digital human real-time dialogue method fused with a multi-modal technology. The method comprises the following steps: S1, constructing a digital human image and tone; s2, propagation content generation and problem guidance; s3, semantic analysis and intention clarification based on the real-time voice dialogue; s4, geographic preference modeling and path planning; s5, cross-context propagation content generated based on retrieval enhancement is generated; s6, visually displaying the propagation content; and S7, carrying out digital human-driven multi-language propagation content real-time output and feedback closed-loop optimization. Through accurate utterance expression analysis, accurate international propagation problem recommendation is provided, cross-context propagation content generation based on semantic understanding is realized, digital people with voice features and visual images are constructed, real-time dialogue interaction of users is realized, and user experience is improved. The system can carry out geographic modeling according to the region where the accurate problem is located, language preference and propagation object culture characteristics, and differential propagation path planning is achieved.

The embodiment of the invention provides a voice conversation method and system, electronic equipment, a storage medium and a program product. One method is implemented as follows: a voice dialogue stream (which can be an inquiry voice stream input by a user for a target commodity) of a user is subjected to streaming response, a plurality of dialogue text segments are generated in a segmented manner in the streaming response process, and the dialogue text segments are output in real time after being generated and stored in a first cache queue; and when it is detected that the reply text fragment is stored in the first cache queue, the stored reply text fragment is converted into a reply voice fragment in real time and played. The streaming response may be performed by an agent. Visibly, according to the response of the voice dialogue stream of the user, the dialogue is played while being generated, the waiting time from text to voice synthesis can be greatly shortened, and therefore the response time can be shortened.

The invention discloses a digital human real-time interaction system and an AI voice dialogue chip circuit system thereof, and the system employs a modular design, and comprises the following modules: a BaseReal module which is responsible for audio frame management, TTS service initialization, video recording and customized audio and video cycle management; the media stream management module has an audio stream processing function and is responsible for real-time transmission of audio and video streams; the speech recognition module is responsible for speech feature extraction and text conversion; the text-to-voice module is used for converting the collected audio data into a text; the lip shape synchronization module is used for realizing real-time lip shape synchronization animation generation through lip shape synchronization algorithm optimization and inheritance of a basic class; and the large model module comprises an LLM large model based on mass data fine tuning, an api interface or a workflow agent. The method has the effects of voice recognition, text-to-voice conversion, lip shape synchronization, real-time audio and video interaction, real-time translation and the like.

The invention provides a method and a system for dynamically evaluating competency of civil aviation flight trainees, and relates to the technical field of information collection and analysis systems.The method comprises the steps of firstly, obtaining various observation data including operation behavior data, physiological signal data, eye movement tracking data, face video data and voice dialogue data; a pre-trained multi-modal feature extraction model is utilized to mine the performance features of the trainees from different angles, and comprehensive feature representation is generated; secondly, a technical ability score, a psychological quality score, a team cooperation score and an emergency response ability score are obtained through a multi-task scoring model, a teacher score is obtained according to pre-constructed threat and error management classification evaluation indexes, then a comprehensive score comprehensively reflecting the ability of students is obtained through weighted fusion, and finally, the comprehensive score of the students is obtained. And combining the comprehensive score change rate and the score difference progressive value through a competency index calculation formula to obtain a competency index capable of dynamically reflecting a trainee ability development trajectory.

The invention relates to the technical field of voice dialogues, and discloses a robot dialogue intelligent early warning system based on voice outbound, which comprises a voice recognition module for receiving a voice signal in a real-time call initiated by an outbound robot and converting the voice signal into a dialogue text; the intention analysis module is used for extracting semantic features and emotional features from the dialogue text, inputting the semantic features and the emotional features into a pre-trained intention classification model and outputting a user intention label and confidence; the emergency degree evaluation module is used for generating a dialogue emergency level based on the user intention label, the confidence coefficient and a preset emergency keyword library; the early warning execution module is used for triggering real-time early warning operation when the conversation emergency level reaches a preset threshold value; the real-time early warning operation comprises at least one of dynamically adjusting a robot dialogue strategy, generating a manual seat transfer instruction and upgrading a dialogue priority. The method can solve the problems that an existing voice outbound robot cannot accurately judge the user intention, and the conversation emergency degree recognition is insufficient, and improves the user experience and the overall efficiency.

The invention discloses a voice dialogue system based on a large model, and relates to the technical field of voice interaction, and the system comprises an intelligent scheduling module which determines a task type according to a to-be-recognized text, and dynamically adapts to a fast and slow thinking module according to the task type; the fast and slow thinking module dynamically adapts to the calculation unit according to the task type and outputs a feedback result; the voice interaction module generates a to-be-recognized text according to the voice characteristics of the user and generates a voice instruction based on a feedback result; the context management module stores the interaction log to a corresponding storage unit according to the index key value; the interaction log comprises a task type belonging to the same data flow, an adaptive path and a feedback result; and the data centralized control module determines a parameter adjustment strategy according to the performance index factors in the interaction logs, and the parameter adjustment strategy acts on the intelligent scheduling module or / and the fast and slow thinking module to optimize corresponding parameters. The intelligent balance of quick response and deep reasoning of the dialogue system under a single architecture is realized, and the real-time performance and accuracy of interaction are remarkably improved.

The invention discloses an intelligent accompanying method and an emotion monitoring device, which are applied to an emotion monitoring intelligent bed, and the method comprises the steps: obtaining emotion data of a user, the emotion data comprising voice features, expression features and posture change features; judging different emotion data to obtain different emotional states, and obtaining a target emotional state of the user according to the different emotional states and preset weight information of the different emotional states; when the target emotional state is an abnormal state, sending out a first feedback signal; and according to the first feedback signal, guiding the user to perform voice interaction through a semantic analysis model, and outputting an audio reply corresponding to the user voice information. According to the method, the emotion state of the user can be detected and fed back in real time, the emotion of the user is adjusted in a voice dialogue communication mode, and the nursing quality of the user is improved.

The invention discloses an electric power customer service work order abstract generation method based on voice dialogue, and relates to the technical field of voice recognition, and the method comprises the steps: receiving voice dialogue content; key entities related to electric power are extracted; performing deep understanding on the voice text based on the semantic analysis model; based on the extracted key information, automatically generating an abstract of the electric power customer service work order; submitting the automatically generated work order abstract for manual auditing; the invention discloses an electric power customer service work order abstract generation system based on voice conversation. The system comprises a voice receiving module, a voice text processing module, a keyword extraction module, a semantic analysis module, a work order generation module, a work order auditing module and a work order publishing module. Through advanced voice recognition, natural language processing, semantic analysis and emotion recognition technologies, automation of the work order generation process can be achieved, the work order processing quality and efficiency can be optimized, and therefore remarkable benefits and improvement are brought to electric customer service work.

The present application relates to the technical field of speech recognition, and discloses a speech dialogue method and system, an electronic device, and a storage medium. The method of the present application comprises: in response to a speech audio stream of a first preset duration, generating a first streaming recognition result; obtaining a prediction result of a user speech request on the basis of the first streaming recognition result, obtaining a pre-request response result on the basis of the prediction result, and caching the pre-request response result; and in response to a speech audio stream of a second preset duration, acquiring the cached pre-request response result as a dialogue response result, the speech audio stream of the first preset duration corresponding to the incomplete user voice request, and the speech audio stream of the second preset duration corresponding to the complete user voice request. The speech dialogue method disclosed in the present application is a new and more efficient response acceleration scheme for a speech dialogue system, and can effectively improve the response speed of the speech dialogue system.

An embodiment of the present application provides a voice interaction method and an electronic device, which relate to the fields of artificial intelligence (AI) technology and voice processing technology, and can improve the performance of the interaction between the electronic device and the user, thereby providing a personalized voice interaction experience for the user. The specific solution includes: the electronic device can receive the first voice information sent by the second user; and in response to the first voice information, the electronic device recognizes the first voice information. Among them, the first voice information is used to request a voice conversation with the first user. Based on the fact that the electronic device recognizes that the first voice information is the voice information of the second user, the electronic device can simulate the voice of the first user and conduct a voice conversation with the second user in the way of the first user having a voice conversation with the second user. This method can be applied to the scenario where a smart robot replaces parents to accompany and educate children.

The invention provides a GAN-based digital doctor expression, action and emotion interaction simulation system. The system comprises a data set construction module, an audio encoder, an action mask encoder, a Transform converter, a training module and a digital doctor generation module. The data set construction module is used for constructing a high-quality medical scene data set; the audio encoder extracts advanced feature representation from the audio by adopting a pre-training model; the motion mask encoder is used for extracting motion feature information in a binary image with the same size as a video frame; the Transform converter is used for realizing the fusion of the multi-modal data; the training module is used for training the GAN network; and the digital doctor generation module generates a vivid whole body action and expression sequence through a trained generator. According to the system, expressions and actions of the digital doctor can be more accurately generated according to voice dialogues in the current system, and the problems that in a traditional online diagnosis and treatment system, users are not satisfactory, and the expressions of the digital doctor are not generated well are solved.

The invention relates to the technical field of voice interaction, in particular to a method and system for determining a voice reply strategy based on dynamic risk assessment, and the method comprises the steps: obtaining user input and calling a pre-trained large language model to output a risk score in response to a voice interaction request initiated by a user; grading the risk scores based on a preset risk threshold, wherein the risk scores comprise high risks, medium risks and low risks; if the risk is high, immediately entering a strategic refusal response process; if the risk is medium risk, entering a multi-round clarification dialogue process, and continuously recalculating a risk score based on the constructed clarification question and the answer of the user until the risk score is not in a medium risk interval; and if the risk is low, directly entering a normal interaction process. According to the invention, a strategy which can perform fine-grained evaluation and dynamic management and control on a multi-level risk scene and can adaptively switch rejection and clarification in the same closed-loop process can be constructed in a voice dialogue system, so as to give consideration to the interactive experience of safety compliance and natural smoothness.

The invention relates to the field of network live broadcast, and discloses a live broadcast room voice dialogue voice changing method and device, equipment and a medium, and the method comprises the steps: responding to a voice speaking event of a speaking user in a live broadcast room, and detecting and determining a voice segment in target audio data; according to a preset target pitch value, performing tone modification processing on the segment pitch feature of the voice segment to obtain a tone tuning pitch feature; based on the tuning pitch feature and the target timbre feature, performing voice changing processing on the voice segment to obtain a voice changing segment; and replacing the corresponding voice segment in the target audio data with the variable voice segment to obtain variable voice audio data, and sending the variable voice audio data to a called user in the live broadcast room. According to the method and the device, the performance of a voice changing technology is remarkably improved, and the problems in the aspects of real-time performance, personalized service, pitch adjustment naturalness, tone true restoration and the like in the traditional technology are solved.

The invention relates to a method for operating a speech dialogue system. First, a request (1) from a user (6) is detected via an interface of the speech dialogue system. Then, a domain recognition unit (8) is activated by a control logic (7) of the speech dialogue system, which domain recognition unit is configured to assign the detected request (1) to at least one domain (24). The request (1) is assigned to at least one domain (24) by means of domain recognition. Subsequently, the at least one domain (24) is associated with at least one language model (5) within the speech dialogue system using a decision matrix (3) of the control logic (7), each domain (24) being associated with at least one language model (5).The request (1) is then forwarded to the at least one corresponding language model (5), wherein a feedback (9) regarding the request (1) is generated based on information stored in the at least one selected language model (5). This feedback (9) is then output to the user (6).

The invention discloses a vehicle voice interaction method, apparatus and device, and a medium. The method comprises the steps of obtaining dialogue information of a driver and passengers in a target vehicle; inputting the dialogue information into a local large model deployed in the target vehicle, and analyzing and processing the dialogue information through the local large model to obtain standard question information; wherein the local large model is an offline large model, and the standard question information comprises at least one of optimization question information, a first prompt word and answer form information; sending the standard question information to a cloud big model for analysis and processing to obtain first reply information fed back by the cloud big model; and playing the first reply information based on the voice dialogue function of the target vehicle. According to the technical scheme, through cooperative processing of the local large model and the cloud large model, the accuracy, safety and user experience of vehicle voice interaction are effectively improved. The method can be widely applied to the technical field of vehicles.

The invention relates to the technical field of customer management, in particular to an intelligent customer relationship management system based on multi-modal data fusion, and the system comprises a multi-modal data acquisition module which is used for extracting a semantic contradiction index, a service request frequency change rate and a historical performance quality score of a text work order of a voice conversation from a customer interaction log; the state vector generation module is used for generating customer relationship state vectors based on health degree scores; the customer relationship network topology construction module is used for constructing a causal association edge of'customer-service defect-relationship attenuation 'based on the customer relationship state vector, outputting a customer relationship network topology structure with an interpretable path, and forming a relationship maintenance knowledge graph; and a self-adaptive relation repair strategy generation module. According to the invention, through identification of the high-weight relation attenuation path, key nodes and paths of customer relation degradation can be positioned.

The invention discloses a model training method, a voice recommendation method and electronic equipment. The model training method comprises the following steps: clustering user data of a target user obtained from a plurality of data sources to obtain user tag system data; according to the user tag system data, obtaining corresponding user preference data of the target user in the plurality of data sources; and migrating the user preference data of the target user corresponding to the plurality of data sources to the vehicle-mounted voice scene through migration learning to obtain sample data, training the initial prediction model through the sample data to obtain a user preference prediction model for predicting the user preference data in different voice conversation scenes in the vehicle-mounted voice scene, and predicting the user preference data in the vehicle-mounted voice scene according to the user preference prediction model. The user preference data of the user in different vehicle-mounted voice conversation scenes are predicted through the user preference prediction model, voice recommendation is carried out based on the user preference data, and personalized vehicle-mounted voice recommendation is realized.

The invention provides a voice dialogue interaction method and system for industrial equipment, and relates to the technical field of man-machine interaction, and the method comprises the following steps: obtaining a voice signal of a user, and carrying out the voice enhancement processing of the voice signal of the user through an incremental adaptive filtering algorithm, and obtaining an enhanced voice; according to the enhanced voice, using an information gain transfer learning method to identify an interaction intention of the user, and generating a to-be-interacted voice based on the interaction intention of the user; based on the enhanced voice, utilizing a time delay estimation method to identify an interaction position of the user, and performing position prediction on the position of the user; according to the position prediction result, an industrial equipment horn output strategy is constructed; and outputting the to-be-interacted voice based on the industrial equipment loudspeaker output strategy so as to realize voice dialogue interaction between the user and the industrial equipment. According to the invention, convenience, high efficiency and accuracy of interaction between the user and the equipment in an industrial scene are greatly improved, and intelligent development of industrial production is facilitated.

The invention relates to the technical field of medical emotion early warning, and discloses an early warning system used based on a hospital platform. The system comprises a data acquisition module which is used for acquiring a patient identifier and treatment time, and acquiring voice dialogue and text record content and associated information to generate a patient data set; the sentiment analysis module is used for analyzing semantic and acoustic features of the voice and text contents and generating a sentiment analysis result set; the grading processing module is used for carrying out hierarchical division on emotions and generating a patient emotion grading result set; the information synchronization module is used for synchronizing the grading labels and the patient identifiers to a hospital information system display terminal according to specifications and generating grading information synchronization records; the intervention execution module is used for matching an AI automatic pacifying or manual intervention strategy for the patients higher than the basic warning level, and generating an intervention execution record set; and the early warning triggering module is used for marking the patient with the emotion activation risk and generating an early warning triggering result. The system can sense emotion changes of a patient in real time, active intervention is achieved, and the diagnosis and treatment order is maintained.

The invention discloses a telecommunication fraud treatment method and system based on model pre-judgment and AI outbound forensics, and the method comprises the steps: introducing a characteristic factor of a scene user based on the specific behavior data of a telecommunication fraud case, constructing a scene user portrait, and constructing a user group prediction model through a machine learning algorithm; acquiring a risk number in real time, and predicting a suspicious number by adopting a user group prediction model; determining a called number according to the suspicious number, and carrying out call return visit on the called number by adopting an AI outbound robot; acquiring and analyzing the voice content of the revisited person in the call revisit process, performing voice conversation according to preset interaction logic, analyzing to determine a revisit result, and determining a telecommunication fraud judgment result of the suspicious number; and performing preset governance processing on the suspicious number according to the telecommunication fraud judgment result. Through the technical scheme of the invention, the timeliness and coverage rate of the local drainage fraud model are improved, and the judgment accuracy of telecommunication fraud is improved.

The invention relates to the technical field of artificial intelligence and voice dialogue, and discloses a voice interaction method based on retrieval enhancement generation and multi-model collaboration and application thereof. Performing local multi-round intention recognition; performing local RAG retrieval and candidate aggregation; decision making and cloud generation; performing feedback, synthesis and persistence; the invention further provides application of the voice interaction method based on retrieval enhancement generation and multi-model collaboration. According to the method, through local structured intention recognition and RAG retrieval aggregation, in combination with the generation capability of a cloud large model, response timeliness and privacy protection are considered, and the recommendation accuracy and robustness in a multi-round voice interaction scene are remarkably improved; the retrieval redundancy is reduced and the context quality is improved through entity-level combination duplicate removal, rearrangement and fuzzy matching strategies; and the consistency of data updating and the stability of online service are ensured through an offline atom replacement mechanism, so that the terminal user experience is improved, and engineering deployment and expansion are facilitated.

The invention relates to the technical field of 3D digital humans, and discloses a real-time interaction method and device of a 3D digital human. First dialogue data initiated by a target user is converted into a first dialogue text through a voice recognition unit in sequence, and then a response text is recognized by inputting a language recognition model; and the response voice is synthesized into response voice, target lip language data is generated through the lip language generator, and finally the 3D digital human is driven to make voice response, so that the target user can obtain accurate, natural and real feedback. Moreover, interaction is carried out in a voice conversation mode, the method is simple and convenient, a highly vivid conversation scene can be displayed, and the interaction experience of the target user is enhanced.

A method and system for observing the presence, location, movement and / or posture of a person in a monitored area is provided. The system comprises: at least one sensor (101, 201, 301, 401, 601); and means for processing a measurement signal of the sensor, such as measurement electronics; and means for transmitting the measurements and / or data relating to said measurements for further processing. The system further comprises means (105, 205, 405, 605) for generating and capturing audio, and the system is configured to: determine, using the at least one sensor (101, 201, 301, 401, 601), that the security of the monitored person (206, 402, 602) is threatened; and based on the determined that the security of the monitored person (206, 402, 602) is threatened, initiating a voice conversation with the monitored person (206, 402, 602) using a means for generating audio, and listening to a reply from the monitored person (206, 402, 602) using a means for capturing audio. Based on the captured audio, the system is configured to evaluate a situation and determine whether an action, such as an alarm, needs to be taken, where the system is configured to make the voice conversation with the monitored person using a large language model (LLM).

The invention relates to an automatic depression risk assessment method based on a natural language processing technology. The method comprises the following steps: performing selective conversation content interviews with a subject by using a voice conversation function of artificial intelligence; dialogue recording and text data between the patient and artificial intelligence are obtained; performing automatic transcription and speaker separation on the original speech transcription text data, and constructing a structured question and answer dialogue sequence; generating a structured question prompt related to the depressive symptom by using a large language model, and splicing the structured question prompt with the original dialogue content; designing an artificial neural network pre-training language model coding word vector, fusing position coding, a self-attention mechanism, a problem-guided attention mechanism and a bidirectional long short-term memory network, and extracting deep semantic features related to depression risks; finally, continuous depression risk scores are output, correlation verification is carried out on the continuous depression risk scores and standard PHQ-8 scores, and the accuracy, robustness and interpretability of depression risk assessment are effectively improved.

A vehicle control system executing a voice control system for facilitating voice-based dialog with a driver to enable the driver or autonomous vehicle to control certain operational aspects of an autonomous vehicle is provided. Using environmental and sensor input, the vehicle control system can select optimal routes for operating the vehicle in an autonomous mode or choose a preferred operational mode. Occupants of the autonomous vehicle can change a destination, route or driving mode by engaging with the vehicle control system in a dialog enabled by the voice control system.

The invention provides a voice dialogue task handover method and device, a server and a system, and the method comprises the steps: obtaining historical dialogue data of a voice dialogue task between a target user and first voice equipment; monitoring sensor data of a plurality of second voice devices of the first voice device in a preset neighborhood range; selecting a target voice device to which the first voice device is to be handed over from the plurality of second voice devices according to the sensor data; and handing over the voice dialogue task from the first voice device to the target voice device according to the historical dialogue data, so that the target voice device continues to perform voice interaction with a target user based on the historical dialogue data. By adopting the scheme of the embodiment, the technical problem that continuous voice interaction can only be carried out on a single voice device in the prior art is solved, and the fluency and intelligence of man-machine interaction are improved.

The invention discloses a robot control system and method based on AI voice, and belongs to the technical field of robot intelligent control. The system comprises a local voice processing module, a dialogue management and context understanding module, an instruction mapping and generating module and a robot communication and control module. The local voice processing module is integrated with a 16 kHz sampling ASR engine, voice-to-text conversion is realized through a 6-layer LSTM network and beam search decoding, and the TTS engine adopts a WaveNet architecture to synthesize voice. A 7 billion parameter large language model is arranged in the dialogue management module, a 32-layer Transform decoder architecture is adopted, 10 rounds of dialogue history is maintained, and intention recognition and entity extraction are achieved through a multi-head self-attention mechanism. And the instruction mapping module performs trajectory planning by adopting quintic polynomial interpolation and quaternion spherical linear interpolation. According to the invention, full-link localization processing is realized, the speech recognition delay is 180 milliseconds, the position error is less than 0.5 mm, and the problems of high delay, high privacy risk and weak complex instruction understanding ability in the prior art are effectively solved.