526 results about "Speech output" patented technology

The invention provides a multi-modal intelligent question-answering and recommendation system supporting emotional speech output, and relates to the technical field of visual question-answering, and the system specifically comprises a multi-modal question-answering module, a multi-modal recommendation module, a speech recognition module, and a speech synthesis module. Wherein the multi-mode question answering module converts an image input by a user into a natural language form, and final question answering is completed in combination with a comet knowledge base COMET; the voice recognition module is used for recognizing voice data input into the multi-mode intelligent question answering and recommending system and processing the voice data into a text form; the multi-modal recommendation module identifies whether modal missing exists in input data or not, generates representation of the missing modal, obtains vector representation of the user and the article through a graph convolutional neural network and learning according to an interaction matrix of the user and the article, carries out reliability calculation, and finally sends a multi-modal recommendation result to the user and the article. Sending back to the multi-mode question and answer module; and the voice synthesis module outputs the voice to the user.

Provided in the embodiments of the present application are a video generation method and interaction method based on a digital human, and a device, a storage medium and a program product. In the embodiments of the present application, text-to-speech processing is performed on the basis of voice features of a user and an emotion label, speech-to-expression processing is performed on the basis of a mapping relationship between the voice features of the user and expression coefficients, and a digital human model is rendered on the basis of speech signals and the expression coefficients, so as to obtain video data of the digital human model. Thus, voice features of a user are accurately simulated, so as to ensure that a speech output of a digital human sounds natural and is also highly personalized, thereby realizing personalized driving of the digital human, and improving the realism of the digital human in terms of voice and dynamic images. Thus, the user experience is improved, and the interactivity of the digital human and the authenticity and immersion are enhanced.

The invention relates to the technical field of computer vision and human-computer interaction, and discloses an AI digital human intelligent interaction method, system and equipment based on multiple modalities, and the method comprises the steps: pre-awakening a digital human when a human face is detected, and further thoroughly awakening the digital human based on recognized preset voice information or preset gesture information; voice and video information of a user in the interaction process is obtained, a keyword extraction result, a gesture recognition result and an emotional state tag are generated, a pre-constructed knowledge base is utilized to retrieve related information, a big language generation model module is combined to generate an answer text, and the answer text is input into a preset voice synthesis model to generate emotional voice output. And based on the current emotional state label of the user, driving the digital human animation to be output in an emotional manner. According to the method and the system, the digital human for understanding the emotion of the user, generating personalized answers, providing voices with rich emotions and displaying natural expressions and actions can be created, better interaction with the user can be realized, and more humanized and effective services can be provided.

The invention discloses a touch and talk pen intelligent children education method based on emotion recognition. The method comprises the steps that S1, facial expression data, voice and intonation data and touch behavior data of children are collected; s2, multi-modal high-quality emotion data matched with the actual emotion state is generated; s3, generating deep emotional features related to facial expressions, voices and tones and touch behaviors; s4, generating an emotional state label corresponding to the current emotional state of the child; s5, the generation model generates voice feedback content, image animation content and interactive game content which are adaptive to the current emotional state; s6, generating a personalized recommendation content list conforming to the emotional state of the child; and S7, displaying the education content in the personalized recommendation content list through a display screen, a voice output unit and a touch interaction unit of the touch and talk pen. According to the method, the fuzzy classification problem of the emotional state is solved in a touch and talk pen emotion recognition scene, and the adaptability of the model to the complex emotional state is remarkably improved.

The embodiment of the invention provides a digital human interaction system and method based on multi-modal emotion recognition, and belongs to the technical field of digital human interaction. The system comprises a multi-modal sensing module used for collecting multi-modal data and preprocessing the multi-modal data to generate a standardized data stream; the cross-modal fusion and emotion recognition module is used for carrying out interactive modeling on the multi-modal features and outputting a current emotion label and emotion intensity; the reaction planning module is used for generating a composite reaction strategy; and the digital human rendering module is used for mapping the composite reaction strategy into control signals corresponding to the voice, the facial expression and the action respectively, and driving a digital human to execute corresponding voice output, facial expression change and limb action through the control signals so as to realize interaction. According to the method, multi-modal data are deeply fused through the cross-modal graph neural network and comparative learning, the weight is dynamically adjusted in combination with the modal confidence, and the emotion recognition accuracy and robustness are improved.

The invention discloses a blind person auxiliary interaction method and system based on a multi-modal large model and a storage medium, and belongs to the technical field of artificial intelligence, and the method comprises the steps: obtaining a user voice instruction and an external environment image, and carrying out the preprocessing of the obtained data; inputting the preprocessed data into a pre-trained multi-modal large model to obtain an environment understanding result; integrating the environment understanding result into feedback information for voice output, and adjusting output intonations according to environment characteristics; the acquisition method of the pre-trained multi-modal large model comprises the following steps: acquiring historical multi-modal data; and inputting the historical multi-modal data into a pre-constructed multi-modal large model, carrying out fine tuning on the trained multi-modal large model through a base layer in a frozen model to obtain a trained multi-modal large model, fusing multi-modal information through the multi-modal large model, accurately identifying a user intention and a corresponding object in a complex environment, and carrying out real-time identification on the user intention and the corresponding object in the complex environment. Therefore, accurate scene description is fed back.

The invention relates to the technical field of voice processing, can be applied to business scenes of financial science and technology, medical health and the like, and discloses a voice text bidirectional conversion method, device, equipment and medium, and the method comprises the steps: respectively executing voice recognition or voice synthesis operation according to the type of input information; for the voice information, noise suppression parameters are generated in combination with the lip movement video data, noise reduction processing is executed, and the recognition accuracy is improved; for text information, a pre-generated speaker style vector is obtained, the vector is cited in the speech synthesis process to generate natural personalized speech, and lip movement information and tactile feedback which are synchronous with speech output are generated. According to the method, complex noise is suppressed by fusing lip movement data, personalized voice is generated by using the style vector, and lip movement and touch information is output, so that bidirectional real-time conversion of voice and text in a complex environment is realized, and recognition accuracy, voice naturalness and interaction synchronism are effectively improved.

The invention relates to the field of artificial intelligence medical application, and discloses a lightweight intelligent traditional Chinese medicine inquiry system and a construction method thereof, and the system comprises a multi-dialect adaptive speech recognition module, a traditional Chinese medicine intelligent dialogue large language model module, a natural speech synthesis module, and a continuous learning mechanism module. The multi-dialect adaptive speech recognition module is used for converting dialect speech input of a patient into a standard text; the traditional Chinese medicine intelligent dialogue big language model module is the core of the system and is used for carrying out natural language understanding, dialectical reasoning and inquiry dialogue generation, and the natural speech synthesis module is used for converting a text response generated by the system into speech output; and the continuous learning mechanism module realizes continuous optimization of the large language model through incremental learning architecture and clinical feedback integration. According to the method, while the professional traditional Chinese medicine diagnosis capability is maintained, the calculation complexity is remarkably reduced, and the universality and sustainable development capability of system application are improved.

The invention relates to the technical field of flight simulation training, in particular to an air traffic control response system based on a flight simulator and a response training method. The method comprises the steps that a voice input module receives input voice audio data and sends the voice audio data to a voice recognition module; the voice recognition module recognizes and converts the voice audio data into voice text data; the instruction understanding module processes the voice text data to generate structured data corresponding to air-ground communication and sends the structured data to the instruction text generation module; the instruction text generation module generates a corresponding air traffic control instruction text according to the structured data; the voice synthesis module converts the air traffic control instruction text into corresponding voice audio data and sends the voice audio data to the voice output module; and the voice output module outputs the generated voice audio data. According to the invention, the recitation condition of the student can be intelligently evaluated, the workload of a teacher is effectively reduced, and the teacher can concentrate on guiding the flight operation of the student.

The invention discloses a sign language recognition system based on a dynamic time warping algorithm, and the system comprises a data collection module which is used for collecting gesture data in real time through a camera and a depth sensor; the feature extraction and time sequence conversion module is used for converting the gesture data into a time sequence; the dynamic time warping matching module is used for matching the input gesture with the gesture in the template library by utilizing a dynamic time warping algorithm; the matching result selection and template classification module is used for selecting the most matched template according to the similarity and determining the gesture category; the embedded platform and hardware acceleration module is used for executing a sign language recognition algorithm on the embedded platform and improving the recognition speed and real-time performance through hardware acceleration; and the character and voice output and real-time feedback optimization module is used for converting the recognition result into characters and voice output. Through hardware acceleration and data compression optimization, the sign language recognition precision, real-time performance and adaptability are improved, and the sign language recognition method is suitable for an embedded system with limited resources.

The invention discloses a privacy information desensitization method and system for a voice generation type large model, and relates to the technical field of artificial intelligence. Input voice data is discretized, Gaussian noise disturbance sensitive features are injected, and a cross-modal voice generation model is constructed in combination with three-stage training; and meanwhile, a cross-modal privacy enhancement mechanism is applied to detect and fuzzify sensitive information in real time in an output stage. According to the invention, the adaptability of a large-scale voice generation model in a privacy protection scene is improved, and comprehensive protection of user privacy is realized. And moreover, the capability of extracting user privacy information by an adversarial attacker is effectively limited, and the risk of sensitive information leakage in the data transmission, storage and generation process of the voice generation model is reduced. On the premise that privacy is ensured, the voice generation model can still keep high-quality generation performance, generated voice output has high naturalness and accuracy, and actual application requirements are met.

The purpose of the present invention is to provide a more suitable artificial intelligence response output technology. The present invention contributes to Sustainable Development Goals (SDGs), "9. Industry, Innovation, and Infrastructure" and "11. Sustainable Cities and Communities" This response output device is provided with: a control unit that acquires, from a large-scale language model, a response to an instruction sentence to the large-scale language model; a display unit; and an audio output unit. The control states by the control unit include a state in which a response from the large-scale language model is output via the display unit or the audio output unit.

The invention relates to the technical field of voice interaction, in particular to a voice interaction method and system for a mobile digital human, and the method comprises the steps: inputting data through a voice collection channel, reading a sound wave signal time sequence, extracting a waveform amplitude value, and recording a timestamp, and synchronously extracting and coding time domain and frequency domain changes implied in a conventional voice input signal. The method improves the capturing precision of tiny mutation characteristics in voice signals, introduces a generation and discrimination mechanism of an adversarial neural network through the generation and screening process of a mutation candidate characteristic group, enables a generator to dynamically generate diversified amplitude and frequency change modes, further improves the expressive force and naturalness of voice output, and improves the voice recognition accuracy. By performing weighting coefficient addition processing on the product of the amplitude and the frequency change, the voice output intensity can be reflected more accurately, the response performance of the voice is optimized, and the response efficiency and the stability of the system are further improved by setting the bidirectional mapping and the freezing state flag.

The invention belongs to the field of intelligent voice, and particularly relates to an intelligent health robot combining intelligent dialogue and health intervention, which comprises a voice recognition module and a rehabilitation guidance module, and is characterized in that the voice recognition module comprises a voice acquisition unit, a voice compensation unit and a multi-mode recognition unit; the method comprises the following steps: acquiring user voice by a voice acquisition unit, compensating formant features and pause discrimination by a discrimination filter, and obtaining a to-be-recognized voice feature space; correcting voice missing words and tones through a voice compensation unit to obtain a corrected to-be-recognized voice feature space; the multi-modal recognition unit outputs the voice text and the demand intention space according to the voice text and the demand intention space, and obtains an intention emotion state space by combining a preset evaluation interval; and finally, the rehabilitation guidance module calls an enhanced reasoning strategy library to generate a rehabilitation strategy intervention scheme according to the demand intention space and the emotional state space, and performs real-time voice output to realize intelligent dialogue and health intervention.

An information processing system, method, and program for conducting interviews with applicants is provided. [Solution] The method includes starting a conference session in an interview preparation mode, and switching the conference session to the interview mode when an instruction to switch to the interview mode is received from a user, starting an interview with an avatar selected in the interview preparation mode as the interviewer, displaying an avatar in the interview preparation mode, outputting a first utterance for breaking the ice as the avatar's speech, outputting a second utterance for setting the avatar as the avatar's speech, and displaying selectable options for multiple setting items of the avatar, and accepting an avatar selection including a first response to the first utterance and a second response or selection of an option to the second utterance, and a switching instruction, a first display control step of displaying an avatar changed in accordance with the avatar selection, and a second display control step of displaying the changed avatar.

The embodiment of the invention provides an AI-based vehicle intelligent interaction method and system, and belongs to the field of artificial intelligence interaction. The method comprises the following steps: dynamically judging a cognitive load level in a current driving scene through collaborative perception of a driver state and an environment state; determining a target man-machine interaction strategy of the vehicle based on the cognitive load level; wherein the target man-machine interaction strategy comprises a voice output mode, information density of screen display content and a feedback prompt mode; determining a corresponding trigger rule based on a switching relationship between the current man-machine interaction strategy and the target man-machine interaction strategy; and responding to a trigger signal of the determined trigger rule, executing man-machine interaction strategy switching, and executing vehicle interaction based on the switched target man-machine interaction strategy. According to the scheme of the invention, closed-loop adaptive control of a human-vehicle interaction strategy on a cognitive state is realized.

The invention belongs to the technical field of artificial intelligence, and particularly relates to a real-time voice interaction and adaptive content generation system based on RTC and AI GC, which comprises the steps of capturing voice input of a user through a microphone of equipment, and converting the voice input into text information by using a voice recognition technology; the recognized text input is transmitted to a natural language understanding module, semantic analysis is carried out, and user intention and information are recognized; according to the intention and demand of the user, the AI GC technology is used for dynamically generating personalized content; and the generated content is optimized in real time according to the feedback in interaction, the demand change of the user is adapted, the generated text content is converted into voice to be output and provided for the user, and the effects of meeting the demands of different types of users and promoting real-time interactive propagation and automatic content generation are achieved.

The invention relates to the technical field of intelligent education, in particular to an interactive English teaching method and system based on AI vision, and the method comprises the steps: obtaining the voice and mouth dynamic image frame segments of a student, extracting an offset frame segment to generate a semantic motion disjunction interval, analyzing the time sequence consistency of a behavior signal and a motion cause word triggering frame segment, and calculating the motion matching degree. Time synchronism of the speech output and the visual action is evaluated. According to the invention, through real-time analysis of the voice and mouth dynamic image of the student, the alignment offset of the pronunciation and the mouth action is identified, and in combination with behavior signals such as eye fixation and head rotation of the student, the interactivity between the language action and the behavior of the learner is comprehensively evaluated, so that the efficient alignment of the language expression and the standard template is ensured; by analyzing the distribution frequency and the matching degree of the voice behaviors, a personalized teaching path is optimized. In addition, delay frames and contour offset are deeply analyzed, the synchronism of voice and visual actions is enhanced, and the real-time feedback and interaction effect in the learning process is improved.

The invention relates to an artificial intelligence technology, and discloses a voice interaction method and system based on a quantum heuristic algorithm, and computer equipment, and the method comprises the steps: carrying out the preprocessing of a voice signal after receiving voice input, and extracting voice features; taking different combinations of the voice features as superposition of quantum states, dynamically adjusting the selection probability of the voice features based on quantum revolving door operation, and iterating and screening feature subsets with the probability meeting a preset requirement; inputting the screened feature subset into a character recognition model for training, and generating a character recognition result; performing semantic understanding on a character recognition result based on a large language model; after the text information is obtained, extracting voice feature parameters based on the text information; the voice feature parameters are optimized and adjusted through quantum evolution operation; and based on the optimized voice feature parameters, performing voice output by adopting a voice synthesis algorithm. The invention further discloses a computer readable storage medium. The invention aims to improve the efficiency and accuracy of voice interaction.

The invention relates to the technical field of voice semantics, can be applied to business scenes of financial science and technology, medical health and the like, and discloses a voice generation method, device and equipment based on pseudo-autoregression modeling and a medium, and the method comprises the steps: obtaining a training sample containing a text sequence, a prompt voice segment and a target semantic token sequence; performing continuous fragment mask training on the text-to-semantic model to obtain a pseudo-autoregression trained text-to-semantic model; generating candidate speech output by using the text-to-semantic model and the initial semantic-to-acoustic model which are subjected to pseudo-autoregression training, and constructing a preference data pair; updating the semantics-to-acoustics model based on the preference data pair to obtain a preference optimized semantics-to-acoustics model; and generating target voice output based on the target text and the target prompt voice. According to the method, the time sequence modeling capability of the model is enhanced through pseudo-autoregression training, and the voice generation quality is directly optimized through the preference data pair, so that the voice alignment precision and the subjective listening feeling performance are improved.

The invention belongs to the technical field of natural language processing, and discloses a natural language processing platform and method based on deep learning, and the platform comprises a data collection module which is used for collecting font data, text data, image data and environment data; the data processing module is used for preprocessing the collected font data, text data, image data and environment data and obtaining a comprehensive feature data set; the character recognition module is used for predicting character recognition results through a character recognition model and judging whether the character recognition results are consistent or not; the semantic recognition module is used for generating corresponding character paraphrases through a semantic recognition model; collecting character recognition results and corresponding character paraphrases through the software terminal; the voice output module is used for converting the character recognition result and the paraphrases of the corresponding characters into natural voices and outputting the natural voices by adopting a TTS technology; through the implementation of the technology, students can be helped to better learn and understand.

The invention discloses an information acquisition system and method suitable for visually impaired people. The system comprises a hardware layer, a driving layer, an operating system layer, a middleware layer and an application layer which are sequentially in communication connection through interfaces. The application layer comprises a voice interaction module, a large model integration module and a braille conversion module; the hardware layer is used for collecting a voice input instruction and outputting a braille character sequence and a voice output result; the voice interaction module is used for converting a voice input instruction into a text input instruction and converting a text output result into a voice output result; the large model integration module is used for analyzing the text input instruction and generating a text output result; and the braille conversion module is used for converting the text output result into a braille character sequence, so that a hardware layer outputs the braille character sequence based on a timestamp alignment technology. Therefore, based on powerful semantic understanding and information retrieval of the large model integration module, accurate information acquisition is realized, and a five-layer decoupling architecture is adopted, so that the convenience of later system maintenance is improved.

The invention provides a speech synthesis method and device, electronic equipment and a storage medium, and relates to the technical field of speech, and the method comprises the steps: inputting an obtained to-be-synthesized text and an emotion attribute into a speech synthesis model, and obtaining a target speech outputted by the speech synthesis model; wherein the speech synthesis model is obtained through training based on a first sample text corresponding to the first sample speech and a first sample emotion feature corresponding to the first sample speech, and the first sample emotion feature is obtained after the first sample speech is input into the emotion coding model; the emotion coding model is obtained by training based on the target code and the minimum mutual information loss of the second sample emotion feature. According to the invention, the emotion coding model can be obtained based on the minimum mutual information loss training, so that the emotion features output by the emotion coding model do not include unrelated information such as timbre and text content, the speech synthesis model can realize fine-grained emotion control, and the control force of speech synthesis is improved.

The invention discloses an intelligent accompanying system and method based on semantic recognition, relates to the technical field of intelligent questions and answers, and solves the problem that an expected dredging effect cannot be achieved due to the fact that emotional changes and psychological needs of old people are often difficult to comprehensively capture in a traditional method. The method comprises the following steps: converting an acquired voice sample into a text form based on Paraformer voice recognition; constructing an emotional model and performing user emotion recognition based on the emotional model; emotional support strategies and voice output verbal skills are set, time, environment data and user emotions are collected in real time, and the corresponding emotional support strategies and voice output verbal skills are started; a sound copying technology of Ali cloud CosyVoice is adopted to simulate sound of relatives of a user to carry out voice output in a user interaction process. In a word, unprecedented care and support are provided for the elderly in the emotional level, and each elderly in a pension community can feel warmth and accompanying brought by science and technology.

The invention belongs to the field of artificial intelligence, and particularly relates to a medical pre-inquiry real-time interaction method and system, a terminal and a medium. Screening medical knowledge contexts similar to the consultation text from a professional medical knowledge base, and recording the medical knowledge contexts as related texts; based on a pre-constructed prompt template, constructing a current consultation prompt through the consultation text and the related text; inputting the current consultation prompt into the large language model, and outputting consultation reply content; performing voice synthesis on the consultation reply content to generate consultation reply voice data, storing the consultation reply voice data in the buffer area, extracting reply voice data from the main buffer area for voice output, and transferring data in the auxiliary buffer area to the main buffer area when the data in the main buffer area is less than a threshold value; speech synthesis and speech output are asynchronously processed. According to the invention, the professionality and accuracy of medical pre-inquiry are improved, the playing lagging caused by waiting for synthesis is effectively avoided, and the interaction experience effect is improved.

The invention discloses a sign language-voice conversion system, and belongs to the technical field of auxiliary communication and wearable computing. The system comprises a wearable myoelectricity acquisition module used for acquiring double-arm myoelectricity signals when a user executes sign language; the mobile terminal module is wirelessly connected with the acquisition module and is used for receiving and preprocessing the signal and uploading the signal; the cloud processing module is used for receiving the signal, converting the signal into text information through a sign language recognition model, and further calling a voice synthesis service to convert the text into voice data; and the wearable audio output module is used for receiving and playing the voice data. Through an innovative end-to-end hardware system architecture, natural, accurate and real-time translation and voice output of sign language gestures are realized, communication barriers between hearing-impaired people and healthy hearing people are effectively solved, and the system has the advantages of flexible deployment, user friendliness and privacy protection.

To improve the accuracy of a questionnaire collected from a user who drove a test-drive vehicle.SOLUTION: A questionnaire apparatus which can be disposed in a test-drive vehicle includes an input unit to which voice can be input, an output unit which can output voice, and a control unit. The control unit outputs, while a user is driving the test-drive vehicle, voice for a questionnaire to a user via the output unit at a predetermined timing, and executes processing to receive answers of the user to the questionnaire via the input unit.SELECTED DRAWING: Figure 4

The invention discloses an automobile seat motor intelligent sliding rail adjusting system based on AI voice interaction, and the system comprises the following modules: a local voice wake-up module which is used for monitoring the voice input of a user, and activating a voice recognition module after detecting a preset wake-up keyword; the voice recognition module is used for generating a standardized text instruction; the semantic modeling module is used for executing word vector modeling processing on the standardized text instruction; the control parameter mapping module is used for inputting the semantic feature vector into a trained neural network model; the slide rail state detection module is used for collecting slide rail state information in real time; the voice synthesis module is used for converting the content in the prompt information field into a voice output content template; the vehicle-mounted audio output module is used for generating broadcast content; and the voice interaction log module is used for recording information involved in the voice interaction process. Semantic modeling and control mapping are fused, and the intelligent voice-driven seat adjusting system is constructed.

The invention discloses a vending machine intelligent shopping guide method and system based on voice interaction. The objective of the invention is to realize accurate recognition of user demands through voice recognition and natural language processing technologies, and perform intelligent recommendation in combination with commodity big data. According to the method, through integration of a high-sensitivity microphone and voice recognition, a voice instruction of a user is converted into text information, and a purchase intention behind the user is analyzed. Meanwhile, detailed information of commodities sold by the vending machine is collected, and after standardization processing and feature extraction, the detailed information serves as training data of the deep learning model. Personalized commodity recommendation can be generated according to user requirements through a trained and optimized model, and the personalized commodity recommendation is fed back to a user through a high-resolution display screen or voice output. According to the invention, the shopping experience of the user is improved, and the purchase conversion rate is improved through intelligent recommendation.

A method for a motor vehicle and for preventing damage to the motor vehicle, comprising: capturing personal information relating to a person in the vicinity of the motor vehicle; determining a hazard indicator describing a risk of damage to the motor vehicle based on the personal information; issuing an audibly perceptible speech output depending on the hazard indicator; capturing a response of the person to the speech output; and issuing a prevention and / or interaction signal depending on the response.