Patents
Literature
Patsnap Eureka AI that helps you search prior art, draft patents, and assess FTO risks, powered by patent and scientific literature data.

2124 results about "Speech characteristics" patented technology

Speech characteristics are features of speech that in varying may affect intelligibility. They include: Articulation. Pronunciation. Speech disfluency. Speech pauses. Speech pitch. Speech rate.

Intelligent multi-mode virtual digital human interaction system based on AI language large model, interaction method and application

The invention discloses an intelligent multi-modal virtual digital human interaction system based on an AI language large model. The system comprises a high-authenticity face generation module; the high-authenticity face generation module uses an AdaAN network, based on adaptive feature fusion and voice driving and time sequence modeling of voice features, feature information related to voice is extracted, the extracted voice features are processed through a deep neural network, it is ensured that the voice and facial expressions are highly aligned in time and space, and the face recognition accuracy is improved. Collecting a bio-electricity signal, mapping the signal to facial muscle movement, generating a final facial expression, and interacting with a user; the system further comprises an intelligent interaction module, a training optimization and efficient generation module, an efficient integration module, a multi-modal data acquisition module, an AI large model core processing module, a digital human image generation and driving module, an interaction scene adaptation module and a feedback optimization module. The invention further discloses a multi-mode digital human interaction method which has wide application value.
Owner:EAST CHINA NORMAL UNIV

Man-machine interaction method and system based on multiple modes

The invention provides a multi-mode-based man-machine interaction method and system, and relates to the technical field of man-machine interaction. Voice input of a user is received, environment information is collected, facial expressions of the user are captured, and voice features, environment features and facial features are obtained based on the input; utilizing a pre-trained emotion recognition model to generate emotion features of the user, and utilizing a pre-trained user behavior model to generate behavior features according to the current operation information of the user; after the features are obtained, the first weights and the second weights of the features are dynamically adjusted, feature weighted fusion is carried out, and comprehensive features are obtained; the multiple candidate responses are generated based on the comprehensive characteristics, the final response is screened out through a preset scoring mechanism and output, comprehensive understanding of the intention and the emotional state of the user is achieved through comprehensive utilization of multiple data sources, and meanwhile the weight of each characteristic can be dynamically adjusted so that more intelligent and humanized interaction experience can be generated.
Owner:GUANGZHOU DAZZLE VIEW INTELLIGENT TECH CO LTD

Voiceprint recognition method, graphical interface, and electronic device

This application discloses a voiceprint recognition method, a graphical interface, and an electronic device. In the voiceprint recognition method, a voiceprint model is preset in the electronic device, and then the electronic device trains and updates the preset voiceprint model based on a voiceprint feature extracted from a voice of a registered user to obtain an exclusive voiceprint model belonging to the registered user. Finally, the electronic device uses the exclusive voiceprint model to generate a registered user representation based on the voiceprint feature of the voice of the registered user, and uses the registered user representation as a reference standard to realize voiceprint recognition on a voice of a speaker. Since the exclusive voiceprint model is trained based on voiceprint features of personal voices of the registered user, the registered user representation generated can accurately express voiceprint features of the user, thereby improving accuracy of voiceprint recognition.
Owner:HONOR DEVICE CO LTD

Face-translator: end-to-end system for speech-translated lip-synchronized and voice preserving video generation

A neural end-to-end system is provided for the face and voice preserving translation of videos. The system is a pipeline of multiple models that produces a video of the original speaker speaking in the target language with modified lip movement to match the target speech, while preserving emphases and prosody of the original speech, and voice characteristics of the original speaker. The pipeline starts with automatic speech recognition including emphasis detection, followed by the translation model. The translated text is then synthesized by a Text-to-Speech model that recreates the original emphases in the target sentence. The resulting synthetic speech is then converted back to the original speakers' voice using a voice conversion model. Finally, to synchronize the lips of the speaker with the translated audio, a generative model generates frames of adapted lip movements which are combined with the audio to produce the final output. The disclosure further describes several use-cases and configurations that apply these techniques to video conferencing, dubbing, low-bandwidth transmission, speech enhancement and assistive technology for the hearing impaired.
Owner:WAIBEL ALEXANDER

Multi-modal sentiment analysis method and system based on attention and graph enhanced text

The invention provides a multi-modal sentiment analysis method and system based on attention and graph enhanced text, and belongs to the technical field of multi-modal sentiment analysis. The method comprises the steps of obtaining text features, image features and voice features in video data and performing preprocessing; an adaptive cross-modal interaction module is adopted to calculate attention weights between the text features and the image features and between the text features and the voice features, and enhanced text features are obtained; inputting the enhanced text features into a hierarchical multi-modal graph fusion network, and constructing an adjacent matrix by using a self-attention mechanism; inputting the adjacent matrix and the enhanced text features into a jump connection graph convolutional network to obtain a final feature matrix; and combining the feature matrix with the adjacent matrix, and obtaining a prediction result of sentiment analysis by using an encoder and a classifier. The problem of deficiency in context alignment among different modes is solved, and emotion polarity prediction is performed more comprehensively and accurately.
Owner:SHANDONG JIAOTONG UNIV

Character-driven lip sound synchronous digital human generation method and device, equipment and medium

The invention relates to a character-driven lip sound synchronous digital human generation method and device, equipment and a medium, and the method comprises the steps: obtaining an input text and a single object video, and carrying out the preprocessing of the input text, and obtaining a phoneme sequence and semantic features; the method comprises the following steps: preprocessing a single-object video to obtain a circularly played video paragraph, a mask face image and a reference face image; generating voice features based on the phoneme sequence and the semantic features; generating an audio sequence based on the voice features; acquiring face deformation parameters according to the mask face image, the reference face image and the voice features, and deforming the reference face image to generate a target face image; and embedding the target face image into a video frame of the video paragraph to generate a video sequence, and splicing the video sequence with the audio sequence to obtain a target digital human video. The digital human video is generated based on text driving, so that the voice and lip-shaped video matched with the text can be generated in time, the time delay for waiting for text to be converted into voice processing is eliminated, and the digital human video production efficiency is greatly improved.
Owner:GUANGZHOU ZIWEIYUN TECH CO LTD

Speech feature processing method and device, equipment and medium

The invention relates to the technical field of voice processing, can be applied to business scenes of financial science and technology, medical health and the like, and discloses a voice feature processing method, device, equipment and medium. Performing time resolution analysis based on the fused Mel band energy to generate a multi-scale Mel spectrum amplitude value, and performing nonlinear transformation on the multi-scale Mel spectrum amplitude value according to the noise intensity parameter to generate a noise suppression Mel component; and generating a perception weighting coefficient according to an auditory perception model, and executing frequency domain energy adjustment on the noise suppression Mel component to generate Mel spectrum representation. On the basis of frequency resolution self-adaption, time resolution dynamic adjustment and auditory perception modeling, nonlinear transformation and perception weighting processing are applied to the multi-scale Mel spectrum amplitude value, the influence of noise interference on voice features can be effectively reduced, and the key information retention capacity of voice signals is enhanced.
Owner:PING AN TECH (SHENZHEN) CO LTD

Audio processing method and system for speech spectrum reconstruction

The invention relates to the technical field of audio processing, and discloses an audio processing method and system for speech spectrum reconstruction, and the method comprises the steps: carrying out the preprocessing of an original speech signal collected by a vehicle-mounted microphone, and obtaining a speech signal time-frequency matrix; performing singular value decomposition on the voice signal time-frequency matrix to obtain a sparse matrix representing voice components and a low-rank matrix representing noise components, and establishing a frequency band characteristic incidence matrix through a mutual information entropy calculation method; inputting the sparse matrix into a selection forgetting extreme learning machine for voice feature prediction to obtain predicted voice feature data; and based on the predicted voice feature data and the frequency band feature incidence matrix, performing voice probability calculation and feature interpolation on each time frequency point by adopting a logarithmic frequency spectrum amplitude calculation method to obtain reconstructed frequency spectrum data, thereby improving the accuracy of voice recognition and meeting the requirement of real-time processing in a vehicle-mounted environment at the same time.
Owner:DONGGUAN HUAZE ELECTRONIC TECH CO LTD

Target person voice extraction method and device fusing voice and electroencephalogram signals

The invention belongs to the technical field of voice processing, and particularly relates to a target person voice extraction method, system and device fusing voice and electroencephalogram signals. According to the method, a voice separation model comprising an electroencephalogram encoder, a voice encoder, a separation network and a voice decoder is designed, and a voice extraction task of a target person is executed by using the trained voice separation model. Wherein the electroencephalogram encoder comprises a one-dimensional convolutional layer and an attention layer based on KAN. The voice encoder comprises a one-dimensional convolution layer and a bidirectional Mama layer. And the separation network adopts a cross attention module to perform feature fusion on the extracted voice features and electroencephalogram features, and analyzes the obtained fusion features to obtain a feature mask of the voice of the target person. And the voice decoder reconstructs the pure voice of the target person by using the feature mask and the voice features. According to the method, the problems that deep electroencephalogram features and global voice features are difficult to effectively capture and the precision of the reconstructed voice of the target person is poor in the existing scheme are solved.
Owner:ANHUI UNIV

Precise international communication digital human real-time dialogue method fused with multi-modal technology

The invention discloses an accurate international communication digital human real-time dialogue method fused with a multi-modal technology. The method comprises the following steps: S1, constructing a digital human image and tone; s2, propagation content generation and problem guidance; s3, semantic analysis and intention clarification based on the real-time voice dialogue; s4, geographic preference modeling and path planning; s5, cross-context propagation content generated based on retrieval enhancement is generated; s6, visually displaying the propagation content; and S7, carrying out digital human-driven multi-language propagation content real-time output and feedback closed-loop optimization. Through accurate utterance expression analysis, accurate international propagation problem recommendation is provided, cross-context propagation content generation based on semantic understanding is realized, digital people with voice features and visual images are constructed, real-time dialogue interaction of users is realized, and user experience is improved. The system can carry out geographic modeling according to the region where the accurate problem is located, language preference and propagation object culture characteristics, and differential propagation path planning is achieved.
Owner:HUNAN NORMAL UNIVERSITY

Aircraft cabin environment personalized adjustment method based on sentiment analysis

The invention discloses an aircraft cabin environment personalized adjustment method based on sentiment analysis, and the method comprises the steps: collecting the facial expression, voice waveform and environment parameters of a passenger in real time through a cabin multi-source sensor, and generating a standardized physiological signal matrix and anonymized voice features through noise reduction and feature extraction; inputting the physiological signal matrix and the voice features into a pre-trained deep learning model, outputting an emotion index and a classification label, and updating model parameters through a federal learning framework; dynamically generating temperature, humidity and oxygen concentration adjusting instructions and dynamic weights by adopting a fuzzy reasoning system in combination with the emotion indexes and passenger preset preferences; environment adjustment is executed through a closed-loop control system, and environment parameter errors are fed back; synchronously updating a fuzzy inference system rule base and deep learning model parameters by utilizing reinforcement learning in combination with environmental parameter errors and emotion index changes, and completing optimization of a closed loop; according to the invention, real-time dynamic regulation and control and continuous adaptation optimization of the personalized cabin environment can be realized.
Owner:WENZHOU DOVER AVIATION IND GROUP CO LTD

Adaptation and training of neural speech synthesis

Disclosed are systems, methods and other implementations for speech generation, including a method that includes obtaining a speech sample for a target speaker, processing, using a trained encoder, the speech sample to produce a parametric representation of the speech sample for the target speaker, receiving configuration data for a speech synthesis system that accepts as an input the parametric representation, and adapting the configuration data for the speech synthesis system according to an input comprising the parametric representation, and a time-domain representation for the speech sample, to generate adapted configuration data for the speech synthesis system. The method further includes causing configuration of the speech synthesis system according to the adapted configuration data, with the speech synthesis system being implemented to generate synthesized speech output data with estimated voice and time-domain speech characteristics approximating actual voice and time-domain speech characteristics for the target speaker.
Owner:CERENCE OPERATING CO

Service content generation method and device based on emotion recognition, equipment and medium

The invention relates to the technical field of artificial intelligence and the field of financial science and technology, and discloses a service content generation method based on emotion recognition, and the method comprises the steps: collecting voice data, extracting voiceprint features, and building a user voiceprint file; extracting Mel frequency cepstrum coefficient features and Mel energy difference coefficient features, and inputting the extracted features into an emotion classification model for emotion classification recognition to obtain an emotion state of the user; the current emotional state is compared with a historical normal emotional state, emotional fluctuation is analyzed, and whether the emotional state of the user is abnormal or not is judged; and when it is detected that the emotional state is abnormal, target service content is determined from the personalized service library in combination with the emotional state and the user voiceprint file. According to the method, the voice features are extracted and the user voiceprint file is established, so that the emotion fluctuation of the user can be accurately analyzed, whether abnormity exists or not is judged, accurate recommendation service is provided in combination with the personalized features and historical behavior data of the user, and the intelligent and personalized degree of vehicle-mounted voice interaction is improved.
Owner:PING AN TECH (SHENZHEN) CO LTD

Voice synthesis method and device based on gated attention mechanism, equipment and medium

The invention relates to the technical field of artificial intelligence and the field of financial science and technology, and discloses a speech synthesis method based on a gated attention mechanism, and the method comprises the steps: obtaining text data and speech data, converting the text data into a text symbol sequence through a text encoder, speech features of the speech data are extracted through a speech encoder trained based on a self-supervised learning model, and the speech features are quantized into a discrete speech symbol sequence; then, respectively organizing the text symbol sequence and the voice symbol sequence into a text sequence and a voice sequence; the corresponding relation between the text symbols and the voice symbols is established through preliminary alignment processing, the matching degree between the text symbols and the voice symbols is dynamically adjusted through a gating attention mechanism, and finally a final voice signal is generated through a decoder. According to the invention, through the voice encoder trained by the self-supervised learning model, the extraction capability of voice features is effectively improved, and effective feature representation can still be learned from unlabeled data especially in a scene lacking a large amount of labeled data.
Owner:PING AN TECH (SHENZHEN) CO LTD

Voice interaction task execution method and device based on large model, equipment and medium

The invention discloses a voice interaction task execution method and device based on a large model, equipment and a medium, and relates to the field of artificial intelligence, and the method comprises the steps: carrying out the voice recognition of voice features, obtaining a text character sequence, and carrying out the optimization of the text character sequence through a preset large language model, and obtaining a target text character sequence; performing entity recognition on the target text character sequence by using a preset large language model to obtain an entity recognition result, determining a relationship type among entities in the target text character sequence according to the entity recognition result, and constructing a knowledge graph according to the relationship type among the entities; generating an initial triple based on the knowledge graph, and optimizing the initial triple by using a preset large language model to obtain a target triple; and fusing the target triple with the initial knowledge graph, and executing a voice interaction task in the target voice interaction scene based on the updated knowledge graph. According to the method and the device, the efficiency and the accuracy of extracting the structured knowledge from the Chinese speech are improved.
Owner:SHANDONG LANGCHAO YUNTOU INFORMATION TECH CO LTD

Multi-modal driven virtual digital human face animation generation method and system

The invention discloses a multi-mode driven virtual digital human face animation generation method and system, and relates to the field of computer graphics, and the method comprises the steps: obtaining voice input and text input, and extracting voice features and text features; dynamically fusing the two features through an attention fusion model to generate facial expression and head posture control parameters, and dynamically adjusting contribution weights of a voice mode and a text mode to the control parameters by adopting a driving strategy of differentiation of facial upper expression and facial lower expression; and performing local deformation on the virtual digital human face image based on the control parameter to generate an initial animation frame, and performing refining processing on the initial animation by using a generative adversarial network to obtain a refined face animation. By means of the technical scheme, the natural and vivid virtual human face animation matched with the voice content and the text semantics can be generated.
Owner:LIANGSHENG DIGITAL CREATIVE DESIGN (HANGZHOU) CO LTD

Intelligent accompanying human-type robot based on multi-modal emotion interaction and sensing method

The invention relates to the technical field of robot perception, in particular to an intelligent accompanying human-type robot based on multi-mode emotion interaction and a perception method, and the method comprises the following steps: collecting facial expression and body posture data of a user through a visual perception unit, collecting intonation, speed, volume and keyword data through a voice perception unit, and carrying out the recognition of the speech, speed, volume and keyword data; data preprocessing and feature extraction are carried out; secondly, carrying out time sequence modeling on the extracted visual and voice features based on a recurrent neural network, and generating visual and voice time sequence feature vectors; the method comprises the steps that firstly, time sequence features are extracted, then, a multi-mode fusion model based on an attention mechanism is adopted, the time sequence features are fused into a comprehensive emotion feature vector used for emotion state classification, and finally, personalized interaction content such as voice, expressions and limb actions matched with the emotion state of a user is generated according to an emotion classification result. According to the method, the accuracy of robot emotion perception and the naturalness of man-machine interaction are remarkably improved, and the method is suitable for multi-scene application of education, medical treatment, old-age care and the like.
Owner:BEIJING HAIBAICHUAN TECH CO LTD

System and method for enhancing speech of target speaker from audio signal in an ear-worn device using voice signatures

An ear-worn device is provided that operates to isolate and individually treat the received speech of a target speaker or multiple target speakers from an audio input signal detected in a multi-speaker environment. The ear-worn device uses a machine learning model that receives a voice signature of each of one or more target speakers as input signals, to identify and isolate the component of the audio input signal attributable to the target speaker(s). Once isolated, the target speaker's speech may be enhanced, de-emphasized, or otherwise processed in a manner desired by the wearer of the ear-worn device. The wearer may use an external electronic device, e.g., a phone, to select one or more target speakers in a conversation and / or configure various settings associated with processing the speech on the ear-worn device.
Owner:FORTELL RESEARCH INC

Video generation method and apparatus, model training method and apparatus, device, storage medium, computer program, and computer program product

Embodiments of the present disclosure provide a video generation method and apparatus, a model training method and apparatus, a device, a storage medium, a computer program, and a computer program product. The video generation method comprises: acquiring a first voice feature and a first head feature of a first character, wherein the first head feature is obtained by performing mask processing on a mouth area of the first character in a first video, and the first voice feature is obtained on the basis of a first audio; on the basis of the first head feature and the first voice feature, generating a second video containing the first character, wherein a speaking action of the first character in the second video matches the first audio.
Owner:MOORE THREADS TECH CO LTD

Vector voice interaction risk control method and system based on voiceprint features and lip synchronization

The invention relates to a borrower voice interaction risk control method and system based on voiceprint features and lip synchronization. The method comprises the following steps: synchronously acquiring a voice signal and face video stream data of a user through acquisition equipment; performing voice feature extraction on the voice signal to generate a voiceprint feature vector and a voice time sequence feature vector; lip motion feature analysis is carried out on the face video stream data, and lip motion track feature vectors representing lip dynamic changes are extracted; performing identity matching on the voiceprint feature vector and a pre-stored reference voiceprint database to generate a first matching degree score; performing cross-modal time sequence alignment analysis on the voice time sequence feature vector and the lip movement trajectory feature vector to generate a second matching degree score; and generating a comprehensive risk control scoring result according to the first matching degree score and the second matching degree score. According to the scheme provided by the invention, a multi-modal cross validation mechanism can be formed, and the limitation of a traditional single-modal detection scheme is effectively overcome.
Owner:FEIHU INTERACTIVE TECH BEIJING CO LTD

Multi-modal emotion recognition method and system based on gating feature selection and expert modeling

The invention discloses a multi-modal emotion recognition method and system based on gating feature selection and expert modeling, and the method comprises the following steps: obtaining an original voice signal and an original text, and then respectively extracting voice features and text features of different levels; dynamically selecting the most suitable speech features and text features from the speech features and text features of different levels through a corresponding gating feature selection network, carrying out weighted fusion, obtaining the fused speech features and the fused text features, and splicing the fused speech features and the fused text features to obtain multi-modal fusion features; the method comprises the following steps: further capturing information of different modals in multi-modal fusion features through a modal hybrid expert Transform module, fusing the information again to obtain new multi-modal fusion features, and then processing the new multi-modal fusion features through an expert model which is most suitable for processing the new multi-modal fusion features in the hybrid expert Transform module. And obtaining a final prediction result and predicting the emotional state based on the prediction result. According to the method, the emotion prediction accuracy is improved.
Owner:HUNAN UNIV +1

Voice generation method and device, equipment and medium

The invention relates to the technical field of artificial intelligence and the field of financial science and technology, and discloses a voice generation method, which comprises the following steps of: acquiring a to-be-converted text and a reference voice sample, extracting voice features of reference voice by using a pre-trained self-supervised learning encoder, converting the to-be-converted text into voice features through a text-to-voice feature model, and generating a voice feature. And analyzing the similarity of the two groups of features, determining the voice features of the target speaker from the two groups of features, fusing the two groups of features to generate final fused voice features, and decoding by using a pre-trained vocoder to generate voice signals. According to the method, feature matching and fusion are carried out under the small sample or zero sample condition, the voice similar to the target voice can be generated without a large amount of voice data, the voice synthesis efficiency and quality are effectively improved, the multi-scene application requirement is met, and the method is particularly suitable for voice generation of unseen speakers.
Owner:PING AN TECH (SHENZHEN) CO LTD

Artificial intelligence speech recognition system

The invention discloses an artificial intelligence speech recognition system, and the system comprises a multi-modal feature extraction module which employs an improved Conformer architecture to synchronously extract the time-frequency features and text embedding vectors of speech signals; the joint training module is used for performing joint optimization on ASR and NMT loss functions through an adversarial training strategy, learning voice recognition and machine translation tasks at the same time through joint training, and completing direct mapping from voice features to a target language; the context perception translation engine is used for integrating an attention mechanism of a pre-training language model, carrying out deep coding on the extracted speech features and generating cross-language semantic representation; the self-adaptive post-processing module is used for dynamically optimizing an output result by adopting a reinforcement learning framework, dynamically adjusting the output result according to a reward function, and optimizing translation quality and a speech synthesis effect; the dynamic language recognition module is a real-time language classifier based on a Wave2Vec 2.0 framework and is used for recognizing the language of the input voice in real time; and the incremental field adaptation module is used for quickly updating a field term library by using a LoRA fine tuning technology.
Owner:ANKANG UNIV

Telephone answering system based on AI

The invention discloses an AI-based telephone answering system, and relates to the technical field of telephone answering. Comprising a real-time voice acquisition and preprocessing module, a voice speed detection and prediction module, a self-adaptive ASR dynamic regulation and control module, a refined voice slicing and decoding module, a keyword detection and voice recognition module, a semantic understanding and intention analysis module and an emergency response and scheduling module. And the real-time voice acquisition and preprocessing module is used for acquiring user voice data of the emergency call in real time, establishing a real-time audio stream transmission channel and quickly preprocessing the acquired audio signal. According to the method, the voice features of the user at the high speed are quickly and accurately captured, and the voice segmentation length of the ASR engine is dynamically adjusted, so that each voice segment is clearer and more accurate, cross-segmentation vocabulary segmentation and recognition errors caused by too high speed are avoided, and the user experience is improved. Therefore, omission and misjudgment risks of the emergency information of the user are effectively avoided in an emergency help-seeking scene.
Owner:SHANDONG ZHIQUN INFORMATION TECH CO LTD

Streaming speech recognition method, device and equipment based on adaptive AI large model

The invention provides a streaming speech recognition method, device and equipment based on an adaptive AI large model, and the method comprises the steps: carrying out the noise reduction and speech enhancement of a collected streaming speech signal, and obtaining a preliminary speech signal; performing feature extraction on the preliminary voice signal to obtain a voice feature vector; performing multi-layer perceptron coding on the voice feature vector to obtain an initial coding vector; dynamically adjusting the clustering center and the clustering number of a clustering algorithm according to the distribution change of the voice feature vector, so as to cluster the initial coding vector, and obtaining a clustered coding vector; dynamically allocating weights for the clustered coding vectors according to voice context information through an attention mechanism of adaptively adjusting the weights, and performing weighting processing; and inputting the weighted coding vector into a neural network based on an adaptive growth structure for decoding to obtain a speech recognition result. According to the invention, the quality of a speech recognition result is effectively improved.
Owner:SHENZHEN BEIBO INTELLIGENT TECH

Multi-modal emotion prompting method and system for online auxiliary voice dialogue scene

The invention discloses a multi-modal emotion prompting method and system for an online auxiliary voice dialogue scene, and relates to the technical field of artificial intelligence, voice signals in the voice dialogue scene are input into a trained multi-modal emotion prompting large model, and emotion labels and voice emotion prompting information are output; the training process of the multi-modal emotion prompt large model comprises the following steps: acquiring original voice data of a real dialogue scene to construct a training data set for emotion recognition; performing feature extraction on the original voice data in the training data set to obtain an original voice feature vector; inputting the original voice feature vector into a sparse bridging Transform module, and outputting a fusion feature; inputting the fusion features into a large language model, generating coherent voice emotion prompt information through text decoding, and associating with corresponding emotion tags; according to the multi-mode emotion prompting method and system, accurate analysis and prompting of voice emotion in an online auxiliary voice dialogue scene are realized.
Owner:ARTIFICIAL INTELLIGENCE RES INST OF HEFEI COMPREHENSIVE NAT SCI CENT (ANHUI ARTIFICIAL INTELLIGENCE LAB)

Automated generation of targeted feedback using speech characteristics extracted from audio samples to address speech defects

Provided herein are systems and methods for providing instructions for speech based on speech classifications of verbal communications from users. Ae computing system can generate speech characteristics for a first verbal communication using a first audio sample from a user. The computing system can determine, from a plurality of speech classifications, a first speech classification for the first verbal communication based on the speech characteristics. The computing system can select, from a plurality of actions, an action comprising modifying one or more of the speech characteristics to define an utterance for the user based on the first speech classification. The computing system can provide an instruction presenting a message to prompt the user to perform the utterance defined by the action selected from the plurality of actions. The efficacy of the medication that the user is taking to address the condition may be increased.
Owner:CLICK THERAPEUTICS INC

Streaming voice interaction method, related device, equipment and storage medium

The invention discloses a streaming voice interaction method, a related device, equipment and a storage medium, and the method comprises the steps: carrying out the end point detection of streaming voice, responding to the detection of a starting end point of the streaming voice, executing the sliding window operation of the streaming voice, obtaining a voice frame, carrying out the feature extraction based on the voice frame, and obtaining a voice frame; voice features of the voice frames are obtained; inputting the speech features of the speech frames to a speech recognition system for recognizing streaming speech to obtain recognition results of the speech frames, and performing classification prediction based on the coding features of the speech frames to obtain classification results of the speech frames; in response to the classification result representing semantic ending, based on the recognition result of each voice frame from the starting endpoint to the ending endpoint, obtaining a recognition text, and processing at least based on the recognition text by the intelligent dialogue model to generate a reply text; and continuously returning to execute the sliding window operation in response to the classification result representing that the semantics is not finished. According to the scheme, the timeliness of streaming voice interaction can be improved.
Owner:IFLYTEK CO LTD

Speech synthesis method and device, computer equipment and storage medium

The invention belongs to the field of artificial intelligence, medical treatment and finance, and relates to a speech synthesis method, which comprises the following steps: performing phoneme conversion and context embedding on a text data set, obtaining a phoneme sequence and a text context vector, extracting reference speech features of a reference audio sample, inputting a phoneme-level potential diffusion model, and generating phoneme-level prior potential features; inputting the phoneme-level prior potential features into a phoneme-level variational self-encoding model to obtain reconstructed speech features; extracting a target voice feature of the target audio sample; performing alignment and up-sampling on the target voice feature and a phoneme sequence to obtain a target audio frame-level feature; and training based on the reconstructed speech features and the target audio frame-level features to obtain a conditional control speech synthesis model, and synthesizing the to-be-converted text speech. The invention further provides a speech synthesis device, computer equipment and a storage medium. In addition, the invention also relates to a block chain technology, and the to-be-converted text can be stored in a block chain. The speaker fidelity and the text understandability are improved.
Owner:PING AN TECH (SHENZHEN) CO LTD

Multi-speaker dialogue voice analysis method and device, equipment and medium

The invention relates to the technical field of voice processing, can be applied to business scenes such as financial science and technology and medical health, and discloses a multi-speaker dialogue voice analysis method, device and equipment and a medium, and the method comprises the steps: obtaining a to-be-analyzed multi-speaker dialogue voice, determining a naturalness score based on an acoustic feature, and obtaining a multi-speaker dialogue voice analysis result; determining a semantic consistency score based on voice embedding and semantic embedding corresponding to a preset text, determining a speaker consistency score based on embedding of a plurality of speakers of the same speaker, determining an interaction rationality score based on voice alternate overlapping duration, determining a diversity score based on a variance of voice features, and fusing the scores, the comprehensive mass fraction is obtained. According to the invention, through quantitative evaluation of five dimensions of naturalness, semantic consistency, speaker consistency, interaction rationality and diversity, a comprehensive quality scoring system is established, so that the evaluation result simultaneously reflects voice fluency, content matching degree, identity stability, interaction rhythm rationality and feature richness.
Owner:PING AN TECH (SHENZHEN) CO LTD