934 results about "Voice transformation" patented technology

The invention discloses a digital human real-time interaction system and method. The interaction system comprises terminal equipment and a cloud server. The terminal device receives voice input by a user and transmits the voice to the cloud server. The cloud server comprises a video acquisition and processing module, an intelligent question and answer and voice generation module, an audio processing module and an image rendering module. A video capture and processing module captures original video data indicative of an object of a digital figure and generates successive image frames. The intelligent question and answer and voice generation module converts the voice into a text; performing semantic analysis on the text, understanding the intention of the user, and generating a corresponding reply text; the reply text is also converted into a reply voice. The audio processing module processes the reply voice and the continuous image frames to generate first video data. And the image rendering module fuses the first video data with the virtual background, and performs real-time processing on the fused video to generate a response video. And the terminal equipment outputs the response video.

The present disclosure relates to systems and methods for enhancing the interaction between users and automated agents, such as digital assistants, by employing Large Language Models (LLMs) to infer the intent of spoken language. The invention involves continuously monitoring ambient audio, converting speech to text, and utilizing LLMs to determine whether spoken language is intended for the automated agent. A structured prompt, including the converted text and specific instructions, is sent to the LLM, which is fine-tuned to process domain-specific prompts. The LLM provides a structured output in a standardized format, indicating the user's intent. The system may involve multiple prompts to perform separate tasks, such as identifying intent and generating additional context-specific data. This approach facilitates a more natural and intuitive user experience by eliminating the need for wake words and allowing seamless conversational interaction with virtual assistants across various platforms and devices.

A neural end-to-end system is provided for the face and voice preserving translation of videos. The system is a pipeline of multiple models that produces a video of the original speaker speaking in the target language with modified lip movement to match the target speech, while preserving emphases and prosody of the original speech, and voice characteristics of the original speaker. The pipeline starts with automatic speech recognition including emphasis detection, followed by the translation model. The translated text is then synthesized by a Text-to-Speech model that recreates the original emphases in the target sentence. The resulting synthetic speech is then converted back to the original speakers' voice using a voice conversion model. Finally, to synchronize the lips of the speaker with the translated audio, a generative model generates frames of adapted lip movements which are combined with the audio to produce the final output. The disclosure further describes several use-cases and configurations that apply these techniques to video conferencing, dubbing, low-bandwidth transmission, speech enhancement and assistive technology for the hearing impaired.

A voice conversion method used in an electronic device, including determining a target speech to be converted and a content representation vector corresponding to the target speech, where the target speech has a first content and a first voiceprint, and the content representation vector is obtained based on the speech waveform of the target speech; determining a reference speech and a voiceprint representation vector corresponding to the reference speech, where the reference speech has a second voiceprint, and the second voiceprint is different from the first voiceprint; generating a converted speech based on a speech generator according to the content representation vector and the voiceprint representation vector; where the converted speech has the first content and the second voiceprint; where the speech generator is obtained by jointly training a preset speech generation network and a preset discriminator network by using a training speech having the second voiceprint.

The invention relates to the technical field of multi-modal large language models, in particular to a government affair service content navigation method and system based on a large language model, and the method comprises the following steps: receiving multi-modal information input by a user through texts, voices or pictures; the voice is converted into a text, and character information in the picture is analyzed by using an OCR (Optical Character Recognition) technology; fusing multi-modal data, and inputting the fused multi-modal data into a large language model for semantic understanding and context association analysis; matching items are retrieved in combination with a local government affair knowledge base, and an initial recommendation list is generated; the recommendation result is displayed through the intelligent assistant, and interaction optimization options are provided; the method has the beneficial effects that the current government affair service content navigation mode is optimized through the multi-modal capability, the retrieval enhancement capability and the content generation capability of the large language model, so that the navigation is more modal and more intelligent, and meanwhile, the privacy and authority of data reply are ensured.

The invention relates to the technical field of voice processing, in particular to a real-time voice synthesis and conversion method in an intelligent voice call system, which comprises the following steps of: acquiring voice data input by a user through a voice recognition module, preprocessing the voice data, performing semantic analysis through a bidirectional recurrent neural network and an attention mechanism, and obtaining a semantic analysis result; the method comprises the following steps: extracting a key semantic unit, selecting a proper voice template based on user historical interaction data and an adaptive template matching algorithm, performing voice synthesis by using a generative adversarial network, realizing cross-language or dialect real-time voice conversion through a multi-task learning algorithm, and combining phoneme-level optimization processing to ensure the naturalness and continuity of the voice, so as to improve the voice conversion efficiency. And through real-time analysis and a self-adaptive adjustment mechanism fed back by the user, the voice output quality is optimized, and the user experience is improved. According to the invention, efficient and natural speech synthesis and conversion services can be provided in various intelligent devices and application scenarios, and complex requirements in different language environments can be met.

Embodiments of the present application provide a speech conversion method and apparatus, a storage medium, and an electronic device. The method includes: acquiring a source speech to be converted and a target speech sample of a target speaker; recognizing a style category of the target speech sample, and extracting a target audio feature from the target speech sample according to the style category; extracting a source audio feature from the source speech; acquiring a first style feature of the target speech sample and determining a second style feature of the target speech sample according to the first style feature; fusing and mapping the source audio feature, the target audio feature, and the second style feature to obtain a joint encoding feature; and decoding the joint encoding feature, to obtain a target speech feature, and converting the source speech based on the target speech feature to obtain a target speech.

The invention relates to the technical field of voice recognition, in particular to a multi-channel voice conversion and synchronous transmission system applied to simultaneous interpretation. Multi-language voice signals are collected through a multi-microphone array, background noise is dynamically eliminated by adopting a noise suppression technology combining spectral subtraction and deep learning, and a dynamic information source verification symbol is generated to ensure data synchronization integrity; a blind source separation technology is combined with time-frequency analysis and time delay estimation to realize multi-language signal separation and synchronization, and a time sequence is dynamically adjusted through voice activity detection; an end-to-end ASR-NMT-TTS model is constructed to realize voice real-time translation and synthesis, and low-delay transmission is carried out based on a 5G network; the real-time monitoring module is adopted to dynamically adjust the output delay and the signal-to-noise ratio, and the translation delay and the synchronization precision are optimized in combination with user feedback. According to the method, dynamic noise reduction, multi-source synchronization and a self-adaptive feedback mechanism are integrated, the problems of distortion and delay of multi-language simultaneous transmission in a complex noise environment are solved, and the obvious technical synergistic effect and practicability are achieved.

To provide technology that can present convincing information that leads to suppression of harassment to a person who has made a statement that raises concerns about harassment in a conversation.SOLUTION: A harassment information providing device is equipped with a transcription unit, a determination unit, and an output unit. The transcription unit converts speaker's remarks into text from voice data of conversation. The determination unit calculates a degree of negativity and positivity of the conversation by analyzing the text data representing the textualized statements, and determines a degree of harassment of the conversation using the negativity and positivity. The output unit outputs harassment information including information representing the degree of harassment.SELECTED DRAWING: Figure 11

The present application provides a model training method and apparatus, a speech-to-speech translation method and apparatus, and a medium. The method includes: obtaining a speech recognition sample and a real speech-to-speech translation sample; generating a pseudo-labeled speech-to-speech translation sample based on the speech recognition sample; and training a speech-to-speech translation model based on the pseudo-labeled speech-to-speech translation sample and the real speech-to-speech translation sample. Therefore, the model training precision can be improved.

The present disclosure provides a pitch-based speech conversion model training method and a speech conversion system, wherein an audio feature code is output by a priori encoder, and a pitch feature is extracted by a pitch extraction module. A linear spectrum corresponding to the reference speech is input into the posteriori encoder to obtain an audio latent variable. In addition, the audio feature code, a speech concatenation feature obtained by concatenation of the audio feature code and the pitch feature, and the audio latent variable are input into a temporal alignment module to obtain a converted speech code, and the converted speech code is decoded by a decoder to obtain a converted speech. The training loss of the converted speech is then calculated to determine the degree of convergence of the speech conversion model.

The invention discloses a judicial field semantic abstracting method, system and device based on a large model and a storage medium. The method comprises the steps that voice recognition is conducted on a court trial video, voice is converted into a court trial text, the court trial text is labeled, and a labeled text set is obtained to serve as a training set; training the large model by using the labeled text set, and finely adjusting the large model after each round of training; performing voice recognition on the actual court trial video, converting the voice into an actual court trial text, and extracting a timestamp; utilizing the trained large model to identify the court hearing process of the actual court hearing text, and reasoning a court hearing process text; matching the reasoned court hearing process text to timestamps by using a matching algorithm, and generating video anchor points when the timestamps at the starting positions of all the processes in the court hearing process text correspond to the anchor points of the processes of the court hearing video; abstracting the reasoned court hearing process text; and combining the abstract text with the video anchor points to generate structured court hearing information. According to the invention, the court trial video processing efficiency is improved.

Systems, apparatuses, methods, and computer program products are disclosed for predicting an entity and intent based on captured speech. An example method includes capturing speech and converting the speech to text. The example method further includes causing generation of one or more entities and one or more intents based on the speech and the text. The example method further includes determining a next action based on each of the one or more entities and each of the one or more intents.

The invention relates to voice processing, and provides a voice conversion method and device, equipment and a storage medium, and the method comprises the steps: carrying out the data enhancement of a to-be-converted voice, and obtaining a sample voice; inputting the sample voice into a rhythm encoder for processing to obtain phoneme-level rhythm features and spectrum features of the sample voice; inputting the sample voice into a content encoder for processing to obtain content features of the sample voice; inputting the rhythm features and the content features into an attention-based alignment model for alignment processing to obtain aligned rhythm features; inputting the aligned rhythm features, content features, frequency spectrum features and speaker vectors with set timbres into a decoder for synthesis to obtain synthesized voice; and intercepting part of content corresponding to the voice to be converted from the synthesized voice as target voice. The invention also relates to a block chain, and aims to improve the precision and naturalness of voice conversion.

The invention relates to the technical field of artificial intelligence and the field of financial science and technology, and discloses a multi-module collaborative voice generation method, which comprises the following steps: acquiring a reference text and reference voice, converting the reference text into a phoneme sequence and generating potential features, converting the reference voice into a Mel spectrogram and extracting style features, and extracting fundamental frequency information. A phoneme sequence and a Mel spectrogram are input into a text alignment module to generate alignment information, voice duration is predicted based on the alignment information and potential features, fundamental frequency of voice is predicted through fundamental frequency information, the predicted duration, fundamental frequency and style features are input into a decoder to generate a reconstructed Mel spectrogram, and finally the reconstructed Mel spectrogram is converted into voice audio data through a vocoder. According to the method, the generated voice can better reflect the natural rhythm and tone change of the text content by combining accurate control on the voice duration, the fundamental frequency and the style characteristics, and the rhythm and expressive force of the voice are enhanced, so that the naturalness and the fidelity of voice synthesis are improved.

The invention relates to the technical field of electronic medical record generation, and discloses an electronic medical record automatic generation method based on voice recognition, which comprises the following steps: S1, initializing voice input, distributing a unique voice acquisition identifier for a medical session, and completing identifier generation, input, storage, association and identity verification; s2, voice information intelligent recognition: converting voice into a text by using a voice recognition engine, and ensuring semantic consistency through a context verification unit and a semantic analysis unit; and S3, performing multi-speaker processing based on intelligent interference detection and resolution, positioning an interference time period and an interference source through an interference detection unit, and realizing time period distribution and priority ranking of multi-speaker voices by using a voice segmentation protocol and a linear weighting model. And finally, extracting related information from the text generated by voice conversion, and filling the related information into a medical record template of a hospital. The method improves the efficiency and accuracy of electronic medical record generation, solves the problems of multi-speaker interference and semantic logic, and is suitable for medical informatization scenes.

The invention provides a control method, which is applied to an artificial intelligence experiment system, at least comprises a robot hardware platform, a controller, a wheeled robot chassis and an industrial mechanical arm, and the method comprises the following steps: starting an operation system and executing hardware self-inspection; loading a large language model service and a traditional AI model, wherein the traditional AI model comprises a speech recognition model, a target detection model and a semantic analysis model; a natural language instruction of a user is received, voice is converted into a structured text through a voice recognition model, an instruction intention is analyzed through a large language model and a semantic analysis model, robot control parameters are generated, and a multi-modal interaction control signal is obtained; based on the multi-mode interaction control signal, a camera is called to collect image data, the position and category of a target object to be grabbed by the robot are recognized through a target detection model, the moving path of the robot and the grabbing track of the mechanical arm are calculated in combination with laser radar data, and the wheeled robot chassis and the industrial mechanical arm are controlled to execute coordinated actions.

The invention relates to the technical field of digital people, and discloses a real-time high-fidelity voice-driven digital people system which integrates six modules of voice acquisition, voice recognition, semantic analysis, sentiment analysis, animation generation and rendering generation. According to the invention, through contrastive analysis with the input voice, determination of accent and language styles in the early stage, conversion of the voice into the text, and word segmentation and part-of-speech tagging by using advanced machine learning and deep learning technologies, intentions and demands of a user are deeply understood, and all modules of the digital human system are driven to cooperate; a series of signal processing and feature extraction methods of the sentiment analysis module are applied, and a machine learning or deep learning model is combined to accurately recognize the sentiment polarity in the user voice, so that the system can make corresponding feedback according to a sentiment analysis result, for example, in an intelligent customer service scene, a reply strategy is adjusted according to the user sentiment, and the user experience is improved. And more personalized and humanized services are provided, and emotion resonance of interaction between the user and the digital human is enhanced.

To improve the flexibility of voice response scenarios or the versatility of application in a voice communication system. [Solution] The voice call device of the present invention is a voice call device that assists a caller in achieving their goal through a voice call with a user, and has a first prompt input means for inputting a prompt to a language model to guide the caller to achieve the goal, an answer acquisition means for acquiring an answer including a question for the user from the language model into which the prompt has been input, a voice conversion means for converting a first text of the question into a first call voice, a call voice sending means for sending the first call voice to the user's telephone, a call voice receiving means for receiving a second call voice in response to the first call voice from the telephone, a text conversion means for converting the second call voice into a second text, and a second prompt input means for inputting a prompt including the second text into the language model.

The invention discloses a speech enhancement method and system based on conditional stream matching and a vocoder, and the method comprises the following steps: S1, constructing a Mel spectrum extraction module which is used for converting an input noisy speech into a noisy Mel spectrum; s2, constructing a condition flow matching noise reduction module which is used for processing the noisy Mel spectrum obtained in the step S1 and outputting an enhanced Mel spectrum; and S3, constructing a neural network vocoder module which is used for restoring the enhanced Mel spectrum obtained in the step S2 into a time domain voice waveform so as to obtain an enhanced voice signal. The speech enhancement method combining conditional stream matching and the vocoder is proposed for the first time, the conditional stream matching method is innovatively introduced into the Mel-frequency spectrum domain, an end-to-end speech enhancement system is constructed, and a complete processing flow from noise speech input to high-quality speech signal output is realized.

The invention provides a multi-scene self-adaptive guide robot interaction method and system in the technical field of guide robots. The method comprises the steps that S1, a guide robot obtains interaction voice and converts the interaction voice into an interaction text; s2, analyzing the interaction text to generate a reply text, converting the reply text into a reply voice, and playing the reply voice for interaction; s3, managing the dialogue content in the interaction process; s4, the guide robot moves based on the guide path in the interaction process, and automatically plays explanation voice when arriving at the guide point; step S5, when the interaction text carries the target point, planning an advancing path to the target point through a bidirectional progressive optimal search algorithm; and S6, displacement is carried out based on the advancing path, and operation information of the robot is displayed in real time. The method has the advantages that the efficiency and accuracy of path planning, the flexibility of voice interaction and the expansibility of the system are greatly improved, and then the user experience of guide robot interaction is greatly improved.

The invention discloses a translation system based on artificial intelligence, intelligent glasses for translation and a translation method. The translation system comprises first intelligent glasses, a first intelligent mobile terminal and a cloud server, and a generative artificial intelligence large language model is configured in the cloud server. The first intelligent glasses pick up a first voice of a first language from a preset direction and send the first voice to the first intelligent mobile terminal. The first intelligent mobile terminal converts the first voice into a first text and sends the first text to the cloud server; and the cloud server translates the first text through the model according to the translation prompt to obtain a second text of a second language and sends the second text to the first intelligent mobile terminal. And the first intelligent mobile terminal converts the second text into a second voice and sends the second voice to the first intelligent glasses for playing. According to the invention, high-precision intelligent translation based on the intelligent glasses is realized, and the product viscosity can be increased.

The invention discloses an information search system based on a large language model, intelligent glasses and an information search method. The system comprises an intelligent glasses system and a model server, and a large language model is configured on the model server. The intelligent glasses system obtains a first voice containing a to-be-queried user question, obtains prompt information, converts the first voice into a first text, and sends the prompt information and the first text to the model server. And the model server determines characteristics of the user question according to the prompt information and the first text through a large language model (LLM), obtains a second text containing question reply according to the characteristics, and sends the second text to the intelligent glasses system. And the intelligent glasses system converts the second text into a second voice and plays the second voice. Real-time information search based on LLM is realized in the intelligent glasses system.

Disclosed are a speaker embedding-based speaker adaptation method and system generated by using global style tokens and a prediction model. The speaker adaptation method performed by the speaker adaptation system, according to an embodiment, may comprise the steps of: generating a plurality of speaker embeddings representing the tone of a speaker from a speaker embedding by using a voice transformation model including a global style token mechanism; and predicting the final speaker embedding representing a new speaker through similarity comparison between a new speaker embedding predicted by using a prediction model for predicting a speaker embedding and the plurality of generated speaker embeddings.

The invention provides a voice evaluation method, a voice evaluation device, a non-transitory storage medium and a computer program product. The voice evaluation method comprises the following steps: extracting a first person voice with a first tone based on a to-be-evaluated voice; converting the voice of the first person into voice with a second tone; and obtaining an evaluation result of the voice of the first person based on the converted voice with the second tone and the standard voice of the one or more accent.

The invention provides an automatic video generation method and device, equipment and a medium, and relates to the technical field of video production. The method comprises the following steps: acquiring video related information input by a user, and generating copywriting content according to the video related information and a preset copywriting script material library; according to a preset sound material library, converting the copywriting content into audio by using a text-to-voice conversion technology to obtain an audio material; according to the audio material, a digital human-driven technology is utilized, and a preset image material library and a video sub-mirror material library are combined, so that a visual material synchronized with the copywriting content is obtained; according to a preset packaging element material library, obtaining a packaging material matched with the copywriting content; and performing video synthesis and packaging on the audio material, the visual material and the packaging material to obtain a final video. According to the method, related materials can be generated through accurate matching and intelligent recommendation according to user requirements and recent popularity, a mixed cutting type and generation type material fusion arrangement mechanism is combined, one-key film formation is achieved, and the video production efficiency is improved.

The invention discloses a lip language-to-voice conversion method and device, a computer storage medium and a terminal, and aims to solve the problems that a lip language-to-voice conversion technology cannot be deployed on terminal equipment and voice quality cannot meet application requirements. The method is combined with a neural vocoder which reduces calculation complexity and resource requirements, system configuration requirements are reduced while system parameters are reduced, a design basis is provided for deploying a lip language-to-speech method on terminal equipment, and context-related visual feature sequences and user audio embedding vectors are fused, so that the user audio-to-speech conversion efficiency is improved, and the user audio-to-speech conversion efficiency is improved. A Mel spectrum acoustic feature sequence used for being converted into a voice waveform is obtained, and a user audio vector fused with the Mel spectrum acoustic feature sequence is obtained, so that the output voice waveform is more consistent with the real voice of a user; according to the embodiment of the invention, technical support is provided for deploying and applying the lip language-to-speech method meeting the speech quality requirement on the terminal equipment.

The invention discloses a voiceprint characterization method and system for resisting voice conversion and voice synthesis based on deep learning. Relates to the field of speech recognition and biological feature security. Comprising the steps of 1, acquiring an input voice sample signal in an automatic speaker verification system, and extracting an FBANK feature of the voice sample signal; 2, performing depth feature extraction on the FBANK features by using a convolutional neural network (CNN) to obtain a depth feature vector; 3, processing the depth feature vector by using a recurrent neural network (RNN), and generating a forged identity vector of the voice; and 4, classifying the counterfeit identity vectors by using a linear discriminant analysis (LDA) module, and outputting a judgment result. According to the invention, the accuracy of counterfeit detection can be effectively improved in both clean and noise environments.

Voice anonymization systems and methods are provided. Voice anonymization is done on the speaker's computing device and can prevent voice theft. The voice anonymization systems and methods are lightweight and run efficiently in real time on a computing device, allowing for speaker anonymity without diminishing system performance during a teleconference or VoIP meeting. The anonymization system outputs a transformed speaker voice. The anonymization system can also generate a voice embedding that can be used to reconstruct the original speaker voice. The voice embedding can be encrypted and transmitted to another device. Sometimes, the voice embedding is not transmitted and the listener receives the anonymized voice. Systems and methods are provided for the detection of voice transformations in received audio. Thus, a listener can be informed whether the speaker voice output from the listener's computing device is the original speaker's voice or a transformed version of the original speaker voice.

The invention relates to the technical field of voice explanation, in particular to an intelligent interactive voice explanation system based on an AIGC large model, which comprises an intelligent interactive voice explanation device main body 1, a memory and a processor are carried in the intelligent interactive voice explanation device main body 1, and a computer program is stored in the memory. The processor executes a computer program to realize the intelligent interactive voice explanation system based on the AIGC large model. The system comprises a tourist intention recognition module, a standard commentary matching module, an explanation content supplementing module and an explanation content broadcasting module. According to the invention, through converting the voice of the tourist into the text and analyzing the demand of the tourist, the matched standard explanation content can be identified in time, the correlation and pertinence of the explanation content are improved by analyzing the information coverage rate of the explanation and searching for the supplementary content, and the naturalness of voice output is enhanced by adjusting the speech speed and tone of the voice, so that the user experience is improved. And the interactive experience of tourists and the convenience of information acquisition are enriched.