710 results about "Voice transformation" patented technology

The present disclosure relates to systems and methods for enhancing the interaction between users and automated agents, such as digital assistants, by employing Large Language Models (LLMs) to infer the intent of spoken language. The invention involves continuously monitoring ambient audio, converting speech to text, and utilizing LLMs to determine whether spoken language is intended for the automated agent. A structured prompt, including the converted text and specific instructions, is sent to the LLM, which is fine-tuned to process domain-specific prompts. The LLM provides a structured output in a standardized format, indicating the user's intent. The system may involve multiple prompts to perform separate tasks, such as identifying intent and generating additional context-specific data. This approach facilitates a more natural and intuitive user experience by eliminating the need for wake words and allowing seamless conversational interaction with virtual assistants across various platforms and devices.

A neural end-to-end system is provided for the face and voice preserving translation of videos. The system is a pipeline of multiple models that produces a video of the original speaker speaking in the target language with modified lip movement to match the target speech, while preserving emphases and prosody of the original speech, and voice characteristics of the original speaker. The pipeline starts with automatic speech recognition including emphasis detection, followed by the translation model. The translated text is then synthesized by a Text-to-Speech model that recreates the original emphases in the target sentence. The resulting synthetic speech is then converted back to the original speakers' voice using a voice conversion model. Finally, to synchronize the lips of the speaker with the translated audio, a generative model generates frames of adapted lip movements which are combined with the audio to produce the final output. The disclosure further describes several use-cases and configurations that apply these techniques to video conferencing, dubbing, low-bandwidth transmission, speech enhancement and assistive technology for the hearing impaired.

The invention relates to the technical field of multi-modal large language models, in particular to a government affair service content navigation method and system based on a large language model, and the method comprises the following steps: receiving multi-modal information input by a user through texts, voices or pictures; the voice is converted into a text, and character information in the picture is analyzed by using an OCR (Optical Character Recognition) technology; fusing multi-modal data, and inputting the fused multi-modal data into a large language model for semantic understanding and context association analysis; matching items are retrieved in combination with a local government affair knowledge base, and an initial recommendation list is generated; the recommendation result is displayed through the intelligent assistant, and interaction optimization options are provided; the method has the beneficial effects that the current government affair service content navigation mode is optimized through the multi-modal capability, the retrieval enhancement capability and the content generation capability of the large language model, so that the navigation is more modal and more intelligent, and meanwhile, the privacy and authority of data reply are ensured.

The invention relates to the technical field of voice recognition, in particular to a multi-channel voice conversion and synchronous transmission system applied to simultaneous interpretation. Multi-language voice signals are collected through a multi-microphone array, background noise is dynamically eliminated by adopting a noise suppression technology combining spectral subtraction and deep learning, and a dynamic information source verification symbol is generated to ensure data synchronization integrity; a blind source separation technology is combined with time-frequency analysis and time delay estimation to realize multi-language signal separation and synchronization, and a time sequence is dynamically adjusted through voice activity detection; an end-to-end ASR-NMT-TTS model is constructed to realize voice real-time translation and synthesis, and low-delay transmission is carried out based on a 5G network; the real-time monitoring module is adopted to dynamically adjust the output delay and the signal-to-noise ratio, and the translation delay and the synchronization precision are optimized in combination with user feedback. According to the method, dynamic noise reduction, multi-source synchronization and a self-adaptive feedback mechanism are integrated, the problems of distortion and delay of multi-language simultaneous transmission in a complex noise environment are solved, and the obvious technical synergistic effect and practicability are achieved.

The present disclosure provides a pitch-based speech conversion model training method and a speech conversion system, wherein an audio feature code is output by a priori encoder, and a pitch feature is extracted by a pitch extraction module. A linear spectrum corresponding to the reference speech is input into the posteriori encoder to obtain an audio latent variable. In addition, the audio feature code, a speech concatenation feature obtained by concatenation of the audio feature code and the pitch feature, and the audio latent variable are input into a temporal alignment module to obtain a converted speech code, and the converted speech code is decoded by a decoder to obtain a converted speech. The training loss of the converted speech is then calculated to determine the degree of convergence of the speech conversion model.

The invention relates to the technical field of electronic medical record generation, and discloses an electronic medical record automatic generation method based on voice recognition, which comprises the following steps: S1, initializing voice input, distributing a unique voice acquisition identifier for a medical session, and completing identifier generation, input, storage, association and identity verification; s2, voice information intelligent recognition: converting voice into a text by using a voice recognition engine, and ensuring semantic consistency through a context verification unit and a semantic analysis unit; and S3, performing multi-speaker processing based on intelligent interference detection and resolution, positioning an interference time period and an interference source through an interference detection unit, and realizing time period distribution and priority ranking of multi-speaker voices by using a voice segmentation protocol and a linear weighting model. And finally, extracting related information from the text generated by voice conversion, and filling the related information into a medical record template of a hospital. The method improves the efficiency and accuracy of electronic medical record generation, solves the problems of multi-speaker interference and semantic logic, and is suitable for medical informatization scenes.

The invention provides a control method, which is applied to an artificial intelligence experiment system, at least comprises a robot hardware platform, a controller, a wheeled robot chassis and an industrial mechanical arm, and the method comprises the following steps: starting an operation system and executing hardware self-inspection; loading a large language model service and a traditional AI model, wherein the traditional AI model comprises a speech recognition model, a target detection model and a semantic analysis model; a natural language instruction of a user is received, voice is converted into a structured text through a voice recognition model, an instruction intention is analyzed through a large language model and a semantic analysis model, robot control parameters are generated, and a multi-modal interaction control signal is obtained; based on the multi-mode interaction control signal, a camera is called to collect image data, the position and category of a target object to be grabbed by the robot are recognized through a target detection model, the moving path of the robot and the grabbing track of the mechanical arm are calculated in combination with laser radar data, and the wheeled robot chassis and the industrial mechanical arm are controlled to execute coordinated actions.

The invention relates to the technical field of digital people, and discloses a real-time high-fidelity voice-driven digital people system which integrates six modules of voice acquisition, voice recognition, semantic analysis, sentiment analysis, animation generation and rendering generation. According to the invention, through contrastive analysis with the input voice, determination of accent and language styles in the early stage, conversion of the voice into the text, and word segmentation and part-of-speech tagging by using advanced machine learning and deep learning technologies, intentions and demands of a user are deeply understood, and all modules of the digital human system are driven to cooperate; a series of signal processing and feature extraction methods of the sentiment analysis module are applied, and a machine learning or deep learning model is combined to accurately recognize the sentiment polarity in the user voice, so that the system can make corresponding feedback according to a sentiment analysis result, for example, in an intelligent customer service scene, a reply strategy is adjusted according to the user sentiment, and the user experience is improved. And more personalized and humanized services are provided, and emotion resonance of interaction between the user and the digital human is enhanced.

The invention discloses a speech enhancement method and system based on conditional stream matching and a vocoder, and the method comprises the following steps: S1, constructing a Mel spectrum extraction module which is used for converting an input noisy speech into a noisy Mel spectrum; s2, constructing a condition flow matching noise reduction module which is used for processing the noisy Mel spectrum obtained in the step S1 and outputting an enhanced Mel spectrum; and S3, constructing a neural network vocoder module which is used for restoring the enhanced Mel spectrum obtained in the step S2 into a time domain voice waveform so as to obtain an enhanced voice signal. The speech enhancement method combining conditional stream matching and the vocoder is proposed for the first time, the conditional stream matching method is innovatively introduced into the Mel-frequency spectrum domain, an end-to-end speech enhancement system is constructed, and a complete processing flow from noise speech input to high-quality speech signal output is realized.

The invention provides a multi-scene self-adaptive guide robot interaction method and system in the technical field of guide robots. The method comprises the steps that S1, a guide robot obtains interaction voice and converts the interaction voice into an interaction text; s2, analyzing the interaction text to generate a reply text, converting the reply text into a reply voice, and playing the reply voice for interaction; s3, managing the dialogue content in the interaction process; s4, the guide robot moves based on the guide path in the interaction process, and automatically plays explanation voice when arriving at the guide point; step S5, when the interaction text carries the target point, planning an advancing path to the target point through a bidirectional progressive optimal search algorithm; and S6, displacement is carried out based on the advancing path, and operation information of the robot is displayed in real time. The method has the advantages that the efficiency and accuracy of path planning, the flexibility of voice interaction and the expansibility of the system are greatly improved, and then the user experience of guide robot interaction is greatly improved.

The invention discloses a translation system based on artificial intelligence, intelligent glasses for translation and a translation method. The translation system comprises first intelligent glasses, a first intelligent mobile terminal and a cloud server, and a generative artificial intelligence large language model is configured in the cloud server. The first intelligent glasses pick up a first voice of a first language from a preset direction and send the first voice to the first intelligent mobile terminal. The first intelligent mobile terminal converts the first voice into a first text and sends the first text to the cloud server; and the cloud server translates the first text through the model according to the translation prompt to obtain a second text of a second language and sends the second text to the first intelligent mobile terminal. And the first intelligent mobile terminal converts the second text into a second voice and sends the second voice to the first intelligent glasses for playing. According to the invention, high-precision intelligent translation based on the intelligent glasses is realized, and the product viscosity can be increased.

Disclosed are a speaker embedding-based speaker adaptation method and system generated by using global style tokens and a prediction model. The speaker adaptation method performed by the speaker adaptation system, according to an embodiment, may comprise the steps of: generating a plurality of speaker embeddings representing the tone of a speaker from a speaker embedding by using a voice transformation model including a global style token mechanism; and predicting the final speaker embedding representing a new speaker through similarity comparison between a new speaker embedding predicted by using a prediction model for predicting a speaker embedding and the plurality of generated speaker embeddings.

The invention provides a voice evaluation method, a voice evaluation device, a non-transitory storage medium and a computer program product. The voice evaluation method comprises the following steps: extracting a first person voice with a first tone based on a to-be-evaluated voice; converting the voice of the first person into voice with a second tone; and obtaining an evaluation result of the voice of the first person based on the converted voice with the second tone and the standard voice of the one or more accent.

The invention provides an automatic video generation method and device, equipment and a medium, and relates to the technical field of video production. The method comprises the following steps: acquiring video related information input by a user, and generating copywriting content according to the video related information and a preset copywriting script material library; according to a preset sound material library, converting the copywriting content into audio by using a text-to-voice conversion technology to obtain an audio material; according to the audio material, a digital human-driven technology is utilized, and a preset image material library and a video sub-mirror material library are combined, so that a visual material synchronized with the copywriting content is obtained; according to a preset packaging element material library, obtaining a packaging material matched with the copywriting content; and performing video synthesis and packaging on the audio material, the visual material and the packaging material to obtain a final video. According to the method, related materials can be generated through accurate matching and intelligent recommendation according to user requirements and recent popularity, a mixed cutting type and generation type material fusion arrangement mechanism is combined, one-key film formation is achieved, and the video production efficiency is improved.

The invention discloses a lip language-to-voice conversion method and device, a computer storage medium and a terminal, and aims to solve the problems that a lip language-to-voice conversion technology cannot be deployed on terminal equipment and voice quality cannot meet application requirements. The method is combined with a neural vocoder which reduces calculation complexity and resource requirements, system configuration requirements are reduced while system parameters are reduced, a design basis is provided for deploying a lip language-to-speech method on terminal equipment, and context-related visual feature sequences and user audio embedding vectors are fused, so that the user audio-to-speech conversion efficiency is improved, and the user audio-to-speech conversion efficiency is improved. A Mel spectrum acoustic feature sequence used for being converted into a voice waveform is obtained, and a user audio vector fused with the Mel spectrum acoustic feature sequence is obtained, so that the output voice waveform is more consistent with the real voice of a user; according to the embodiment of the invention, technical support is provided for deploying and applying the lip language-to-speech method meeting the speech quality requirement on the terminal equipment.

The invention discloses a voiceprint characterization method and system for resisting voice conversion and voice synthesis based on deep learning. Relates to the field of speech recognition and biological feature security. Comprising the steps of 1, acquiring an input voice sample signal in an automatic speaker verification system, and extracting an FBANK feature of the voice sample signal; 2, performing depth feature extraction on the FBANK features by using a convolutional neural network (CNN) to obtain a depth feature vector; 3, processing the depth feature vector by using a recurrent neural network (RNN), and generating a forged identity vector of the voice; and 4, classifying the counterfeit identity vectors by using a linear discriminant analysis (LDA) module, and outputting a judgment result. According to the invention, the accuracy of counterfeit detection can be effectively improved in both clean and noise environments.

Voice anonymization systems and methods are provided. Voice anonymization is done on the speaker's computing device and can prevent voice theft. The voice anonymization systems and methods are lightweight and run efficiently in real time on a computing device, allowing for speaker anonymity without diminishing system performance during a teleconference or VoIP meeting. The anonymization system outputs a transformed speaker voice. The anonymization system can also generate a voice embedding that can be used to reconstruct the original speaker voice. The voice embedding can be encrypted and transmitted to another device. Sometimes, the voice embedding is not transmitted and the listener receives the anonymized voice. Systems and methods are provided for the detection of voice transformations in received audio. Thus, a listener can be informed whether the speaker voice output from the listener's computing device is the original speaker's voice or a transformed version of the original speaker voice.

Disclosed is an dynamic speech recognition system and associated methods for improving speech recognition accuracy by biasing, tuning, and / or otherwise adjusting a speech recognition model to account or compensate for different speech characteristics of individual speakers and / or different environmental factors that have different effects on the characteristics of the audio recorded from each speaker. The system receives an audio stream, identifies a speaker in the audio stream, selects a first vector that is generated by a first machine learning model and that encodes speech characteristics of the speaker, selects a second vector that is generated by a second machine learning model and that encodes audio characteristics that affect a capture of the audio stream, and adjusts a third machine learning model based on the first vector and the second vector. The system uses the third machine learning model after it is adjusted to convert speech into text.

The invention relates to an information processing method and device for man-machine practice and a medium, and belongs to the technical field of man-machine interaction, and the method comprises the following steps: obtaining interaction voice of a user for question reply in real time, and converting the interaction voice into text information with a timestamp; performing preprocessing and key semantic information extraction on the text information, and performing emotion recognition, mute recognition and speech speed recognition; evaluating the practice process in real time according to the quality inspection rule, and judging whether interaction needs to be interrupted or not according to an evaluation result; performing intention recognition based on the extracted key semantic information, mapping the recognized intention to a corresponding node of a knowledge graph, iteratively updating an associated question-chasing path according to the knowledge graph, generating a response question in combination with a scene label and a process rule, and sending the response question to a user; and obtaining the interactive voice of the user for the response question, and carrying out the next round of man-machine practice interaction. Compared with the prior art, the method has the advantages of being capable of achieving personalized practice and the like.

The invention belongs to the technical field of text-to-speech conversion, and particularly relates to a solution for TTS (text-to-speech) in a high-concurrency scene through tone cloning. According to the method, the text can be generated and returned to the client by segmenting and parallelizing the text and reducing the delay of the first segment, compared with a traditional serial whole segment synthesis mode, the waiting time of a user is greatly shortened, the audio is returned after being generated, the waiting time of the user can be effectively shortened under high concurrency, the network bandwidth and the delay pressure are reduced, and the user experience is improved. The real-time performance is guaranteed, reconnection disasters are avoided, underlying model services can be fully utilized, the influence of the serial processing characteristic of a CosyVoice2 model is avoided, a client can immediately start to play a first segment of audio, even if subsequent segments are still synthesized, the continuous playing experience of previous content is not influenced, and when the concurrency is increased, the continuous playing experience of the previous content is not influenced. The model gateway can dynamically disperse requests to a plurality of GPU instances, and long queue waiting after a single-card video memory is fully occupied is avoided.

The invention provides a game assisting method and device.The method comprises the steps that in the process that a user plays a game, voice of the user is recognized, the voice is converted into a text, and a game image corresponding to the text is matched and recorded as a target game image; retrieving a preset game strategy knowledge base by utilizing the target game image to obtain target game strategy information; the target game image, the text and the target game strategy information are submitted to the large-scale multi-mode model for analysis, the instructive text is generated, the instructive text is synthesized into the voice and played, and a game assisting scheme which does not affect the game experience of the user in the game playing process of the user and efficiently and effectively guides the user is provided.

Systems and methods for orchestrating interaction with an artificial intelligence (AI) application in a contact center environment receive, via an AI agent, a voice message from a user; convert the message from voice to text; generate an initial computational inference process based on the text message; determine whether or not all information required to execute the initial computational inference process is available to the processor; when a determination is made that all information required is available: execute the initial computational inference process; generate a text reply based on the initial computational inference process; convert the text reply to a voice reply; and send the voice reply to the user via the AI agent; when a determination is made that information is unavailable: generate a text query requesting the information; convert the text query to a voice query; and send the voice query to the user via the AI agent.

The invention discloses a singing voice conversion method based on VITS. The method comprises the following steps of: 1, respectively extracting a bottleneck feature, a fundamental frequency, a singing style feature and a linear spectrogram from the singing of a source singer by utilizing a Whisper superficial layer encoder, an automatic singing transcription model AST and an AutoVC-based pitch-energy encoder, and taking the bottleneck feature, the fundamental frequency, the singing style feature and the linear spectrogram as the input of a VITS-based singing voice conversion model; reconstructing the singing voice by using a VITS-based singing voice conversion model in combination with the voiceprint characteristics of the singer to obtain converted singing voice of the target singer; and 2, performing pitch adjustment according to the pitch difference between the source singer and the target singer through a pitch shifter, and generating a target singing sound with the characteristics of the target singer by taking the bottleneck characteristics, the F0 and the singing style characteristics as the input of a voice conversion model. The method has the characteristics of multi-level refined acoustic feature decoupling, explicit singing style migration, high-fidelity waveform generation and dynamic adaptive pitch conversion.

Techniques for video streaming are disclosed. Audio input related to products for sale is received from a user within an embedded interface. The embedded interface includes an artificial intelligence virtual assistant represented by a synthetic human host. The audio input is captured by a natural language processing engine, creating a data segment which is sent to a large language model (LLM). The data segment can be divided into one or more subsegments. Each subsegment can be processed independently by the LLM to generate a response based on product information stored in the LLM database. The LLM responses are converted by a text-to-speech (TTS) converter into an audio response. The audio responses are used to produce a video segment including the synthetic host performing the audio stream. The capturing and processing of user audio continues to allow a human-like dialogue with the synthetic human host.

The invention discloses a speaker segmentation method based on multi-scale feature fusion and voiceprint perception density clustering, and relates to the technical field of speaker segmentation. The method comprises the following steps: extracting at an extraction point of a target voice to obtain an extraction point voiceprint; selecting a reference point, and collecting a reference speaker and a reference voiceprint; if the voiceprint of the extraction point is not consistent with the reference voiceprint, judging whether multi-person talking overlapping exists or not according to the voiceprint perception density, and if the multi-person talking overlapping does not exist, judging whether the extraction point is used as a voice conversion point or not according to the health information and the environment information of the reference speaker; if yes, judging whether the extraction point is used as a voice conversion point or not according to the voice content of the extraction point and the background noise probability; and classifying the voice segments according to the voice conversion points to realize speaker segmentation. According to the invention, the resource utilization rate of speaker segmentation based on multi-scale feature fusion and voiceprint perception density clustering is improved.

The invention discloses a training method and system and an application method and system of a voice conversion model based on data distillation. The method comprises the following steps: establishing a voice training data set, and training an original voice conversion model by using the voice training data set to obtain an initial voice conversion model; establishing a voice evaluation data set, and inputting the voice evaluation data set into the initial voice conversion model to obtain conversion voice data of each corresponding speaker; calculating a voice conversion score of each speaker; judging whether the voice conversion score of each speaker reaches a preset condition or not; if not, removing the voice data corresponding to the speaker from the voice training data set, and taking the remaining voice data as a voice database; repeatedly training; and taking the initial voice conversion model and the voice database as a trained voice conversion model and a final voice database respectively. The voice conversion model and the voice database obtained through training of the method can support conversion of various timbres, and the voice conversion effect is good.

The invention belongs to the field of artificial intelligence, and particularly relates to a medical pre-inquiry real-time interaction method and system, a terminal and a medium. Screening medical knowledge contexts similar to the consultation text from a professional medical knowledge base, and recording the medical knowledge contexts as related texts; based on a pre-constructed prompt template, constructing a current consultation prompt through the consultation text and the related text; inputting the current consultation prompt into the large language model, and outputting consultation reply content; performing voice synthesis on the consultation reply content to generate consultation reply voice data, storing the consultation reply voice data in the buffer area, extracting reply voice data from the main buffer area for voice output, and transferring data in the auxiliary buffer area to the main buffer area when the data in the main buffer area is less than a threshold value; speech synthesis and speech output are asynchronously processed. According to the invention, the professionality and accuracy of medical pre-inquiry are improved, the playing lagging caused by waiting for synthesis is effectively avoided, and the interaction experience effect is improved.

The invention discloses an intelligent teaching management system based on IOP Internet of Things, and relates to the technical field of teaching management, the intelligent teaching management system comprises an object identification terminal, a sitting posture judgment terminal, a teaching management terminal and a voice amplification level module, a three-dimensional numerical value of a student is obtained through a three-dimensional scanner, whether the numerical value of the student corresponding to a serial number is abnormal or not is detected in real time, and the teaching management terminal is started. Through real-time tracking of sitting postures of students and measurement of basic data of the sitting postures, whether the sitting postures of the students are standard or not is judged, the lecture attending states of the students during teaching management are monitored in real time, character conversion is performed on voice answered by the students through the voice conversion module, whether the answered content is correct or not is judged, and the teaching management effect is monitored in real time. According to the intelligent teaching management system based on the IOP internet of things, voice reminding can be carried out on students leaving the designated positions of the seats, the teaching management effect is improved, the students with abnormal sitting postures can be conveniently reminded and corrected, the intelligence of the teaching management system is enhanced, the class attending state of each student can be monitored in real time, and the teaching difficulty is reduced.

The invention discloses a sign language-voice conversion system, and belongs to the technical field of auxiliary communication and wearable computing. The system comprises a wearable myoelectricity acquisition module used for acquiring double-arm myoelectricity signals when a user executes sign language; the mobile terminal module is wirelessly connected with the acquisition module and is used for receiving and preprocessing the signal and uploading the signal; the cloud processing module is used for receiving the signal, converting the signal into text information through a sign language recognition model, and further calling a voice synthesis service to convert the text into voice data; and the wearable audio output module is used for receiving and playing the voice data. Through an innovative end-to-end hardware system architecture, natural, accurate and real-time translation and voice output of sign language gestures are realized, communication barriers between hearing-impaired people and healthy hearing people are effectively solved, and the system has the advantages of flexible deployment, user friendliness and privacy protection.

The invention discloses a voice conversion authentic identification method and system based on knowledge distillation alignment, and is applied to the technical field of voice authentic identification. The method comprises the following steps that a double-branch model used for voice authentic identification is constructed, pure voice serves as input of a teacher branch, noisy voice serves as input of a student branch, and the teacher branch and the student branch share a feature extraction network of the same structure; applying a speech enhancement technology at the front end of a student branch to generate an enhanced speech signal; deep features are extracted, and alignment of the deep features in a hidden space is restrained through a knowledge distillation loss function; carrying out dynamic weight fusion on the aligned deep features through a fusion weight; and training a classifier, performing joint optimization in combination with classification loss and knowledge distillation loss, and outputting a voice authentic identification result. According to the method, distribution alignment of pure and noise features is realized through knowledge distillation, forged traces in voice conversion are effectively recognized, and high detection precision is still kept in complex noise and unknown attack scenes.