375 results about "Reference tone" patented technology

The invention relates to the technical field of voice processing, can be applied to business scenes of financial science and technology, medical health and the like, and discloses a voice enhancement method, device and equipment based on noise perception and a medium. Environment feature information is extracted and input into an audio enhancement model to generate an enhanced audio signal; obtaining a reference audio sample, extracting a personalized feature vector, and carrying out personalized processing on the enhanced audio signal; and collecting playing feedback data, determining a playing time domain adjustment parameter and a playing frequency domain adjustment parameter, adjusting the personalized enhanced audio signal, and generating an optimized audio signal. According to the method, dynamic adjustment is realized in combination with the feedback parameters in the playing process by fusing the environmental perception information and the personalized speaker characteristics, clear and natural optimized audio output with personalized styles can be generated in a complex environment, and the voice interaction quality and adaptability are improved.

The invention belongs to the field of artificial intelligence, medical treatment and finance, and relates to a speech synthesis method, which comprises the following steps: performing phoneme conversion and context embedding on a text data set, obtaining a phoneme sequence and a text context vector, extracting reference speech features of a reference audio sample, inputting a phoneme-level potential diffusion model, and generating phoneme-level prior potential features; inputting the phoneme-level prior potential features into a phoneme-level variational self-encoding model to obtain reconstructed speech features; extracting a target voice feature of the target audio sample; performing alignment and up-sampling on the target voice feature and a phoneme sequence to obtain a target audio frame-level feature; and training based on the reconstructed speech features and the target audio frame-level features to obtain a conditional control speech synthesis model, and synthesizing the to-be-converted text speech. The invention further provides a speech synthesis device, computer equipment and a storage medium. In addition, the invention also relates to a block chain technology, and the to-be-converted text can be stored in a block chain. The speaker fidelity and the text understandability are improved.

The invention discloses a speech cloning method and device based on emotion enhancement, and a related medium. The method comprises the following steps: respectively obtaining a reference audio and a prediction text corresponding to the reference audio; respectively preprocessing the reference audio and the prediction text to generate standardized audio data for feature extraction; inputting the standardized audio data into an emotion enhancement module so as to generate acoustic features matched with a target voice style through multi-round feature fusion and an autoregression generation mechanism; and inputting the acoustic features into a speech synthesis module for decoding processing, and outputting predicted speech. According to the invention, by introducing the emotion enhancement module, accurate modeling of the emotion style of the speaker in the acoustic feature generation process is realized, so that the emotion simulation degree and the personality restoration capability of the synthesized speech are significantly improved.

The invention provides a voice synthesis scheme and device, electronic equipment, a storage medium and a program product, and relates to the technical field of voice processing, and the method comprises the steps: inputting a text content into an LLM semantic understanding module, and obtaining a deep semantic feature and multi-modal context semantic information corresponding to the text content; transmitting the deep semantic features to a neural codec, and outputting compressed acoustic features; inputting the compressed acoustic features into an acoustic modeling module, and outputting high-precision acoustic features; inputting the high-precision acoustic features and the multi-mode context semantic information into an emotion and rhythm control module, and inputting voice parameters with emotion and rhythm marks; and inputting the voice parameters with the emotion and rhythm marks and the reference audio into a tone migration module to obtain a synthetic voice of a tone corresponding to the reference audio.

The invention relates to an adaptive emotion-driven timbre clone character-to-speech method and device. Segmenting the target text according to sentences, extracting semantic features, and splitting each sentence into a target phoneme sequence; the method comprises the following steps: splitting a reference text into reference phoneme sequences, splicing the reference phoneme sequences with each target phoneme sequence to obtain a combined phoneme sequence, processing a reference audio to obtain a reference voice feature and a frequency spectrum, processing based on a semantic feature corresponding to a sentence, the combined phoneme sequence and the reference voice feature to obtain a corresponding voice feature, and extracting a corresponding emotional feature from each sentence semantic feature, and processing based on the speech feature, the emotional feature and the target phoneme sequence corresponding to each sentence and the frequency spectrum to obtain a speech signal corresponding to the target text. And extracting emotional features from the semantic features of the target text, enabling the emotional colors of the voice signals to fit the text, and automatically adjusting the emotional colors of the voice signals based on the mapping relationship between the preset output features and the emotional features.

The invention discloses a fault server positioning method and device, a storage medium and electronic equipment, and relates to the technical field of computers, and the method comprises the steps: detecting the operation state of a server in a server room, responding to a target server in a fault operation state, and controlling the target server to output an audio signal meeting a target audio parameter. The characteristic that an audio collector can collect audio signals is utilized, and reference audio signals output by all servers and collected by the audio collector are received. The audio signal meeting the target audio parameter is screened out from the reference audio signal, so that the audio collector which collects the audio signal output by the target server is determined, and the target server is positioned through the position of the audio collector. The technical problem that the positioning efficiency of the fault server is low is solved, and the technical effect of improving the positioning efficiency of the fault server is achieved.

The invention relates to neural network speech generation. Apparatus, systems, and techniques for generating speech audio are disclosed. In at least one embodiment, a processor generates a first speech audio based at least in part on a second speech audio and a reference audio using one or more neural networks.

An audio processing system is disclosed for comparing a query audio sample with a database of multiple reference audio samples using an external normalization. The system includes at least one processor and memory storing instructions that, when executed by the processor, cause the system to determine a bias term of the external normalization based on a spectro temporal pattern of the query audio sample. The system further compares the query audio sample with each of the reference audio samples to generate a similarity score for each comparison. The system combines the bias term with each of the similarity scores to produce normalized similarity scores. The normalized similarity scores are then compared with a threshold to generate a result of comparison, which is subsequently outputted.

The invention relates to the technical field of end-side model deployment, in particular to an end-side voice model deployment method and device, equipment and a storage medium. Comprising the steps of recording reference audio at an end side; extracting a reference semantic token and an embedded vector of the reference audio; obtaining a training text set, integrating and inputting the training text code, the embedded vector and the reference semantic token corresponding to each training text in the training text set into a preset language model, and outputting a comprehensive token sequence corresponding to the training text; performing Mel spectrum conversion on each comprehensive token sequence to obtain a first spectrum representation corresponding to the comprehensive token sequence; generating an audio signal corresponding to the training text according to each first spectrum representation, and integrating to obtain a training data set; inputting the training data set into a to-be-trained model, and training to obtain a lightweight voice model which only retains timbre modeling parameters; and updating the lightweight voice model to the end side to complete end side deployment. According to the invention, the voice synthesis model deployment of the end-side equipment can be realized.

The embodiment of the invention relates to an information interaction method and device, equipment and a storage medium. The method provided by the invention comprises the following steps: presenting an interactive interface with a virtual object; in response to the fact that a first message received in the interactive interface meets a preset condition, music content is provided, songs corresponding to the music content are determined on the basis of the first message, and at least one audio attribute of the music content is generated on the basis of reference audio content associated with the virtual object. In this way, the embodiment of the invention can support the virtual object to provide the music content corresponding to the first message when receiving the first message meeting the preset condition, thereby improving the interaction effect. Besides, in the embodiment of the invention, at least one audio attribute of the music content generated based on the reference audio content associated with the virtual object is provided, so that the interaction quality is improved, and the interaction capability is enriched.

The invention provides a target audio generation method and device, electronic equipment, a computer readable storage medium and a computer program product, and relates to the field of computers, in particular to the technical field of artificial intelligence, deep learning and natural language processing. According to the implementation scheme, reference audio, a reference text and a target text are obtained; the method comprises the following steps: performing feature discretization on a reference audio to obtain a reference audio acoustic marking sequence and a reference audio semantic marking sequence; performing feature discretization on the target text and the reference text to obtain respective corresponding text semantic tag sequences; based on the text semantic tag sequence and the reference audio semantic tag sequence, obtaining a target audio semantic tag sequence through a first generative language model; based on the reference audio semantic marking sequence, the target audio semantic marking sequence and the reference audio acoustic marking sequence, obtaining a target audio acoustic marking sequence through a second generative language model; and decoding based on the target audio acoustic mark sequence to obtain a target audio.

This disclosure provides methods, devices, and systems for audio signal processing. The present implementations more specifically relate to speech enhancement techniques for separating microphone signals into speech, echo, and noise signals. In some aspects, a speech enhancement system may include a delay estimator and an acoustic echo and noise (AEN) decoupling filter. The delay estimator receives a microphone signal via a microphone and a far-end audio signal for output via a speaker and estimates a reference audio signal based on a delay between the microphone signal and the far-end audio signal. In some aspects, the AEN decoupling filter may determine a speech mask, an echo mask, and a noise mask based on the microphone signal and the reference audio signal and may suppress an echo component and a noise component of the microphone signal based on the determined set of masks.

The invention relates to a voice synthesis device and a training method thereof, electronic equipment and a storage medium, and the method comprises a voiceprint coding module which receives an input reference audio and extracts voiceprint features of the reference audio; the audio code generation module is used for determining a target expert network from a plurality of expert networks to execute a voice synthesis operation according to the input text to be synthesized and the voiceprint features so as to obtain a generated audio code sequence; wherein the plurality of expert networks correspond to different application scenes; and the audio decoding module is used for converting the audio coding sequence output by the audio coding generation module into an audio signal. According to the embodiment of the invention, through the dynamically selected target expert network, the voice synthesis of the application scene suitable for the input to-be-synthesized text and the voiceprint feature can be executed, and the scene adaptability of the synthesized voice is remarkably improved.

The invention provides a mediator timbre cloning method and system, electronic equipment and a storage medium. The method comprises the following steps: selecting a tone of a mediator in response to a selection instruction input by a user; obtaining an input text; predicting a target audio feature vector of the input text by using an autoregression model; a target clustering center matched with the target audio feature vector is searched in an audio dictionary, the audio dictionary comprises a plurality of clustering clusters, and each clustering cluster comprises a plurality of audio feature vectors; and inputting the input text phonemes of the input text, the frequency characteristics of a reference audio and the target clustering center into the trained acoustic model to obtain a target output audio, the reference audio being a mediator audio corresponding to the tone of the mediator. According to the method, the mediation efficiency can be improved, the service consistency is ensured, the user experience is improved, and the method has better practicability and market competitiveness.

This disclosure provides methods, devices, and systems for audio signal processing. The present implementations more specifically relate to speech enhancement techniques that can adapt to varying signal-to-noise ratio (SNR) conditions. In some aspects, a speech enhancement system may include a low SNR detector and a spatial filter. The spatial filter receives a multi-channel audio signal via a microphone array and produces an enhanced audio signal based on a beamforming filter. The low SNR detector tracks an SNR of a reference audio signal of the multi-channel audio signal. In some implementations, the spatial filter may substitute at least part of the reference audio signal for an auxiliary audio signal, received from an auxiliary microphone separate from the microphone array, when the SNR falls below a wideband SNR threshold. In some other implementations, the spatial filter may refrain from updating the beamforming filter when the SNR falls below a narrowband SNR threshold.

A method of audio generation based on a large language model is disclosed, which involves the fields of artificial intelligence such as large language models, natural language processing, deep learning, and audio generation. The method of audio generation based on a large language model comprises: acquiring a text to be processed; parsing the text to be processed using the large language model to obtain role information and emotional information corresponding to the text to be processed; obtaining a target reference text and a target reference audio according to the role information and the emotional information; and generating a target audio corresponding to the text to be processed according to the text to be processed, the target reference text, and the target reference audio.

A method for training a non-autoregressive TTS model includes receiving training data that includes a reference audio signal and a corresponding input text sequence. The method also includes encoding the reference audio signal into a variational embedding that disentangles the style / prosody information from the reference audio signal and encoding the input text sequence into an encoded text sequence. The method also includes predicting a phoneme duration for each phoneme in the input text sequence and determining a phoneme duration loss based on the predicted phoneme durations and a reference phoneme duration. The method also includes generating one or more predicted mel-frequency spectrogram sequences for the input text sequence and determining a final spectrogram loss based on the predicted mel-frequency spectrogram sequences and a reference mel-frequency spectrogram sequence. The method also includes training the TTS model based on the final spectrogram loss and the corresponding phoneme duration loss.

The invention discloses an audio editing method and system, a terminal and a storage medium, and the method comprises the steps: determining an original text corresponding to an original audio, and obtaining a target text after part of the original text is modified; deleting an audio segment corresponding to the partial text in the original audio to obtain a reference audio; and based on the original text, the target text and the reference audio, generating a target audio corresponding to the target text. The invention provides an intelligent voice editing processing method which does not depend on a locally installed voice editing system and is high in usability. Moreover, the reference audio is generated after the audio segments corresponding to the partial text are deleted, and then the target audio is generated based on the reference audio, the original text and the target text after the partial text is modified, so that the calculation burden is reduced, and the voice editing effect is improved.

The invention discloses a voice wake-up data processing method and device and a computer readable storage medium, and the method comprises the steps: determining the interval time between voice interaction processes, and the voice interaction processes are the interaction processes of successfully waking up the device through voice; determining the confidence of the reference audio data in the interval time; if a difference value between the confidence coefficient and a preset confidence coefficient threshold value is within a preset range, obtaining target audio data of the voice interaction process of a preset number of times adjacent to the reference audio data; and taking the target audio data as sample data, wherein the sample data is used for voice wake-up training. According to the invention, real and effective wake-up audio data can be efficiently collected, the audio data is used for later training, the wake-up accuracy of the user is improved, and the user demand is met.

The embodiment of the invention provides a training method of a singing sound conversion system, a method for generating audio based on the singing sound conversion system and a related device, which are used for improving the similarity between the timbre of a synthesized singing sound and the timbre of a to-be-converted singing sound. The method provided by the embodiment of the invention comprises the following steps: acquiring a plurality of reference audios of a first target timbre, and inputting the plurality of reference audios into a timbre encoder to obtain a timbre encoding vector; inputting the phoneme posterior probability and the fundamental frequency of the to-be-converted singing sound into a text encoder to obtain a prior distribution parameter of the to-be-converted singing sound content; sampling according to the prior distribution parameter to obtain a text sampling value vector of the singing content to be converted; inputting the text sampling value vector and the timbre coding vector into a timbre perception attention mechanism module to determine a new timbre coding vector; and taking the new tone coding vector as new input added in the singing sound conversion system, calculating the reconstruction loss of the singing sound conversion system, and training the singing sound conversion system according to the reconstruction loss.

The invention relates to the technical field of voice signal processing, and discloses a method for realizing personalized singing synthesis model training through multi-modal voice driving, and the method comprises the following steps: obtaining multi-modal input data, including text, reference audio, speaker and emotional features; processing the data to obtain coding features; the inter-modal redundant information is evaluated and suppressed through redundant perception coding; compressing coding features by using an information bottleneck model, and retaining effective information; fusing the compressed features to generate personalized singing features; the input decoding module generates Mel spectrum features; and converting the Mel spectrum into an audio waveform through a vocoder, and outputting the audio waveform. And converting the Mel spectrum into an audio waveform through a vocoder, and outputting the audio waveform. According to the method, multi-modal data can be effectively fused, the individuation and emotion expression ability of the singing sound is improved, redundant information interference is reduced, the quality and efficiency of singing sound synthesis are improved, and finally, high-fidelity individualized singing sound is output.

The invention discloses a method and system for ensuring smooth communication in a large private car, and relates to the technical field of vehicle-mounted central control. Comprising the steps of collecting audio data in a vehicle and extracting reference sound data of central control equipment; performing denoising processing on the audio data to generate communication voice data; separating the alternating current voice data according to the significant sound source; calculating an output sound pressure level of each loudspeaker according to a microphone position corresponding to the separated alternating current voice data, and performing voice gain compensation; accurate selection of communication voice data among passengers is realized by removing reference voice data in the audio data; accurate matching between passenger sound and seats is realized through accurate extraction of a significant sound source, and the position of the sound source is updated in real time; through the cooperation of the output sound pressure level and the voice gain compensation, the audio output is optimized, echoes caused by sound amplification of a loudspeaker are avoided, and the communication experience of passengers is improved.

The present disclosure relates to systems, methods, and products for using machine-learning networks to generate trustworthy audio and face mesh. A system, serving as a digital avatar, generates a trust audio and trust face mesh corresponding to an input text. A method includes generating a set of trust embedding vectors based on a reference audio; generate a text embedding vector based on the input text; generate a conditioned vector based on the set of trust embedding vectors and the text embedding vector; synthesize an audio representation based on the conditioned vector; generate the trust audio based on the synthesized audio representation; obtain a speech feature representation based on the trust audio; obtain an abstract feature vector based on the speech feature representation; and generate positions of vertices based on the abstract feature vector, the positions of vertices being used for generating the trust face mesh.

The invention relates to the technical field of digital people, and provides a method and equipment for generating a digital people video, which can be applied to a digital marketing scene. According to the method, the generation of the digital human video is divided into an audio generation process and a video generation process, when the audio is generated, a target text, an emotion parameter and a reference audio are taken as input, and the target audio of the target text is broadcasted by the sound of a reference person in the reference video and a set emotion, so that sound cloning is realized, the degree of violation of the generated digital human video is reduced, and the user experience is improved. And the text features are extracted from two dimensions of characters and pronunciation, so that the pronunciation accuracy during sound cloning is ensured. When the video is generated, the cloned target audio and the cloned reference video are used as input, and the fidelity of the digital human video is improved through expression and mouth shape matching, so that a reference person can quickly and efficiently generate vivid digital human videos with different broadcast contents only by shooting one reference video without shooting one by one, thereby reducing the manufacturing cost of the digital human video, and improving the user experience. And the video production efficiency is improved.

The invention provides a vehicle whistling control method and system, and the method comprises the steps: obtaining the environment noise data of a vehicle, and determining a reference whistling parameter according to the environment noise data; in response to a whistling instruction sent by a driver, acquiring the duration of the whistling instruction, and marking the whistling type of the whistling instruction according to the duration; under the condition that the whistling instruction is short whistling, the whistling loudness and the whistling tone are set according to a first rule, and the first rule comprises at least one of reference whistling parameters, the vehicle speed, the vehicle position and a vehicle front pedestrian sign; under the condition that the whistling instruction is long whistling, the whistling loudness is set to be first loudness, the whistling tone is set to be first tone, the first loudness is larger than the reference loudness, and the first tone is higher than the reference tone. According to the method, the whistling loudness and tone can be adjusted according to the vehicle whistling duration, the vehicle speed, whether the vehicle is in the no-whistling area or not and whether pedestrians exist in front of the vehicle or not, and then noise pollution is reduced.

Embodiments are disclosed for generating a gesture reenactment video sequence corresponding to a target audio sequence using a trained network based on a video motion graph generated from a reference speech video. In particular, in one or more embodiments, the disclosed systems and methods comprise receiving a first input including a reference speech video and generating a video motion graph representing the reference speech video, where each node is associated with a frame of the reference video sequence and reference audio features of the reference audio sequence. The disclosed systems and methods further comprise receiving a second input including a target audio sequence, generating target audio features, identifying a node path through the video motion graph based on the target audio features and the reference audio features, and generating an output media sequence based on the identified node path through the video motion graph paired with the target audio sequence.

The invention relates to an audio data evaluation method and device, equipment, a storage medium and a program product. The method comprises the following steps: acquiring an original audio, wherein the original audio comprises a reference audio and a to-be-evaluated target audio; human ear perception processing is carried out on the original audio to obtain human ear perception domain audio, and the human ear perception domain audio comprises a reference human ear perception domain audio corresponding to the reference audio and a target human ear perception domain audio corresponding to the target audio; performing evaluation processing on the original audio and the human ear perception domain audio by adopting an evaluation network; and determining difference information between the reference audio and the target audio according to an evaluation processing result, and determining an evaluation result of the target audio according to the difference information. By adopting the method, the accuracy of audio evaluation can be improved, so that the evaluation result better fits the subjective characteristics of human ears.

The invention relates to the field of interaction, in particular to a medical information interaction terminal and an interaction method based on AI analysis, and aims to eliminate emotion recognition influence caused by personal vocal characteristics by performing target audio feature extraction and character extraction on request voice information and performing differentiated calculation on target audio features and reference audio features of the request voice information. And then an AI model is used to carry out emotion analysis on the reference audio features, and an accurate emotion recognition result is obtained in combination with an emotion entity extracted from the text. And then determining which interaction mode is adopted based on an emotion analysis result and the extracted key entity. When the emotion of the patient or the family member is stable and the intention is inquiry, a question and answer template or a language large model is adopted to respond. And when the emotion of the patient or the family member is unstable or the patient or the family member intends to request, forwarding the request voice information to the corresponding medical staff. Therefore, response of medical staff can be reduced, and doctor-patient contradiction excitation caused by the adoption of a single question-answering robot can be avoided.

The invention provides an audio processing method and device, and the method comprises the steps: collecting to-be-detected audio data related to a target audio based on to-be-detected audio equipment, and determining reference audio data related to the target audio; determining a to-be-tested audio index corresponding to the to-be-tested audio data in an audio consistency dimension, determining a reference audio index corresponding to the reference audio data, and generating an index difference score based on the to-be-tested audio index and the reference audio index; and under the condition that the to-be-tested audio data is determined to be abnormal based on the index difference score, generating difference processing prompt information corresponding to the to-be-tested audio data based on a difference mode rule and the index difference score. The audio processing method can be widely applied to the field of digital music in the digital creative industry.

A method includes receiving target audio data captured by a first audio input device, the target audio data comprising a target audio signal and a first version of an interfering audio signal, and receiving reference audio data captured by a second audio input device different from the first audio input device, the reference audio data comprising a second version of the interfering audio signal. The method also includes processing, using a trained neural network, the target audio data and the reference audio data to generate enhanced audio data, the neural network attenuating the interfering audio signal in the enhanced audio data.