485 results about "Reference tone" patented technology

The invention relates to the technical field of voice processing, can be applied to business scenes of financial science and technology, medical health and the like, and discloses a voice enhancement method, device and equipment based on noise perception and a medium. Environment feature information is extracted and input into an audio enhancement model to generate an enhanced audio signal; obtaining a reference audio sample, extracting a personalized feature vector, and carrying out personalized processing on the enhanced audio signal; and collecting playing feedback data, determining a playing time domain adjustment parameter and a playing frequency domain adjustment parameter, adjusting the personalized enhanced audio signal, and generating an optimized audio signal. According to the method, dynamic adjustment is realized in combination with the feedback parameters in the playing process by fusing the environmental perception information and the personalized speaker characteristics, clear and natural optimized audio output with personalized styles can be generated in a complex environment, and the voice interaction quality and adaptability are improved.

The invention belongs to the field of artificial intelligence, medical treatment and finance, and relates to a speech synthesis method, which comprises the following steps: performing phoneme conversion and context embedding on a text data set, obtaining a phoneme sequence and a text context vector, extracting reference speech features of a reference audio sample, inputting a phoneme-level potential diffusion model, and generating phoneme-level prior potential features; inputting the phoneme-level prior potential features into a phoneme-level variational self-encoding model to obtain reconstructed speech features; extracting a target voice feature of the target audio sample; performing alignment and up-sampling on the target voice feature and a phoneme sequence to obtain a target audio frame-level feature; and training based on the reconstructed speech features and the target audio frame-level features to obtain a conditional control speech synthesis model, and synthesizing the to-be-converted text speech. The invention further provides a speech synthesis device, computer equipment and a storage medium. In addition, the invention also relates to a block chain technology, and the to-be-converted text can be stored in a block chain. The speaker fidelity and the text understandability are improved.

The invention provides a speech synthesis method, device and equipment and a computer readable medium, and the method comprises the steps: obtaining input data, the input data comprises a reference audio and a target synthesis text, and then determining a target tone feature and a reference speech content feature sequence according to the reference audio, reference voice content features in the reference voice content feature sequence are used for representing at least one of audio content, intonation, rhythm and speaking habits of the reference voice, and inputting the reference voice content feature sequence and the target synthetic text into a pre-trained voice content feature generation model, according to the method, the target voice content feature sequence is generated, and then the target synthetic audio is generated according to the target timbre feature and the target voice content feature sequence, so that the target synthetic text can be converted into the target synthetic audio conforming to the timbre, the intonation, the rhythm, the speaking habit and the like of the reference audio, and the authenticity of voice synthesis is improved.

The invention relates to the technical field of artificial intelligence, and discloses a speech synthesis method and device, equipment and a medium, and the method comprises the steps: obtaining a to-be-synthesized text and a reference audio; dividing the to-be-synthesized text into a plurality of text segments by adopting a sliding window algorithm; generating an audio waveform corresponding to each text segment based on the reference audio and a pre-trained speech synthesis model; and performing speech synthesis on the audio waveform corresponding to each text segment to obtain a target speech. Through the mode, the input text is segmented by adopting the sliding window algorithm to form a plurality of smaller and easier-to-manage text segments, and the plurality of text segments and the parameter audio are input into the speech synthesis model, so that the speech synthesis model generates the audio sentence by sentence based on the shorter text segments; therefore, the model can focus on accurate voice and intonation details of each segment, the accuracy and natural fluency of audio output are ensured, the degradation problem caused by the fact that the text length exceeds a typical length parameter is solved, and the stability of voice generation is improved.

The invention discloses a speech cloning method and device based on emotion enhancement, and a related medium. The method comprises the following steps: respectively obtaining a reference audio and a prediction text corresponding to the reference audio; respectively preprocessing the reference audio and the prediction text to generate standardized audio data for feature extraction; inputting the standardized audio data into an emotion enhancement module so as to generate acoustic features matched with a target voice style through multi-round feature fusion and an autoregression generation mechanism; and inputting the acoustic features into a speech synthesis module for decoding processing, and outputting predicted speech. According to the invention, by introducing the emotion enhancement module, accurate modeling of the emotion style of the speaker in the acoustic feature generation process is realized, so that the emotion simulation degree and the personality restoration capability of the synthesized speech are significantly improved.

The invention provides a voice synthesis scheme and device, electronic equipment, a storage medium and a program product, and relates to the technical field of voice processing, and the method comprises the steps: inputting a text content into an LLM semantic understanding module, and obtaining a deep semantic feature and multi-modal context semantic information corresponding to the text content; transmitting the deep semantic features to a neural codec, and outputting compressed acoustic features; inputting the compressed acoustic features into an acoustic modeling module, and outputting high-precision acoustic features; inputting the high-precision acoustic features and the multi-mode context semantic information into an emotion and rhythm control module, and inputting voice parameters with emotion and rhythm marks; and inputting the voice parameters with the emotion and rhythm marks and the reference audio into a tone migration module to obtain a synthetic voice of a tone corresponding to the reference audio.

A method includes receiving, by a processing device, an audio sample of a distorted voice. The method further includes extracting a first set of characteristics from the audio sample. The method further includes selecting a second set of characteristics. The second set of characteristics is associated with one of a set of reference audio samples. The method further includes generating an undistorted audio sample. The undistorted audio sample is based on the distorted audio sample, the first set of characteristics, and the second set of characteristics.

The invention provides a voice conversion method, device and equipment and a readable storage medium, and the method comprises the steps: carrying out the compression quantization of a to-be-converted audio and a reference audio through an audio quantizer, so as to generate a semantic code of the to-be-converted audio and a reference code of the reference audio; aligning the semantic code and the reference code according to the length of the Mel spectrum so as to splice a group of feature vectors, calling an audio large language model to carry out autoregression decoding on the feature vectors so as to generate an output code, inputting the Mel spectrum of the reference audio into a tone style network so as to generate a perception vector, and outputting the perception vector; fusing the output code and the sensing vector to generate a vector code; then, a flow prediction network is called to process the vector codes in combination with a time sequence embedding and prior distribution, so that a Mel spectrum is generated step by step; and finally, restoring the Mel spectrum into a target voice signal through a vocoder, thereby solving the problem that the existing small-scale model is not ideal in fine sound feature restoration effect.

The invention discloses an audio signal adjustment method and device, equipment, a medium and a program product, and belongs to the field of audio processing. The method is executed by a client, and comprises the following steps: sending a reference audio signal to airborne equipment in a target vehicle; recording to obtain a recorded audio signal, the recorded audio signal being a signal recorded by the client in the cabin of the target vehicle after the airborne equipment plays the reference audio signal; adjusting an audio filter with the recording audio signal and the reference audio signal being the same as each other as a target; and filtering the subsequently played first audio signal through an audio filter to obtain a second audio signal. According to the method, a recorded audio signal is obtained through recording, the recorded audio signal is an audio signal associated with a reference audio signal, and an audio filter is adjusted with the two audio signals being the same as each other, so that a first audio signal played subsequently can be filtered and optimized, and a second audio signal more suitable for being listened in a current cabin is obtained.

The invention discloses a zero-sample voice cloning method based on a dynamic neural network and feature modulation. The method comprises the following steps: extracting a speaker style vector from a reference audio by a speaker style encoder; performing feature modulation on the speaker style vector based on an SGF algorithm, and inputting the modulated speaker style vector into a generator; the generator synthesizes the audio of the target speaker based on the dynamic neural network. According to the method, the audio of any speaker can be cloned in a zero sample situation, and the smooth and natural target audio with high similarity is synthesized.

The invention provides a sound wave generation method, device and system. In the method, after a user instruction is obtained, to-be-simulated sound wave information in the user instruction is determined, then a reference audio signal corresponding to the to-be-simulated sound wave information is determined, and after current driving state information of a vehicle is determined, the reference audio signal is processed based on the driving state information, so that the current driving state information of the vehicle is obtained. The to-be-played simulated sound corresponding to the to-be-simulated sound information is obtained, so that the vehicle plays the to-be-played simulated sound, and the sound playing effect of the vehicle is achieved. According to the scheme of the invention, the corresponding reference audio signal is determined based on the user instruction, and the to-be-played simulated sound is obtained in a mode of processing the reference audio signal by using the driving state information. Therefore, when the sound wave is generated through the scheme of the invention, the audio frequency as the sound wave does not need to be stored in the memory space of the vehicle, so that the occupation of the memory space of the vehicle can be reduced.

This disclosure provides methods, devices, and systems for audio signal processing. The present implementations more specifically relate to speech enhancement techniques that are agnostic to varying signal levels in near-end audio signals. In some aspects, a speech enhancement system may include a delay estimator, an input normalizer, and an acoustic echo and noise (AEN) decoupling filter. The delay estimator receives a near-end audio signal via a microphone and a far-end audio signal for output via a speaker and estimates a reference audio signal based on a delay between the near-end audio signal and the far-end audio signal. The input normalizer normalizes a loudness of the near-end audio signal and the reference audio signal. The AEN decoupling filter determines a set of masks based on the normalized audio signals and suppresses an echo component and a noise component of the near-end audio signal based on the set of masks.

The invention relates to an adaptive emotion-driven timbre clone character-to-speech method and device. Segmenting the target text according to sentences, extracting semantic features, and splitting each sentence into a target phoneme sequence; the method comprises the following steps: splitting a reference text into reference phoneme sequences, splicing the reference phoneme sequences with each target phoneme sequence to obtain a combined phoneme sequence, processing a reference audio to obtain a reference voice feature and a frequency spectrum, processing based on a semantic feature corresponding to a sentence, the combined phoneme sequence and the reference voice feature to obtain a corresponding voice feature, and extracting a corresponding emotional feature from each sentence semantic feature, and processing based on the speech feature, the emotional feature and the target phoneme sequence corresponding to each sentence and the frequency spectrum to obtain a speech signal corresponding to the target text. And extracting emotional features from the semantic features of the target text, enabling the emotional colors of the voice signals to fit the text, and automatically adjusting the emotional colors of the voice signals based on the mapping relationship between the preset output features and the emotional features.

The invention discloses a fault server positioning method and device, a storage medium and electronic equipment, and relates to the technical field of computers, and the method comprises the steps: detecting the operation state of a server in a server room, responding to a target server in a fault operation state, and controlling the target server to output an audio signal meeting a target audio parameter. The characteristic that an audio collector can collect audio signals is utilized, and reference audio signals output by all servers and collected by the audio collector are received. The audio signal meeting the target audio parameter is screened out from the reference audio signal, so that the audio collector which collects the audio signal output by the target server is determined, and the target server is positioned through the position of the audio collector. The technical problem that the positioning efficiency of the fault server is low is solved, and the technical effect of improving the positioning efficiency of the fault server is achieved.

The invention relates to neural network speech generation. Apparatus, systems, and techniques for generating speech audio are disclosed. In at least one embodiment, a processor generates a first speech audio based at least in part on a second speech audio and a reference audio using one or more neural networks.

An audio processing system is disclosed for comparing a query audio sample with a database of multiple reference audio samples using an external normalization. The system includes at least one processor and memory storing instructions that, when executed by the processor, cause the system to determine a bias term of the external normalization based on a spectro temporal pattern of the query audio sample. The system further compares the query audio sample with each of the reference audio samples to generate a similarity score for each comparison. The system combines the bias term with each of the similarity scores to produce normalized similarity scores. The normalized similarity scores are then compared with a threshold to generate a result of comparison, which is subsequently outputted.

The invention relates to the technical field of end-side model deployment, in particular to an end-side voice model deployment method and device, equipment and a storage medium. Comprising the steps of recording reference audio at an end side; extracting a reference semantic token and an embedded vector of the reference audio; obtaining a training text set, integrating and inputting the training text code, the embedded vector and the reference semantic token corresponding to each training text in the training text set into a preset language model, and outputting a comprehensive token sequence corresponding to the training text; performing Mel spectrum conversion on each comprehensive token sequence to obtain a first spectrum representation corresponding to the comprehensive token sequence; generating an audio signal corresponding to the training text according to each first spectrum representation, and integrating to obtain a training data set; inputting the training data set into a to-be-trained model, and training to obtain a lightweight voice model which only retains timbre modeling parameters; and updating the lightweight voice model to the end side to complete end side deployment. According to the invention, the voice synthesis model deployment of the end-side equipment can be realized.

The invention provides a method for generating a voice cloning model and a related device, and the method is applied to the field of voice cloning, and comprises the steps: obtaining a scoring result of a plurality of reference audios inputted by a user through a terminal device; training based on the plurality of reference audios and the scoring result of the user on the plurality of reference audios to obtain an acoustic feedback unit, the acoustic feedback unit being used for measuring the auditory feeling of the user on different audios; acquiring a first voice data set input by a user through a terminal device; and based on the first voice data set and the acoustic feedback unit, training the voice clone model to obtain the voice clone model. In the voice clone model training process, the preference degree of the user for different audios is considered, the scoring results of the user for the multiple reference audios are added into the voice clone model training process, the voice clone model obtained through training better meets the use requirements of the user, and the user experience is improved. And the voice generated by using the voice cloning model obtained by training can better meet the auditory feeling of the user.

The invention provides a closed container leakage detection method, device and system, and belongs to the technical field of signal processing, and the method comprises the steps: receiving a first audio signal collected by a microphone array disposed in an outer side space of a closed container, and determining a first angle range of the microphone array according to the audio characteristics of the first audio signal; collecting multiple paths of second audio signals in the first angle range; performing primary beamforming on the multiple paths of second audio signals to determine a third audio signal when the signal intensity is the strongest; and judging whether leakage exists in the closed container according to the signal gain between the third audio signal and the reference audio signal. According to the method, the device and the system for detecting the leakage of the closed container, provided by the invention, the non-dead-corner leakage detection of the detected container on a production line can be quickly realized by carrying out self-adaptive frequency range and intensity range scanning beam forming on the microphone array, the detection precision is higher, the detection cost is lower, and the production efficiency is effectively improved.

The invention provides a zero-sample speech synthesis method and device based on an autoregressive large language model, and the method comprises the steps: inputting an audio coding feature of a reference audio, a text coding feature of a reference text, and a target text coding feature of a target text into a pre-trained large language model, obtaining a target sequence output by the large language model; inputting the target sequence into a pre-trained diffusion model to obtain a target Mel spectrum corresponding to the target sequence output by the diffusion model; and inputting the target Mel spectrum into a pre-trained vocoder to obtain a target audio corresponding to the target Mel spectrum output by the vocoder, and determining the target audio as a speech synthesis result corresponding to the reference audio and the target text. Therefore, a user only needs to add a small amount of reference audio of the target pronunciation object in a speech synthesis stage, and does not need to use speech data of the target pronunciation object to repeatedly train an original speech synthesis model, so that the zero-sample speech synthesis capability is effectively improved.

The embodiment of the invention relates to an information interaction method and device, equipment and a storage medium. The method provided by the invention comprises the following steps: presenting an interactive interface with a virtual object; in response to the fact that a first message received in the interactive interface meets a preset condition, music content is provided, songs corresponding to the music content are determined on the basis of the first message, and at least one audio attribute of the music content is generated on the basis of reference audio content associated with the virtual object. In this way, the embodiment of the invention can support the virtual object to provide the music content corresponding to the first message when receiving the first message meeting the preset condition, thereby improving the interaction effect. Besides, in the embodiment of the invention, at least one audio attribute of the music content generated based on the reference audio content associated with the virtual object is provided, so that the interaction quality is improved, and the interaction capability is enriched.

The invention provides a target audio generation method and device, electronic equipment, a computer readable storage medium and a computer program product, and relates to the field of computers, in particular to the technical field of artificial intelligence, deep learning and natural language processing. According to the implementation scheme, reference audio, a reference text and a target text are obtained; the method comprises the following steps: performing feature discretization on a reference audio to obtain a reference audio acoustic marking sequence and a reference audio semantic marking sequence; performing feature discretization on the target text and the reference text to obtain respective corresponding text semantic tag sequences; based on the text semantic tag sequence and the reference audio semantic tag sequence, obtaining a target audio semantic tag sequence through a first generative language model; based on the reference audio semantic marking sequence, the target audio semantic marking sequence and the reference audio acoustic marking sequence, obtaining a target audio acoustic marking sequence through a second generative language model; and decoding based on the target audio acoustic mark sequence to obtain a target audio.

This disclosure provides methods, devices, and systems for audio signal processing. The present implementations more specifically relate to speech enhancement techniques for separating microphone signals into speech, echo, and noise signals. In some aspects, a speech enhancement system may include a delay estimator and an acoustic echo and noise (AEN) decoupling filter. The delay estimator receives a microphone signal via a microphone and a far-end audio signal for output via a speaker and estimates a reference audio signal based on a delay between the microphone signal and the far-end audio signal. In some aspects, the AEN decoupling filter may determine a speech mask, an echo mask, and a noise mask based on the microphone signal and the reference audio signal and may suppress an echo component and a noise component of the microphone signal based on the determined set of masks.

The invention belongs to the field of artificial intelligence, and relates to a speech synthesis method, which comprises the following steps: acquiring a text of a speech to be synthesized and a reference audio representing a target emotion; performing feature extraction on the reference audio by adopting an emotion coding model, and constructing a target three-dimensional emotion coordinate for describing a target emotion; performing coordinate conversion processing on the target three-dimensional emotion coordinate to obtain a target spherical emotion coordinate describing the target emotion; carrying out coding conversion on the target spherical emotion coordinates by adopting an emotion encoder to obtain a target emotion embedding vector for describing target emotion; and fusing the text and the target emotion embedding vector by adopting a speech synthesis model to generate a target synthesized speech with the semantics of the text and the target emotion. The invention further provides a device, equipment and a medium. In addition, the invention also relates to a block chain technology, and the text, the reference audio and the target synthetic voice can be stored in a block chain. According to the method and the device, more precise and flexible control on emotion expression in speech synthesis can be realized.

The invention discloses a music generation method and device, electronic equipment and a storage medium. The method comprises the steps of obtaining reference music and target lyrics in response to triggering of a music generation event; generating at least one reference coding vector based on the reference music; wherein the reference coding vector is a coding vector of at least one layer; performing word segmentation processing on the target lyrics, and converting each word segmentation unit in the target lyrics into a corresponding word segmentation unit vector; and inputting the at least one reference coding vector and the word segmentation unit vector into a pre-constructed music generation large model, and generating target music according to an output result of the music generation large model. According to the scheme, the large model is generated based on the music, the target music of any length, any content and any type can be generated at a time according to the obtained reference music and the target lyrics, the generation is free, the flexibility is high, and the diversity of the generated target music is greatly improved.

The invention relates to a voice synthesis device and a training method thereof, electronic equipment and a storage medium, and the method comprises a voiceprint coding module which receives an input reference audio and extracts voiceprint features of the reference audio; the audio code generation module is used for determining a target expert network from a plurality of expert networks to execute a voice synthesis operation according to the input text to be synthesized and the voiceprint features so as to obtain a generated audio code sequence; wherein the plurality of expert networks correspond to different application scenes; and the audio decoding module is used for converting the audio coding sequence output by the audio coding generation module into an audio signal. According to the embodiment of the invention, through the dynamically selected target expert network, the voice synthesis of the application scene suitable for the input to-be-synthesized text and the voiceprint feature can be executed, and the scene adaptability of the synthesized voice is remarkably improved.

The invention provides a mediator timbre cloning method and system, electronic equipment and a storage medium. The method comprises the following steps: selecting a tone of a mediator in response to a selection instruction input by a user; obtaining an input text; predicting a target audio feature vector of the input text by using an autoregression model; a target clustering center matched with the target audio feature vector is searched in an audio dictionary, the audio dictionary comprises a plurality of clustering clusters, and each clustering cluster comprises a plurality of audio feature vectors; and inputting the input text phonemes of the input text, the frequency characteristics of a reference audio and the target clustering center into the trained acoustic model to obtain a target output audio, the reference audio being a mediator audio corresponding to the tone of the mediator. According to the method, the mediation efficiency can be improved, the service consistency is ensured, the user experience is improved, and the method has better practicability and market competitiveness.

This disclosure provides methods, devices, and systems for audio signal processing. The present implementations more specifically relate to speech enhancement techniques that can adapt to varying signal-to-noise ratio (SNR) conditions. In some aspects, a speech enhancement system may include a low SNR detector and a spatial filter. The spatial filter receives a multi-channel audio signal via a microphone array and produces an enhanced audio signal based on a beamforming filter. The low SNR detector tracks an SNR of a reference audio signal of the multi-channel audio signal. In some implementations, the spatial filter may substitute at least part of the reference audio signal for an auxiliary audio signal, received from an auxiliary microphone separate from the microphone array, when the SNR falls below a wideband SNR threshold. In some other implementations, the spatial filter may refrain from updating the beamforming filter when the SNR falls below a narrowband SNR threshold.

A method of audio generation based on a large language model is disclosed, which involves the fields of artificial intelligence such as large language models, natural language processing, deep learning, and audio generation. The method of audio generation based on a large language model comprises: acquiring a text to be processed; parsing the text to be processed using the large language model to obtain role information and emotional information corresponding to the text to be processed; obtaining a target reference text and a target reference audio according to the role information and the emotional information; and generating a target audio corresponding to the text to be processed according to the text to be processed, the target reference text, and the target reference audio.

A method for training a non-autoregressive TTS model includes receiving training data that includes a reference audio signal and a corresponding input text sequence. The method also includes encoding the reference audio signal into a variational embedding that disentangles the style / prosody information from the reference audio signal and encoding the input text sequence into an encoded text sequence. The method also includes predicting a phoneme duration for each phoneme in the input text sequence and determining a phoneme duration loss based on the predicted phoneme durations and a reference phoneme duration. The method also includes generating one or more predicted mel-frequency spectrogram sequences for the input text sequence and determining a final spectrogram loss based on the predicted mel-frequency spectrogram sequences and a reference mel-frequency spectrogram sequence. The method also includes training the TTS model based on the final spectrogram loss and the corresponding phoneme duration loss.