980 results about "Timbre" patented technology

The invention relates to the technical field of artificial intelligence, can be applied to business scenes of medical health, financial science and technology, cultural transmission and the like, and discloses a voice generation method based on multi-modal fusion, which comprises the following steps: collecting audio data to extract timbre features, and training a field feature timbre generation model; analyzing text semantic recognition emotion information, adjusting speech synthesis parameters, combining personalized information to construct a parameter mapping table, fusing to generate a synthesis control parameter sequence, aligning the synthesis control parameter sequence with character labels, visual elements and background music data, driving a domain feature timbre generation model, and generating synthesis data of synchronous speech, text, vision and music. Domain timbres are generated through timbre feature training, speech expression is optimized by combining semantic analysis and emotion recognition, user requirements are matched based on personalized information, and time alignment is performed by fusing text, vision and music data, so that the synthesized speech has domain features, emotion adaptability and personalization, and the speech quality is improved. And the voice immersion and the information transmission capability are improved.

The invention provides a pet voice translation method and system, electronic equipment and a storage medium, and the method achieves the accurate recognition of the emotion state of a pet through collecting a high-quality pet sound audio sample, extracting acoustic features and constructing a deep learning model. The intention and demand of the pet can be deduced in combination with the variety, age and other information of the pet, and a proper pacifying statement is selected from a statement library. Through a deep neural network technology, synthetic voice similar to pet sound in tone and emotion is generated, and personalized voice interaction is realized. The voice interaction between a person and the pet is realized, and the sound of the pet is converted into the voice, so that an owner of the pet can timely and accurately know the current demand of the pet, and the current demand of the pet can be met more quickly.

The invention relates to the technical field of artificial intelligence, can be applied to the field of medical health, and discloses a timbre conversion method, device, equipment and medium based on cultural semanteme, the method comprises the following steps: constructing a cultural semantic timbre library comprising a semantic label and timbre characteristic parameter mapping relationship, the semantic label characterizing emotional semanteme of a target timbre, and the timbre characteristic parameter mapping relationship between the semantic label and the timbre characteristic parameter; the timbre characteristic parameters comprise a pitch range, rhythm rhythm and a harmonic structure; performing feature extraction based on text, image and audio multi-mode information to obtain semantic keywords, visual emotion features and audio acoustic features; performing attention weight fusion on the features through a multi-modal fusion deep learning model, and dynamically adjusting model parameters in combination with a semantic timbre library to generate a target timbre; and finally, intelligent conversion from the multi-mode information to the adaptive tone is realized. Through semantic-driven multi-modal feature collaborative optimization, the defect that timbre conversion machinery is stiff and lacks emotional expression is overcome, and the integrating degree of timbre expression and semantic scenes is improved.

The invention relates to the technical field of voice processing, can be applied to business scenes such as financial science and technology, medical health and the like, and discloses a cross-language voice migration synthesis method, device, equipment and medium. And training an acoustic model by adopting a hierarchical self-adaptive fine tuning strategy, fusing the phoneme sequence for reasoning and the tone mark to generate a representation sequence, and finally synthesizing a target language speech signal. According to the method, a sharing and separation parallel cross-language modeling structure is constructed, the accuracy of phoneme and tone modeling in a low-resource language is effectively improved, the target language acoustic model has higher generalization ability and migration efficiency in combination with a multi-stage self-adaptive fine tuning and training data enhancement strategy, and the method is suitable for being applied to the field of multi-language modeling. And finally, synchronous improvement of the voice naturalness and the tone fidelity is realized.

The invention provides a speech synthesis method, device and equipment and a computer readable medium, and the method comprises the steps: obtaining input data, the input data comprises a reference audio and a target synthesis text, and then determining a target tone feature and a reference speech content feature sequence according to the reference audio, reference voice content features in the reference voice content feature sequence are used for representing at least one of audio content, intonation, rhythm and speaking habits of the reference voice, and inputting the reference voice content feature sequence and the target synthetic text into a pre-trained voice content feature generation model, according to the method, the target voice content feature sequence is generated, and then the target synthetic audio is generated according to the target timbre feature and the target voice content feature sequence, so that the target synthetic text can be converted into the target synthetic audio conforming to the timbre, the intonation, the rhythm, the speaking habit and the like of the reference audio, and the authenticity of voice synthesis is improved.

The invention discloses a video translation method and system based on artificial intelligence. The method relates to the technical field of video translation and comprises the following steps of original sound track extraction, target AI speaker adaptation, AI dubbing generation and mouth shape synchronization and video synthesis. According to the method, independent audio and video streams are obtained by adopting an audio and video separation technology, and multiple original sound tracks are extracted through a voice separation model; matching or generating an adaptive target AI speaker module in a preset tone library; converting the original language voice into a text, translating the text into a target language text, and synthesizing an AI dubbing audio track in combination with a target AI speaker module; and finally, the independent video stream and the multi-AI dubbing audio track are input into the mouth shape synchronization model to output a translated video, so that the timbre fitting degree, the voice quality and the voice consistency of the same speaker of AI dubbing are improved, and meanwhile, the resource utilization rate of video translation and the processing efficiency under batch tasks are improved. The problem that in the prior art, video translation is low in quality and efficiency is solved.

The invention discloses a voice cloning system and method fusing rhythm characteristics, belongs to the technical field of voice synthesis and natural language, and is applied to the aspect of fine-grained rhythm control in zero-sample voice synthesis. The implementation method comprises the following steps of: 1, extracting rhythm features and audio features of an audio file, and further respectively acquiring pause features, speed features and tone features in the rhythm features by sequentially adopting transcriptional text inverse coding, syllable-level speed registration quantization and pitch sequence feature splicing modes; 2, fusing the features of the audio files in a manner of discarding feature screening without guidance of a classifier; 3, generating a target Mel spectrogram based on conditional flow matching; generating a target audio file from a to-be-cloned audio file through the trained voice cloning model controlled by the fusion rhythm; compared with the prior art, fine-grained rhythm control of tone and rhythm feature decoupling is realized in zero-sample speech synthesis, so that intonation accuracy based on a context scene is improved.

A full-frequency band high quality speaker with a bar and sound tunnels includes a voice coil, magnet, diaphragm and magnetically conductive structure; the diaphragm is conical, a bridge-style bar is on the surface of at least one side of the diaphragm; the diaphragm includes at least two grooves, each horizontally radial; each groove crosses the bridge-type bar in the horizontal direction and the length direction of the groove is perpendicular to the bridge-type bar; the groove is concave on the surface of the diaphragm to form a sound tunnel. The diaphragm is changed to the current standard vibration mode, and the timbre of treble register is not bright and the timbre of bass register is not sonorous and mellow enough from the perspective of vibration, resonance and phonation. The practice proves that the improved scheme has outstanding substantive characteristics and remarkable technical progress, and has obtained obvious technical results.

The invention provides a voice synthesis scheme and device, electronic equipment, a storage medium and a program product, and relates to the technical field of voice processing, and the method comprises the steps: inputting a text content into an LLM semantic understanding module, and obtaining a deep semantic feature and multi-modal context semantic information corresponding to the text content; transmitting the deep semantic features to a neural codec, and outputting compressed acoustic features; inputting the compressed acoustic features into an acoustic modeling module, and outputting high-precision acoustic features; inputting the high-precision acoustic features and the multi-mode context semantic information into an emotion and rhythm control module, and inputting voice parameters with emotion and rhythm marks; and inputting the voice parameters with the emotion and rhythm marks and the reference audio into a tone migration module to obtain a synthetic voice of a tone corresponding to the reference audio.

The invention relates to the technical field of voice processing, and provides a rhythm migration method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining a decoupled rhythm feature based on a source rhythm voice, and a decoupled tone feature based on the voice of a target speaker, the decoupled rhythm feature represents the rhythm of the source rhythm voice, and the decoupled tone feature represents the tone of the target speaker; the decoupled timbre features represent the timbre of the voice of the target speaker; generating a target voice vector sequence based on the text features of the target text, the voice features of the voice of the target speaker, the decoupled rhythm features and the decoupled timbre features; and synthesizing a target audio based on the target voice vector sequence. According to the method and the device, the decoupled rhythm features and the decoupled timbre features are acquired, and the target voice is generated based on the features, so that the problem of feature mixing is effectively relieved, the timbre purity of the target speaker in cross-person rhythm migration is ensured, the expressive force of rhythm migration is improved, and the synthesized audio is more natural and vivid.

The invention relates to a song conversion model training method, a song timbre conversion method, computer equipment, a computer readable storage medium and a computer program product, relates to the technical field of song synthesis, and can improve the efficiency of song timbre conversion by a user. The singing conversion model training method comprises the steps of obtaining a sample song and a singer timbre description text paired with the sample song; obtaining tone characteristics of the sample song by an audio coding module in the singing conversion model, and obtaining text characteristics corresponding to the tone description text of the singer by a text coding module in the singing conversion model; according to the difference between the timbre features and the text features, adjusting model parameters of a singing sound conversion model to obtain a trained singing sound conversion model; the trained song conversion model is used for extracting text features corresponding to the input timbre description text through the text coding module, obtaining target timbre features according to the text features corresponding to the timbre description text, and performing song timbre conversion.

The invention provides a real-time sound copying method and system based on end-cloud fusion. The method comprises the following steps: a cloud end carries out real-time tone copying and voice synthesis on a small amount of voice data of a user based on an AI large model; timbre samples are collected when a user registers voice audio data, and user timbre voice data of a preset text are synchronously generated by a large model and are used as fine tuning training data of an end-side voice synthesis model; user tone voice data of a preset text and voice audio data registered by a user are utilized to carry out migration fine tuning training on an end-side voice synthesis model to adapt to the personalized tone of the user, so that high-quality output of the end-side voice synthesis model is ensured, and personalized voice replication is realized; and issuing the trained end-side speech synthesis model to the user equipment, and independently completing speech replication in a network-free or weak network environment. According to the invention, through automatic generation of the user tone data and adaptive fine tuning of the model, the tone of the user is deployed to the end side after fine tuning, and high-quality and high-adaptability sound replication of end-cloud collaboration is realized.

The invention discloses a music generation method and device. The method comprises the steps of obtaining tone information in first audio data in response to triggering of a music generation event, and generating at least one tone coding vector based on the tone information; obtaining rhythm information in the second audio data, and generating at least one rhythm coding vector based on the rhythm information; obtaining target lyrics, performing word segmentation processing on the target lyrics, and converting each word segmentation unit in the target lyrics into a corresponding word segmentation unit vector; and inputting the at least one timbre coding vector, the at least one rhythm coding vector and the word segmentation unit vector into a pre-constructed music generation large model, and generating target music according to an output result of the music generation large model. According to the scheme, the decoupling of the timbre and the rhythm can be realized, so that the timbre duplicating of any timbre on any rhythm is realized, and the free generation of music by using the duplicated timbre and any rhythm can be realized.

To effectively resolve the issue of inappropriate handovers during communication sessions, methods, systems, and machine-readable mediums which utilize speech features captured by microphones in the original wireless peripheral device and / or the wireless peripheral device to which the communication session is to be handed over to determine if a handover should proceed or should be reversed. Speech features refer to the composite attributes of spoken language that encompass both acoustic and linguistic features. Acoustic features characterize the sound properties of speech and include, but are not limited to, timbre, pitch, intonation, speaking rate, articulation, prosody, melody, spectral features, formant frequencies, and the like. Linguistic features pertain to the actual content conveyed, comprising words, phrases, syntax, and semantics. This includes the analysis of words, phrases, syntax, and semantics to understand the context and continuity of the conversation.

The utility model relates to a novel empty drum. The novel empty drum comprises a drum body, the drum body comprises an upper drum body and a lower drum body which are fixedly connected, and eight peripheral tongues are arranged around the geometric center of the upper drum body; each peripheral tone tongue is provided with a first sub-tone tongue, a second sub-tone tongue, a third sub-tone tongue, a fourth sub-tone tongue and a fifth sub-tone tongue, and the first sub-tone tongue, the second sub-tone tongue, the third sub-tone tongue and the fourth sub-tone tongue are arranged close to the fifth sub-tone tongue; the geometric center of the lower drum body is provided with a timbre control hole and a rubber plug matched with the timbre control hole. According to the novel empty drum, the structure of the peripheral tongues is improved and designed, so that the tone of the empty drum in the playing process is richer, and the use experience of a user is improved.

The invention belongs to the technical field of audio equipment design, and relates to a microphone for AI voice cloning based on pseudo tone of acoustic shielding, and the microphone comprises an audio collection module which is used for collecting an environment audio signal in real time; the analog-to-digital converter is used for converting the environment audio signal into a digital signal; the feature extraction module is used for performing frequency domain conversion on the digital signals and extracting frequency spectrum features and tone features; the acoustic shielding calculation module is used for calculating a shielding threshold value of the environment audio signal and determining a frequency band which cannot be heard by human ears in the environment audio signal according to the shielding threshold value; the pseudo tone generation module is used for generating pseudo tones based on the extracted tone features; the synthesis module is used for replacing a corresponding frequency band of the environment audio signal with the generated pseudo tone to form a synthesized audio; and the audio output module is used for outputting the synthesized audio. According to the invention, the pseudo tone can be generated in real time in the audio recording process, and it is ensured that a user is always protected during audio recording.

The invention relates to the technical field of piano teaching and computers, discloses a multi-dimensional scoring auxiliary system and method in piano teaching, and aims to solve the problems that existing piano teaching is single in evaluation dimension, one-sided in data acquisition and lagged in feedback. According to the method, a five-dimensional scoring model covering rhythm, strength, timbre, posture and fingering is constructed by synchronously collecting a key triggering time sequence, an audio frequency spectrum, limb posture and finger joint tracks, and quantitative scores are output through time alignment, feature extraction and neural network fusion calculation. The system comprises a key sensing module, an audio acquisition module, a posture capture module, a score calculation module and an AR visual feedback module, and supports teaching suggestion generation, historical trend analysis, difficulty self-adaption and multi-person comparison functions. Through multi-modal data closed-loop evaluation and intelligent intervention, the teaching accuracy and individuation level are remarkably improved, and scientific quantification and efficient improvement of the piano playing ability are achieved.

The invention relates to the technical field of ancient music score intelligent processing, and discloses an ancient music score intelligent identification and visual playing system, which comprises an image preprocessing module, a stroke enhancement module, a symbol segmentation module, a symbol identification module, a semantic analysis module, a visualization module, an acoustic synthesis module and the like. Through a specially designed image processing and symbol analysis method, the aging problem of the ancient music score can be accurately repaired, adhered symbols are separated, a complex structure is identified, music semantics are inferred in combination with a knowledge graph, and a multi-modal visualization effect and a high-sampling ancient charm tone are generated. The system supports feedback optimization and automatic workflow, significantly improves the efficiency and accuracy of ancient music score recognition, translation and presentation, and provides technical support for ancient music research and propagation.

The invention provides a voice evaluation method, a voice evaluation device, a non-transitory storage medium and a computer program product. The voice evaluation method comprises the following steps: extracting a first person voice with a first tone based on a to-be-evaluated voice; converting the voice of the first person into voice with a second tone; and obtaining an evaluation result of the voice of the first person based on the converted voice with the second tone and the standard voice of the one or more accent.

The invention relates to a dubbing method and device, electronic equipment and a storage medium. The method comprises the following steps: acquiring a target dubbing table book of a to-be-dubbed video; wherein the target dubbing table book comprises at least one line and the time and role corresponding to each line; the language corresponding to the target dubbing table book is different from the language corresponding to the video to be dubbed; performing voice synthesis on the target dubbing table book to generate target audio data; wherein the language corresponding to the target audio data is the same as the language corresponding to the target dubbing table book; performing timbre conversion on the target audio data; and performing audio and picture synchronization processing on the audio data after tone conversion and the video to be dubbed to generate a dubbed video. According to the method and the device, the cross-language automatic dubbing of the video is realized through processing such as speech synthesis, tone conversion and audio and picture synchronization.

The invention provides a video dubbing language conversion method, a video dubbing language conversion system and related equipment. The method comprises the following steps: acquiring audio track data from a video to be converted; carrying out human voice extraction on the audio track data and classifying according to roles to obtain a single speaker audio of each role; performing voice-to-text conversion on the single speaker audio of each role to obtain an original language copywriting of each role; performing sound cloning on the single speaker audio of each role to obtain a timbre model of each role; performing target language translation on the original language copywriting of each role to obtain a translated copywriting of each role; based on the translation copywriting of each role and the tone model of each role, performing text-to-voice conversion to obtain a translation audio of each role; and performing replacement of each role translation audio on the audio track data in the to-be-converted video to obtain a dubbing conversion video. According to the technical scheme, language video dubbing conversion combined with the tone of the speaker is achieved, the video is more diversified, and the user requirements can be better met.

The invention provides a method for generating a virtual character and interacting with a user, and solves the problems of insufficient personification degree and limited interaction capability of the virtual character in the prior art. The method comprises the steps of obtaining information of a virtual character to be generated, generating a cue word Prompt used for generating a virtual character image, generating a character image through the cue word, generating timbre, fixed verbal skill voice and video based on the virtual character image, and optimizing the timbre, the fixed verbal skill voice and the video; according to the virtual character and the large language model, interaction between the user and the virtual character is achieved, the anthropomorphic degree of the virtual character is improved, and the interaction ability of the virtual character is improved.

The invention belongs to the technical field of text-to-speech conversion, and particularly relates to a solution for TTS (text-to-speech) in a high-concurrency scene through tone cloning. According to the method, the text can be generated and returned to the client by segmenting and parallelizing the text and reducing the delay of the first segment, compared with a traditional serial whole segment synthesis mode, the waiting time of a user is greatly shortened, the audio is returned after being generated, the waiting time of the user can be effectively shortened under high concurrency, the network bandwidth and the delay pressure are reduced, and the user experience is improved. The real-time performance is guaranteed, reconnection disasters are avoided, underlying model services can be fully utilized, the influence of the serial processing characteristic of a CosyVoice2 model is avoided, a client can immediately start to play a first segment of audio, even if subsequent segments are still synthesized, the continuous playing experience of previous content is not influenced, and when the concurrency is increased, the continuous playing experience of the previous content is not influenced. The model gateway can dynamically disperse requests to a plurality of GPU instances, and long queue waiting after a single-card video memory is fully occupied is avoided.

The invention relates to the technical field of music information management, in particular to a flute ethnic music library construction method and system, and the method comprises the following steps: obtaining flute track audio data, dividing paragraphs according to a time axis, extracting a tone waveform change track, carrying out the cumulative statistics of tone turning, intercepting melody main line information, and recognizing the rising and falling and continuous changes of musical scales. Defining path nodes, marking melody structures, comparing timbre fluctuation segments, recording style change positions, and generating style music library field combinations. According to the method, the audio data of the flute track is finely divided, the tone waveform change is counted, the key tone change in the track is captured, and the node division is performed through the directivity of the musical scale trend, so that the multi-dimensional visualization of the information is increased, and the structural characteristics of the melody are clearer; and national style sample tracks are called for structure comparison, and fragments with consistent styles are counted, so that the convenience and efficiency of searching and identifying specific style tracks by a user are greatly improved.

The invention provides a self-adaptive rhythm electronic piano accompaniment system based on artificial intelligence, and the system comprises a single-key triggering module which is used for indicating an accompaniment process; the speed-control signal conversion module is used for converting the speed signal acquired by the inductive sensor into a music control signal; the storage module is used for storing accompaniment note sequence data of the song; and the artificial intelligence accompaniment generation module adjusts the accompaniment rhythm according to the trigger signal and the playing speed, and generates accompaniment sound or harmony in real time. The system further comprises a pitch calculation unit, a timbre parameter adjustment module, a timbre generation module, a dynamic range adjustment module, a timbre smooth transition module, an automatic harmony generation module, a volume dynamic adjustment module and an audio output module. The control unit receives a user instruction and adjusts system setting parameters. Intelligentization and self-adaption of the accompaniment can be achieved, self-adjustment of the playing speed of the accompaniment is achieved, the playing speed of the accompaniment is kept consistent with the singing speed of the user, the playing flexibility and expressive force are improved, and the playing experience is improved.

The invention relates to the field of network live broadcast, and discloses a live broadcast room voice dialogue voice changing method and device, equipment and a medium, and the method comprises the steps: responding to a voice speaking event of a speaking user in a live broadcast room, and detecting and determining a voice segment in target audio data; according to a preset target pitch value, performing tone modification processing on the segment pitch feature of the voice segment to obtain a tone tuning pitch feature; based on the tuning pitch feature and the target timbre feature, performing voice changing processing on the voice segment to obtain a voice changing segment; and replacing the corresponding voice segment in the target audio data with the variable voice segment to obtain variable voice audio data, and sending the variable voice audio data to a called user in the live broadcast room. According to the method and the device, the performance of a voice changing technology is remarkably improved, and the problems in the aspects of real-time performance, personalized service, pitch adjustment naturalness, tone true restoration and the like in the traditional technology are solved.

The embodiment of the invention discloses an intelligent voice interaction system and method based on streaming multi-mode fusion and an equipment control protocol, the system comprises a voice input processing module, a voice understanding and generating module and a voice synthesis module, the voice input processing module is used for converting an audio signal into a first token sequence, and the first token sequence is used for converting the audio signal into a second token sequence; the voice understanding and generating module is used for determining a response token sequence according to the first token sequence on the basis of a multi-modal Transform architecture so as to realize voice understanding and generation; and the voice synthesis module is used for synthesizing the response token sequence into an output audio so as to carry out at least one of the following adjustments on the converted audio of the response token sequence: emotion parameter adjustment, tone adjustment and rhythm adjustment. By adopting the embodiment of the invention, low-delay and high-naturalness intelligent voice interaction can be realized, multi-modal fusion and equipment control are supported, and the user experience is remarkably improved.

A metal-free signal cable made of at least 24 carbon rovings twisted together for use in music reproduction as a loudspeaker cable, small signal cable or bridge, which is kept free from external influences by spacers made of electromagnetically practically neutral aluminum-magnesium alloy. Even the highest-quality hi-fi systems sound significantly more energetic, clearer, more accurate, more tangible, and even more "in the room" when using my invention. Transients are much more realistic, the reproduced stage size increases significantly, the sound and positioning of the instruments involved are noticeably improved, and each individual note becomes significantly more subtle, complex, and rich in nuance. Even the smallest sound events become perceptible, which are lost with conventional solutions.

The invention relates to the technical field of artificial intelligence, in particular to a sound conversion method and device based on artificial intelligence, equipment and a medium. According to the method, the pitch feature of the source speaker is difficult to obtain, the frame number of the pitch feature is determined, the timbre feature extraction and the emotion feature of the target speaker are obtained, and the frame number of the timbre feature and the frame number of the emotion feature are expanded to be equal to the frame number of the pitch feature; and aligning the pitch feature, the expanded timbre feature and the expanded emotion feature, fusing the aligned features to obtain a first fusion feature, extracting the source voice content, fusing the source voice content with the first fusion feature to obtain a second fusion feature, and obtaining the converted voice according to the second fusion feature. According to the method and the device, in the voice conversion process, the emotion information and the pitch information corresponding to the target speaker are fused into the content information of the source speaker, so that the obtained converted target voice can better reflect the voice of the target speaker, and the voice conversion effect is improved.

The invention relates to the technical field of end-side model deployment, in particular to an end-side voice model deployment method and device, equipment and a storage medium. Comprising the steps of recording reference audio at an end side; extracting a reference semantic token and an embedded vector of the reference audio; obtaining a training text set, integrating and inputting the training text code, the embedded vector and the reference semantic token corresponding to each training text in the training text set into a preset language model, and outputting a comprehensive token sequence corresponding to the training text; performing Mel spectrum conversion on each comprehensive token sequence to obtain a first spectrum representation corresponding to the comprehensive token sequence; generating an audio signal corresponding to the training text according to each first spectrum representation, and integrating to obtain a training data set; inputting the training data set into a to-be-trained model, and training to obtain a lightweight voice model which only retains timbre modeling parameters; and updating the lightweight voice model to the end side to complete end side deployment. According to the invention, the voice synthesis model deployment of the end-side equipment can be realized.