A voice changing method and system

Abstract

The application discloses a voice changing method and system, acquires audio data and at least one target timbre, pre-processes the audio data to obtain pre-processed audio data, separates human voice in the pre-processed audio data using a deep learning model to obtain a human voice signal, performs timbre recognition on the human voice signal based on a feature extraction model and a target timbre library to obtain a timbre recognition result, converts the human voice signal according to the target timbre based on the timbre recognition result using a generation model to obtain an initial timbre output, and adjusts the initial timbre output to obtain a final timbre output. After timbre recognition, the generation model is used for timbre conversion, the human voice signal can be accurately and efficiently converted into multiple target timbres, the initial timbre output is adjusted, the timbre effect can be effectively optimized, a natural tone effect is achieved, and the voice changing quality and flexibility are improved.

Description

Technical Field

[0001] This invention relates to the field of audio processing technology, and in particular to a voice-changing method and system. Background Technology

[0002] Currently, AI (Artificial Intelligence) voice changing technology has been applied in fields such as voice assistants, online entertainment, and music creation. Most existing voice changing solutions rely on basic pitch adjustment or simple timbre processing, failing to achieve more complex timbre recognition and conversion functions. This results in a lack of natural sound quality, and the converted voice often sounds mechanical, lacking realism and expressiveness. These technologies can only provide single-timbre conversion, and when blending human voices with background music or other timbres, the effect is often unsatisfactory, easily leading to sound distortion or incoherence. They also have limitations in scenarios such as speech synthesis and virtual character performance, especially in simultaneously outputting multiple timbres and achieving natural blending. Furthermore, existing voice changing applications lack personalization and intelligence in user interaction, failing to automatically adjust and optimize sound effects according to user needs. Summary of the Invention

[0003] The technical problem to be solved by the present invention is to provide a voice changing method and system that can improve the quality and flexibility of voice changing.

[0004] To solve the above-mentioned technical problems, the present invention adopts the following technical solution: A voice-changing method, comprising the following steps: Acquire audio data and at least one target timbre; The audio data is preprocessed to obtain preprocessed audio data, and a deep learning model is used to separate the human voice in the preprocessed audio data to obtain the human voice signal. Based on the feature extraction model and the target timbre library, the human voice signal is timbre recognition is performed to obtain the timbre recognition result; Based on the timbre recognition result, a generative model is used to convert the human voice signal according to the target timbre to obtain an initial timbre output, and the initial timbre output is adjusted to obtain the final timbre output.

[0005] To solve the above-mentioned technical problems, another technical solution adopted by the present invention is as follows: A voice-changing system includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it performs the following steps: Acquire audio data and at least one target timbre; The audio data is preprocessed to obtain preprocessed audio data, and a deep learning model is used to separate the human voice in the preprocessed audio data to obtain the human voice signal. Based on the feature extraction model and the target timbre library, the human voice signal is timbre recognition is performed to obtain the timbre recognition result; Based on the timbre recognition result, a generative model is used to convert the human voice signal according to the target timbre to obtain an initial timbre output, and the initial timbre output is adjusted to obtain the final timbre output.

[0006] The beneficial effects of this invention are as follows: It acquires audio data and at least one target timbre; preprocesses the audio data to obtain preprocessed audio data; uses a deep learning model to separate the human voice from the preprocessed audio data to obtain a human voice signal; performs timbre recognition on the human voice signal based on a feature extraction model and a target timbre library to obtain a timbre recognition result; uses a generative model based on the timbre recognition result to convert the human voice signal according to the target timbre to obtain an initial timbre output; and adjusts the initial timbre output to obtain a final timbre output. By using a deep learning model to separate a pure human voice signal after preprocessing, the quality of the audio signal is improved. Then, after timbre recognition, a generative model is used for timbre conversion, which can accurately and efficiently convert the human voice signal into multiple target timbres. Finally, adjusting the initial timbre output can effectively optimize the timbre effect, achieving a natural sound quality, thereby improving the quality and flexibility of voice changing. Attached Figure Description

[0007] Figure 1 This is a flowchart illustrating the steps of a voice-changing method according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the structure of a voice-changing system according to an embodiment of the present invention. Detailed Implementation

[0008] To explain in detail the technical content, objectives, and effects of the present invention, the following description is provided in conjunction with the embodiments and accompanying drawings.

[0009] Please refer to Figure 1 A voice-changing method, comprising the following steps: Acquire audio data and at least one target timbre; The audio data is preprocessed to obtain preprocessed audio data, and a deep learning model is used to separate the human voice in the preprocessed audio data to obtain the human voice signal. Based on the feature extraction model and the target timbre library, the human voice signal is timbre recognition is performed to obtain the timbre recognition result; Based on the timbre recognition result, a generative model is used to convert the human voice signal according to the target timbre to obtain an initial timbre output, and the initial timbre output is adjusted to obtain the final timbre output.

[0010] As can be seen from the above description, the beneficial effects of the present invention are as follows: Audio data and at least one target timbre are acquired; the audio data is preprocessed to obtain preprocessed audio data; a deep learning model is used to separate the human voice in the preprocessed audio data to obtain a human voice signal; timbre recognition is performed on the human voice signal based on a feature extraction model and a target timbre library to obtain a timbre recognition result; a generative model is used to convert the human voice signal according to the target timbre based on the timbre recognition result to obtain an initial timbre output; and the initial timbre output is adjusted to obtain a final timbre output. Thus, by using a deep learning model to separate a pure human voice signal after preprocessing, the quality of the audio signal is improved; and by using a generative model to convert the timbre after timbre recognition, the human voice signal can be accurately and efficiently converted into multiple target timbres. Finally, adjusting the initial timbre output can effectively optimize the timbre effect, achieving a natural sound quality effect, thereby improving the quality and flexibility of voice changing.

[0011] Further, the preprocessing of the audio data to obtain preprocessed audio data includes: The audio data is denoised to obtain denoised audio data, and the denoised audio data is normalized to obtain normalized audio data. The background noise in the normalized audio data is initially filtered to obtain the filtered audio data. Signal processing techniques are used to convert the filtered audio data into a spectral representation to obtain preprocessed audio data.

[0012] As described above, performing noise reduction, volume normalization, and preliminary background noise filtering on audio data, followed by converting the audio data into a spectral representation, improves the quality of the audio data and facilitates subsequent processing.

[0013] Furthermore, the step of using a deep learning model to separate the human voice from the preprocessed audio data to obtain the human voice signal includes: The human voice in the preprocessed audio data is separated using a waveform U-shaped codec network model to obtain the human voice signal.

[0014] As described above, using a Wave-U-Net model to separate human voices from preprocessed audio data can effectively remove background music and environmental noise, obtaining a clean human voice signal to ensure the quality of subsequent timbre conversion.

[0015] Furthermore, the timbre recognition of the human voice signal based on the feature extraction model and the target timbre library, to obtain the timbre recognition result, includes: The human voice signal is used to extract features using a feature extraction model to obtain a timbre feature vector; The timbre category and personalized matching parameters of the human voice signal are determined from the target timbre library based on the timbre feature vector.

[0016] As described above, using a feature extraction model to extract features from human voice signals yields timbre feature vectors, which can reveal characteristics such as pitch, timbre, gender, and age. Then, based on the timbre feature vectors, the timbre category and personalized matching parameters of the human voice signal are determined from the target timbre library. Utilizing the timbre category and personalized matching parameters makes the final timbre conversion result more accurate and better meets user expectations.

[0017] Further, the step of using a generative model to convert the human voice signal according to the target timbre based on the timbre recognition result to obtain the initial timbre output includes: Based on the timbre category and personalized matching parameters, a generative adversarial network or variational autoencoder is used to convert the human voice signal according to the target timbre to obtain the initial timbre output.

[0018] As described above, using Generative Adversarial Networks (GANs) or Variational Autoencoders (VAEs) based on timbre category and personalized matching parameters to convert human voice signals according to the target timbre can accurately map the input timbre to the target timbre, achieving a natural and smooth voice-changing effect. This avoids the mechanical feel or distortion problems in traditional timbre processing methods, ensuring the quality of voice changing. The converted sound is more realistic and expressive, meeting the needs of various scenarios such as entertainment, education, and performance.

[0019] Furthermore, adjusting the initial timbre output to obtain the final timbre output includes: The spectral characteristics of the initial timbre output are calculated, and the spectral characteristics are compared with the spectrum of the target timbre to adjust the pitch and formant positions of the initial timbre output, thereby obtaining the adjusted timbre output; The adjusted timbre output is subjected to spectral smoothing to obtain the processed timbre output; The timbre style of the processed timbre output is optimized to obtain the optimized timbre output; Calculate the matching degree between the optimized timbre output and the target timbre; If the matching degree reaches a preset threshold, the optimized timbre output is used as the final timbre output. If not, the optimized timbre output is optimized using Bayesian optimization or a genetic algorithm, and then the step of calculating the spectral characteristics of the initial timbre output is returned to be executed until the matching degree reaches the preset threshold.

[0020] As described above, a good voice-changing effect has been achieved after the initial timbre conversion, but some problems may still exist. Therefore, the pitch and formant position of the initial timbre output are adjusted, spectral smoothing is performed, and the timbre style is optimized. When the matching degree does not reach the preset threshold, Bayesian optimization or genetic algorithm optimization is used on the optimized timbre output until the matching degree between the optimized timbre output and the target timbre reaches the preset threshold, so as to make the converted timbre more natural, smooth, and in line with the expected characteristics of the target timbre.

[0021] Furthermore, after adjusting the initial timbre output to obtain the final timbre output, the method further includes: The final timbre output is post-processed using audio smoothing filtering and audio equalization techniques to obtain the post-processed timbre output.

[0022] As described above, using audio smoothing filtering and audio equalization techniques to post-process the final timbre output can improve the naturalness and clarity of the sound, reduce mechanical or artificial traces, and thus improve the naturalness of voice changing.

[0023] Furthermore, after adjusting the initial timbre output to obtain the final timbre output, the method further includes: If a timbre fusion request is received, the final timbre output and the timbre to be fused in the timbre fusion request are aligned using spectral registration technology to obtain timbre alignment data. Then, the timbre alignment data is fused using a multi-track mixing algorithm to obtain timbre fusion data.

[0024] As described above, when a timbre fusion request is received, if it needs to be blended with background accompaniment or other timbres, spectral registration technology is used to align the final timbre output with the timbre to be blended in the timbre fusion request, avoiding abrupt or out-of-sync sounds. Then, a multi-track mixing algorithm is used to blend the aligned timbre data, ensuring a natural and harmonious fusion effect. Especially in accompaniment fusion scenarios, the user's changed voice timbre can be seamlessly integrated with the background accompaniment, producing high-quality audio effects and enhancing the overall listening experience.

[0025] Furthermore, after adjusting the initial timbre output to obtain the final timbre output, the method further includes: Receive user feedback on the final sound output; The final timbre output is dynamically adjusted based on the feedback information.

[0026] As described above, it can also receive user feedback and make further adjustments to the final timbre output based on the feedback, thereby better optimizing the output results and improving the personalized experience of voice changing.

[0027] Please refer to Figure 2 Another embodiment of the present invention provides a voice changing system, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the various steps in the above-described voice changing method.

[0028] The voice-changing method and system described above are applicable to various scenarios requiring voice changing, such as voice assistants and smart devices, virtual idols and entertainment applications, music creation and performance, and social media and content creation. The following detailed embodiments illustrate these applications: Please refer to Figure 1 Embodiment 1 of the present invention is as follows: A voice-changing method, comprising the following steps: S1. Acquire audio data and at least one target timbre.

[0029] The audio data can be humming, speech, or other types of audio input including human voice. The target timbre can be one or more. When there are multiple target timbres, the audio data is converted according to the target timbre to obtain multiple final timbre outputs.

[0030] S2. Preprocess the audio data to obtain preprocessed audio data, and use a deep learning model to separate the human voice in the preprocessed audio data to obtain the human voice signal, specifically including S21-S24: S21. The audio data is denoised to obtain denoised audio data, and the denoised audio data is normalized to obtain normalized audio data.

[0031] S22. Perform preliminary filtering on the background noise in the normalized audio data to obtain filtered audio data.

[0032] S23. Use signal processing technology to convert the filtered audio data into a spectral representation to obtain preprocessed audio data.

[0033] In one alternative implementation, the signal processing technique is Fourier transform or wavelet transform.

[0034] S24. Use a waveform U-shaped codec network model to separate the human voice in the preprocessed audio data to obtain the human voice signal.

[0035] Among them, the Wave-U-Net model is a deep learning model specifically designed for audio source separation and timbre conversion, particularly suitable for processing speech and music signals. Wave-U-Net can help separate noise from the original speech and extract clean human voices.

[0036] S3. Based on the feature extraction model and the target timbre library, perform timbre recognition on the human voice signal to obtain the timbre recognition result, specifically including S31-S32: S31. Use a feature extraction model to extract features from the human voice signal to obtain a timbre feature vector.

[0037] In some alternative implementations, the feature extraction model is a convolutional neural network (CNN) or a recurrent neural network (RNN). The timbre feature vector includes features such as pitch, timbre, gender, and age.

[0038] S32. Determine the timbre category and personalized matching parameters of the human voice signal from the target timbre library based on the timbre feature vector.

[0039] In one optional implementation, the target timbre library is used for timbre recognition, classification, and matching, and includes multiple reference timbre files. These reference timbre files are pre-recorded or extracted standard timbre samples that can be used as a benchmark for timbre recognition and conversion. The multiple reference timbre files may include standard speech samples of different genders, ages, and timbre styles.

[0040] For example, reference sound file 1 corresponds to the adult male sound category, reference sound file 2 corresponds to the adult female sound category, reference sound file 3 corresponds to the child sound category, and other reference sound files can correspond to the sound categories of cartoon sound, virtual character sound, instrument sound, etc.

[0041] Specifically, the similarity between each reference timbre file in the target timbre library and the human voice signal is calculated based on the timbre feature vector to determine the timbre category and personalized matching parameters of the human voice signal. For example, if the input timbre has the highest similarity to reference timbre file 1 (adult male), it will be classified as male timbre by default. If the input timbre is similar to both reference timbre file 2 (female) and reference timbre file 3 (child), further personalized adjustments may be required.

[0042] In one optional implementation, the reference timbre file in the target timbre library can also store spectral characteristics, formant information, and time-domain characteristics of various timbres, such as formant frequency data, Mel-frequency cepstral coefficients (MFCC), pitch distribution, and timbre envelope of the target timbre. By calculating the similarity between the input timbre and the reference timbre, the most suitable conversion direction and adjustment parameters are determined.

[0043] For example, the parameters of the reference timbre file can serve as a benchmark for the conversion target, ensuring that the output timbre meets expectations. For instance, when converting an adult male timbre to a child's timbre, the pitch, formant distribution, timbre envelope, and other information stored in the reference timbre file 3 will be referenced to adjust the parameters of the converted timbre, making it closer to the target timbre. Users can also manually adjust these parameters, such as making it "closer to a female timbre" or "more like a specific character."

[0044] In one alternative implementation, the target timbre library may further include target timbre vectors trained by models such as GAN or VAE.

[0045] For example, during model training, these reference timbre files can be used as part of the training dataset, providing high-quality target timbre samples.

[0046] In practical applications, the reference timbre library can be continuously expanded, for example, to include different age groups (such as the elderly and teenagers), to add stylized timbres (such as deep voices, anime characters, and robot timbres), to add multi-language support (such as different timbre characteristics of Chinese, Japanese, and English), to support users to define timbre goals, and to store personalized reference timbre files.

[0047] The personalized matching parameter is used to adjust the timbre conversion result to better match the user's expectations. The function of this parameter is to make fine adjustments during the voice changing process based on the user's preferences or needs, so as to ensure that the final timbre is more natural and meets personalized needs.

[0048] In one optional implementation, the personalized matching parameters may include the following: (1) Timbre feature parameters, used to match the user's input timbre and target timbre to ensure that the conversion result meets expectations, specifically including: Timbre Similarity: Measures the degree of similarity between an input timbre and a target timbre, and can be calculated using Mel-frequency cepstral coefficients (MFCC) and cosine similarity.

[0049] Formant Shift: Used to adjust the harmonic structure of a sound, making the converted sound closer to the target timbre (e.g., increasing the formant when converting an adult timbre to a child's timbre).

[0050] Pitch Matching: Ensures that the base pitch of the converted timbre matches the average pitch of the target timbre.

[0051] Energy Distribution: Adjusts the dynamic range of the timbre to make the converted timbre sound more realistic.

[0052] (2) User-personalized control parameters allow users to adjust the voice-changing effect according to their own needs, specifically including: Pitch Shift: Users can adjust the pitch of the converted sound, such as +2 semitones, -3 semitones, etc.

[0053] Timbre Style: Users can choose to prefer a certain style (such as deep, sharp, soft, magnetic).

[0054] Gender Adjustment: Allows users to adjust their voice from male to female, or from female to male.

[0055] Age Adjustment: Allows users to select the age range of the target tone (e.g., toddler, teenager, adult, senior).

[0056] Resonance Boost: Enhances the resonance of the timbre, making the sound more expressive, especially suitable for virtual broadcasters and musical performances.

[0057] (3) Audio fusion parameters: When the voice-changing result needs to be fused with background music and ambient sound, personalized matching parameters can be used to optimize the fusion effect, specifically including: Background Mixing Ratio: Controls the proportion of the altered voice timbre in the background music, such as 80% vocals and +20% accompaniment.

[0058] Dynamic EQ: Automatically adjusts the EQ curve based on timbre characteristics to make the converted timbre smoother.

[0059] Track Synchronization: Ensures that the audio track is time-aligned with the accompaniment or background music after voice change, avoiding timing discrepancies.

[0060] (4) Machine learning to optimize parameters: In order to further optimize the timbre conversion effect, the system can automatically adjust personalized parameters based on user habits and feedback, including: Adaptive Timbre Enhancement: Automatically optimizes the timbre style based on user feedback, making the conversion more in line with personal habits.

[0061] Target Timbre Match Score: Dynamically calculates the matching degree between the changed timbre and the target timbre, and provides personalized adjustment suggestions.

[0062] Noise Suppression Level: Allows users to control the intensity of background noise removal for a clearer voice changer effect.

[0063] S4. Based on the timbre recognition result, a generative model is used to convert the human voice signal according to the target timbre to obtain an initial timbre output, and the initial timbre output is adjusted to obtain the final timbre output, specifically including S41-S46: S41. Based on the timbre category and personalized matching parameters, a generative adversarial network or variational autoencoder is used to convert the human voice signal according to the target timbre to obtain an initial timbre output.

[0064] Among them, the generative adversarial network or variational autoencoder has been preprocessed and trained with a large number of target timbre samples during training, and can map the input timbre to the target timbre.

[0065] S42. Calculate the spectral characteristics of the initial timbre output and compare the spectral characteristics with the spectrum of the target timbre to adjust the pitch and formant position of the initial timbre output, thereby obtaining the adjusted timbre output.

[0066] In one alternative implementation, the spectral characteristics of the initial timbre output are calculated, and the formant frequency is adjusted to match the timbre quality of the target timbre. For example, when converting a male voice to a female voice, the formant frequency needs to be increased. The spectral characteristics (such as MFCC and spectral envelope) of the target timbre are compared and optimized for matching. Dynamic Time Warping (DTW) is used to ensure that the converted timbre is temporally aligned with the characteristics of the target timbre. Pitch Normalization is used to avoid unstable pitch shifts in the converted sound. Harmonic-to-Noise Ratio (HNR) analysis is used to enhance the clarity of the timbre and make the sound more realistic. Pitch Shift is applied to ensure that the converted sound does not lose its timbre characteristics due to fundamental frequency drift. After the above processing, the adjusted timbre output is obtained.

[0067] This step, through frequency and pitch adjustments, can make the spectral characteristics of the timbre closer to the target timbre, optimize the formants, make the sound more natural, and correct the pitch of the converted timbre so that it does not deviate from the key of the target timbre.

[0068] S43. Perform spectral smoothing on the adjusted timbre output to obtain the processed timbre output.

[0069] In one alternative implementation, spectral smoothing is used to remove any glitches or abrupt frequency bands that may occur during voice alteration. Adaptive equivalence (AI) is applied to adjust the brightness and clarity of the timbre, making it more natural. Post-processing models such as WaveNet or HiFi-GAN are then used to further enhance the sound quality, making the sound fuller and more realistic. This results in the processed timbre output.

[0070] This step reduces artifacts generated during the conversion process, resulting in a smoother tone.

[0071] S44. Optimize the timbre style of the processed timbre output to obtain the optimized timbre output.

[0072] In one alternative implementation, prosody modeling preserves the original timbre's emotional information, ensuring the correct emotion is conveyed even after voice alteration. Combined with StyleGAN for Speech, more nuanced timbre adjustments are achieved, such as a more magnetic or warmer style. Furthermore, users can further optimize the timbre by adjusting personalized parameters (such as breath control and resonance enhancement).

[0073] This step ensures that the emotional expression, tone, and timbre style of the changed voice meet the user's expectations.

[0074] S45. Calculate the matching degree between the optimized timbre output and the target timbre.

[0075] The matching degree is the MFCC similarity.

[0076] S46. Determine whether the matching degree has reached a preset threshold. If yes, use the optimized timbre output as the final timbre output. If no, optimize the optimized timbre output using Bayesian optimization or a genetic algorithm, and then return to execute S45 until the matching degree reaches the preset threshold.

[0077] After the initial timbre conversion, the converted timbre may still have some problems, such as: the spectral characteristics do not match the target timbre perfectly (e.g., the formant distribution of the sound is inconsistent with the target timbre); the pitch shift may not be precise enough, resulting in pitch deviation or instability; the sound quality may still have a mechanical feel and unnatural conversion traces; the timbre style may not fully match the target, for example, the converted sound may be too sharp or have lost its original resonance.

[0078] The converted timbre is further adjusted using S42-S46 to make it more natural, smooth, and in line with the expected characteristics of the target timbre.

[0079] In an alternative implementation, after S4, the following may also be included: The final timbre output is post-processed using audio smoothing filtering and audio equalization techniques to obtain the post-processed timbre output.

[0080] This improves the naturalness and clarity of the sound, reducing mechanical or artificial traces.

[0081] In an alternative implementation, after S4, the following may also be included: If a timbre fusion request is received, the final timbre output and the timbre to be fused in the timbre fusion request are aligned using spectral registration technology to obtain timbre alignment data. Then, the timbre alignment data is fused using a multi-track mixing algorithm to obtain timbre fusion data.

[0082] In one alternative implementation, the timbre to be blended can be background accompaniment or other timbres.

[0083] In an alternative implementation, after S4, the following may also be included: Receive user feedback on the final sound output; The final timbre output is dynamically adjusted based on the feedback information.

[0084] In one optional implementation, the feedback information includes timbre-related parameters and pitch and tone-related parameters. The timbre-related parameters determine the overall timbre style of the voice changer, which users can adjust according to their preferences. These include timbre brightness, formant adjustment, and timbre depth. Through timbre brightness, users can choose to make the voice brighter (high-frequency enhancement) or deeper (low-frequency enhancement). By adjusting the formants, the timbre can be changed without affecting the pitch, which can be used for gender or age conversion. Timbre depth adjusts the spatial sense of the timbre, making the voice sound heavier or lighter. The pitch and tone-related parameters are used to adjust the fundamental pitch, pitch stability, etc., including pitch adjustment, pitch stability, and speech rate and intonation control. Pitch adjustment changes the pitch after voice change, raising or lowering it by a few degrees to better match the user's desired timbre. Pitch stability optimizes pitch fluctuations after voice change, making the voice smoother and avoiding sudden changes in pitch. Speech rate and intonation control allows users to adjust their speech rate and intonation, making the voice changer more in line with their speaking habits.

[0085] When multiple target timbres exist, the system enables simultaneous output of multiple timbres, converting the user's input vocals into timbres appropriate for the target age and gender, as well as instrumental accompaniment timbres. The system utilizes multi-channel audio processing technology to ensure that each timbre is processed independently and maintains consistency during final fusion.

[0086] In one optional implementation, to ensure real-time voice-changing processing, parallel processing and optimized algorithm structures (such as GPU acceleration) are employed to reduce computational latency and ensure a smooth voice-changing experience for users in live streaming, interactive scenarios, and other similar situations. In high-concurrency scenarios (such as multiple users using the system simultaneously), the system uses distributed computing resources for load balancing to ensure low-latency processing even when multiple users are using the system at the same time.

[0087] The system stores users' personalized tone settings and usage preferences for quick recall in subsequent use, providing a customized experience. Users can also save the tone conversion results as audio files for easy editing or sharing later.

[0088] Through these specific implementation steps, the present invention can effectively achieve high-quality and flexible voice changing processing, support intelligent conversion and output of multiple timbres, and provide a good user experience in various application scenarios.

[0089] Specific application scenarios of the above-described method of the present invention include, but are not limited to: (1) Virtual Reality (VR) and Augmented Reality (AR): In VR / AR, this voice-changing method can provide virtual characters with a variety of timbres and sound effects, enhancing the immersive experience. For example, users can change their voices to become different characters in VR games, such as warriors or magicians, each with its own unique voice style. In AR teaching, the instructor's voice can be automatically adjusted to different timbres, enhancing the expressiveness of the teaching content and making it easier for students to immerse themselves in the learning scenario.

[0090] (2) Voice Assistants and Smart Homes: The above methods can be used in devices such as voice assistants and smart speakers, enabling smart devices to have rich tonal expression capabilities. For example, voice assistants can automatically switch between different tones (such as soft, formal, or entertaining tones) to respond based on user preferences or scenarios, enhancing interactivity and personalized experience. In smart homes, voice assistants can use voice changing to simulate the voices of family members, enhancing the device's sense of intimacy.

[0091] (3) Music creation and performance: AI voice changing technology provides a brand-new creative tool for music creation. For example, musicians can hum or record simple sound clips, convert them into the accompaniment timbres of different instruments, and integrate them into a complete musical work. In live performances, performers can switch between different instrument timbres through the real-time voice changing function to create diverse sound effects and enhance stage presence.

[0092] (4) Social media and content creation: In social media such as short video platforms, live streaming, and podcasts, voice changing technology can provide content creators with more possibilities for voice expression. For example, video creators can use voice changing functions to transform their voices into the voices of cartoon characters, animals, or other characters, adding fun and entertainment to their works. During live streaming, the host can switch timbres in real time, making the voice style more diverse and increasing the audience's sense of participation and interaction.

[0093] (5) Virtual anchors and virtual idols: The above methods can be widely applied to the creation of virtual anchors and virtual idols. For example, virtual anchors can use voice-changing technology to perform multiple roles and interact with the audience using different timbres to enhance the entertainment effect. When performing, virtual idols can switch timbres according to the needs of the scene to make their performance more diversified and enhance the audience's sense of participation and resonance.

[0094] (6) Interactive entertainment and games: In interactive entertainment and games, voice changing technology can be applied to a variety of sound effects production and character interaction. For example, players can change their voices to become different characters in multiplayer online games, enhancing the sense of immersion and personalized experience. Game developers can give NPC characters a variety of voice expressions, improving the immersion and interactivity of the game.

[0095] (7) Film dubbing and animation production: In film and animation production, the above methods can provide efficient and flexible solutions for character dubbing. For example, voice actors can use this technology to quickly switch between different timbres and dub multiple characters, saving production time and costs. Animators can provide unique sound effects for characters, enhancing their personality and expressiveness.

[0096] Through these application scenarios, this invention can be widely used in entertainment, education, social networking, medical and other fields, bringing more innovative audio experiences.

[0097] Please refer to Figure 2 Embodiment two of the present invention is as follows: A voice-changing system includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the various steps of the voice-changing method in Embodiment 1.

[0098] In summary, this invention provides a voice-changing method and system. It acquires audio data and at least one target timbre, preprocesses the audio data to obtain preprocessed audio data, uses a deep learning model to separate the human voice from the preprocessed audio data to obtain a human voice signal, performs timbre recognition on the human voice signal based on a feature extraction model and a target timbre library to obtain a timbre recognition result, uses a generative model based on the timbre recognition result to convert the human voice signal according to the target timbre to obtain an initial timbre output, and adjusts the initial timbre output to obtain a final timbre output. This method improves the quality of the audio signal by using a deep learning model to separate a pure human voice signal after preprocessing, and then uses a generative model for timbre conversion after timbre recognition, enabling accurate and efficient conversion of the human voice signal into multiple target timbres. Finally, adjusting the initial timbre output effectively optimizes the audio signal. The system achieves a natural sound quality through enhanced timbre effects, thereby improving the quality and flexibility of voice changing. Furthermore, a feature extraction model is used to extract features from the human voice signal, obtaining timbre feature vectors. This reveals characteristics such as pitch, timbre, gender, and age. Based on these timbre feature vectors, the timbre category and personalized matching parameters of the human voice signal are determined from a target timbre library. Utilizing timbre category and personalized matching parameters ensures a more accurate and user-predictable final timbre conversion result. Simultaneously, based on the timbre category and personalized matching parameters, GAN or VAE is used to convert the human voice signal according to the target timbre. This accurately maps the input timbre to the target timbre, achieving a natural and smooth voice changing effect. This avoids the mechanical feel or distortion problems of traditional timbre processing methods, ensuring voice changing quality. The converted sound is more realistic and expressive, meeting the needs of various scenarios such as entertainment, education, and performance.

[0099] The above description is merely an embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent modifications made based on the content of the present invention specification and drawings, or direct or indirect applications in related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A voice-changing method, characterized in that, Including the following steps: Acquire audio data and at least one target timbre; The audio data is preprocessed to obtain preprocessed audio data, and a deep learning model is used to separate the human voice in the preprocessed audio data to obtain the human voice signal; Based on the feature extraction model and the target timbre library, the human voice signal is timbre recognition is performed to obtain the timbre recognition result; Based on the timbre recognition result, a generative model is used to convert the human voice signal according to the target timbre to obtain an initial timbre output, and the initial timbre output is adjusted to obtain the final timbre output.

2. The voice-changing method according to claim 1, characterized in that, The preprocessing of the audio data to obtain preprocessed audio data includes: The audio data is denoised to obtain denoised audio data, and the denoised audio data is normalized to obtain normalized audio data. The background noise in the normalized audio data is initially filtered to obtain the filtered audio data. Signal processing techniques are used to convert the filtered audio data into a spectral representation to obtain preprocessed audio data.

3. The voice-changing method according to claim 1, characterized in that, The step of using a deep learning model to separate human voices from the preprocessed audio data to obtain human voice signals includes: The human voice in the preprocessed audio data is separated using a waveform U-shaped codec network model to obtain the human voice signal.

4. The voice-changing method according to claim 1, characterized in that, The timbre recognition of the human voice signal based on the feature extraction model and the target timbre library yields the following results: The human voice signal is used to extract features using a feature extraction model to obtain a timbre feature vector; The timbre category and personalized matching parameters of the human voice signal are determined from the target timbre library based on the timbre feature vector.

5. The voice-changing method according to claim 4, characterized in that, The step of using a generative model to convert the human voice signal according to the target timbre based on the timbre recognition result to obtain the initial timbre output includes: Based on the timbre category and personalized matching parameters, a generative adversarial network or variational autoencoder is used to convert the human voice signal according to the target timbre to obtain the initial timbre output.

6. The voice-changing method according to claim 1, characterized in that, The adjustment of the initial timbre output to obtain the final timbre output includes: The spectral characteristics of the initial timbre output are calculated, and the spectral characteristics are compared with the spectrum of the target timbre to adjust the pitch and formant positions of the initial timbre output, thereby obtaining the adjusted timbre output; The adjusted timbre output is subjected to spectral smoothing to obtain the processed timbre output; The timbre style of the processed timbre output is optimized to obtain the optimized timbre output; Calculate the matching degree between the optimized timbre output and the target timbre; If the matching degree reaches a preset threshold, the optimized timbre output is used as the final timbre output. If not, the optimized timbre output is optimized using Bayesian optimization or a genetic algorithm, and then the step of calculating the spectral characteristics of the initial timbre output is returned to be executed until the matching degree reaches the preset threshold.

7. The voice-changing method according to claim 1, characterized in that, After adjusting the initial timbre output to obtain the final timbre output, the method further includes: The final timbre output is post-processed using audio smoothing filtering and audio equalization techniques to obtain the post-processed timbre output.

8. The voice-changing method according to claim 1, characterized in that, After adjusting the initial timbre output to obtain the final timbre output, the method further includes: If a timbre fusion request is received, the final timbre output and the timbre to be fused in the timbre fusion request are aligned using spectral registration technology to obtain timbre alignment data. Then, the timbre alignment data is fused using a multi-track mixing algorithm to obtain timbre fusion data.

9. The voice-changing method according to claim 1, characterized in that, After adjusting the initial timbre output to obtain the final timbre output, the method further includes: Receive user feedback on the final sound output; The final timbre output is dynamically adjusted based on the feedback information.

10. A voice-changing system, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements each step of the voice-changing method according to any one of claims 1 to 9.

Images