Methods for generating singing voices, computer equipment, and storage media
By acquiring and processing musical score information, using a pre-trained model to generate audio of the score and correct out-of-tune singing, the problem of poor matching between singing and musical score in traditional singing synthesis methods is solved, and more natural singing is generated.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-04-26
- Publication Date
- 2026-04-03
AI Technical Summary
Traditional vocal synthesis methods rely heavily on the original singer, resulting in poor matching between the synthesized vocals and the musical score, thus leading to insufficient naturalness in the generated vocals.
By acquiring the syllable, note, and tuplet information of the target musical score, and using a pre-trained score vector transformation model and feature determination model, the phoneme encoding features, phoneme duration, and phoneme-level fundamental frequency of the score are determined. The score audio is then generated, and frame-level fundamental frequency and phoneme information are processed to correct out-of-tune phenomena. Finally, the singing voice is generated by combining timbre information.
It improves the fit between the singing and the musical score, resulting in a more natural singing voice and enhancing the naturalness and accuracy of the singing.
Smart Images
Figure CN116486765B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of computer technology, and in particular to a method, apparatus, computer device, storage medium, and computer program product for generating singing voice. Background Technology
[0002] With the development of computer technology, vocal synthesis technology has emerged in the music field, which can synthesize audio based on lyrics and sheet music.
[0003] Traditional vocal synthesis methods can create musical scores based on existing audio. The fundamental frequency comes from the human voice portion of the audio, while the phonemes are obtained from the audio through a forced alignment algorithm.
[0004] However, traditional vocal synthesis methods rely heavily on the original singer and the synthesized vocals do not match the musical score well, which is not conducive to improving the naturalness of the generated vocals. Summary of the Invention
[0005] Therefore, it is necessary to provide a singing voice generation method, apparatus, computer equipment, computer-readable storage medium, and computer program product that can improve the naturalness of generated singing voices in response to the above-mentioned technical problems.
[0006] Firstly, this application provides a method for generating singing voice, the method comprising:
[0007] Obtain syllable information, note information, and tuplet information of the target musical score;
[0008] Based on the syllable information, the note information, and the tuplet information, determine the musical notation phoneme encoding features, musical notation phoneme duration, and phoneme level fundamental frequency of the target musical score;
[0009] Based on the musical score phoneme encoding features, the musical score phoneme duration, and the phoneme level fundamental frequency, generate the musical score audio corresponding to the target musical score;
[0010] Obtain the frame-level fundamental frequency corresponding to the music score audio and the music score phoneme information corresponding to the target music score, and generate the target singing audio based on the frame-level fundamental frequency and the music score phoneme information.
[0011] In one embodiment, determining the musical notation phoneme encoding features, musical notation phoneme duration, and phoneme-level fundamental frequency of the target musical score based on the syllable information, the note information, and the truncation information includes:
[0012] The syllable information, the note information, and the tuplet information are input into a pre-trained music score vector conversion model to obtain the music score feature vector of the target music score;
[0013] The musical score feature vector is input into a pre-trained musical score feature determination model to obtain the musical score phoneme encoding features, musical score phoneme duration, and phoneme level fundamental frequency of the target musical score.
[0014] In one embodiment, the music score feature determination model includes a pre-trained music score synthesis encoder, a phoneme duration determination model, and a fundamental frequency determination model. The step of inputting the music score feature vector into the pre-trained music score feature determination model to obtain the music score phoneme encoding features, music score phoneme duration, and phoneme-level fundamental frequency of the target music score includes:
[0015] The musical score feature vector is input into the musical score synthesis encoder to obtain the musical score encoding features of the target musical score;
[0016] The music score encoding features are input into the phoneme duration determination model to obtain the music score phoneme duration, and the music score encoding features are input into the fundamental frequency determination model to obtain the phoneme level fundamental frequency.
[0017] In one embodiment, generating the audio of the target score based on the score phoneme encoding features, the score phoneme duration, and the phoneme level fundamental frequency includes:
[0018] The musical score phoneme encoding features, the musical score phoneme duration, and the phoneme level fundamental frequency are input into a pre-trained musical score synthesis decoder to obtain the musical score acoustic features corresponding to the target musical score.
[0019] The acoustic features of the musical score are input into a vocoder to obtain the audio of the musical score corresponding to the target musical score.
[0020] In one embodiment, before the step of obtaining the frame-level fundamental frequency and musical note phoneme information corresponding to the musical score audio, the method further includes:
[0021] Determine the fundamental frequency deviation information of the musical score audio; the fundamental frequency deviation information is used to characterize the degree of deviation between the fundamental frequency information in the musical score audio and the fundamental frequency information of the target musical score;
[0022] If it is determined that the score audio is out of tune based on the fundamental frequency deviation information, the score audio is corrected according to the fundamental frequency deviation information to obtain the corrected score audio, which is used as the new score audio.
[0023] In one embodiment, determining the fundamental frequency deviation information of the musical score audio includes:
[0024] The phoneme-level fundamental frequency is input into a moving average filter to obtain the phoneme-level fundamental frequency mean value;
[0025] Determine the fundamental frequency deviation between the mean fundamental frequency of the phoneme and the standard pitch of the target musical score;
[0026] The fundamental frequency deviation is used as the fundamental frequency deviation information.
[0027] In one embodiment, the step of performing pitch correction processing on the score audio based on the fundamental frequency deviation information to obtain the pitch-corrected score audio includes:
[0028] Based on the fundamental frequency deviation information, determine the pitch compensation value for the musical score audio;
[0029] The frame-level fundamental frequency of the musical score audio is corrected based on the pitch compensation value to obtain the corrected frame-level fundamental frequency;
[0030] The corrected audio of the musical score is determined based on the corrected frame-level fundamental frequency.
[0031] In one embodiment, determining the pitch-corrected score audio based on the corrected frame-level fundamental frequency includes:
[0032] Determine the position of the corrected frame-level fundamental frequency in the fundamental frequency sequence of the musical score;
[0033] Determine the fundamental frequency connection node corresponding to the fundamental frequency correction position;
[0034] The fundamental frequency connection node is subjected to transition processing to obtain the transition-processed fundamental frequency sequence;
[0035] The corrected musical score audio is generated based on the fundamental frequency sequence after the transition processing.
[0036] In one embodiment, the method further includes:
[0037] Obtain the modified sheet music information; the modified sheet music information is determined in response to a change operation on the target sheet music; the modified sheet music information includes at least one of lyrics or melody;
[0038] Based on the modified score information, the target score is updated to obtain the updated target score, which serves as the new target score.
[0039] In one embodiment, the method further includes:
[0040] Obtain the timbre information corresponding to the target object;
[0041] Based on the frame-level fundamental frequency, the musical score phoneme information, and the timbre information, the target singing audio corresponding to the target object is generated.
[0042] Secondly, this application also provides a singing voice generation device, the device comprising:
[0043] The information acquisition module is used to acquire syllable information, note information, and legato information of the target musical score;
[0044] The feature determination module is used to determine the music score phoneme encoding features, music score phoneme duration, and phoneme level fundamental frequency of the target music score based on the syllable information, the note information, and the tuplet information;
[0045] An audio generation module is used to generate the audio of the target score based on the phoneme encoding features of the score, the phoneme duration of the score, and the fundamental frequency of the phoneme level.
[0046] The vocal synthesis module is used to obtain the frame-level fundamental frequency corresponding to the music score audio and the music score phoneme information corresponding to the target music score, and generate the target vocal audio based on the frame-level fundamental frequency and the music score phoneme information.
[0047] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program that, when executed by the processor, implements the steps of the method described above.
[0048] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the steps of the above-described method.
[0049] Fifthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the steps of the above-described method.
[0050] The aforementioned singing generation method, apparatus, computer equipment, storage medium, and computer program product obtain the parsed score information by acquiring syllable information, note information, and tether information of the target score. Based on the syllable information, note information, and tether information, they determine the score's phoneme encoding features, phoneme duration, and phoneme-level fundamental frequency, thereby determining the score's phoneme characteristics, phoneme duration characteristics, and phoneme-level fundamental frequency characteristics. Based on these phoneme encoding features, phoneme duration, and phoneme-level fundamental frequency, they generate the corresponding audio of the target score, thus generating audio based on the target score and acquiring the singing voice. The system uses the frame-level fundamental frequency corresponding to the audio score and the phoneme information of the target score to generate the target vocal audio. This process involves analyzing the score to determine the syllable, note, and legato information, determining the phoneme encoding features, phoneme duration, and phoneme-level fundamental frequency, and generating the target score audio based on these features. Then, based on the fundamental frequency features of the audio score and the phoneme information of the target score, a vocal performance with a preset timbre is generated, resulting in a vocal performance with a high degree of consistency with the target score and improving the naturalness of the synthesized vocal performance. Attached Figure Description
[0051] Figure 1 This is a flowchart illustrating a singing voice generation method in one embodiment;
[0052] Figure 2 This is a schematic diagram of a musical score information in one embodiment;
[0053] Figure 3 This is a schematic diagram of a music score synthesis system in one embodiment;
[0054] Figure 4 This is a schematic diagram of a pitch detection system in one embodiment;
[0055] Figure 5 This is a schematic diagram of the fundamental frequency curve of a phoneme in one embodiment;
[0056] Figure 6 This is a schematic diagram of a singing voice synthesis system in one embodiment;
[0057] Figure 7 This is a flowchart illustrating a singing voice generation method in another embodiment;
[0058] Figure 8 This is a structural block diagram of a singing voice generation device in one embodiment;
[0059] Figure 9 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation
[0060] To make the objectives, technical solutions, and advantages of this application clearer and more understandable, the following further elaborates on this application in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely used to explain this application and are not used to limit this application.
[0061] It can be understood that this method can also be applied to a server, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server.
[0062] In some embodiments, as Figure 1 shown, a singing generation method is provided. In this embodiment, an example is given where this method is applied to an electronic device. It can be understood that this method can also be applied to a server, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. Among them, the electronic device can be a terminal. In this embodiment, the method includes the following steps:
[0063] Step S102, obtain the syllable information, note information, and tie note information of the target musical score.
[0064] Among them, the target musical score can refer to the musical score corresponding to the musical score information required for generating singing. In practical applications, the target musical score can include the original musical score with unmodified musical score information, the modified musical score after modifying the lyrics and / or modifying the melody. Common musical score formats can include midi and musicxml. The target musical score can include lyrics and pitch information.
[0065] Among them, the syllable information can refer to the phonetic unit formed by the combination of phonemes (including consonants and vowels) in the musical score. For example, the syllable of the character "好" is hao. In practical applications, the syllable information can include the syllable and the time corresponding to the syllable, start, end, syl.
[0066] Among them, the note information can refer to the pitch information. For example: C4, D4, etc. In practical applications, the note information can include the note and the time corresponding to the note.
[0067] Among them, the tie note information can refer to a special form of rhythm division. In practical applications, the tie note can evenly divide the time value of the note freely, and the number thereof is inconsistent with the basic division. <000015As shown, after the electronic device parses the target musical score, the parsed musical score obtained by the electronic device may include several musical score units. The combination form of each musical score unit can be represented as: word_syllable_pitch. The format of the parsed musical score is convenient for the parsing program to parse the musical score.
[0069] Step S104: Based on syllable information, note information, and tuplet information, determine the musical score phoneme encoding features, musical score phoneme duration, and phoneme level fundamental frequency of the target score.
[0070] Among them, the musical score phoneme encoding feature can refer to the encoding that represents the phoneme features of the musical score output by the encoder when extracting features from the musical score.
[0071] Among them, the duration of a musical score phoneme can refer to the information representing the duration of each phoneme in the musical score output by the phoneme duration prediction module when extracting features from the musical score.
[0072] Among them, the phoneme-level fundamental frequency can refer to the phoneme-level fundamental frequency of each phoneme in the score output by the fundamental frequency prediction module when the score is feature extracted based on the fundamental frequency extraction technology.
[0073] As an example, the electronic device inputs syllable information, note information, and tuplet information into the score vector conversion model to obtain the score feature vector. The electronic device then inputs the score feature vector into a pre-trained encoder. The electronic device inputs the encoder's output into a phoneme duration prediction module to obtain the score phoneme duration. Finally, the electronic device inputs the encoder's output into a pre-trained fundamental frequency prediction module to obtain the phoneme-level fundamental frequency. The encoder's output can be used as the score phoneme encoding feature.
[0074] Step S106: Generate the audio of the target score based on the phoneme encoding features, phoneme duration, and phoneme level fundamental frequency.
[0075] Among them, the music score audio can refer to the audio obtained by a music score synthesis system based on the target music score.
[0076] As an example, when the encoder output is the phoneme encoding feature of the musical score, the electronic device expands the encoder output, the musical score phoneme duration, and the phoneme level fundamental frequency to obtain the expanded result. The electronic device inputs the expanded result into the decoder to obtain the decoding result. The electronic device inputs the decoding result into the linear layer to obtain the acoustic feature of the target musical score. The electronic device inputs the acoustic feature of the target musical score into the vocoder to obtain the musical score audio corresponding to the target musical score.
[0077] Step S108: Obtain the frame-level fundamental frequency corresponding to the music score audio and the music score phoneme information corresponding to the target music score, and generate the target singing audio based on the frame-level fundamental frequency and music score phoneme information.
[0078] Among them, the frame-level fundamental frequency can refer to the fundamental frequency obtained by extracting the fundamental frequency of the score audio frame by frame based on the fundamental frequency extraction technology. In practical applications, when processing audio signals, the audio can be divided into frames at the ms level for fundamental frequency detection. Generally, the length of each frame is 256 or 512 sampling points. Combining with the sampling frequency, duration = frame length / sampling rate.
[0079] Among them, the score phoneme information can refer to the smallest speech unit divided according to the natural attributes of speech in the target score. It is analyzed based on the pronunciation actions in the syllable. For example, the character "好" contains two phonemes, "h" and "ao". In practical applications, the score phoneme information can include phonemes and the corresponding time of the phonemes.
[0080] Among them, the target singing audio can refer to the audio output by the singing synthesis system that sings the target score with the system's preset timbre.
[0081] As an example, the electronic device extracts the fundamental frequency of the score audio frame by frame to obtain the frame-level fundamental frequency corresponding to the score audio. The electronic device analyzes the score audio or the target score to obtain the score phoneme information. The electronic device inputs the frame-level fundamental frequency and the score phoneme information into the singing synthesis system to obtain the audio of singing the target score with the target preset timbre.
[0082] In the above singing generation method, by obtaining the syllable information, note information, and tie note information of the target score, the score information after parsing the score is obtained. According to the syllable information, note information, and tie note information, the score phoneme coding feature, score phoneme duration, and phoneme-level fundamental frequency of the target score are determined, so as to determine the phoneme feature, phoneme duration feature, and phoneme-level fundamental frequency feature of the score. According to the score phoneme coding feature, score phoneme duration, and phoneme-level fundamental frequency, the score audio corresponding to the target score is generated, so as to generate the audio based on the target score. The frame-level fundamental frequency corresponding to the score audio and the score phoneme information corresponding to the target score are obtained. According to the frame-level fundamental frequency and the score phoneme information, the target singing audio is generated, realizing the determination of the syllable information, note information, and tie note information of the score by parsing the score, determining the score phoneme coding feature, score phoneme duration, and phoneme-level fundamental frequency of the score according to the parsed score information, generating the score audio of the target score according to the score features, generating the singing of singing the target score with the preset timbre based on the fundamental frequency feature of the score audio and the phoneme information of the target score, obtaining the singing with a high degree of fit with the target score, and improving the naturalness of the synthesized singing.
[0083] In some embodiments, determining the musical notation phoneme encoding features, musical notation phoneme durations, and phoneme-level fundamental frequencies of a target musical score based on syllable information, note information, and tethered note information includes: inputting syllable information, note information, and tethered note information into a pre-trained musical notation vector conversion model to obtain a musical notation feature vector of the target musical score; and inputting the musical notation feature vector into a pre-trained musical notation feature determination model to obtain the musical notation phoneme encoding features, musical notation phoneme durations, and phoneme-level fundamental frequencies of the target musical score.
[0084] Among them, the musical score vector conversion model can refer to a way to convert discrete variables into continuous vector representations. In practical applications, embedding can be used in neural networks to convert discrete variables into continuous vectors. Embedding can not only reduce the spatial dimension of discrete variables, but also represent variables meaningfully.
[0085] Among them, the musical score feature vector can refer to the vector output by the musical score vector conversion model, which is used to characterize the musical score features in terms of syllables, notes, tuplets, phonemes, fundamental frequency, etc.
[0086] Among them, the music score feature determination model can refer to the model used to extract features such as music score phonemes and fundamental frequency from the music score feature vector. In practical applications, the music score feature determination model can include an encoder, a phoneme duration prediction module and a fundamental frequency prediction module.
[0087] As an example, the electronic device inputs the syllable information, note information, and tuplet information obtained from parsing the target score into a pre-trained score vector conversion model. The electronic device uses the output of the score vector conversion model as the score feature vector of the target score. The electronic device inputs the score feature vector into a pre-trained score feature determination model. The output of the score feature determination model can represent the score phoneme encoding features, score phoneme duration, and phoneme level fundamental frequency of the target score, respectively.
[0088] In this embodiment, by inputting syllable information, note information, and tuplet information into a pre-trained score vector conversion model, the score feature vector of the target score is obtained. The score feature vector is then input into a pre-trained score feature determination model to obtain the score phoneme encoding features, score phoneme duration, and phoneme level fundamental frequency of the target score. This enables feature extraction of the score information obtained from parsing the score, acquisition of key score features, removal of irrelevant information from the score features, and improvement of the accuracy of score features.
[0089] In some embodiments, the music score feature determination model includes a pre-trained music score synthesis encoder, a phoneme duration determination model, and a fundamental frequency determination model. The music score feature vector is input into the pre-trained music score feature determination model to obtain the music score phoneme encoding features, music score phoneme duration, and phoneme-level fundamental frequency of the target music score. This includes: inputting the music score feature vector into the music score synthesis encoder to obtain the music score encoding features of the target music score; inputting the music score encoding features into the phoneme duration determination model to obtain the music score phoneme duration; and inputting the music score encoding features into the fundamental frequency determination model to obtain the phoneme-level fundamental frequency.
[0090] Among them, a music score synthesizer encoder can refer to a device that encodes signals (such as bit streams) or data and converts them into a signal form that can be used for communication, transmission and storage. In practical applications, the music score synthesizer encoder can determine the music score phoneme encoding features based on the music score feature vector in the music score synthesis system.
[0091] Among them, the phoneme duration determination model can refer to a model used to predict the number of frames that should be allocated to each phoneme in the musical score. For example, if the input syllable is "jin tian hao", and the number of frames corresponding to each syllable is 5, 5, and 8, the phoneme duration determination model can break down the above syllable into the phoneme "j in tian hao", and the frame length allocated to each phoneme can be 2, 3, 2, 3, 3, and 5.
[0092] Among them, the fundamental frequency determination model can refer to the model that extracts or detects the fundamental frequency of a musical score. In practical applications, the fundamental frequency determination model can predict the fundamental frequency at the phoneme level, that is, each phoneme corresponds to a pitch.
[0093] As an example, Figure 3 A music score synthesis system is proposed, such as Figure 3 As shown, the electronic device inputs syllable information, note information, and truncation information into a pre-trained music score vector conversion model (embedding). The electronic device uses the output of the music score vector conversion model as the music score feature vector. The electronic device inputs the music score feature vector into a pre-trained music score synthesis encoder to obtain the music score encoding features of the target music score. The electronic device inputs the music score encoding features into a phoneme duration determination model (phoneme duration prediction module). The electronic device uses the output of the phoneme duration determination model as the music score phoneme duration. The electronic device inputs the music score encoding features into a fundamental frequency determination model (fundamental frequency prediction module). The electronic device uses the output of the fundamental frequency determination model as the phoneme-level fundamental frequency. In this case, the music score synthesis system places the fundamental frequency prediction module before expansion, predicting the phoneme-level fundamental frequency. That is, each phoneme corresponds to a pitch. This can ensure the accuracy of the overall pitch of the phonemes while giving a certain tolerance to the fundamental frequency at the phoneme intra-frame level.
[0094] In this embodiment, the music score feature vector is input into the music score synthesis encoder to obtain the music score encoding features of the target music score; the music score encoding features are input into the phoneme duration determination model to obtain the music score phoneme duration; the music score encoding features are input into the fundamental frequency determination model to obtain the phoneme level fundamental frequency. The music score encoding features can be processed based on the pre-trained module to obtain the music score phoneme duration and phoneme level fundamental frequency, thereby improving the accuracy of the phoneme duration and phoneme level fundamental frequency.
[0095] In some embodiments, generating the audio of a target score based on the score phoneme encoding features, score phoneme duration, and phoneme-level fundamental frequency includes: inputting the score phoneme encoding features, score phoneme duration, and phoneme-level fundamental frequency into a pre-trained score synthesis decoder to obtain the score acoustic features corresponding to the target score; and inputting the score acoustic features into a vocoder to obtain the audio of the target score.
[0096] Among them, a music score synthesizer / decoder can refer to a hardware / software device that decodes and restores digital audio and video data streams into analog audio and video signals.
[0097] Among them, the acoustic characteristics of a musical score can refer to the physical quantities that characterize the acoustic properties of a musical score.
[0098] A vocoder can be a system that converts acoustic features into playable speech waveforms.
[0099] As an example, such as Figure 3 As shown, the electronic device expands the phoneme encoding features, phoneme duration, and phoneme-level fundamental frequency of the musical score. The electronic device then inputs the expanded phoneme encoding features, phoneme duration, and phoneme-level fundamental frequency into a pre-trained musical score synthesizer. The electronic device inputs the decoding result of the musical score synthesizer into a linear layer to obtain the acoustic features of the target musical score. The electronic device then converts the acoustic features of the musical score into the corresponding audio of the target musical score through a vocoder.
[0100] In this embodiment, by inputting the musical score phoneme encoding features, musical score phoneme duration, and phoneme level fundamental frequency into a pre-trained musical score synthesizer, the acoustic features of the target musical score are obtained; the acoustic features of the musical score are then input into a vocoder to obtain the audio of the target musical score. Based on the acoustic features of the musical score determined by the musical score information, the audio of the target musical score can be generated by the vocoder, thereby improving the fit between the audio and the musical score.
[0101] In some embodiments, before the step of obtaining the frame-level fundamental frequency and musical note phoneme information corresponding to the musical score audio, the above method further includes: determining the fundamental frequency deviation information of the musical score audio; if it is determined that the musical score audio is out of tune based on the fundamental frequency deviation information, performing pitch correction processing on the musical score audio based on the fundamental frequency deviation information to obtain the pitch-corrected musical score audio as a new musical score audio.
[0102] Among them, the fundamental frequency deviation information is used to characterize the degree of deviation between the fundamental frequency information in the music score audio and the fundamental frequency information of the target music score.
[0103] As an example, Figure 4 A pitch detection system is proposed, such as Figure 4 As shown, the electronic device performs frame-level fundamental frequency detection on the music score audio to obtain the frame-level fundamental frequency. Based on the syllable and note information of the target music score and the frame-level fundamental frequency, the electronic device estimates the pitch deviation of the music score audio to determine the fundamental frequency deviation information. Based on the fundamental frequency deviation information, the electronic device determines whether the music score audio is out of tune. When the electronic device determines that the music score audio is out of tune based on the fundamental frequency deviation information, it performs pitch correction processing on the music score audio to obtain a pitch-corrected music score audio. The electronic device uses the pitch-corrected music score audio as the new music score audio, which includes the pitch-corrected frame-level fundamental frequency. The electronic device generates the target object's singing voice based on the pitch-corrected music score audio, music score phoneme information, and the timbre information of the target object. When the electronic device determines that the music score audio is not out of tune based on the fundamental frequency deviation information, it directly outputs the frame-level fundamental frequency. The electronic device then generates the target object's singing voice based on the frame-level fundamental frequency of the music score audio, music score phoneme information, and the timbre information of the target object.
[0104] In this embodiment, the fundamental frequency deviation information of the musical score audio is determined. If it is determined that the musical score audio is out of tune based on the fundamental frequency deviation information, the musical score audio is corrected according to the fundamental frequency deviation information to obtain the corrected musical score audio as a new musical score audio. This allows for pitch detection and correction of the musical score audio based on the musical score, thereby improving the accuracy of the musical score audio.
[0105] In some embodiments, determining the fundamental frequency deviation information of the musical score audio includes: inputting the phoneme-level fundamental frequency into a moving average filter to obtain the phoneme-level fundamental frequency mean; determining the fundamental frequency deviation between the phoneme-level fundamental frequency mean and the standard pitch of the target musical score; and using the fundamental frequency deviation as the fundamental frequency deviation information.
[0106] Among them, the moving average filter can refer to a type of digital filter. In practical applications, the moving average filter can effectively suppress noise and smooth changes in the input signal, thereby improving the signal-to-noise ratio of the signal.
[0107] The mean fundamental frequency within a phoneme can refer to the numerical value representing the pitch of a phoneme in the fundamental frequency curve. In practical applications, the mean fundamental frequency within a phoneme can be represented as note_hat, and the mean fundamental frequency within a phoneme can include the average value of the fundamental frequency within the phoneme.
[0108] Among them, the fundamental frequency deviation can refer to the degree of deviation between the mean fundamental frequency of the phonemes in the audio of the musical score and the standard pitch of the target musical score.
[0109] As an example, Figure 5 A schematic diagram of the fundamental frequency curve within a phoneme is proposed, such as... Figure 5 As shown, taking the phoneme "ai" as an example, the electronic device inputs the phoneme-level fundamental frequency of the musical score into a moving average filter to obtain the mean fundamental frequency (note_hat) within the phoneme. The standard pitch marked in the musical score can be represented as note_ori = 56. Based on the mean fundamental frequency within the phoneme and the standard pitch of the corresponding phoneme in the musical score, the electronic device determines the fundamental frequency deviation between the mean fundamental frequency within the phoneme and the standard pitch of the target musical score. The electronic device uses the fundamental frequency deviation as fundamental frequency deviation information. Therefore, the fundamental frequency deviation of the phoneme "ai" can be represented as note_hat. _diff = note_hat - note_ori. Generally, note_diff < 0.5 is considered to be in tune. 0.5 refers to half a semitone. One semitone = 100 cents, so 0.5 semitones is 50 cents. 50 cents is a tolerance threshold. The mean fundamental frequency within a phoneme can be calculated using a moving average filter or simply by using the mean. The mean fundamental frequency within a phoneme can represent the actual fundamental frequency value of the audio. The actual fundamental frequency value of the audio and the phoneme-level fundamental frequency predicted by the model are not equivalent.
[0110] In this embodiment, by inputting the phoneme-level fundamental frequency into a moving average filter, the mean fundamental frequency within the phoneme is obtained; the fundamental frequency deviation between the mean fundamental frequency within the phoneme and the standard pitch of the target musical score is determined; using the fundamental frequency deviation as fundamental frequency deviation information, the mean fundamental frequency within the phoneme can be determined based on the phoneme-level fundamental frequency, avoiding the influence of extreme fundamental frequency values on the fundamental frequency deviation, and improving the accuracy of the fundamental frequency deviation information.
[0111] In some embodiments, the score audio is tuned according to the fundamental frequency deviation information to obtain tuned score audio, including: determining the pitch compensation value for the score audio according to the fundamental frequency deviation information; correcting the frame-level fundamental frequency of the score audio according to the pitch compensation value to obtain the corrected frame-level fundamental frequency; and determining the tuned score audio according to the corrected frame-level fundamental frequency.
[0112] Among them, the pitch compensation value can refer to the modification value of the fundamental frequency at the frame level when the audio of the musical score needs to be corrected. In practical applications, the pitch compensation value can increase and / or decrease the fundamental frequency.
[0113] As an example, if an electronic device detects a pitch deviation within a phoneme, it determines the pitch compensation value for the musical score audio based on the fundamental frequency deviation information. The electronic device then corrects the frame-level fundamental frequency of the musical score audio based on this compensation value, pulling the out-of-tone pitch back into the correct range, resulting in the corrected frame-level fundamental frequency. Based on this corrected frame-level fundamental frequency, the electronic device determines the corrected musical score audio. For example, the frame-level fundamental frequency can be represented as `frame_note`, and the pitch compensation value as `note_diff`. Therefore, the corrected frame-level fundamental frequency can be represented as `new_note = frame_note + note_diff`. Based on the pitch compensation value `note_diff`, the electronic device performs a complete translation operation on the frame-level fundamental frequency `frame_note` to obtain the corrected frame-level fundamental frequency `new_note`, thus obtaining a new fundamental frequency sequence that meets the pitch requirements of the target musical score.
[0114] In this embodiment, the pitch compensation value for the score audio is determined based on the fundamental frequency deviation information; the frame-level fundamental frequency of the score audio is corrected based on the pitch compensation value to obtain the corrected frame-level fundamental frequency; and the corrected score audio is determined based on the corrected frame-level fundamental frequency. This allows for the determination of an accurate pitch compensation value based on the fundamental frequency deviation information, thereby improving the accuracy of the frame-level fundamental frequency of the score audio.
[0115] In some embodiments, determining the corrected score audio based on the corrected frame-level fundamental frequency includes: determining the fundamental frequency correction position of the corrected frame-level fundamental frequency in the score fundamental frequency sequence; determining the fundamental frequency transition node corresponding to the fundamental frequency correction position; performing transition processing on the fundamental frequency transition node to obtain a transitioned fundamental frequency sequence; and generating the corrected score audio based on the transitioned fundamental frequency sequence.
[0116] Among them, the fundamental frequency sequence of the musical score can refer to the sequence composed of the fundamental frequencies at the frame level of the musical score.
[0117] The fundamental frequency correction position can refer to the position of the corrected frame-level fundamental frequency in the fundamental frequency sequence of the musical score.
[0118] Here, the fundamental frequency connection node can refer to the position of the fundamental frequency in the musical score fundamental frequency sequence that is connected to the corrected frame-level fundamental frequency.
[0119] As an example, after correcting the frame-level fundamental frequency, the electronic device determines the position of the corrected frame-level fundamental frequency in the fundamental frequency sequence of the musical score. The electronic device uses the position of the corrected frame-level fundamental frequency in the fundamental frequency sequence of the musical score as the fundamental frequency correction position. The electronic device detects whether there is a fundamental frequency connection before and after the fundamental frequency correction position. When the electronic device detects a fundamental frequency connection before and after the fundamental frequency correction position, the electronic device determines the fundamental frequency connection node corresponding to the fundamental frequency correction position. The electronic device performs transition processing on the fundamental frequency connection node to obtain the transitioned fundamental frequency sequence. The electronic device generates the tuned audio of the musical score based on the transitioned fundamental frequency sequence.
[0120] In this embodiment, by determining the fundamental frequency correction position of the corrected frame-level fundamental frequency in the fundamental frequency sequence of the musical score; determining the fundamental frequency connection node corresponding to the fundamental frequency correction position; performing transition processing on the fundamental frequency connection node to obtain the fundamental frequency sequence after transition processing; and generating the corrected musical score audio based on the fundamental frequency sequence after transition processing, the transition processing of the fundamental frequency connection point before and after the corrected frame-level fundamental frequency can be performed to improve the naturalness of the musical score audio.
[0121] In some embodiments, the above method further includes: obtaining modified score information; updating the target score according to the modified score information to obtain the updated target score as the new target score.
[0122] The modified score information is determined in response to a change operation on the target score; the modified score information includes at least one of lyrics or melody.
[0123] As an example, when sheet music information needs to be modified, the electronic device acquires the modified sheet music information. Based on the modified sheet music information, the electronic device modifies and updates the lyrics and / or melody information in the target sheet music to obtain the updated target sheet music. The electronic device then uses the updated target sheet music as the new target sheet music. For example: Figure 2 As shown, after the electronic device parses the target score, the parsed score obtained by the electronic device may include several score units. The combination form of each score unit can be represented as: word_syllable_pitch. The format of the parsed score is convenient for the parsing program to parse the score. When it is necessary to modify the lyrics in the score, the electronic device replaces the information at the positions of word and syllable with new word and syllable. When it is necessary to modify the melody in the score, the electronic device replaces the information at the position of pitch with new pitch.
[0124] In this embodiment, the modified score information is obtained; the target score is updated based on the modified score information to obtain the updated target score as the new target score. Based on the modified score information, the modified score information can be updated to the original score in a timely and convenient manner, thereby improving the efficiency of score updating.
[0125] In some embodiments, the above method further includes: obtaining timbre information corresponding to the target object; and generating target singing audio corresponding to the target object based on frame-level fundamental frequency, musical notation phoneme information, and timbre information.
[0126] The target object can refer to the singer whose voice timbre corresponds to the vocal timbre during vocal synthesis.
[0127] Among them, timbre information can refer to information that characterizes the singer's vocal features. In practical applications, since different sounds have unique waveform characteristics, each singer's timbre can be different.
[0128] Among them, the target singing audio corresponding to the target object can refer to the audio output by the singing synthesis system that sings the target song in the timbre of the target object.
[0129] As an example, the electronic device obtains the singer's identity identifier as the target object, determines the timbre information of the target object based on the singer's identity identifier, and inputs the timbre information, frame-level fundamental frequency and music score phoneme information of the target object into the vocal synthesis system to generate a vocal audio of the target object singing the target music score in the timbre of the target object.
[0130] In this embodiment, by acquiring the timbre information corresponding to the target object, and generating the target singing audio corresponding to the target object based on the frame-level fundamental frequency, musical score phoneme information and timbre information, it is possible to generate singing audio of the target score sung with the timbre of the target object, thereby realizing the modification of the timbre of the singing audio and improving the flexibility of singing synthesis.
[0131] For the ease of understanding of those skilled in the art, Figure 6 An exemplary singing voice synthesis system is proposed, such as... Figure 6As shown, the electronic device acquires the frame-level fundamental frequency, musical note phoneme information, and the target object's identity (singer ID) corresponding to the musical score audio. Based on the target object's identity, the electronic device determines the target object's timbre information. It then inputs the frame-level fundamental frequency, musical note phoneme information, and the target object's timbre information into the vocal vector conversion model (embedding) to obtain a vocal feature vector. This feature vector is then input into a vocal synthesis encoder to obtain vocal encoding features. The encoded features are input into a vocal synthesis decoder, and the decoder's output is input into a linear layer to obtain the target vocal acoustic features. Finally, the acoustic features are input into a vocoder to obtain the target vocal audio. This target vocal audio represents the audio of the target musical score sung in the target object's timbre. The frame-level fundamental frequency and musical note phoneme information can serve as a template. After determining the target object's timbre, the electronic device can generate audio of the target musical score sung in the target object's timbre based on this template.
[0132] As an example, Figure 7 A flowchart illustrating a method for generating singing voices is presented, as follows: Figure 7 As shown, the electronic device acquires the syllable information, note information, and legato information of the target score. The electronic device inputs the syllable information, note information, and legato information into the score synthesis system. The electronic device inputs the output of the score synthesis system into the pitch detection system for pitch detection and correction, obtaining frame-level information and phoneme information. The electronic device inputs the frame-level information and phoneme information into the vocal synthesis system to obtain the target vocal audio of the target object.
[0133] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.
[0134] Based on the same inventive concept, this application also provides a singing voice generation device for implementing the singing voice generation method described above. The solution provided by this device is similar to the implementation described in the above method; therefore, the specific limitations in one or more singing voice generation device embodiments provided below can be found in the limitations of the singing voice generation method described above, and will not be repeated here.
[0135] In one embodiment, such as Figure 8 As shown, a singing voice generation device is provided, including: an information acquisition module 802, a feature determination module 804, an audio generation module 806, and a singing voice synthesis module 808, wherein:
[0136] The information acquisition module 802 is used to acquire syllable information, note information, and legato information of the target musical score;
[0137] The feature determination module 804 is used to determine the music score phoneme encoding features, music score phoneme duration, and phoneme level fundamental frequency of the target music score based on the syllable information, the note information, and the tuplet information;
[0138] The audio generation module 806 is used to generate the audio of the target score based on the phoneme encoding features of the score, the phoneme duration of the score, and the fundamental frequency of the phoneme level.
[0139] The vocal synthesis module 808 is used to obtain the frame-level fundamental frequency corresponding to the music score audio and the music score phoneme information corresponding to the target music score, and generate the target vocal audio based on the frame-level fundamental frequency and the music score phoneme information.
[0140] In an exemplary embodiment, the feature determination module 804 is further configured to input the syllable information, the note information, and the tuplet information into a pre-trained score vector conversion model to obtain the score feature vector of the target score; and input the score feature vector into a pre-trained score feature determination model to obtain the score phoneme encoding features, score phoneme duration, and phoneme level fundamental frequency of the target score.
[0141] In an exemplary embodiment, the music score feature determination model includes a pre-trained music score synthesis encoder, a phoneme duration determination model, and a fundamental frequency determination model. Specifically, the feature determination module 804 is further configured to input the music score feature vector into the music score synthesis encoder to obtain the music score encoding features of the target music score; input the music score encoding features into the phoneme duration determination model to obtain the music score phoneme duration; and input the music score encoding features into the fundamental frequency determination model to obtain the phoneme-level fundamental frequency.
[0142] In an exemplary embodiment, the audio generation module 806 is further configured to input the musical score phoneme encoding features, the musical score phoneme duration, and the phoneme level fundamental frequency into a pre-trained musical score synthesis decoder to obtain the musical score acoustic features corresponding to the target musical score; and input the musical score acoustic features into a vocoder to obtain the musical score audio corresponding to the target musical score.
[0143] In one exemplary embodiment, the above-described apparatus further includes a pitch correction module, which is specifically used to determine the fundamental frequency deviation information of the score audio; the fundamental frequency deviation information is used to characterize the degree of deviation between the fundamental frequency information in the score audio and the fundamental frequency information of the target score; if it is determined that the score audio is out of tune based on the fundamental frequency deviation information, the score audio is pitch-corrected based on the fundamental frequency deviation information to obtain the pitch-corrected score audio, which is used as the new score audio.
[0144] In an exemplary embodiment, the above-mentioned pitch correction module is further configured to input the phoneme-level fundamental frequency to a moving average filter to obtain the phoneme-level fundamental frequency mean; determine the fundamental frequency deviation between the phoneme-level fundamental frequency mean and the standard pitch of the target musical score; and use the fundamental frequency deviation as the fundamental frequency deviation information.
[0145] In an exemplary embodiment, the pitch correction module is further configured to determine a pitch compensation value for the score audio based on the fundamental frequency deviation information; correct the frame-level fundamental frequency of the score audio based on the pitch compensation value to obtain a corrected frame-level fundamental frequency; and determine the pitch-corrected score audio based on the corrected frame-level fundamental frequency.
[0146] In an exemplary embodiment, the above-mentioned pitch correction module is further configured to determine the pitch correction position of the corrected frame-level fundamental frequency in the fundamental frequency sequence of the musical score; determine the fundamental frequency connection node corresponding to the fundamental frequency correction position; perform transition processing on the fundamental frequency connection node to obtain the transition-processed fundamental frequency sequence; and generate the pitch-corrected musical score audio based on the transition-processed fundamental frequency sequence.
[0147] In one exemplary embodiment, the above-described apparatus further includes a score update module, which is specifically used to obtain modified score information; the modified score information is determined in response to a change operation on the target score; the modified score information includes at least one of lyrics or melody; and the target score is updated according to the modified score information to obtain an updated target score, which serves as the new target score.
[0148] In an exemplary embodiment, the above-described apparatus further includes a timbre fusion module, which is specifically used to acquire timbre information corresponding to the target object; and generate target singing audio corresponding to the target object based on the frame-level fundamental frequency, the musical score phoneme information and the timbre information.
[0149] Each module in the aforementioned singing generation device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.
[0150] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 9 As shown, the computer device includes a processor, memory, input / output interfaces, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interfaces. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a method for generating singing voices. The display unit is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.
[0151] Those skilled in the art will understand that Figure 9 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.
[0152] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.
[0153] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.
[0154] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.
[0155] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data shall comply with the relevant laws, regulations and standards of the relevant countries and regions.
[0156] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.
[0157] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0158] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.
Claims
1. A method for generating singing voice, characterized in that, The method includes: Obtain syllable information, note information, and tuplet information of the target musical score; Based on the syllable information, the note information, and the tuplet information, determine the musical notation phoneme encoding features, musical notation phoneme duration, and phoneme level fundamental frequency of the target musical score; Based on the musical score phoneme encoding features, the musical score phoneme duration, and the phoneme level fundamental frequency, generate the musical score audio corresponding to the target musical score; Obtain the frame-level fundamental frequency corresponding to the music score audio and the music score phoneme information corresponding to the target music score, and generate the target singing audio based on the frame-level fundamental frequency and the music score phoneme information.
2. The method according to claim 1, characterized in that, The step of determining the musical notation phoneme encoding features, musical notation phoneme duration, and phoneme level fundamental frequency of the target musical score based on the syllable information, the note information, and the truncation information includes: The syllable information, the note information, and the tuplet information are input into a pre-trained music score vector conversion model to obtain the music score feature vector of the target music score; The musical score feature vector is input into a pre-trained musical score feature determination model to obtain the musical score phoneme encoding features, musical score phoneme duration, and phoneme level fundamental frequency of the target musical score.
3. The method according to claim 2, characterized in that, The music score feature determination model includes a pre-trained music score synthesis encoder, a phoneme duration determination model, and a fundamental frequency determination model. The step of inputting the music score feature vector into the pre-trained music score feature determination model to obtain the music score phoneme encoding features, music score phoneme duration, and phoneme-level fundamental frequency of the target music score includes: The musical score feature vector is input into the musical score synthesis encoder to obtain the musical score phoneme encoding features of the target musical score; The musical score phoneme encoding features are input into the phoneme duration determination model to obtain the musical score phoneme duration, and the musical score phoneme encoding features are input into the fundamental frequency determination model to obtain the phoneme level fundamental frequency.
4. The method according to claim 1, characterized in that, The step of generating the audio of the target score based on the phoneme encoding features of the score, the phoneme duration, and the fundamental frequency of the phoneme level includes: The musical score phoneme encoding features, the musical score phoneme duration, and the phoneme level fundamental frequency are input into a pre-trained musical score synthesis decoder to obtain the musical score acoustic features corresponding to the target musical score. The acoustic features of the musical score are input into a vocoder to obtain the audio of the musical score corresponding to the target musical score.
5. The method according to claim 1, characterized in that, Before the step of obtaining the frame-level fundamental frequency and musical note phoneme information corresponding to the musical score audio, the method further includes: Determine the fundamental frequency deviation information of the musical score audio; the fundamental frequency deviation information is used to characterize the degree of deviation between the fundamental frequency information in the musical score audio and the fundamental frequency information of the target musical score; If it is determined that the score audio is out of tune based on the fundamental frequency deviation information, the score audio is corrected according to the fundamental frequency deviation information to obtain the corrected score audio, which is used as the new score audio.
6. The method according to claim 5, characterized in that, Determining the fundamental frequency deviation information of the musical score audio includes: The phoneme-level fundamental frequency is input into a moving average filter to obtain the phoneme-level fundamental frequency mean value; Determine the fundamental frequency deviation between the mean fundamental frequency of the phoneme and the standard pitch of the target musical score; The fundamental frequency deviation is used as the fundamental frequency deviation information.
7. The method according to claim 5, characterized in that, The step of performing pitch correction processing on the score audio based on the fundamental frequency deviation information to obtain the pitch-corrected score audio includes: Based on the fundamental frequency deviation information, determine the pitch compensation value for the musical score audio; The frame-level fundamental frequency of the musical score audio is corrected based on the pitch compensation value to obtain the corrected frame-level fundamental frequency; The corrected audio of the musical score is determined based on the corrected frame-level fundamental frequency.
8. The method according to claim 7, characterized in that, Determining the corrected musical score audio based on the corrected frame-level fundamental frequency includes: Determine the position of the corrected frame-level fundamental frequency in the fundamental frequency sequence of the musical score; Determine the fundamental frequency connection node corresponding to the fundamental frequency correction position; The fundamental frequency connection node is subjected to transition processing to obtain the transition-processed fundamental frequency sequence; The corrected musical score audio is generated based on the fundamental frequency sequence after the transition processing.
9. The method according to claim 1, characterized in that, The method further includes: Obtain the modified sheet music information; the modified sheet music information is determined in response to a change operation on the target sheet music; the modified sheet music information includes at least one of lyrics or melody; Based on the modified score information, the target score is updated to obtain the updated target score, which serves as the new target score.
10. The method according to claim 1, characterized in that, The method further includes: Obtain the timbre information corresponding to the target object; Based on the frame-level fundamental frequency, the musical score phoneme information, and the timbre information, the target singing audio corresponding to the target object is generated.
11. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 10.
12. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 10.
Citation Information
Patent Citations
Singing synthesis method, device and equipment
CN112750420A
Audio processing method and related device
CN115862592A