Synthetic song processing method, computer device and storage medium

Abstract

This application relates to a method for synthesizing songs, a computer device, and a readable storage medium. The method includes: determining accompaniment resources and vocal resources for a target song; the vocal resources carrying the user's pronunciation characteristics; executing the task corresponding to the current frame through a first thread to generate the accompaniment audio signal for the current frame; executing the task corresponding to the current frame through a second thread to generate the vocal audio signal for the current frame; executing the first frame playback task through a main process: when both the accompaniment audio signal and the vocal audio signal of the first frame have been generated, starting a preview engine, synthesizing the song audio signal of the first frame through the preview engine, and playing it; during the execution of the first frame playback task in the main process, simultaneously executing the subsequent frame tasks of the first thread and the second thread; executing the subsequent frame playback task through the main process until the subsequent frame is the end frame. This method can improve the efficiency of previewing or editing synthesized songs.

Description

Technical Field

[0001] This application relates to the field of audio processing technology, and in particular to a method for synthesizing songs, a computer device, a computer-readable storage medium, and a computer program product. Background Technology

[0002] With the development of audio processing technology, a technique for generating synthesized songs has emerged. This technique can pre-download the accompaniment of the synthesized song and the vocal resource file used to generate the vocals in the synthesized song. By reasoning through the vocal resource file, the vocal audio of the synthesized song is generated. Afterwards, users can preview the synthesized song, synthesize the accompaniment and vocal audio to obtain the song audio for playback, and can also edit and process the synthesized song audio.

[0003] In traditional technology, if a user needs to preview or edit the synthesized song audio, they usually have to wait for the accompaniment and vocal resources of the synthesized song to be downloaded and for the vocal audio to be generated. That is, the user has to wait for the accompaniment and vocal resources to be downloaded before the vocal audio is inferred. Only after the vocal audio inference is completed can the song audio be previewed or edited. Therefore, the current preview or editing efficiency of synthesized songs is relatively low. Summary of the Invention

[0004] Therefore, it is necessary to provide a method, apparatus, computer device, computer-readable storage medium, and computer program product for processing synthesized songs that can improve the preview or editing efficiency of synthesized songs, in response to the above-mentioned technical problems.

[0005] Firstly, this application provides a method for processing synthesized songs, applied to a client, including:

[0006] For the target song selected by the user, determine the accompaniment resources and vocal resources stored on the server for synthesizing the target song; the vocal resources carry the user's pronunciation characteristics;

[0007] The first thread executes the task corresponding to the current frame: downloading the accompaniment data of the accompaniment resource in the current frame from the server, and parsing the accompaniment data of the accompaniment resource in the current frame to generate the accompaniment audio signal of the current frame;

[0008] The second thread executes the task corresponding to the current frame: downloading the voice data of the voice resource in the current frame from the server, and inferring the voice data of the voice resource in the current frame to generate the voice audio signal of the current frame; wherein: the current frame includes the first frame and subsequent frames, and the first frame task of the first thread and the first frame task of the second thread are executed synchronously.

[0009] The first frame playback task is executed through the main process: when it is detected that the accompaniment audio signal and the human voice audio signal of the first frame have been generated, the preview engine is started, and the accompaniment audio signal and the human voice audio signal of the first frame are combined by the preview engine to obtain the song audio signal of the first frame and the song audio signal of the first frame is played.

[0010] During the execution of the first frame playback task in the main process, the subsequent frame task of the first thread is executed synchronously to generate the accompaniment audio signal of the subsequent frame, and the subsequent frame task of the second thread is executed synchronously to generate the human voice audio signal of the subsequent frame.

[0011] The main process executes subsequent frame playback tasks: whenever the accompaniment audio signal and the vocal audio signal of the same subsequent frame are detected to be generated, the preview engine synthesizes the accompaniment audio signal and the vocal audio signal of the same subsequent frame to obtain the song audio signal of the same subsequent frame and plays the song audio signal of the same subsequent frame until the same subsequent frame is the end frame of the target song.

[0012] In one embodiment, the step of downloading the accompaniment data of the accompaniment resource in the current frame from the server and parsing the accompaniment data of the accompaniment resource in the current frame to generate the accompaniment audio signal of the current frame includes: obtaining the audio metadata of the target song, and downloading the accompaniment data of the accompaniment resource in the current frame from the server based on the audio metadata; and parsing the accompaniment data of the accompaniment resource in the current frame using the audio metadata to generate the accompaniment audio signal of the current frame.

[0013] In one embodiment, downloading the accompaniment data of the accompaniment resource in the current frame from the server based on the audio metadata includes: obtaining the signal duration, sampling rate, sampling bit depth, and number of channels of the current frame from the audio metadata; obtaining the data volume corresponding to the accompaniment data of the current frame according to the signal duration, the sampling rate, the sampling bit depth, and the number of channels; and downloading the accompaniment data of the accompaniment resource in the current frame from the server according to the data volume.

[0014] In one embodiment, when the current frame is the first frame, obtaining the audio metadata of the target song includes: downloading the accompaniment resource from the server and parsing the accompaniment resource to obtain the audio metadata of the target song.

[0015] In one embodiment, downloading the voice data of the voice resource in the current frame from the server includes: obtaining the data frame length and signal duration of the current frame; obtaining the data volume corresponding to the voice data of the current frame based on the data frame length and signal duration; and downloading the voice data of the voice resource in the current frame from the server according to the data volume.

[0016] In one embodiment, the number of vocal resources is multiple; during the process of synchronously executing the subsequent frame task of the first thread to generate the accompaniment audio signal of the subsequent frame and synchronously executing the subsequent frame task of the second thread to generate the vocal audio signal of the subsequent frame, the method further includes: obtaining the resource download progress of each vocal resource and obtaining the resource download progress of the accompaniment resource; if the difference between the download progress of each resource is greater than a preset threshold, adjusting the resource download efficiency of each vocal resource and the accompaniment resource, wherein the difference between the adjusted download progress of each resource meets a preset difference requirement.

[0017] In one embodiment, during the execution of subsequent frame playback tasks through the main process, the method further includes: responding to the user's request to adjust the playback progress of the target song, obtaining the playback progress frame corresponding to the adjusted playback progress; and when it is detected that the song audio signal of the playback progress frame has been synthesized, playing the song audio signal of the playback progress frame through the preview engine.

[0018] In one embodiment, after obtaining the playback progress frame corresponding to the adjusted playback progress, the method further includes: when it is detected that the accompaniment audio signal of the playback progress frame has not been fully generated, executing the task corresponding to the playback progress frame through a first thread to generate the accompaniment audio signal of the playback progress frame; when it is detected that the vocal audio signal of the playback progress frame has not been fully generated, executing the task corresponding to the playback progress frame through a second thread to generate the vocal audio signal of the playback progress frame; and executing the playback progress frame task through the main process: when it is detected that both the accompaniment audio signal and the vocal audio signal of the playback progress frame have been fully generated, synthesizing the accompaniment audio signal and the vocal audio signal of the playback progress frame through the preview engine to obtain the song audio signal of the playback progress frame and playing the song audio signal of the playback progress frame.

[0019] Secondly, this application also provides a synthesized song processing device for use on a client side, comprising:

[0020] The synthesis resource acquisition module is used to determine, for example, the accompaniment resources and vocal resources stored on the server for synthesizing the target song selected by the user; the vocal resources carry the user's pronunciation characteristics.

[0021] The first thread execution module is used to execute the task corresponding to the current frame through the first thread: download the accompaniment data of the accompaniment resource in the current frame from the server, and parse the accompaniment data of the accompaniment resource in the current frame to generate the accompaniment audio signal of the current frame;

[0022] The second thread execution module is used to execute the task corresponding to the current frame through the second thread: download the human voice data of the human voice resource in the current frame from the server, and perform inference on the human voice data of the human voice resource in the current frame to generate the human voice audio signal of the current frame; wherein: the current frame includes the first frame and subsequent frames, and the first frame task of the first thread and the first frame task of the second thread are executed synchronously;

[0023] The main process execution module is used to execute the first frame playback task through the main process: when it is detected that the accompaniment audio signal and the human voice audio signal of the first frame have been generated, the preview engine is started, and the accompaniment audio signal and the human voice audio signal of the first frame are combined through the preview engine to obtain the song audio signal of the first frame and the song audio signal of the first frame is played.

[0024] The first thread execution module is also used to synchronously execute the subsequent frame tasks of the first thread to generate the accompaniment audio signal of the subsequent frames during the execution of the first frame playback task in the main process.

[0025] The second thread execution module is also used to synchronously execute the subsequent frame tasks of the second thread to generate the human voice audio signal of the subsequent frames during the execution of the first frame playback task in the main process;

[0026] The main process execution module is also used to execute subsequent frame playback tasks through the main process: whenever the accompaniment audio signal and the human voice audio signal of the same subsequent frame are detected to be generated, the accompaniment audio signal and the human voice audio signal of the same subsequent frame are synthesized by the preview engine to obtain the song audio signal of the same subsequent frame and the song audio signal of the same subsequent frame is played until the same subsequent frame is the end frame of the target song.

[0027] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the method described in any embodiment of the first aspect.

[0028] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in any embodiment of the first aspect.

[0029] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the method described in any embodiment of the first aspect.

[0030] The aforementioned song synthesis processing method, apparatus, computer equipment, computer-readable storage medium, and computer program product obtain the accompaniment and vocal resources of the target song selected by the user from the server through a client. The process of obtaining the first frame's accompaniment audio signal using the accompaniment resources and the first frame's vocal audio signal using the vocal resources is executed synchronously through a first thread and a second thread. After the main process detects that the first frame's accompaniment and vocal audio signals have been generated, a preview engine is started to synthesize and play the first frame's song audio signal. Furthermore, while the main process is executing the first frame playback task, subsequent frame tasks can be executed synchronously through the first thread and the second thread to generate... The system generates the accompaniment audio signal and vocal audio signal for subsequent frames, and executes the playback task for subsequent frames through the main process until the subsequent frame is the end frame of the target song, thus completing the playback of the target song. It can be seen that this method only requires the acquisition of the accompaniment audio signal and vocal audio signal of the starting frame to start the preview engine to play the first frame of the song audio signal. Furthermore, during the playback of the song audio signal, the acquisition of the accompaniment audio signal and vocal audio signal of subsequent frames can be carried out simultaneously. This achieves parallel processing of accompaniment and vocal resource download, accompaniment parsing, vocal inference, and song audio signal synthesis and playback, thereby improving the preview or editing efficiency of synthesized songs. Attached Figure Description

[0031] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0032] Figure 1 This is a flowchart illustrating a song synthesis processing method in one embodiment;

[0033] Figure 2 This is a schematic diagram of the process for generating the accompaniment audio signal of the current frame in one embodiment;

[0034] Figure 3 This is a schematic diagram of the process of playing the song audio signal with the adjusted progress frame in one embodiment.

[0035] Figure 4 This is a schematic diagram of the process of playing the song audio signal with the adjusted progress frame in another embodiment;

[0036] Figure 5 This is a flowchart illustrating a method for real-time previewing and editing of inference vocals in one embodiment;

[0037] Figure 6 This is a schematic diagram illustrating the download volume of the first frame of accompaniment data in one embodiment.

[0038] Figure 7 This is a schematic diagram illustrating the implementation of the secondary buffering process in one embodiment;

[0039] Figure 8 This is a structural block diagram of a song synthesis processing device in one embodiment;

[0040] Figure 9 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0041] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0042] In one embodiment, such as Figure 1 As shown, a method for synthesizing songs is provided. This embodiment illustrates the method by applying it to a client, and demonstrates its implementation through interaction between the client and server. In this embodiment, the method includes the following steps:

[0043] Step S101: For the target song selected by the user, determine the accompaniment resources and vocal resources stored on the server for synthesizing the target song; the vocal resources carry the user's pronunciation characteristics.

[0044] The target song refers to the song selected by the user and to be generated through song synthesis. The audio of the target song can consist of two parts: accompaniment audio and vocal audio. Accompaniment resources refer to the resources used to generate the accompaniment audio signal of the target song, such as accompaniment audio files. Vocal resources refer to the resources used to generate the vocal audio signal of the target song, which carry the user's pronunciation characteristics, such as PPG (Polyphonic Performance Generator) files, timbre files, pitch files, etc.

[0045] Specifically, when a user selects a target song to be synthesized, the client can first determine the accompaniment resource file and vocal resource file stored on the server for synthesizing the target song.

[0046] Step S102: The task corresponding to the current frame is executed by the first thread: download the accompaniment data of the accompaniment resource in the current frame from the server, and parse the accompaniment data of the accompaniment resource in the current frame to generate the accompaniment audio signal of the current frame.

[0047] Step S103: The second thread executes the task corresponding to the current frame: downloads the voice data of the voice resource in the current frame from the server, and performs inference on the voice data of the voice resource in the current frame to generate the voice audio signal of the current frame; wherein: the current frame includes the first frame and subsequent frames, and the first frame task of the first thread and the first frame task of the second thread are executed synchronously.

[0048] The current frame refers to any song signal frame of the target song, which can include the first frame and subsequent frames of the target song. The first thread is used to generate the accompaniment audio signal, while the second thread is used to generate the vocal audio signal, that is, the audio signal of the pure vocal part of the target song. The first frame task executed by the first thread and the second thread is executed synchronously, that is, the process of the first thread generating the accompaniment audio signal of the first frame and the second thread generating the vocal audio signal of the first frame can be performed synchronously.

[0049] Specifically, the client can start the first thread to generate the accompaniment audio signal for the current frame. This can be done by first downloading the accompaniment data corresponding to the current frame from the server, and then parsing the accompaniment data for the current frame to generate the accompaniment audio signal for the current frame.

[0050] At the same time, the client can also start a second thread to generate the human voice audio signal for the current frame. This can be done by first downloading the human voice data of the current frame from the server, and then using the singing inference engine to perform singing inference on the human voice data of the current frame to generate the human voice audio signal for the current frame.

[0051] Step S104: Execute the first frame playback task through the main process: When it is detected that the accompaniment audio signal and the vocal audio signal of the first frame have been generated, start the preview engine, and use the preview engine to synthesize the accompaniment audio signal and the vocal audio signal of the first frame to obtain the song audio signal of the first frame and play the song audio signal of the first frame.

[0052] The main process is the thread used to synthesize and play the song's audio signal. After the first frame of the accompaniment audio signal and the vocal audio signal are detected to be generated, the preview engine can be started through the main process. The preview engine is used to preview the target song, that is, to play the target song. Through the preview engine, the first frame of the accompaniment audio signal obtained by the first thread can be synthesized with the first frame of the vocal audio signal obtained by the second thread. After obtaining the first frame of the target song's audio signal, the first frame of the song's audio signal can be played.

[0053] During the execution of the first frame playback task in the main process, the subsequent frame task of the first thread is executed synchronously to generate the accompaniment audio signal of the subsequent frame, and the subsequent frame task of the second thread is executed synchronously to generate the human voice audio signal of the subsequent frame.

[0054] Step S105: Execute the subsequent frame playback task through the main process: Whenever the accompaniment audio signal and the vocal audio signal of the same subsequent frame are detected to be generated, the accompaniment audio signal and the vocal audio signal of the same subsequent frame are synthesized by the preview engine to obtain the song audio signal of the same subsequent frame and played until the same subsequent frame is the end frame of the target song.

[0055] The end frame refers to the last song signal frame in the song to be synthesized. While the client is executing the first frame playback task through the main process, it can also simultaneously execute the subsequent frame tasks of the first thread and the subsequent frame tasks of the second thread, thereby generating the accompaniment audio signal of the subsequent frame through the first thread and the vocal audio signal of the subsequent frame through the second thread.

[0056] After the main process detects that the accompaniment audio signal and the vocal audio signal of the same subsequent frame have been generated, the preview engine can synthesize the accompaniment audio signal and the vocal audio signal of the same subsequent frame to obtain the song audio signal of the same subsequent frame and play it until the same subsequent frame is the end frame of the target song.

[0057] For example, while the main process is executing the first frame playback task, the client can simultaneously execute the subsequent frame tasks of the first thread and the second thread. That is, it can simultaneously execute the accompaniment audio signal for the second frame generated by the first thread and the vocal audio signal for the second frame generated by the second thread. Since the accompaniment audio signal and vocal audio signal for the second frame have been generated at this time, the main thread can also execute the playback task for the second frame, synthesizing the accompaniment audio signal and vocal audio signal of the second frame to obtain the song audio signal for the second frame, and then playing the song audio signal for the second frame. At this time, the client can also simultaneously execute the accompaniment audio signal for the third frame generated by the first thread and the vocal audio signal for the third frame generated by the second thread, and so on, until the main process completes the synthesis and playback of the song audio signal for the final frame, thus completing the synthesis and playback of the target song.

[0058] The aforementioned song synthesis processing method, apparatus, computer equipment, computer-readable storage medium, and computer program product, through a client-side selection of a target song by a user, determine the accompaniment resources and vocal resources stored on the server for synthesizing the target song; the vocal resources carry the user's pronunciation characteristics; the first thread executes the task corresponding to the current frame: downloading the accompaniment data of the accompaniment resources in the current frame from the server and parsing the accompaniment data of the accompaniment resources in the current frame to generate the accompaniment audio signal of the current frame; the second thread executes the task corresponding to the current frame: downloading the vocal data of the vocal resources in the current frame from the server and inferring from the vocal data of the vocal resources in the current frame to generate the vocal audio signal of the current frame; wherein: the current frame includes the first frame and subsequent frames, and the first frame task of the first thread and the first frame task of the second thread are executed synchronously; the main process executes the first frame playback. Task execution: When both the accompaniment audio signal and the vocal audio signal of the first frame are detected as being generated, the preview engine is started. The preview engine synthesizes the accompaniment audio signal and the vocal audio signal of the first frame to obtain the song audio signal of the first frame, and plays the song audio signal of the first frame. During the execution of the first frame playback task in the main process, the subsequent frame task of the first thread is executed synchronously to generate the accompaniment audio signal of the subsequent frame, and the subsequent frame task of the second thread is executed synchronously to generate the vocal audio signal of the subsequent frame. The subsequent frame playback task is executed through the main process: Whenever both the accompaniment audio signal and the vocal audio signal of the same subsequent frame are detected as being generated, the accompaniment audio signal and the vocal audio signal of the same subsequent frame are synthesized by the preview engine to obtain the song audio signal of the same subsequent frame, and the song audio signal of the same subsequent frame is played, until the same subsequent frame is the end frame of the target song. This application obtains the accompaniment and vocal resources of the target song selected by the user from the server through the client. It then synchronously executes the processes of obtaining the accompaniment audio signal of the first frame using the accompaniment resources and the vocal audio signal of the first frame using the vocal resources through a first thread and a second thread. After the main process detects that the accompaniment and vocal audio signals of the first frame have been generated, it starts the preview engine to synthesize and play the song audio signal of the first frame. Furthermore, while the main process is executing the first frame playback task, it can also synchronously execute subsequent frame tasks through the first and second threads to generate the accompaniment and vocal audio signals of subsequent frames. The system executes subsequent frame playback tasks through the main process until the subsequent frame is the end frame of the target song, thus completing the playback of the target song. It can be seen that this method only requires the acquisition of the accompaniment audio signal and vocal audio signal of the starting frame to start the preview engine to play the first frame of the song audio signal. Furthermore, during the playback of the song audio signal, the acquisition of the accompaniment audio signal and vocal audio signal of subsequent frames can be performed simultaneously. This achieves parallel processing of accompaniment and vocal resource download, accompaniment parsing, vocal inference, and song audio signal synthesis and playback, thereby improving the preview or editing efficiency of synthesized songs.

[0059] In one embodiment, such as Figure 2 As shown, step S102 may further include:

[0060] Step S201: Obtain the audio metadata of the target song, and download the accompaniment data of the accompaniment resource in the current frame from the server based on the audio metadata.

[0061] Among them, audio metadata refers to the metadata of the accompaniment audio signal of the target song. When the client downloads the accompaniment data of the accompaniment resource in the current frame from the server, it first needs to obtain the audio metadata of the target song. Then, it can download the accompaniment data of the accompaniment resource in the current frame from the server based on the metadata.

[0062] Step S202: Use audio metadata to parse the accompaniment data of the accompaniment resource in the current frame to generate the accompaniment audio signal of the current frame.

[0063] After obtaining the audio metadata, the client can also use the aforementioned audio metadata to parse the accompaniment data of the current frame, thereby generating the accompaniment audio signal of the current frame.

[0064] In this embodiment, when obtaining the accompaniment audio signal of the current frame, it is first necessary to obtain the audio metadata of the target song. This audio metadata can be used to instruct the client to download the accompaniment data of the current frame from the server, and to instruct the process of parsing the accompaniment data of the current frame to generate the accompaniment audio signal of the current frame. This method can improve the accuracy of generating the accompaniment audio signal of the current frame.

[0065] Furthermore, step S201 may further include: obtaining the signal duration, sampling rate, sampling bit depth, and number of channels of the current frame from the audio metadata; obtaining the data volume corresponding to the accompaniment data of the current frame based on the signal duration, sampling rate, sampling bit depth, and number of channels; and downloading the accompaniment data of the accompaniment resource in the current frame from the server according to the data volume.

[0066] In this embodiment, the audio metadata can record information such as duration, sampling rate, sampling bit depth, number of channels, etc. The duration can refer to the signal duration of the current frame, that is, the frame length of the current frame, while the data volume corresponding to the accompaniment resource of the current frame refers to the data size of the accompaniment resource corresponding to the current frame.

[0067] Specifically, the client can obtain information such as the signal duration of the current frame, the sampling rate, sampling bit depth, and number of channels of the accompaniment audio signal based on the audio metadata. After obtaining the above information, the client can calculate the data volume corresponding to the accompaniment data of the current frame based on the signal duration of the current frame, the sampling rate, sampling bit depth, and number of channels of the accompaniment audio signal. For example, the data volume corresponding to the accompaniment resource of the current frame can be calculated by the following formula: Current frame size = Current frame duration * Sampling rate * Sampling bit depth * Number of channels / 8. Thus, the client can download the data corresponding to the above data volume from the accompaniment resource as the accompaniment data of the current frame.

[0068] In this embodiment, the client can determine the amount of data corresponding to the accompaniment data of the current frame based on information such as the signal duration of the current frame, the sampling rate of the accompaniment audio signal, the sampling bit depth, and the number of channels stored in the audio metadata. Then, according to the amount of data, the client can obtain the accompaniment data of the current frame from the accompaniment resources. This method can improve the accuracy of obtaining the accompaniment data of the current frame.

[0069] In addition, if the current frame is the first frame, step S201 may further include: downloading the accompaniment resource from the server and parsing the accompaniment resource to obtain the audio metadata of the target song.

[0070] In this embodiment, audio metadata can also be carried in the accompaniment resource, and this audio metadata is set at the very beginning of the accompaniment resource. If the current frame is the first frame, the client needs to first download the accompaniment resource from the server, and parse the downloaded accompaniment resource while downloading it to obtain the audio metadata of the target song. Then, the client downloads the accompaniment data of the first frame based on the parsed audio metadata. If it is not the first frame, it indicates that the audio metadata has been downloaded and parsed, and the pre-parsed audio metadata can be directly obtained.

[0071] In this embodiment, when the current frame is the first frame, the client can also obtain audio metadata by downloading and parsing the accompaniment resources, which can improve the efficiency of obtaining audio metadata.

[0072] In one embodiment, step S103 may further include: obtaining the data frame length and signal duration of the current frame; obtaining the data volume corresponding to the human voice data of the current frame based on the data frame length and signal duration; and downloading the human voice data of the human voice resource in the current frame from the server according to the data volume.

[0073] For downloading the voice data of the current frame, it is necessary to first determine the amount of voice data corresponding to the current frame. This amount of data can be determined based on the data frame length and signal duration of the current frame, which can be obtained by the server sending the data frame length and signal duration to the client.

[0074] Specifically, the backend server can send the data frame length and signal duration of the current frame to the client. The client can then obtain the data frame length and signal duration of the current frame and calculate the amount of data corresponding to the vocal resource of the current frame based on the data frame length and signal duration. For example, the amount of data corresponding to the vocal resource of the current frame can be calculated using the following formula: Data size required for the vocal data of the current frame = Data frame length * First frame duration. Then, the vocal data of the current frame can be downloaded from the vocal resource file according to the above data size.

[0075] In this embodiment, the client can determine the amount of data corresponding to the vocal resource of the current frame based on the data frame length and signal duration of the current frame sent by the backend server. Then, according to the amount of data, the client can obtain the vocal data of the current frame from the vocal resource. This method can improve the accuracy of obtaining the vocal data of the current frame.

[0076] In one embodiment, the number of vocal resources is multiple; during the execution of step S104, it may further include: obtaining the resource download progress of each vocal resource and obtaining the resource download progress of the accompaniment resource. If the difference between the download progress of each resource is greater than a preset threshold, the resource download efficiency of each vocal resource and the accompaniment resource is adjusted, wherein the difference between the adjusted download progress of each resource meets the preset difference requirement.

[0077] In this embodiment, the number of vocal resources can be multiple. For example, vocal resources can include multiple resource files such as PPG, timbre files, and pitch files. The resource download progress represents the download progress of each vocal resource and accompaniment resource. Since it is necessary to acquire each vocal resource and accompaniment resource simultaneously during the playback of the song audio signal through the preview engine in order to generate and play the song audio signal of each audio frame simultaneously, it is necessary to synchronize the download progress of various resources.

[0078] Specifically, while the client is synchronously executing the subsequent frame tasks of the first thread and the subsequent frame tasks of the second thread, it can also obtain the download progress of each vocal resource and accompaniment resource in real time. If the difference between the download progress of two resources is greater than a preset threshold, the download efficiency of each vocal resource and accompaniment resource can be adjusted accordingly to reduce the difference between the download progress of each resource.

[0079] For example, during the download process, if the download progress of the timbre file is much higher than that of the pitch file, and the difference in download progress exceeds a preset threshold, then the download efficiency of each resource can be adjusted. For example, the download of the timbre file can be paused, thereby increasing the download speed of the pitch file. This method balances the download progress of each resource and reduces the difference between their download progress.

[0080] In this embodiment, during the download of vocal and accompaniment resources, the download progress of each vocal and accompaniment resource can be obtained. If the difference between the download progress is large, the download efficiency of each vocal and accompaniment resource can be adjusted to reduce the difference between the download progress. This method can further improve the overall efficiency of resource download, thereby improving the efficiency of generating song audio signals.

[0081] In one embodiment, such as Figure 3 As shown, the subsequent frame playback task can also include the following:

[0082] Step S301: In response to the user's request to adjust the playback progress of the target song, obtain the playback progress frame corresponding to the adjusted playback progress.

[0083] A playback progress adjustment request refers to a user's request to adjust the playback progress of a song's audio signal during playback. The adjusted playback progress refers to the playback progress at the time the user triggers the request. For example, if the current playback progress of the target song is 10 seconds, and the user needs to adjust the playback progress to 30 seconds, a playback progress adjustment request for the song to be synthesized can be triggered, and 30 seconds will be used as the adjusted playback progress.

[0084] The playback progress frame corresponding to the adjusted playback progress refers to the song signal frame represented by the adjusted playback progress, that is, the song signal frame corresponding to a playback time of 30 seconds. Specifically, after the client receives the user's request to adjust the playback progress of the target song, it can obtain the playback progress frame corresponding to the adjusted playback progress.

[0085] Step S302: When it is detected that the song audio signal of the playback progress frame has been synthesized, the song audio signal of the playback progress frame is played through the preview engine.

[0086] Generally, synthesizing a song's audio signal is far more efficient than playing it. For example, synthesizing a 1-second song audio signal might only take 0.1 seconds. In this case, even though the audio signal for the adjusted playback frame hasn't been played yet, it's possible that the audio signal for that frame has already been synthesized. In such a situation, the client can directly play the audio signal for the adjusted playback frame through the preview engine.

[0087] For example, if the current playback progress of the target song is 10 seconds, when the user adjusts the playback progress to 30 seconds, if the audio signal of the song corresponding to the 30-second playback progress frame has been synthesized, the client can directly play the audio signal of the song corresponding to the 30-second playback progress frame through the preview engine.

[0088] In this embodiment, after the playback progress is adjusted, if the song audio signal corresponding to the playback progress frame corresponding to the adjusted playback progress has been synthesized, the song audio signal can be played directly. In this way, the song audio signal can be played immediately after the playback progress is adjusted, thereby improving the playback efficiency of the song audio signal after the playback progress is adjusted.

[0089] In addition, such as Figure 4 As shown, after step S301, the following may also be included:

[0090] Step S401: When it is detected that the accompaniment audio signal of the playback progress frame has not been fully generated, the task corresponding to the playback progress frame is executed through the first thread to generate the accompaniment audio signal of the playback progress frame.

[0091] In this embodiment, the incomplete synthesis of the song audio signal for the playback progress frame is usually caused by the following two factors: the accompaniment audio signal corresponding to the playback progress frame has not been fully generated, or the vocal audio signal corresponding to the playback progress frame has not been fully generated, or both the accompaniment audio signal and the vocal audio signal corresponding to the playback progress frame have not been fully generated.

[0092] Specifically, if the accompaniment audio signal for a playback progress frame has not been fully generated—for example, if the current playback progress of the target song is 1 second, and the user adjusts the playback progress to 100 seconds—although the synthesis efficiency of the song's audio signal is higher than the playback efficiency, the client may still not have obtained the accompaniment audio signal for the current playback frame. In this case, the client can execute the task corresponding to the playback progress frame through the first thread to generate the accompaniment audio signal for that frame; that is, prioritize generating the accompaniment audio signal for the playback progress frame.

[0093] For example, when a user adjusts the playback progress to 100 seconds, the accompaniment audio signal for the song signal frame corresponding to 100 seconds may not be available. In this case, the client can execute the task for the song signal frame corresponding to 100 seconds through the first thread, thereby prioritizing the generation of the accompaniment audio signal for the song signal frame corresponding to 100 seconds.

[0094] Step S402: When it is detected that the human voice audio signal of the playback progress frame has not been fully generated, the task corresponding to the playback progress frame is executed through the second thread to generate the human voice audio signal of the playback progress frame.

[0095] Similarly, if the vocal audio signal of the playback progress frame has not been fully generated, the client can also execute the task corresponding to the playback progress frame through a second thread to generate the vocal audio signal of the playback progress frame. That is, the client first downloads the vocal data of the song signal frame corresponding to the vocal resource at 100s from the server, and performs inference on the vocal data of the song signal frame corresponding to 100s to generate the vocal audio signal of the song signal frame corresponding to 100s.

[0096] Step S403: Execute the playback progress frame task through the main process: When it is detected that the accompaniment audio signal and the vocal audio signal of the playback progress frame have been generated, the preview engine synthesizes the accompaniment audio signal and the vocal audio signal of the playback progress frame to obtain the song audio signal of the playback progress frame and plays the song audio signal of the playback progress frame.

[0097] Finally, after the main process detects that the accompaniment audio signal and the vocal audio signal of the playback progress frame have been generated, the preview engine can synthesize the accompaniment audio signal and the vocal audio signal of the playback progress frame to obtain the song audio signal of the playback progress frame and play it.

[0098] In this embodiment, if the human voice audio signal or accompaniment audio signal of the adjusted playback progress frame is not obtained, the step of obtaining the human voice audio signal or accompaniment audio signal of the adjusted playback progress frame can be performed first, and the signal can be played when the acquisition is completed. This method can reduce the playback buffering time of the song audio signal after the playback progress is adjusted.

[0099] In one embodiment, a method for real-time previewing and editing of inferred vocals on a client-side is also provided. This method can be applied to AI vocal synthesis scenarios. Currently, to generate AI-synthesized vocals, it is typically necessary to download resource files such as PPG files, timbre files, pitch files, and accompaniment data used for vocal inference. The inference task can only be started and executed after the resources are downloaded. Generally, the RTF of vocal inference is 0.1, meaning that 5 minutes of vocals requires a 30-second wait. If a user wants to preview or edit the inferred vocals, they need to wait for the resource download and vocal inference task to complete, resulting in a relatively long total waiting time throughout the entire process.

[0100] The method for real-time previewing and editing of vocal inference provided in this embodiment allows resource files such as PPG files, timbre files, pitch files, and accompaniment files to be downloaded in parallel after the first frame of data is downloaded. Since resource downloading uses network resources and vocal inference uses client hardware resources, there is no contention between the two. Once the first frame of data is obtained through vocal inference, the vocal preview is started in real time, while the remaining resources and vocal inference are downloaded and processed in parallel during the preview process. By parallel processing of the three processes—resource downloading, vocal inference, and real-time previewing and editing—and resolving issues such as resource dependencies and state conflicts among them, the waiting time for users to preview vocals is effectively reduced, and issues such as resource dependencies and state conflicts caused by parallel processing are also optimized.

[0101] 1. The overall process can be as follows: Figure 5 As shown, the following steps may be included:

[0102] (1) After the user obtains the AI ​​work, the client will start the inference of the singing voice. It is necessary to download the inference singing voice resource files (PPG, timbre file, pitch file). After the first frame data of all resources is downloaded, inference can be performed in parallel. The inferred data is saved in the memory of the inference singing voice.

[0103] (2) Download the accompaniment in parallel. Once the metadata of the audio can be parsed, the parameters of the initial preview engine can be obtained through the metadata of the audio.

[0104] (3) Once the accompaniment audio metadata and the first frame data have been inferred, the preview engine can be started to preview the singing data in real time, while also supporting users to edit the singing (pitch shifting, sound effects and other sound processing).

[0105] (4) During the preview process, if there is insufficient voice data for reasoning or insufficient voice data for playback, a secondary buffer will be triggered (insufficient data causes playback to pause), and the situation will be handled to reduce the impact.

[0106] (5) If a user encounters an error at any stage, the user will be guided to retry based on the error message.

[0107] 2. Detailed implementation of loading the first frame of data and starting inference:

[0108] (1) Calculation of accompaniment download volume:

[0109] like Figure 6 As shown, the preview requires accompaniment. Based on the implementation scheme of downloading and playing simultaneously, it is necessary to download the audio metadata (including duration, sampling rate, sampling bit depth, number of channels, etc.), and then calculate the size of the first frame data using the audio metadata, as shown in the following formula:

[0110] First frame size = First frame duration * Sampling rate * Sampling bit depth * Number of channels / 8.

[0111] In summary, the total amount of data that needs to be downloaded to play the first frame of the accompaniment is (metadata size + first frame size).

[0112] (2) Implementation of the principle of downloading and reasoning simultaneously

[0113] PPG files, timbre files, pitch files, etc. The underlying inference engine retrieves the necessary resource files according to the inference progress. The resource files contain raw data (unlike accompaniment files, which require decapsulation). Therefore, starting inference does not require downloading the complete resource file; it is sufficient to ensure that the corresponding data segments in the resource file exist when inferring the first frame of vocal data. The resource size required for inferring the first frame of vocal data can be calculated using the inference data frame length (SampleSize) and the first frame duration (duration) sent from the background, as shown in the following formula:

[0114] The data size required for the first frame resource = SampleSize * duration.

[0115] (3) Dynamic allocation of download resources:

[0116] Inference resources include backing track files, PPG files, and timbre files. During download, the download speed of each task cannot remain completely consistent, and the progress differences between them may gradually increase over time. This can lead to some resource data volumes far exceeding the current inference requirements, while other resource data is insufficient. In this embodiment, when the progress of related resources exceeds a threshold (supporting dynamic server distribution), the faster-progressing task will be automatically paused, freeing up network resources for the slower-progressing task. Once the progress balance is achieved, all paused tasks will be restarted to ensure that the progress differences between tasks are within a reasonable range.

[0117] 3. Detailed implementation of fast Seek and double buffering:

[0118] If the inferred RTF is less than 1, there will be no secondary buffering during normal playback. However, during preview, users can manually seek to a specific position to start playback. If the inferred vocal data or resource data is insufficient at this time, secondary buffering will occur, and the system will need to wait for sufficient inferred vocal data and resource data.

[0119] If the inference of the song and the download of resources continue to proceed linearly, the user's waiting time could be very long, in the worst case requiring them to wait for all resources to download and the inference to complete. Therefore, adopting a method such as Figure 7As shown, when a user seeks, the download and inference of vocals will prioritize that location, ensuring the user doesn't experience long wait times. After optimization, the longest wait time for a user seeking is only the time it takes to download one frame of resource data plus the time it takes to infer one frame of vocal data, making the user's wait time manageable.

[0120] 4. Comparison of startup time between the original solution and this embodiment:

[0121] In the original solution, resource download and inference were processed sequentially, and previewing could only begin after inference was complete. This embodiment significantly reduces the user's preview time by enabling simultaneous downloading and inference, and simultaneous inference and previewing. In this embodiment, since the inference RTF is less than 1, the inference speed is normally faster than the playback speed, so playback can begin once the first frame of vocal data is inferred. Because the preview requires playing the accompaniment, based on the principle of downloading and playing the accompaniment simultaneously, playback can begin as soon as the accompaniment audio metadata is obtained. The overall calculation time formula is as follows:

[0122] The time taken to download the first frame of the accompaniment is equal to the time taken to download the metadata of the accompaniment resource plus the time taken to download the first frame of the accompaniment resource data; the parsing time of the accompaniment data is very short and can be ignored.

[0123] The time taken for inferring the first frame of human voice = the time taken to download the first frame of human voice resource data + the time taken to infer the first frame of human voice audio signal;

[0124] First frame data preparation time = MAX(first frame accompaniment download time, first frame vocal inference time). After the first frame data (including the first frame accompaniment and the first frame vocal) is ready, the preview engine is started to synthesize the first frame accompaniment and the first frame vocal into the first frame AI singing voice and play it to the user. Since the preview engine takes a certain amount of time to start, but the time to synthesize the AI ​​singing voice is very short and can be ignored, the time for the user to wait for the first frame AI singing voice to play = first frame data preparation time + preview engine start time.

[0125] In this embodiment, by downloading and reasoning simultaneously on the client side and previewing while reasoning, the efficiency of song previewing can be improved. Furthermore, song reasoning uses the client device's hard drive resources, while resource downloading uses network resources. There is no preemption relationship between the two, and parallel processing can maximize the device's performance and effectively reduce the total time spent in the client reasoning process.

[0126] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0127] Based on the same inventive concept, this application also provides a synthesized song processing apparatus for implementing the synthesized song processing method described above. The solution provided by this apparatus is similar to the implementation described in the above method; therefore, the specific limitations in one or more embodiments of the synthesized song processing apparatus provided below can be found in the limitations of the synthesized song processing method described above, and will not be repeated here.

[0128] In one embodiment, such as Figure 8 As shown, a song synthesis processing device is provided, comprising: a synthesis resource acquisition module 801, a first thread execution module 802, a second thread execution module 803, and a main process execution module 804, wherein:

[0129] The synthesis resource acquisition module 801 is used to determine the accompaniment resources and vocal resources stored on the server for synthesizing the target song selected by the user; the vocal resources carry the user's pronunciation characteristics.

[0130] The first thread execution module 802 is used to execute the task corresponding to the current frame through the first thread: download the accompaniment data of the accompaniment resource in the current frame from the server, and parse the accompaniment data of the accompaniment resource in the current frame to generate the accompaniment audio signal of the current frame.

[0131] The second thread execution module 803 is used to execute the task corresponding to the current frame through the second thread: download the human voice data of the human voice resource in the current frame from the server, and perform inference on the human voice data of the human voice resource in the current frame to generate the human voice audio signal of the current frame; wherein: the current frame includes the first frame and subsequent frames, and the first frame task of the first thread and the first frame task of the second thread are executed synchronously.

[0132] The main process execution module 804 is used to execute the first frame playback task through the main process: when it is detected that the accompaniment audio signal and the vocal audio signal of the first frame have been generated, the preview engine is started, and the accompaniment audio signal and the vocal audio signal of the first frame are combined through the preview engine to obtain the song audio signal of the first frame and the song audio signal of the first frame is played.

[0133] The first thread execution module 802 is also used to synchronously execute the subsequent frame tasks of the first thread to generate the accompaniment audio signal of the subsequent frames during the execution of the first frame playback task in the main process.

[0134] The second thread execution module 803 is also used to synchronously execute the subsequent frame tasks of the second thread to generate the human voice audio signal of the subsequent frames during the execution of the first frame playback task in the main process.

[0135] The main process execution module 804 is also used to execute subsequent frame playback tasks through the main process: whenever it is detected that the accompaniment audio signal and the vocal audio signal of the same subsequent frame have been generated, the preview engine synthesizes the accompaniment audio signal and the vocal audio signal of the same subsequent frame to obtain the song audio signal of the same subsequent frame and plays the song audio signal of the same subsequent frame until the same subsequent frame is the end frame of the target song.

[0136] Each module in the aforementioned song synthesis processing device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0137] In one embodiment, a computer device is provided, which may be a client, and its internal structure diagram may be as follows: Figure 9As shown, the computer device includes a processor, memory, input / output interfaces, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interfaces. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, Near Field Communication (NFC), or other technologies. When the computer program is executed by the processor, it implements a method for synthesizing songs. The display unit is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device casing, or an external keyboard, touchpad or mouse.

[0138] Those skilled in the art will understand that Figure 9 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0139] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0140] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0141] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0142] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0143] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0144] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0145] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A method for processing synthesized songs, characterized in that, Applied to a client, the method includes: For the target song selected by the user, determine the accompaniment resources and vocal resources stored on the server for synthesizing the target song; the vocal resources carry the user's pronunciation characteristics; The first thread executes the task corresponding to the current frame: downloading the accompaniment data of the accompaniment resource in the current frame from the server, and parsing the accompaniment data of the accompaniment resource in the current frame to generate the accompaniment audio signal of the current frame; The second thread executes the task corresponding to the current frame: downloading the voice data of the voice resource in the current frame from the server, and inferring the voice data of the voice resource in the current frame to generate the voice audio signal of the current frame; wherein: the current frame includes the first frame and subsequent frames, and the first frame task of the first thread and the first frame task of the second thread are executed synchronously. The first frame playback task is executed through the main process: when it is detected that the accompaniment audio signal and the human voice audio signal of the first frame have been generated, the preview engine is started, and the accompaniment audio signal and the human voice audio signal of the first frame are combined by the preview engine to obtain the song audio signal of the first frame and the song audio signal of the first frame is played. During the execution of the first frame playback task in the main process, the subsequent frame task of the first thread is executed synchronously to generate the accompaniment audio signal of the subsequent frame, and the subsequent frame task of the second thread is executed synchronously to generate the human voice audio signal of the subsequent frame. The main process executes subsequent frame playback tasks: whenever the accompaniment audio signal and the vocal audio signal of the same subsequent frame are detected to be generated, the preview engine synthesizes the accompaniment audio signal and the vocal audio signal of the same subsequent frame to obtain the song audio signal of the same subsequent frame and plays the song audio signal of the same subsequent frame until the same subsequent frame is the end frame of the target song.

2. The method according to claim 1, characterized in that, The step of downloading the accompaniment data of the accompaniment resource in the current frame from the server and parsing the accompaniment data of the accompaniment resource in the current frame to generate the accompaniment audio signal of the current frame includes: Obtain the audio metadata of the target song, and download the accompaniment data of the accompaniment resource in the current frame from the server based on the audio metadata; The accompaniment data of the accompaniment resource in the current frame is parsed using the audio metadata to generate the accompaniment audio signal for the current frame.

3. The method according to claim 2, characterized in that, The step of downloading the accompaniment data of the accompaniment resource in the current frame from the server based on the audio metadata includes: Obtain the signal duration, sampling rate, sampling bit depth, and number of channels of the current frame from the audio metadata; and obtain the data volume corresponding to the accompaniment data of the current frame based on the signal duration, sampling rate, sampling bit depth, and number of channels. Based on the stated data volume, download the accompaniment data of the accompaniment resource in the current frame from the server.

4. The method according to claim 2, characterized in that, When the current frame is the first frame, obtaining the audio metadata of the target song includes: The accompaniment resource is downloaded from the server, and the accompaniment resource is parsed to obtain the audio metadata of the target song.

5. The method according to claim 1, characterized in that, The step of downloading the human voice data of the human voice resource in the current frame from the server includes: Obtain the data frame length and signal duration of the current frame; based on the data frame length and signal duration, obtain the data volume corresponding to the human voice data of the current frame; According to the stated data volume, download the voice data of the voice resource in the current frame from the server.

6. The method according to claim 1, characterized in that, The number of human voice resources is multiple; during the process of synchronously executing the subsequent frame tasks of the first thread to generate the accompaniment audio signal of the subsequent frame and synchronously executing the subsequent frame tasks of the second thread to generate the human voice audio signal of the subsequent frame, the method further includes: Obtain the download progress of each of the aforementioned vocal resources, and obtain the download progress of the aforementioned accompaniment resources; If the difference between the download progress of each resource is greater than a preset threshold, the download efficiency of each vocal resource and the accompaniment resource is adjusted, wherein the difference between the download progress of each resource after adjustment meets the preset difference requirement.

7. The method according to claim 1, characterized in that, In executing subsequent frame playback tasks through the main process, the method further includes: In response to the user's request to adjust the playback progress of the target song, obtain the playback progress frame corresponding to the adjusted playback progress; Once it is detected that the song audio signal of the playback progress frame has been synthesized, the song audio signal of the playback progress frame is played through the preview engine.

8. The method according to claim 7, characterized in that, After obtaining the playback progress frame corresponding to the adjusted playback progress, the method further includes: When it is detected that the accompaniment audio signal of the playback progress frame has not been fully generated, the task corresponding to the playback progress frame is executed by the first thread to generate the accompaniment audio signal of the playback progress frame. When it is detected that the human voice audio signal of the playback progress frame has not been fully generated, the task corresponding to the playback progress frame is executed through the second thread to generate the human voice audio signal of the playback progress frame. The main process executes the playback progress frame task: when it is detected that the accompaniment audio signal and the vocal audio signal of the playback progress frame have been generated, the preview engine synthesizes the accompaniment audio signal and the vocal audio signal of the playback progress frame to obtain the song audio signal of the playback progress frame and plays the song audio signal of the playback progress frame.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 8.

11. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 8.

Images