Synthetic song processing method, computer equipment and storage medium

Abstract

The invention relates to a synthetic song processing method, computer equipment and a readable storage medium. The method comprises the steps of determining accompaniment resources and human voice resources for a target song; the voice resources carry pronunciation characteristics of the user; executing a task corresponding to the current frame through the first thread to generate an accompaniment audio signal of the current frame; executing a task corresponding to the current frame through a second thread to generate a human voice audio signal of the current frame; when the accompaniment audio signal of the first frame and the human voice audio signal of the first frame are monitored to be completely generated, starting a preview engine, synthesizing through the preview engine to obtain a song audio signal of the first frame, and playing the song audio signal of the first frame; in the process of executing the first frame playing task in the main process, executing the subsequent frame task of the first thread and the subsequent frame task of the second thread synchronously; and executing the follow-up frame playing task through the main process until the follow-up frame is an end frame. By adopting the method, the preview or editing processing efficiency of the synthesized song can be improved.

Description

Technical Field

[0001] The present application relates to the technical field of audio processing, and in particular to a method for processing a synthesized song, a computer device, a computer-readable storage medium, and a computer program product. Background Art

[0002] With the development of audio processing technology, a technology for generating synthesized songs has emerged. This technology can pre-download the accompaniment of the synthesized song and the vocal resource file used to generate the singing in the synthesized song, and generate the vocal audio of the synthesized song by inferring the vocal resource file. After that, the user can preview the synthesized song, synthesize the accompaniment and the vocal audio to obtain the song audio for playback, and can also edit the synthesized song audio.

[0003] In traditional technology, if a user needs to preview or edit the synthesized song audio, he usually needs to wait for the accompaniment and vocal resources of the synthesized song to be downloaded, and the vocal audio to be generated. That is, the user needs to wait for the download of the accompaniment and vocal resources to be completed, and then infer the vocal audio. Only after the vocal audio inference is completed, can the song audio be previewed or edited. Therefore, the current preview or editing processing efficiency of synthesized songs is low. Summary of the invention

[0004] Based on this, it is necessary to provide a synthetic song processing method, device, computer equipment, computer-readable storage medium and computer program product that can improve the preview or editing efficiency of synthetic songs in response to the above technical problems.

[0005] In a first aspect, the present application provides a method for processing a synthesized song, which is applied to a client and includes:

[0006] For a target song selected by a user, determining an accompaniment resource and a vocal resource stored in a server for synthesizing the target song; the vocal resource carries the pronunciation characteristics of the user;

[0007] Executing the task corresponding to the current frame through the first thread: downloading the accompaniment data of the accompaniment resource in the current frame from the server, and parsing the accompaniment data of the accompaniment resource in the current frame to generate an accompaniment audio signal of the current frame;

[0008] Execute the task corresponding to the current frame through the second thread: download the human voice data of the human voice resource in the current frame from the server, and infer the human voice data of the human voice resource in the current frame to generate a human voice audio signal of the current frame; wherein: the current frame includes a first frame and subsequent frames, and the first frame task of the first thread is executed synchronously with the first frame task of the second thread;

[0009] The first frame playing task is executed through the main process: when it is monitored that the accompaniment audio signal of the first frame and the human voice audio signal of the first frame are both generated, the preview engine is started, the accompaniment audio signal of the first frame and the human voice audio signal of the first frame are synthesized by the preview engine to obtain the song audio signal of the first frame, and the song audio signal of the first frame is played;

[0010] During the execution of the first frame playing task by the main process, the subsequent frame tasks of the first thread are synchronously executed to generate the accompaniment audio signals of the subsequent frames, and the subsequent frame tasks of the second thread are synchronously executed to generate the human voice audio signals of the subsequent frames;

[0011] The subsequent frame playback task is executed through the main process: whenever it is monitored that the accompaniment audio signal and the vocal audio signal of the same subsequent frame are generated, the accompaniment audio signal and the vocal audio signal of the same subsequent frame are synthesized by the preview engine to obtain the song audio signal of the same subsequent frame and play the song audio signal of the same subsequent frame until the same subsequent frame is the end frame of the target song.

[0012] In one of the embodiments, downloading the accompaniment data of the accompaniment resource in the current frame from the server, and parsing the accompaniment data of the accompaniment resource in the current frame to generate an accompaniment audio signal of the current frame, includes: acquiring audio metadata of the target song, and downloading the accompaniment data of the accompaniment resource in the current frame from the server based on the audio metadata; and parsing the accompaniment data of the accompaniment resource in the current frame using the audio metadata to generate an accompaniment audio signal of the current frame.

[0013] In one of the embodiments, downloading the accompaniment data of the accompaniment resource in the current frame from the server based on the audio metadata includes: acquiring the signal duration, sampling rate, sampling bit depth and number of channels of the current frame from the audio metadata; obtaining the data volume corresponding to the accompaniment data of the current frame according to the signal duration, the sampling rate, the sampling bit depth and the number of channels; and downloading the accompaniment data of the accompaniment resource in the current frame from the server according to the data volume.

[0014] In one of the embodiments, when the current frame is the first frame, obtaining the audio metadata of the target song includes: downloading the accompaniment resource from the server, and parsing the accompaniment resource to obtain the audio metadata of the target song.

[0015] In one of the embodiments, downloading the vocal data of the vocal resource in the current frame from the server includes: obtaining the data frame length and signal duration of the current frame; obtaining the data volume corresponding to the vocal data of the current frame according to the data frame length and signal duration; and downloading the vocal data of the vocal resource in the current frame from the server according to the data volume.

[0016] In one of the embodiments, there are multiple vocal resources; in the process of synchronously executing the subsequent frame tasks of the first thread to generate the accompaniment audio signals of the subsequent frames and synchronously executing the subsequent frame tasks of the second thread to generate the vocal audio signals of the subsequent frames, the method also includes: obtaining the resource download progress of each of the vocal resources, and obtaining the resource download progress of the accompaniment resources; when the difference between the download progress of each of the resources is greater than a preset threshold, adjusting the resource download efficiency of each of the vocal resources and the accompaniment resources, wherein the difference between the adjusted download progress of each of the resources meets the preset difference requirement.

[0017] In one of the embodiments, in executing subsequent frame playback tasks through the main process, the method also includes: responding to the user's request to adjust the playback progress of the target song, obtaining a playback progress frame corresponding to the adjusted playback progress; when it is monitored that the song audio signal of the playback progress frame has been synthesized, playing the song audio signal of the playback progress frame through the preview engine.

[0018] In one of the embodiments, after obtaining the playback progress frame corresponding to the adjusted playback progress, it also includes: when it is monitored that the accompaniment audio signal of the playback progress frame has not been completely generated, executing the task corresponding to the playback progress frame through the first thread to generate the accompaniment audio signal of the playback progress frame; when it is monitored that the human voice audio signal of the playback progress frame has not been completely generated, executing the task corresponding to the playback progress frame through the second thread to generate the human voice audio signal of the playback progress frame; executing the playback progress frame task through the main process: when it is monitored that the accompaniment audio signal and the human voice audio signal of the playback progress frame have been completely generated, synthesizing the accompaniment audio signal and the human voice audio signal of the playback progress frame through the preview engine to obtain the song audio signal of the playback progress frame and play the song audio signal of the playback progress frame.

[0019] In a second aspect, the present application further provides a synthetic song processing device, which is applied to a client, comprising:

[0020] A synthesis resource acquisition module, for determining, for a target song selected by a user, an accompaniment resource and a vocal resource stored on a server for synthesizing the target song; the vocal resource carries the pronunciation characteristics of the user;

[0021] A first thread execution module is used to execute a task corresponding to the current frame through the first thread: downloading the accompaniment data of the accompaniment resource in the current frame from the server, and parsing the accompaniment data of the accompaniment resource in the current frame to generate an accompaniment audio signal of the current frame;

[0022] The second thread execution module is used to execute the task corresponding to the current frame through the second thread: download the human voice data of the human voice resource in the current frame from the server, and infer the human voice data of the human voice resource in the current frame to generate a human voice audio signal of the current frame; wherein: the current frame includes a first frame and subsequent frames, and the first frame task of the first thread is executed synchronously with the first frame task of the second thread;

[0023] The main process execution module is used to execute the first frame playback task through the main process: when it is monitored that the accompaniment audio signal of the first frame and the human voice audio signal of the first frame are both generated, the preview engine is started, the accompaniment audio signal of the first frame and the human voice audio signal of the first frame are synthesized by the preview engine to obtain the song audio signal of the first frame and the song audio signal of the first frame is played;

[0024] The first thread execution module is further used to synchronously execute the subsequent frame tasks of the first thread to generate accompaniment audio signals of subsequent frames during the execution of the first frame playing task by the main process;

[0025] The second thread execution module is further used to synchronously execute the subsequent frame tasks of the second thread during the main process executing the first frame playback task to generate human voice audio signals of subsequent frames;

[0026] The main process execution module is also used to execute the subsequent frame playback task through the main process: whenever it is monitored that the accompaniment audio signal and the human voice audio signal of the same subsequent frame are generated, the accompaniment audio signal and the human voice audio signal of the same subsequent frame are synthesized by the preview engine to obtain the song audio signal of the same subsequent frame and play the song audio signal of the same subsequent frame until the same subsequent frame is the end frame of the target song.

[0027] In a third aspect, the present application further provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the method described in any one of the embodiments of the first aspect when executing the computer program.

[0028] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the method described in any one of the embodiments of the first aspect.

[0029] In a fifth aspect, the present application further provides a computer program product, comprising a computer program, which, when executed by a processor, implements the steps of the method described in any one of the embodiments of the first aspect.

[0030] The above-mentioned synthesized song processing method, device, computer equipment, computer-readable storage medium and computer program product obtain the accompaniment resources and vocal resources of the target song selected by the user from the server through the client, and synchronously execute the process of obtaining the accompaniment audio signal of the first frame by using the accompaniment resources and obtaining the vocal audio signal of the first frame by using the vocal resources through the first thread and the second thread, and after the main process detects that the accompaniment audio signal and the vocal audio signal of the first frame are generated, start the preview engine to synthesize the song audio signal of the first frame and play it, and can also synchronously execute the subsequent frame tasks to generate the first frame when the main process executes the first frame playback task. The accompaniment audio signal and the vocal audio signal of the subsequent frames are generated, and the subsequent frame playback task is executed through the main process until the subsequent frame is the end frame of the target song to complete the playback of the target song. It can be seen that through this method, it is only necessary to complete the acquisition of the accompaniment audio signal and the vocal audio signal of the starting frame, and then the preview engine can be started to play the song audio signal of the first frame, and the accompaniment audio signal and the vocal audio signal of the subsequent frames can be acquired during the playback of the song audio signal, thereby realizing the parallel processing of downloading the accompaniment resources and the vocal resources, accompaniment analysis, singing voice reasoning and synthesis playback of the song audio signal, thereby improving the preview or editing processing efficiency of the synthesized song. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0032] Figure 1 A schematic diagram of a process flow of a method for processing a synthesized song in one embodiment;

[0033] Figure 2 A schematic diagram of a flow chart of generating an accompaniment audio signal of a current frame in one embodiment;

[0034] Figure 3 A schematic diagram of a flow chart of playing a song audio signal of an adjusted progress frame in one embodiment;

[0035] Figure 4 A schematic diagram of a flow chart of playing a song audio signal of an adjusted progress frame in another embodiment;

[0036] Figure 5 A schematic flow chart of a method for real-time preview and editing of inference singing in one embodiment;

[0037] Figure 6 A schematic diagram of the amount of data downloaded for the first frame of an accompaniment in one embodiment;

[0038] Figure 7 A schematic diagram of a secondary buffering process implementation in one embodiment;

[0039] Figure 8 is a structural block diagram of a synthetic song processing device in one embodiment;

[0040] Fig. 9 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0042] In one embodiment, Figure 1 As shown, a method for processing a synthesized song is provided. This embodiment uses the method applied to a client as an example and is implemented through interaction between the client and the server. In this embodiment, the method includes the following steps:

[0043] Step S101, for the target song selected by the user, determine the accompaniment resources and vocal resources stored in the server for synthesizing the target song; the vocal resources carry the pronunciation characteristics of the user.

[0044] Among them, the target song refers to the song selected by the user that needs to be generated through song synthesis. The song audio of the target song can be composed of two parts, namely accompaniment audio and vocal audio. The accompaniment resource refers to the resource used to generate the accompaniment audio signal of the target song, such as the accompaniment audio file, and the vocal resource refers to the resource carrying the pronunciation characteristics of the user and used to generate the vocal audio signal of the target song, such as PPG, i.e. Polyphonic Performance Generator file, timbre file and pitch file, etc.

[0045] Specifically, when the user selects a target song to be synthesized, the client may first determine the accompaniment resource file and the vocal resource file stored in the server for synthesizing the target song.

[0046] Step S102, executing a task corresponding to the current frame through the first thread: downloading the accompaniment data of the accompaniment resource in the current frame from the server, and parsing the accompaniment data of the accompaniment resource in the current frame to generate an accompaniment audio signal of the current frame;

[0047] Step S103, execute the task corresponding to the current frame through the second thread: download the vocal data of the vocal resource in the current frame from the server, and infer the vocal data of the vocal resource in the current frame to generate the vocal audio signal of the current frame; wherein: the current frame includes the first frame and subsequent frames, and the first frame task of the first thread is executed synchronously with the first frame task of the second thread.

[0048] The current frame refers to any song signal frame of the target song, which may include the first frame and subsequent frames of the target song. The first thread is a thread for generating an accompaniment audio signal, and the second thread is a thread for generating a vocal audio signal, that is, a thread for generating an audio signal of the pure vocal part of the target song, and the first frame tasks executed by the first thread and the second thread are executed synchronously, that is, the process of the first thread generating the accompaniment audio signal of the first frame and the second thread generating the vocal audio signal of the first frame can be performed synchronously.

[0049] Specifically, the client can start the first thread to execute the generation of the accompaniment audio signal for the current frame, which can be to first download the accompaniment data corresponding to the accompaniment resource in the current frame from the server, and then parse the accompaniment data of the current frame to generate the accompaniment audio signal of the current frame.

[0050] At the same time, the client can also synchronously start the second thread to execute the generation of the human voice audio signal for the current frame. It can first download the human voice data of the human voice resource in the current frame from the server, and use the singing inference engine to perform singing inference on the above-mentioned human voice data of the current frame to generate the human voice audio signal of the current frame.

[0051] Step S104, execute the first frame playback task through the main process: when it is monitored that the accompaniment audio signal of the first frame and the vocal audio signal of the first frame are generated, start the preview engine, and synthesize the accompaniment audio signal of the first frame and the vocal audio signal of the first frame through the preview engine to obtain the song audio signal of the first frame and play the song audio signal of the first frame.

[0052] The main process is a thread used to synthesize song audio signals and play them. After detecting that the first frame accompaniment audio signal and the vocal audio signal have been generated, the preview engine can be started through the main process. The preview engine refers to a playback engine used to preview the target song, that is, to play the target song. Through the preview engine, the first frame accompaniment audio signal obtained by the first thread and the first frame vocal audio signal obtained by the second thread can be synthesized. After obtaining the first frame song audio signal of the target song, the first frame song audio signal can be played.

[0053] Wherein, during the main process executing the first frame playing task, the subsequent frame tasks of the first thread are synchronously executed to generate the accompaniment audio signals of the subsequent frames, and the subsequent frame tasks of the second thread are synchronously executed to generate the human voice audio signals of the subsequent frames;

[0054] Step S105, execute the subsequent frame playback task through the main process: whenever it is monitored that the accompaniment audio signal and the vocal audio signal of the same subsequent frame are generated, the accompaniment audio signal and the vocal audio signal of the same subsequent frame are synthesized by the preview engine to obtain the song audio signal of the same subsequent frame and play the song audio signal of the same subsequent frame until the same subsequent frame is the end frame of the target song.

[0055] The end frame refers to the last song signal frame in the song to be synthesized. While the client executes the first frame playback task through the main process, it can also synchronously execute the subsequent frame tasks of the first thread and the subsequent frame tasks of the second thread, thereby generating the accompaniment audio signal of the subsequent frame through the first thread and the vocal audio signal of the subsequent frame through the second thread.

[0056] Afterwards, when the main process detects that the accompaniment audio signal and the vocal audio signal of the same subsequent frame have been generated, the accompaniment audio signal and the vocal audio signal of the same subsequent frame can be synthesized through the preview engine to obtain the song audio signal of the same subsequent frame and play it until the same subsequent frame is the end frame of the target song.

[0057] For example, when the main process executes the first frame playback task, the client can also synchronously execute the subsequent frame tasks of the first thread and the subsequent frame tasks of the second thread, that is, synchronously execute the accompaniment audio signal of the second frame generated by the first thread, and the vocal audio signal of the second frame generated by the second thread. Since the accompaniment audio signal and the vocal audio signal of the second frame are both generated at this time, the main thread can also execute the playback task of the second frame, synthesize the accompaniment audio signal and the vocal audio signal of the second frame to obtain the song audio signal of the second frame, and play the song audio signal of the second frame. At this time, the client can also synchronously execute the accompaniment audio signal of the third frame generated by the first thread, and the vocal audio signal of the third frame generated by the second thread, and so on, until the main process completes the synthesis and playback of the song audio signal of the end frame, and completes the synthesis and playback of the target song.

[0058] The above-mentioned song synthesis processing method, device, computer equipment, computer-readable storage medium and computer program product determine the accompaniment resources and vocal resources stored in the server for synthesizing the target song selected by the user through the client; the vocal resources carry the pronunciation characteristics of the user; the task corresponding to the current frame is executed through the first thread: the accompaniment data of the accompaniment resource in the current frame is downloaded from the server, and the accompaniment data of the accompaniment resource in the current frame is parsed to generate the accompaniment audio signal of the current frame; the task corresponding to the current frame is executed through the second thread: the vocal data of the vocal resource in the current frame is downloaded from the server, and the vocal data of the vocal resource in the current frame is inferred to generate the vocal audio signal of the current frame; wherein: the current frame includes the first frame and subsequent frames, and the first frame task of the first thread is executed synchronously with the first frame task of the second thread; the first frame playback is executed through the main process Playback task: When it is detected that the accompaniment audio signal of the first frame and the vocal audio signal of the first frame are both generated, start the preview engine, and synthesize the accompaniment audio signal of the first frame and the vocal audio signal of the first frame through the preview engine to obtain the song audio signal of the first frame and play the song audio signal of the first frame; in the process of executing the first frame playback task in the main process, synchronously execute the subsequent frame tasks of the first thread to generate the accompaniment audio signal of the subsequent frame and synchronously execute the subsequent frame tasks of the second thread to generate the vocal audio signal of the subsequent frame; execute the subsequent frame playback task through the main process: whenever it is detected that the accompaniment audio signal and the vocal audio signal of the same subsequent frame are both generated, synthesize the accompaniment audio signal and the vocal audio signal of the same subsequent frame through the preview engine to obtain the song audio signal of the same subsequent frame and play the song audio signal of the same subsequent frame, until the same subsequent frame is the end frame of the target song. The present application obtains the accompaniment resources and vocal resources of the target song selected by the user from the server through the client, and synchronously executes the process of obtaining the accompaniment audio signal of the first frame by using the accompaniment resources and obtaining the vocal audio signal of the first frame by using the vocal resources through the first thread and the second thread, and after the main process detects that the accompaniment audio signal and the vocal audio signal of the first frame are generated, the preview engine is started to synthesize the song audio signal of the first frame and play it, and when the main process executes the first frame playback task, the first thread and the second thread can synchronously execute the subsequent frame tasks to generate the accompaniment audio signal and the vocal audio signal of the subsequent frames. number, and executes subsequent frame playback tasks through the main process until the subsequent frame is the end frame of the target song to complete the playback of the target song. It can be seen that through this method, it is only necessary to complete the acquisition of the accompaniment audio signal and the vocal audio signal of the starting frame, and then the preview engine can be started to play the song audio signal of the first frame, and the accompaniment audio signal and the vocal audio signal of the subsequent frames can be acquired during the playback of the song audio signal, thereby realizing the parallel processing of downloading accompaniment resources and vocal resources, accompaniment analysis, singing voice reasoning and synthetic playback of song audio signals, thereby improving the preview or editing processing efficiency of the synthesized song.

[0059] In one embodiment, Figure 2 As shown, step S102 may further include:

[0060] Step S201, obtaining audio metadata of the target song, and downloading accompaniment data of the accompaniment resource in the current frame from the server based on the audio metadata.

[0061] Among them, audio metadata refers to the metadata of the target song accompaniment audio signal. When the client downloads the accompaniment data of the accompaniment resource in the current frame from the server, it first needs to obtain the audio metadata of the target song, and then can download the accompaniment data of the accompaniment resource in the current frame from the server based on the metadata.

[0062] Step S202: parsing the accompaniment data of the accompaniment resource in the current frame by using the audio metadata to generate an accompaniment audio signal of the current frame.

[0063] After obtaining the audio metadata, the client may also use the audio metadata to parse the accompaniment data of the current frame, thereby generating an accompaniment audio signal of the current frame.

[0064] In this embodiment, when obtaining the accompaniment audio signal of the current frame, it is first necessary to obtain the audio metadata of the target song. The audio metadata can be used to instruct the client to download the accompaniment data of the current frame from the server, and to instruct the process of parsing the accompaniment data of the current frame to generate the accompaniment audio signal of the current frame. In this way, the accuracy of generating the accompaniment audio signal of the current frame can be improved.

[0065] Furthermore, step S201 may further include: obtaining the signal duration, sampling rate, sampling bit depth and number of channels of the current frame from the audio metadata; obtaining the data volume corresponding to the accompaniment data of the current frame according to the signal duration, sampling rate, sampling bit depth and number of channels; and downloading the accompaniment data of the accompaniment resource in the current frame from the server according to the data volume.

[0066] In this embodiment, the audio metadata may record information such as duration, sampling rate, sampling bit depth, number of channels, etc., where the duration may refer to the signal duration of the current frame, that is, the frame length of the current frame, and the amount of data corresponding to the accompaniment resource of the current frame refers to the data size of the accompaniment resource corresponding to the current frame.

[0067] Specifically, the client can obtain the signal duration of the current frame, the sampling rate, sampling bit depth, and number of channels of the accompaniment audio signal based on the audio metadata. After obtaining the above information, the client can calculate the data volume corresponding to the accompaniment data of the current frame based on the signal duration of the current frame, the sampling rate, sampling bit depth, and number of channels of the accompaniment audio signal. For example, the data volume corresponding to the accompaniment resource of the current frame can be calculated by the following formula: current frame size = current frame duration * sampling rate * sampling bit depth * number of channels / 8, so that the client can download the data corresponding to the above data volume from the accompaniment resource as the accompaniment data of the current frame.

[0068] In this embodiment, the client can determine the data volume corresponding to the accompaniment data of the current frame based on information such as the signal duration of the current frame, the sampling rate of the accompaniment audio signal, the sampling bit depth, and the number of channels stored in the audio metadata, and thus obtain the accompaniment data of the current frame from the accompaniment resource according to the data volume. In this way, the accuracy of obtaining the accompaniment data of the current frame can be improved.

[0069] In addition, when the current frame is the first frame, step S201 may further include: downloading accompaniment resources from the server, and parsing the accompaniment resources to obtain audio metadata of the target song.

[0070] In this embodiment, the audio metadata may also be carried in the accompaniment resource, and the audio metadata is set at the front end of the accompaniment resource. If the current frame is the first frame, the client needs to first download the accompaniment resource from the server, and parse the downloaded accompaniment resource while downloading the accompaniment resource to obtain the audio metadata of the target song, and then download the accompaniment data of the first frame according to the parsed audio metadata. If it is not the first frame, it means that the audio metadata has been downloaded and parsed, and the pre-parsed audio metadata can be directly obtained.

[0071] In this embodiment, when the current frame is the first frame, the client can also obtain the audio metadata by downloading the accompaniment resource and parsing the accompaniment resource. This method can improve the efficiency of obtaining the audio metadata.

[0072] In one embodiment, step S103 may further include: obtaining the data frame length and signal duration of the current frame; obtaining the data volume corresponding to the human voice data of the current frame according to the data frame length and signal duration; and downloading the human voice data of the human voice resource in the current frame from the server according to the data volume.

[0073] For downloading the vocal data of the current frame, it is necessary to first determine the data volume corresponding to the vocal data of the current frame. The data volume can be determined based on the data frame length and signal duration of the current frame, where the data frame length and signal duration can be sent from the server to the client.

[0074] Specifically, the background server can send the data frame length and signal duration of the current frame to the client, and the client can obtain the data frame length and signal duration of the current frame, and then calculate the data volume corresponding to the singing resource of the current frame based on the data frame length and signal duration. For example, the data volume corresponding to the singing resource of the current frame can be calculated by the following formula: The data size required for the current frame vocal data = data frame length * first frame duration, and then the vocal data of the current frame can be downloaded from the singing resource file according to the above data volume.

[0075] In this embodiment, the client can determine the data volume corresponding to the singing resource of the current frame based on the data frame length and signal duration of the current frame sent by the background server, and then obtain the vocal data of the current frame from the vocal resource according to the data volume. In this way, the accuracy of obtaining the vocal data of the current frame can be improved.

[0076] In one embodiment, there are multiple vocal resources; the process of executing step S104 may also include: obtaining the resource download progress of each vocal resource, and obtaining the resource download progress of the accompaniment resource. When the difference between the download progress of each resource is greater than a preset threshold, adjusting the resource download efficiency of each vocal resource and the accompaniment resource, wherein the difference between the adjusted download progress of each resource meets the preset difference requirement.

[0077] In this embodiment, the number of vocal resources can be multiple, for example, the vocal resources can include multiple resource files such as PPG, timbre files and pitch files, and the resource download progress represents the resource download progress of each vocal resource and accompaniment resource. In the process of playing the song audio signal through the preview engine, it is necessary to synchronously obtain the vocal resources and accompaniment resources, so that the song audio signal of each audio frame can be synchronously generated and played, so the download progress of various resources needs to be synchronized.

[0078] Specifically, during the process of synchronously executing the subsequent frame tasks of the first thread and the subsequent frame tasks of the second thread, the client can also obtain the resource download progress of each vocal resource and the accompaniment resource in real time. If the difference between the download progress of two resources is greater than a preset threshold, the resource download efficiency of each vocal resource and the accompaniment resource can be adjusted accordingly to reduce the difference between the download progress of each resource.

[0079] For example, during the resource download process, if the download progress of the timbre file is much higher than the download progress of the pitch file, and the difference in download progress is greater than a preset threshold, in this case, the resource download efficiency of each resource can be adjusted, such as pausing the download of the timbre file to increase the download speed of the pitch file. In this way, the resource download progress of each resource can be balanced to reduce the difference between the download progress of each resource.

[0080] In this embodiment, in the process of downloading vocal resources and accompaniment resources, the resource download progress of each vocal resource and accompaniment resource can also be obtained. Therefore, when the difference between the resource download progress is large, the resource download efficiency of each vocal resource and accompaniment resource is adjusted to reduce the difference between each download progress. In this way, the overall efficiency of resource downloading can be further improved, thereby improving the efficiency of generating song audio signals.

[0081] In one embodiment, Figure 3 As shown, in executing the subsequent frame playback task through the main process, the following may also be included:

[0082] Step S301, in response to a user's request to adjust the play progress of a target song, obtaining a play progress frame corresponding to the adjusted play progress.

[0083] The playback progress adjustment request refers to a request by a user to adjust the playback progress of a song audio signal during the playback of the target song audio signal, and the adjusted playback progress refers to the adjusted playback progress when the user triggers the playback progress adjustment request. For example, if the current playback progress of the target song is 10s, when the user needs to adjust the playback progress to 30s, the playback progress adjustment request for the song to be synthesized can be triggered, so that 30s is used as the adjusted playback progress.

[0084] The playback progress frame corresponding to the adjusted playback progress refers to the song signal frame represented by the adjusted playback progress, that is, the song signal frame corresponding to the playback time of 30s. Specifically, after the client receives the playback progress adjustment request for the target song triggered by the user, it can obtain the playback progress frame corresponding to the adjusted playback progress.

[0085] Step S302, when it is detected that the song audio signal of the playing progress frame has been synthesized, the song audio signal of the playing progress frame is played through the preview engine.

[0086] Generally speaking, the efficiency of synthesizing song audio signals is much higher than the efficiency of playing song audio signals. For example, it may only take 0.1 seconds to synthesize a song audio signal of 1 second. In this case, although it is not the turn of the song audio signal of the playback progress frame corresponding to the adjusted playback progress to be played, it is possible that the song audio signal of this frame has been synthesized. In this case, the client can directly play the song audio signal of the playback progress frame corresponding to the adjusted playback progress through the preview engine.

[0087] For example, if the current playback progress of the target song is 10s, when the user adjusts the playback progress to 30s, if the song audio signal of the playback progress frame corresponding to 30s has been synthesized, the client can directly play the song audio signal of the playback progress frame corresponding to 30s through the preview engine.

[0088] In this embodiment, after the playback progress is adjusted, if the song audio signal corresponding to the playback progress frame corresponding to the adjusted playback progress has been synthesized, the above song audio signal can be played directly. In this way, the playback of the song audio signal can be achieved after the playback progress is adjusted, thereby improving the playback efficiency of the song audio signal after the playback progress is adjusted.

[0089] In addition, if Figure 4 As shown, after step S301, the following steps may also be included:

[0090] Step S401: when it is detected that the accompaniment audio signal of the playing progress frame has not been completely generated, the task corresponding to the playing progress frame is executed by the first thread to generate the accompaniment audio signal of the playing progress frame.

[0091] In this embodiment, the song audio signal of the playback progress frame has not been synthesized, which is usually caused by the following two factors, namely, the accompaniment audio signal corresponding to the playback progress frame has not been completely generated, or the human voice audio signal corresponding to the playback progress frame has not been completely generated, or both the accompaniment audio signal and the human voice audio signal corresponding to the playback progress frame have not been completely generated.

[0092] Specifically, if the accompaniment audio signal of the playback progress frame has not been generated, for example, the playback progress of the current target song is 1s, and the user adjusts the playback progress to 100s, at this time, although the synthesis efficiency of the song audio signal is higher than the playback efficiency, the client may still not obtain the accompaniment audio signal of the current playback frame. In this case, the client can execute the task corresponding to the playback progress frame through the first thread to generate the accompaniment audio signal of the playback progress frame, that is, the accompaniment audio signal of the playback progress frame is generated first.

[0093] For example, when the user adjusts the playback progress to 100s, the accompaniment audio signal of the song signal frame corresponding to 100s may not be obtained at this time. Then the client can execute the task of the song signal frame corresponding to 100s through the first thread, thereby giving priority to generating the accompaniment audio signal of the song signal frame corresponding to 100s.

[0094] Step S402: when it is detected that the human voice audio signal of the playing progress frame has not been completely generated, the task corresponding to the playing progress frame is executed by the second thread to generate the human voice audio signal of the playing progress frame.

[0095] Similarly, if the vocal audio signal of the playback progress frame has not been generated, the client can also execute the task corresponding to the playback progress frame through the second thread to generate the vocal audio signal of the playback progress frame, that is, the client preferentially downloads the vocal data of the song signal frame corresponding to 100s of the vocal resource from the server, and infers the vocal data of the song signal frame corresponding to 100s to generate the vocal audio signal of the song signal frame corresponding to 100s.

[0096] Step S403, execute the playing progress frame task through the main process: when it is monitored that the accompaniment audio signal and the vocal audio signal of the playing progress frame are generated, the accompaniment audio signal and the vocal audio signal of the playing progress frame are synthesized by the preview engine to obtain the song audio signal of the playing progress frame and play the song audio signal of the playing progress frame.

[0097] Finally, after the main process detects that the accompaniment audio signal and the vocal audio signal of the playback progress frame have been generated, the accompaniment audio signal and the vocal audio signal of the playback progress frame can be synthesized through the preview engine to obtain the song audio signal of the playback progress frame and play it.

[0098] In this embodiment, if the vocal audio signal or accompaniment audio signal of the adjusted playback progress frame is not obtained, the step of obtaining the vocal audio signal or accompaniment audio signal of the adjusted playback progress frame can be performed preferentially and played when the acquisition is completed. In this way, the playback buffering time of the song audio signal after the playback progress is adjusted can be reduced.

[0099] In one embodiment, a method for real-time preview and editing of inferred singing on a client is also provided. This method can be applied in the scenario of AI singing synthesis. Currently, to generate AI synthesized singing, it is usually necessary to download resource files such as PPG files, timbre files, pitch files, and accompaniment data used for inferred singing. Only when the resources are downloaded can the inference task be started and executed. Under normal circumstances, the RTF of singing inference is 0.1, that is, 5 minutes of singing requires 30 seconds of waiting. If the user wants to preview or edit the inference-generated singing, he needs to wait for the resource download to be completed and the singing inference task to be completed. The total waiting time on the entire link is relatively long.

[0100] The method for real-time preview and editing of inferred singing provided by this embodiment, resource files such as PPG files, timbre files, pitch files and accompaniment files, only need to download the first frame data to start the singing inference task in parallel. Since resource download uses network resources and singing inference uses client hardware resources, there is no preemption between the two. When the singing inference gets the first frame data, the singing is previewed in real time, and the remaining resources and singing inference are downloaded in parallel during the preview process. Through the parallel processing of the three processes of resource download, singing inference, real-time preview and editing, and solving the problems of resource dependence, state conflict and so on among the three, the waiting time for users to preview singing is effectively reduced, and the problems of resource dependence, state conflict and so on caused by parallel processing are also optimized.

[0101] 1. The overall process can be as follows Figure 5 As shown, the following steps may be included:

[0102] (1) After the user obtains the AI ​​work, the client will start to infer the singing voice. It is necessary to download the inferred singing voice resource files (PPG, timbre file, pitch file). After downloading the first frame data of all resources, parallel inference can be performed. The inferred data is saved in the inferred singing voice storage.

[0103] (2) The accompaniment is downloaded in parallel. When the accompaniment can be parsed into the audio metadata, the preview engine parameters can be initialized through the audio metadata.

[0104] (3) Once the accompaniment audio metadata and the first frame data have been inferred, the preview engine can be started to preview the singing data in real time, while supporting users to edit the singing (pitch change, sound effects, and other sound effects processing).

[0105] (4) During the preview process, if there is insufficient inferred voice data or insufficient playback voice data, secondary buffering will be triggered (insufficient data causes playback to be paused), and the situation will be processed to reduce the impact.

[0106] (5) If an exception occurs at any stage, the user will be guided and asked to try again based on the error message.

[0107] 2. Detailed implementation of first frame data loading and inference startup:

[0108] (1) Calculation of accompaniment download volume:

[0109] like Figure 6 As shown, the preview requires the use of accompaniment. Based on the implementation of downloading and playing, the metadata of the audio (including duration, sampling rate, sampling bit depth, number of channels, etc.) needs to be downloaded, and then the size of the first frame data is calculated through the metadata of the audio. The formula is as follows:

[0110] First frame size = first frame duration * sampling rate * sampling bit depth * number of channels / 8.

[0111] In summary, the total amount of data that needs to be downloaded to play the first frame of the accompaniment is (metadata size + first frame size).

[0112] (2) Implementation of the principle of downloading and reasoning

[0113] PPG files, timbre files, pitch files, etc. The underlying inference engine obtains the required resource files according to the inference progress. The content of the resource file is raw data (no need to decapsulate like accompaniment), so it is not necessary to download the complete resource file to start the inference. It is sufficient to ensure that the corresponding data segment in the resource file exists when the first frame of singing data is inferred. The resource size required for inferring the first frame of singing can be calculated by the inference data frame length (SampleSize) and the first frame duration (duration) sent by the background. The formula is as follows:

[0114] The data size required for the first frame resource = SampleSize*duration.

[0115] (3) Dynamic allocation of download resources:

[0116] Reasoning resources include accompaniment files, PPG files, timbre files, PPG files, etc. When downloading, the download speed of each task cannot be kept completely consistent. The progress difference between them may gradually increase over time, resulting in the amount of data of some resources far exceeding the current reasoning requirements, but some resources are insufficient. In this embodiment, when the progress between related resources exceeds a threshold (supporting dynamic distribution by the server), the task with fast progress will be automatically paused, giving network resources to the task with slow progress. When the tasks are balanced, all paused tasks will be restarted to ensure that the progress difference between tasks is at a reasonable value.

[0117] 3. Detailed implementation of fast Seek and secondary buffering:

[0118] If the inferred RTF is less than 1, secondary buffering will not occur in normal playback. However, the user can manually seek to a specified position for playback during the preview process. If the inferred singing data or resource data is insufficient at this time, secondary buffering will occur and you need to wait for sufficient inferred singing data and resource data.

[0119] If the inference of songs and resource downloads continue to be executed linearly, the user's waiting time may be very long. In the worst case, the user needs to wait until all resources are downloaded and inferred. Figure 7When the user seeks, the location will be prioritized for downloading and inferring the vocals, ensuring that the user does not have to wait for a long time. After optimization, when the user seeks, the longest waiting time is only the time it takes to download one frame of resources plus the time it takes to infer one frame of singing data, and the user's waiting time is controllable.

[0120] 4. Comparison of startup time between the original solution and this embodiment:

[0121] In the original solution, resource download and reasoning are processed serially, and preview can only start after reasoning is completed. This embodiment greatly reduces the user's preview time by providing the ability to download while reasoning and preview while reasoning. In this embodiment, since the inferred RTF is less than 1, the inference speed is faster than the playback speed under normal conditions, so it is only necessary to infer the first frame of singing data for playback. Since mixed accompaniment is required for playback during preview, based on the principle of downloading and playing the accompaniment at the same time, it is necessary to obtain the accompaniment audio metadata before downloading and playing. The time-consuming formula for calculating the above is as follows:

[0122] The time taken to download the first frame of accompaniment = the time taken to download the metadata of the accompaniment resource + the time taken to download the first frame of data of the accompaniment resource; the time taken to parse the accompaniment data is very short and can be ignored.

[0123] Time consumed for inferring the first frame of human voice = time consumed for downloading the first frame of human voice resource data + time consumed for inferring the first frame of human voice audio signal;

[0124] Time spent preparing the first frame of data = MAX (time spent downloading the first frame of accompaniment, time spent inferring the first frame of vocals). After the first frame of data (including the first frame of accompaniment and the first frame of vocals) is prepared, the preview engine is started to synthesize the first frame of accompaniment and the first frame of vocals into the first frame of AI singing and play it to the user. Since it takes a certain amount of time to start the preview engine, but the time spent synthesizing the AI ​​singing is very short and can be ignored, the time spent by the user waiting for the first frame of AI singing to play = time spent preparing the first frame of data + time spent starting the preview engine.

[0125] In this embodiment, the efficiency of song preview can be improved by downloading and inferring at the same time and previewing at the same time by the client. The song inference uses the hard disk resources of the client device, and the resource download uses network resources. There is no preemptive relationship between the two. Parallel processing can maximize the performance of the device and effectively reduce the total time spent on the client inference process.

[0126] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0127] Based on the same inventive concept, the embodiment of the present application also provides a synthetic song processing device for implementing the synthetic song processing method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more synthetic song processing device embodiments provided below can refer to the limitations of the synthetic song processing method above, and will not be repeated here.

[0128] In one embodiment, Figure 8 As shown, a synthetic song processing device is provided, including: a synthetic resource acquisition module 801, a first thread execution module 802, a second thread execution module 803 and a main process execution module 804, wherein:

[0129] The synthesis resource acquisition module 801 is used to determine the accompaniment resource and vocal resource stored in the server for synthesizing the target song selected by the user; the vocal resource carries the pronunciation characteristics of the user;

[0130] The first thread execution module 802 is used to execute the task corresponding to the current frame through the first thread: downloading the accompaniment data of the accompaniment resource in the current frame from the server, and parsing the accompaniment data of the accompaniment resource in the current frame to generate an accompaniment audio signal of the current frame;

[0131] The second thread execution module 803 is used to execute the task corresponding to the current frame through the second thread: download the human voice data of the human voice resource in the current frame from the server, and infer the human voice data of the human voice resource in the current frame to generate the human voice audio signal of the current frame; wherein: the current frame includes the first frame and subsequent frames, and the first frame task of the first thread is synchronously executed with the first frame task of the second thread;

[0132] The main process execution module 804 is used to execute the first frame playback task through the main process: when it is detected that the accompaniment audio signal of the first frame and the human voice audio signal of the first frame are generated, the preview engine is started, the accompaniment audio signal of the first frame and the human voice audio signal of the first frame are synthesized by the preview engine to obtain the song audio signal of the first frame, and the song audio signal of the first frame is played;

[0133] The first thread execution module 802 is further used to synchronously execute the subsequent frame tasks of the first thread to generate accompaniment audio signals of subsequent frames during the execution of the first frame playing task by the main process;

[0134] The second thread execution module 803 is further used to synchronously execute the subsequent frame tasks of the second thread to generate the human voice audio signals of the subsequent frames during the execution of the first frame playing task by the main process;

[0135] The main process execution module 804 is also used to execute the subsequent frame playback task through the main process: whenever it is monitored that the accompaniment audio signal and the vocal audio signal of the same subsequent frame are generated, the accompaniment audio signal and the vocal audio signal of the same subsequent frame are synthesized through the preview engine to obtain the song audio signal of the same subsequent frame and play the song audio signal of the same subsequent frame until the same subsequent frame is the end frame of the target song.

[0136] Each module in the above-mentioned synthesized song processing device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0137] In one embodiment, a computer device is provided. The computer device may be a client, and its internal structure diagram may be as follows: Fig. 9As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, near field communication (NFC) or other technologies. When the computer program is executed by the processor, a method for processing a synthesized song is implemented. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse.

[0138] Those skilled in the art will understand that Fig. 9 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0139] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0140] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0141] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0142] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0143] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0144] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0145] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for processing a synthesized song, characterized in that: Applied to a client, the method comprises: For a target song selected by a user, determining an accompaniment resource and a vocal resource stored in a server for synthesizing the target song; the vocal resource carries the pronunciation characteristics of the user; Executing the task corresponding to the current frame through the first thread: downloading the accompaniment data of the accompaniment resource in the current frame from the server, and parsing the accompaniment data of the accompaniment resource in the current frame to generate an accompaniment audio signal of the current frame; Execute the task corresponding to the current frame through the second thread: download the human voice data of the human voice resource in the current frame from the server, and infer the human voice data of the human voice resource in the current frame to generate a human voice audio signal of the current frame; wherein: the current frame includes a first frame and subsequent frames, and the first frame task of the first thread is executed synchronously with the first frame task of the second thread; The first frame playing task is executed through the main process: when it is monitored that the accompaniment audio signal of the first frame and the human voice audio signal of the first frame are both generated, the preview engine is started, the accompaniment audio signal of the first frame and the human voice audio signal of the first frame are synthesized by the preview engine to obtain the song audio signal of the first frame, and the song audio signal of the first frame is played; During the execution of the first frame playing task by the main process, the subsequent frame tasks of the first thread are synchronously executed to generate the accompaniment audio signals of the subsequent frames, and the subsequent frame tasks of the second thread are synchronously executed to generate the human voice audio signals of the subsequent frames; The subsequent frame playback task is executed through the main process: whenever it is monitored that the accompaniment audio signal and the vocal audio signal of the same subsequent frame are generated, the accompaniment audio signal and the vocal audio signal of the same subsequent frame are synthesized by the preview engine to obtain the song audio signal of the same subsequent frame and play the song audio signal of the same subsequent frame until the same subsequent frame is the end frame of the target song.

2. The method according to claim 1, characterized in that The downloading of the accompaniment data of the accompaniment resource in the current frame from the server, and parsing the accompaniment data of the accompaniment resource in the current frame to generate an accompaniment audio signal of the current frame includes: Acquire the audio metadata of the target song, and download the accompaniment data of the accompaniment resource in the current frame from the server based on the audio metadata; The audio metadata is used to parse the accompaniment data of the accompaniment resource in the current frame to generate an accompaniment audio signal of the current frame.

3. The method according to claim 2, characterized in that The downloading of the accompaniment data of the accompaniment resource in the current frame from the server based on the audio metadata includes: Acquire the signal duration, sampling rate, sampling bit depth and number of channels of the current frame from the audio metadata; obtain the data volume corresponding to the accompaniment data of the current frame according to the signal duration, the sampling rate, the sampling bit depth and the number of channels; According to the data amount, the accompaniment data of the accompaniment resource in the current frame is downloaded from the server.

4. The method according to claim 2, characterized in that: In the case where the current frame is the first frame, obtaining the audio metadata of the target song includes: The accompaniment resource is downloaded from the server, and the accompaniment resource is parsed to obtain the audio metadata of the target song.

5. The method according to claim 1, characterized in that The downloading of the vocal data of the vocal resource in the current frame from the server includes: Obtaining a data frame length and a signal duration of the current frame; and obtaining a data volume corresponding to the human voice data of the current frame according to the data frame length and the signal duration; According to the data amount, the human voice data of the human voice resource in the current frame is downloaded from the server.

6. The method according to claim 1, characterized in that The number of human voice resources is multiple; in the process of synchronously executing the subsequent frame tasks of the first thread to generate the accompaniment audio signal of the subsequent frame and synchronously executing the subsequent frame tasks of the second thread to generate the human voice audio signal of the subsequent frame, the method further includes: Obtaining the resource download progress of each vocal resource, and obtaining the resource download progress of the accompaniment resource; When the difference between the download progress of each resource is greater than a preset threshold, the resource download efficiency of each vocal resource and the accompaniment resource is adjusted, wherein the difference between the download progress of each resource after adjustment meets the preset difference requirement.

7. The method according to claim 1, characterized in that In executing the subsequent frame playback task through the main process, the method further includes: In response to the user's request for adjusting the playing progress of the target song, obtaining a playing progress frame corresponding to the adjusted playing progress; When it is detected that the song audio signal of the playing progress frame has been synthesized, the song audio signal of the playing progress frame is played through the preview engine.

8. The method according to claim 7, characterized in that After obtaining the playback progress frame corresponding to the adjusted playback progress, the method further includes: When it is detected that the accompaniment audio signal of the playing progress frame has not been completely generated, executing the task corresponding to the playing progress frame through the first thread to generate the accompaniment audio signal of the playing progress frame; When it is detected that the human voice audio signal of the playback progress frame has not been completely generated, executing the task corresponding to the playback progress frame through the second thread to generate the human voice audio signal of the playback progress frame; The playback progress frame task is executed through the main process: when it is monitored that the accompaniment audio signal and the vocal audio signal of the playback progress frame are generated, the accompaniment audio signal and the vocal audio signal of the playback progress frame are synthesized by the preview engine to obtain the song audio signal of the playback progress frame and the song audio signal of the playback progress frame is played.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

11. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images