Song recording recommendation method, computer equipment, readable storage medium and program product

Abstract

The invention relates to a song recording recommendation method, computer equipment, a readable storage medium and a program product. The method comprises the following steps: in a process of displaying a recommended song to a user through a song recording recommendation interface, playing a reasoning song sound for simulating the user to singe the recommended song; in the process of playing the reasoning singing sound, displaying first song recording auxiliary information and a song recording entrance on a song recording recommendation interface; the first song recording auxiliary information comprises lyric information of the recommended song and / or intonation evaluation information corresponding to the reasoned song; in response to a trigger operation on the song recording entrance, switching from the song recording recommendation interface to a song recording interface for displaying recommended songs, and displaying second song recording auxiliary information and a song recording operation control on the song recording interface; the second song recording auxiliary information comprises lyric information of the recommended song and / or intonation evaluation information corresponding to the currently sung song of the user. By adopting the method, the user can perceive the song recording effect in advance when the song is recommended to the user, and the user is attracted to record the song.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a song recommendation method, device, computer equipment, computer-readable storage medium and computer program product. Background Art

[0002] Users can use microphones and mobile devices to sing songs with accompaniment music on the online platform, and use the functions provided by the platform to record, edit, rate, share and interact with the singing voice. Users can browse the songs on the online platform and select specific songs to sing and record songs.

[0003] Traditional technology mainly attracts users to record songs by recommending song accompaniments or other users' singing works to users. However, traditional technology cannot enable users to perceive the final effect of song recording in advance by relying solely on song accompaniments or other users' singing works. The overall perception of users is weak, which is not conducive to attracting users to record songs. Summary of the invention

[0004] Based on this, it is necessary to provide a song recording recommendation method, device, computer equipment, computer-readable storage medium and computer program product that can attract users to record songs in response to the above technical problems.

[0005] In a first aspect, the present application provides a method for recommending recorded songs, comprising:

[0006] In the process of presenting the recommended songs to the user through the recorded song recommendation interface, playing the inference singing voice simulating the user singing the recommended songs;

[0007] During the playing of the inferred song, the first song recording auxiliary information and the song recording entrance are displayed on the song recording recommendation interface; wherein the first song recording auxiliary information includes lyrics information of the recommended song and / or pitch evaluation information corresponding to the inferred song;

[0008] In response to the triggering operation of the song recording entrance, the song recording recommendation interface switches to a song recording interface displaying the recommended song, and the second song recording auxiliary information and song recording operation controls are displayed on the song recording interface; the second song recording auxiliary information includes the lyrics information of the recommended song and / or the pitch evaluation information corresponding to the singing voice currently sung by the user.

[0009] In a second aspect, the present application also provides a song recommendation device, comprising:

[0010] A playing module, used for playing a reasoning singing voice simulating the user singing the recommended song in the process of displaying the recommended song to the user through the song recommendation interface;

[0011] A display module, used for displaying the first song recording auxiliary information and the song recording entrance on the song recording recommendation interface during the playing of the inferred song; wherein the first song recording auxiliary information includes the lyrics information of the recommended song and / or the pitch evaluation information corresponding to the inferred song;

[0012] A switching module is used to respond to a trigger operation on the song recording entrance, switch from the song recording recommendation interface to a song recording interface displaying the recommended song, and display second song recording auxiliary information and song recording operation controls on the song recording interface; the second song recording auxiliary information includes the lyrics information of the recommended song and / or the pitch evaluation information corresponding to the singing voice currently sung by the user.

[0013] In a third aspect, the present application further provides a computer device, wherein the computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the steps of the above method are implemented.

[0014] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the steps of the above method when executed by a processor.

[0015] In a fifth aspect, the present application further provides a computer program product, wherein the computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0016] The above-mentioned song recommendation method, device, computer equipment, computer-readable storage medium and computer program product, in the process of displaying recommended songs to users through the song recommendation interface, play the inferred singing voice simulating the user singing the recommended song, and display the lyrics and the pitch evaluation of the inferred singing voice in the song recommendation interface. Auxiliary song information such as can be used to enable users to perceive the song recording effect in advance when recommending songs to users. At the same time, combined with the auxiliary song information in the song recommendation interface, users can also experience the same song recording effect as in the real song recording scene, thereby enhancing users' interest in song recording and attracting users to record songs. The song recommendation interface also displays a song recording entrance. In response to the triggering operation of the song recording entrance, the song recommendation interface switches to the song recording interface displaying the recommended song. It can respond to the song recording operation triggered by the user in a timely manner and switch from the song recommendation interface to the song recording interface. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 A diagram of an application environment of a song recording recommendation method in one embodiment;

[0019] Figure 2 A schematic diagram of a process of a song recording recommendation method in one embodiment;

[0020] Figure 3 1 is a flow chart of a method for recommending song recordings based on inferenced singing voices on a client in an embodiment;

[0021] Figure 4 A schematic diagram of a song recommendation interface in one embodiment;

[0022] Figure 5 A schematic diagram of a song recording interface in one embodiment;

[0023] Figure 6 A schematic diagram of another song recording interface in one embodiment;

[0024] Figure 7 is a schematic diagram of a preloading mechanism in one embodiment;

[0025] Figure 8 A schematic diagram of a flow chart of a secondary cache access in one embodiment;

[0026] Fig. 9 A structural block diagram of a song recommendation device in one embodiment;

[0027] Fig.10 The figure is a diagram of the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0029] The song recommendation method provided in the embodiment of the present application can be applied to Figure 1In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. In the process of displaying the recommended songs to the user through the song recording recommendation interface, the terminal 102 plays the inference singing voice used to simulate the user singing the recommended songs; in the process of playing the inference singing voice, the terminal 102 displays the first song recording auxiliary information and the song recording entrance on the song recording recommendation interface; wherein, the first song recording auxiliary information includes the lyrics information of the recommended song and / or the pitch evaluation information corresponding to the inference singing voice. In response to the triggering operation of the song recording entrance, the terminal 102 switches from the song recording recommendation interface to the song recording interface displaying the recommended song, and displays the second song recording auxiliary information and the song recording operation control on the song recording interface; the second song recording auxiliary information includes the lyrics information of the recommended song and / or the pitch evaluation information corresponding to the song currently sung by the user. The terminal 102 may be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, IoT devices, and portable wearable devices. The IoT devices may be smart speakers, smart TVs, smart car devices, projection devices, etc. Portable wearable devices may be smart watches, head-mounted devices, etc. Head-mounted devices may be virtual reality (VR) devices, augmented reality (AR) devices, smart glasses, etc. The server 104 may be an independent physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server that provides cloud computing services.

[0030] In an exemplary embodiment, Figure 2 As shown, a song recommendation method is provided, which is described by taking the method applied to a terminal as an example, and includes the following steps S202 to S206. Among them:

[0031] Step S202, in the process of displaying the recommended songs to the user through the recorded song recommendation interface, play the inference singing voice used to simulate the user singing the recommended songs.

[0032] Among them, the song recommendation interface may refer to an interface used to display recommended songs to users. In actual applications, the information displayed on the song recommendation interface may include but is not limited to the name of the recommended song, the singer and other information.

[0033] Here, the user may refer to the user of the terminal. In actual applications, the user can log in to the user account of the song recording platform through the terminal, so that the song recording platform can recommend songs to the user based on the user information corresponding to the user account (such as song preferences, etc.).

[0034] Among them, recommended songs may refer to songs that need to be recommended to users so as to attract users to record songs. In actual applications, recommended songs may include songs in a preset song database that meet the user's song preference requirements.

[0035] Among them, the inferred singing voice may refer to a singing voice that simulates a user singing a recommended song. In actual applications, the inferred singing voice may include a singing voice output by a singing synthesis model that sings a recommended song with the user's timbre.

[0036] As an example, when a user uses a song recording platform, the user can log in to the user's account on the user end of the song recording platform for the terminal used by the user. The user end has the function of recommending songs to the user. In actual applications, the user end can display a song recording recommendation interface through the terminal, and the song recording recommendation interface can show recommended songs to the user. In the process of showing recommended songs to the user through the song recording recommendation interface, the terminal can play an inference song that simulates the user singing the recommended song.

[0037] Step S204, during the process of playing the inferred singing voice, the first song recording auxiliary information and the song recording entrance are displayed on the song recording recommendation interface; wherein the first song recording auxiliary information includes the lyrics information of the recommended song and / or the pitch evaluation information corresponding to the inferred singing voice.

[0038] Among them, the first song recording auxiliary information may refer to information used to assist users in recording songs. In actual applications, the first song recording auxiliary information can show users the singing effect of the inferred singing voice. The first song recording auxiliary information may include information displayed on the song recording interface corresponding to the recommended song and used to assist song recording, such as: singing score, lyrics, pitch analysis and other information. The pitch evaluation information corresponding to the inferred singing voice may include the pitch score of the inferred singing voice, etc. In specific implementation, when determining the pitch score through pitch analysis, the terminal can use the pitch of the recommended song as the standard for analysis.

[0039] Among them, the song recording entrance can refer to a specific interactive element in the song recording recommendation interface, such as a button, link or icon. In actual applications, the song recording entrance can be used as the starting point to the song recording interface. The song recording entrance can be used to control the terminal to jump from the song recording recommendation interface to the song recording interface.

[0040] As an example, in order to attract users to record songs, during the process of playing the inferred singing voice, the terminal can display the first song recording auxiliary information matching the inferred singing voice in the song recording recommendation interface, such as lyrics, singing score and pitch analysis, etc. In actual applications, the song recording auxiliary information for the inferred singing voice displayed in the song recording recommendation interface can be matched with the song recording auxiliary information for the user's real singing voice displayed in the song recording interface, so that the user can feel the same experience as the real song recording scene. In a specific implementation, the terminal can also display the song recording entrance in the song recording recommendation interface, and the user can interact with the song recording entrance, so that the terminal's display interface switches to the song recording interface.

[0041] Step S206, in response to the triggering operation on the song recording entrance, the song recording recommendation interface is switched to the song recording interface displaying the recommended songs, and the second song recording auxiliary information and song recording operation controls are displayed on the song recording interface; the second song recording auxiliary information includes the lyrics information of the recommended song and / or the pitch evaluation information corresponding to the user's current singing voice.

[0042] The trigger operation may include a click or other operation.

[0043] Among them, the song recording interface may refer to an interface for users to sing recommended songs and record their singing voices. In actual applications, the information displayed on the song recording interface may include but is not limited to the name of the recommended song, the singer, the user's singing rating and other information.

[0044] Among them, the second song recording auxiliary information may refer to information used to assist users in recording songs. In actual applications, the second song recording auxiliary information can show users the singing effect of the user's singing. The second song recording auxiliary information may include information displayed on the song recording interface corresponding to the recommended song and used to assist song recording, such as: singing score, lyrics, pitch analysis and other information. The pitch evaluation information corresponding to the user's singing may include the pitch score of the user's singing, etc. In specific implementation, when determining the pitch score through pitch analysis, the terminal may use the pitch of the recommended song as the standard for analysis.

[0045] Among them, the song recording operation control may refer to the control in the song recording interface used to perform specific operations on the song recording process. In actual applications, the specific operations may include but are not limited to sound effect adjustment operations, pause recording operations, re-recording operations, recording end operations, and recording mode adjustment operations.

[0046] As an example, when the terminal plays an inferred song and displays the first song recording auxiliary information and the song recording entrance to the user through the song recording recommendation interface, when the user needs to record a song, the user can perform a trigger operation on the song recording entrance. In response to the trigger operation on the song recording entrance, the terminal switches the display interface from the song recording recommendation interface to the song recording interface displaying the recommended song, and displays the second song recording auxiliary information on the song recording interface. The second song recording auxiliary information may include the lyrics information of the recommended song and / or the pitch evaluation information corresponding to the user's current singing. The terminal can also display the song recording operation control on the song recording interface. Based on the user's personalized needs for the song recording process, the user can control the progress of the song recording and adjust the sound effects of the user's singing by performing a trigger operation on the song recording operation control.

[0047] In the above-mentioned song recording recommendation method, in the process of displaying recommended songs to users through the song recording recommendation interface, the inferred singing voice simulating the user singing the recommended song is played, and the recording auxiliary information such as the pitch evaluation of the lyrics and the inferred singing voice is displayed in the song recording recommendation interface. This can enable the user to perceive the song recording effect in advance when recommending songs to the user. At the same time, combined with the song recording auxiliary information in the song recording recommendation interface, the user can also experience the same song recording effect as in the real song recording scene, thereby enhancing the user's interest in song recording and attracting the user to record songs. The song recording recommendation interface also displays a song recording entrance. In response to the triggering operation of the song recording entrance, the song recording recommendation interface switches to the song recording interface displaying the recommended song, and displays the lyrics and the pitch evaluation of the user's real singing voice and other recording auxiliary information and the song recording operation controls on the song recording interface. This can promptly respond to the song recording operation triggered by the user, and switch from the song recording recommendation interface to the song recording interface. At the same time, the song recording operation controls support adjusting the sound effects of the song recording process, which is conducive to meeting the user's personalized song recording needs.

[0048] In some embodiments, the above method also includes: responding to the sound effect adjustment operation on the song recording recommendation interface, adjusting the sound effect of the inferred singing according to the sound effect adjustment operation, and recording the sound effect configuration of the inferred singing; switching from the song recording recommendation interface to the song recording interface displaying the recommended song, including: according to the sound effect configuration of the inferred singing, synchronously setting the song recording configuration in the process of recording the recommended song.

[0049] Among them, the sound effect adjustment operation may refer to an operation triggered by the user to adjust the sound effect of the inference song. In actual applications, the user can click on the "tuning" button and other song recording operation controls in the song recording recommendation interface to enter the sound effect parameter setting interface. The user can modify the various sound effect parameters in the sound effect parameter setting interface to complete the sound effect adjustment operation.

[0050] Among them, the sound effect configuration of the inference song can refer to the sound effect parameters obtained after adjusting the sound effect of the inference song based on the user's sound effect adjustment operation in the song recording recommendation interface.

[0051] Among them, the song recording configuration may refer to the sound effect parameters of the user's singing voice when the user records a song in the song recording interface.

[0052] As an example, the song recording recommendation interface may also display song recording operation controls such as a "tune" button. When the user needs to adjust the sound effect of the inferred song, the user may click the "tune" button in the song recording recommendation interface, and the terminal may display the sound effect parameter setting interface. The user may modify the various sound effect parameters in the sound effect parameter setting interface by operating the terminal to implement the sound effect adjustment operation. The terminal may respond to the sound effect adjustment operation and adjust the sound effect of the inferred song according to the modified sound effect parameters. In actual applications, when the terminal switches from the song recording recommendation interface to the song recording interface that displays the recommended song, the terminal may also synchronize the song recording configuration in the process of recording the recommended song according to the sound effect configuration of the inferred song, so that the sound effect parameters when the user records the song in the song recording interface are consistent with the sound effect parameters of the inferred song in the song recording recommendation interface, thereby implementing the synchronized setting of the sound effect parameters.

[0053] In this embodiment, by adjusting the sound effect of the inferred singing voice in response to the sound effect adjustment operation, recording the sound effect configuration of the inferred singing voice, and synchronously setting the song recording configuration in the process of recording the recommended song based on the sound effect configuration of the inferred singing voice, the sound effect of the inferred singing voice can be adjusted in a personalized way to meet user needs, so that the user can fully perceive the changes in the song recording effect before and after the sound effect adjustment, enhance the user's interest in song recording, display the song recording effect to the user in advance, and facilitate the user to preview the singing effect of the song recording in advance. At the same time, the song recording configuration of the recommended song can also be synchronized with the sound effect configuration of the inferred singing voice to achieve seamless entry into song recording and synchronized sound effect related settings.

[0054] In some embodiments, the above method also includes: when the generation progress of the inference singing of the recommended song meets the preset progress, playing the generated inference singing of the recommended song; when the playback progress of the inference singing meets the preset preloading conditions, generating an inference singing for simulating the user singing another recommended song; the other recommended song is the next recommended song in the user's song recommendation sequence.

[0055] Among them, the preset progress may refer to information used to determine whether to play the generated inference song. In actual applications, the preset progress may include the time difference between the playback progress of the inference song and the generation progress of the inference song being greater than or equal to the preset time difference threshold.

[0056] Among them, the preset preloading condition may refer to information representing the need to preload the inference songs of other recommended songs. In actual applications, the preset preloading condition includes the time difference between the playback progress of the inference song and the generation progress of the inference song being greater than or equal to a preset time difference threshold.

[0057] As an example, in the process of playing the inference song of the recommended song, the terminal can obtain the playback progress of the inference song and the generation progress of the inference song at the current moment, and then the terminal can calculate the time difference between the playback progress of the inference song and the generation progress of the inference song at the current moment, and compare the time difference with the preset time difference threshold. When the time difference is greater than or equal to the preset time difference threshold, the terminal can determine that the generation progress of the inference song of the recommended song meets the preset progress, and the terminal can play the generated inference song of the recommended song. It is understandable that if the generation progress of the inference song of the recommended song does not meet the preset progress, the terminal can pause playing the inference song until the generation progress of the inference song of the recommended song meets the preset progress. In actual applications, each user is pre-set with a corresponding song recommendation sequence, which may include several recommended songs. In order to improve the start-up speed of the recommended songs, a pre-loading mechanism is pre-set: the terminal may calculate the time difference between the playback progress of the inference song and the generation progress of the inference song at the current moment, and compare the time difference with the preset time difference threshold. When the time difference is greater than or equal to the preset time difference threshold, the terminal may determine that the playback progress of the inference song at the current moment meets the preset pre-loading condition, and the terminal may generate an inference song for simulating the user singing another recommended song. For example: the time difference between the playback progress and the generation progress of the inference song of the recommended song A at the current moment is greater than or equal to the preset time difference threshold, the terminal may generate an inference song for simulating the user singing a recommended song B (and recommended song C, etc.) different from the recommended song A. When the user switches the recommended song in the recording song recommendation interface from recommended song A to recommended song B, the terminal may play the inference song of recommended song B in a timely manner.

[0058] In this embodiment, when the generation progress of the inference song meets the preset progress, the generated inference song is played, which can avoid the problem of smooth playback of the inference song caused by insufficient generated inference song. At the same time, when the playback progress of the inference song meets the preloading condition, the inference song of another recommended song is generated, and the inference song of other recommended songs can be preloaded, so that when the recommended song is switched, the inference song of the switched recommended song can be played in time, thereby improving the start-up speed of the recommended song.

[0059] In some embodiments, playing an inference singing voice simulating a user singing a recommended song includes: caching the generated audio data of the inference singing voice of the recommended song to a preset storage area; in response to the generation progress of the inference singing voice of the recommended song meeting the preset progress, searching the audio data of the inference singing voice from the preset storage area, and playing the audio data of the inference singing voice.

[0060] Among them, the preset storage area may refer to a pre-set area for storing the inferred singing voices of recommended songs. In actual applications, the preset storage area may include but is not limited to a memory area and a pre-set preload pool, etc.

[0061] As an example, the terminal can cache the audio data of the inference singing voice of the generated recommended song to a preset storage area. When the generation progress of the inference singing voice of the recommended song meets the preset progress, the terminal can search for the audio data of the inference singing voice from the preset storage area and play the audio data of the inference singing voice. In actual applications, the terminal can store the inference singing voice of the recommended song that has been recommended to the user in the preset storage area. The inference singing voice of the recommended song can also be pre-inferred and stored in the preset storage area on the background server. For example, for users who often use singing synthesis, the inference singing voice of the recommended song is pre-inferred and generated in advance on the background server. The terminal can give priority to recommending the recommended songs corresponding to the inference singing voice that has been inferred to the user. When the recording song recommendation interface displays the recommended song to the user, the terminal downloads the existing pre-inference generated inference singing voice and plays it. The client does not need to infer, which reduces the user's waiting time. When the terminal needs to play the inference singing voice used to simulate the user singing the recommended song, the terminal can first search in the preset storage area whether there is an inference singing voice of the recommended song. When the terminal successfully finds the inference singing voice of the recommended song from the preset storage area, the terminal can play the found inference singing voice. It is understandable that when it is necessary to preload the inference singing of the recommended song, the terminal can also first search whether the inference singing of the recommended song exists in the preset storage area. If so, the terminal can map the preloading task to the cache file of the found inference singing. When it is necessary to play the inference singing of the recommended song, the data in the cache file is reused to play the inference singing, thereby avoiding resource consumption and time consumption caused by regenerating the inference singing.

[0062] In this embodiment, the inference song of the recommended song is played when searching for the inference song in the preset storage area. During the process of playing the inference song of the recommended song, it can be checked whether the inference song of the recommended song exists in the preset storage area, thereby avoiding resource consumption and time consumption caused by regenerating the inference song, and improving the playback efficiency of the inference song of the recommended song.

[0063] In some embodiments, playing an inferred singing voice simulating a user singing a recommended song includes: when the audio data of the inferred singing voice cannot be found in a preset storage area, obtaining the song accompaniment resource and the singing inference resource of the recommended song; the singing inference resource is used to generate the inferred singing voice; when the audio metadata of the song accompaniment resource is parsed and the inferred singing voice is generated according to the singing inference resource, the song accompaniment of the recommended song is generated according to the audio metadata, and the inferred singing voice and the song accompaniment are played.

[0064] The song accompaniment resource may refer to a resource used to generate an accompaniment audio signal of a recommended song. In practical applications, the song accompaniment resource may include but is not limited to an accompaniment audio file.

[0065] Among them, singing inference resources can refer to information used to generate inferred singing. In practical applications, singing inference resources can include but are not limited to singing synthesis models, timbre models, user's timbre files, etc. The user's timbre files can represent the characteristics of the user's timbre.

[0066] Among them, audio metadata can refer to data used to describe audio content and its attributes. In practical applications, audio metadata can store various parameter information of audio files, including audio bit rate, sampling rate, duration, channels and other information.

[0067] As an example, when the terminal needs to play an inference song that simulates a user singing a recommended song, the terminal can first search in a preset storage area whether there is an inference song for the recommended song. When the terminal cannot find the audio data of the inference song for the recommended song from the preset storage area, the terminal can obtain the song accompaniment resource of the recommended song and the singing inference resource required to generate the inference song. The terminal can then parse the song accompaniment resource. When the audio metadata of the song accompaniment resource is obtained through parsing, the terminal can mark the song accompaniment resource as playable. The terminal can then generate the inference song in real time based on the singing inference resource by converting the lyrics text of the recommended song into speech. After the terminal parses the audio metadata of the song accompaniment resource and generates the inference song based on the singing inference resource, the terminal can play the generated inference song and song accompaniment.

[0068] In this embodiment, when the inference song of the recommended song cannot be found in the preset storage area, the inference song is generated and played in real time based on the inference resources. The inference song can be generated in time in combination with the inference resources, avoiding the impact of the failure to obtain the generated inference song in time on the recorded song recommendation, and improving the generation efficiency of the inference song.

[0069] In some embodiments, inferred singing is generated based on singing inference resources, including: for recommended songs, executing the task corresponding to the current frame through the first thread: obtaining the accompaniment data of the song accompaniment resource of the recommended song in the current frame, and parsing the accompaniment data of the song accompaniment resource of the recommended song in the current frame to generate the accompaniment audio signal of the current frame; executing the task corresponding to the current frame through the second thread: obtaining the vocal data of the singing inference resource in the current frame, and inferring the vocal data of the singing inference resource in the current frame to generate the vocal audio signal of the current frame.

[0070] The current frame includes a first frame and subsequent frames, and the first frame task of the first thread is executed synchronously with the first frame task of the second thread.

[0071] Among them, the current frame refers to any song signal frame of the recommended song, which may include the first frame and subsequent frames of the recommended song. The first thread is a thread for generating an accompaniment audio signal, and the second thread is a thread for generating a vocal audio signal, that is, a thread for generating an audio signal of the pure vocal part of the recommended song, and the first frame tasks executed by the first thread and the second thread are executed synchronously, that is, the process of the first thread generating the accompaniment audio signal of the first frame and the second thread generating the vocal audio signal of the first frame can be performed synchronously.

[0072] As an example, when the terminal needs to generate an inferred singing voice of a recommended song based on the singing voice inference resources of the recommended song, the terminal can start a first thread to execute the generation of an accompaniment audio signal for the current frame. It can be based on the audio metadata of the recommended song and first download the accompaniment data corresponding to the song accompaniment resource in the current frame from the server, and then use the audio metadata to parse the accompaniment data of the song accompaniment resource in the current frame to generate the accompaniment audio signal of the current frame. At the same time, the client can also synchronously start a second thread to execute the generation of a human voice audio signal for the current frame. It can be based on first downloading the human voice data of the human voice resource in the current frame from the server, and using the singing voice inference engine to perform singing voice inference on the above-mentioned human voice data of the current frame to generate the human voice audio signal of the current frame.

[0073] In this embodiment, the first thread and the second thread synchronously execute the use of accompaniment resources to obtain the accompaniment audio signal of the current frame, and the use of vocal resources to obtain the vocal audio signal of the current frame. At the same time, the accompaniment audio signal and the vocal audio signal of each frame are obtained, thereby realizing the parallel processing of downloading accompaniment resources and vocal resources, accompaniment analysis, singing voice reasoning and synthesis and playback of song audio signals, which can improve the preview or editing processing efficiency of the synthesized song.

[0074] In some embodiments, after the step of executing the task corresponding to the current frame through the second thread, the method also includes: executing the first frame playback task through the main process: when it is monitored that the accompaniment audio signal of the first frame and the human voice audio signal of the first frame are both generated, start the preview engine, and synthesize the accompaniment audio signal of the first frame and the human voice audio signal of the first frame through the preview engine to obtain the song audio signal of the first frame and play the song audio signal of the first frame; in the process of executing the first frame playback task by the main process, synchronously execute the subsequent frame task of the first thread to generate the accompaniment audio signal of the subsequent frame and synchronously execute the subsequent frame task of the second thread to generate the human voice audio signal of the subsequent frame; execute the subsequent frame playback task through the main process: whenever it is monitored that the accompaniment audio signal and the human voice audio signal of the same subsequent frame are both generated, synthesize the accompaniment audio signal and the human voice audio signal of the same subsequent frame through the preview engine to obtain the song audio signal of the same subsequent frame and play the song audio signal of the same subsequent frame, until the same subsequent frame is the end frame of the recommended song.

[0075] Among them, the main process may refer to a thread for synthesizing and playing song audio signals. After detecting that the first frame accompaniment audio signal and the vocal audio signal have been generated, the preview engine can be started through the main process. The preview engine refers to a playback engine for previewing recommended songs, that is, playing recommended songs. Through the preview engine, the first frame accompaniment audio signal obtained by the first thread and the first frame vocal audio signal obtained by the second thread can be synthesized. After obtaining the first frame song audio signal of the recommended song, the first frame song audio signal can be played.

[0076] During the execution of the first frame playing task by the main process, the subsequent frame tasks of the first thread are synchronously executed to generate the accompaniment audio signals of the subsequent frames, and the subsequent frame tasks of the second thread are synchronously executed to generate the human voice audio signals of the subsequent frames.

[0077] The end frame may refer to the last song signal frame in the song to be synthesized. While the terminal executes the first frame playback task through the main process, it can also synchronously execute the subsequent frame tasks of the first thread and the subsequent frame tasks of the second thread, thereby generating the accompaniment audio signal of the subsequent frame through the first thread and the vocal audio signal of the subsequent frame through the second thread.

[0078] Afterwards, when the main process detects that the accompaniment audio signal and the vocal audio signal of the same subsequent frame have been generated, the accompaniment audio signal and the vocal audio signal of the same subsequent frame can be synthesized through the preview engine to obtain the song audio signal of the same subsequent frame and play it until the same subsequent frame is the end frame of the recommended song.

[0079] For example, when the main process executes the first frame playback task, the terminal can also synchronously execute the subsequent frame tasks of the first thread and the subsequent frame tasks of the second thread, that is, synchronously execute the accompaniment audio signal of the second frame generated by the first thread, and the vocal audio signal of the second frame generated by the second thread. Since the accompaniment audio signal and the vocal audio signal of the second frame are both generated at this time, the main thread can also execute the playback task of the second frame, synthesize the accompaniment audio signal and the vocal audio signal of the second frame to obtain the song audio signal of the second frame, and play the song audio signal of the second frame. At this time, the client can also synchronously execute the accompaniment audio signal of the third frame generated by the first thread, and the vocal audio signal of the third frame generated by the second thread, and so on, until the main process completes the synthesis and playback of the song audio signal of the end frame, and completes the synthesis and playback of the recommended song.

[0080] In this embodiment, after the main process detects that the accompaniment audio signal and the vocal audio signal of the first frame have been generated, the preview engine is started to synthesize and play the song audio signal of the first frame, and when the main process executes the first frame playback task, the subsequent frame tasks can be synchronously executed by the first thread and the second thread to generate the accompaniment audio signal and the vocal audio signal of the subsequent frames, and the subsequent frame playback tasks are executed by the main process until the subsequent frame is the end frame of the target song, so as to complete the playback of the recommended song. During the playback of the song audio signal, the accompaniment audio signal and the vocal audio signal of the subsequent frame are obtained at the same time, thereby realizing the parallel processing of downloading the accompaniment resources and the vocal resources, accompaniment analysis, singing voice reasoning and synthesis and playback of the song audio signal, thereby improving the preview or editing processing efficiency of the synthesized song.

[0081] In some embodiments, during the process of playing the inferred singing voice of the recommended song, in response to the generation progress of the inferred singing voice of the recommended song meeting the preset progress, the song recording engine is started.

[0082] Among them, the song recording engine is used to record the singing of the user. In actual applications, the song recording engine may include a comprehensive song recording system for accompaniment data decoding and effect processing, vocal recording and effect processing, mixed playback of vocals and accompaniment, and work synthesis.

[0083] As an example, the terminal can preheat the song recording engine while playing the inference song of the recommended song. In actual application, when the generation progress of the inference song of the recommended song meets the preset progress, the terminal can play the generated inference song of the recommended song, and preheat the song recording engine asynchronously in advance during the playback of the generated inference song of the recommended song. In the specific implementation, the preheating song recording engine does not need to wait for the accompaniment data to be downloaded. The preheating song recording engine occupies hardware resources, and the accompaniment download occupies network resources, and there is no preemption between the two.

[0084] In this embodiment, the song recording engine is started during the playing of the inferred singing voice of the recommended song, and the song recording engine can be preheated before the song is recorded so as to quickly respond to the user's song recording operation to record the song, avoid the influence of the song recording engine startup speed on the song recording, and improve the recording efficiency of the song recording.

[0085] In some embodiments, switching from a song recording recommendation interface to a song recording interface displaying recommended songs includes: obtaining a view resource of a display view of first song recording auxiliary information; rendering a song recording interface and displaying the song recording interface according to the view resource.

[0086] Among them, the view resource may refer to a reused rendering resource used to generate a display view of the second song recording auxiliary information.

[0087] As an example, in the process of switching the display interface of the terminal from a song recording recommendation interface to a song recording interface displaying recommended songs, the terminal can first determine the view resources of the display view of the first song recording auxiliary information in the song recording recommendation interface based on the first song recording auxiliary information in the song recording recommendation interface. Then, the terminal can render the second song recording auxiliary information in the song recording interface according to the view resources of the display view of the first song recording auxiliary information in the song recording recommendation interface, and display the song recording interface, thereby avoiding repeated data parsing and repeated view creation, and speeding up page switching.

[0088] In this embodiment, the song recording interface is rendered based on the view resources of the display view of the first song recording auxiliary information, which can avoid repeated creation of views and speed up the interface switching speed.

[0089] In some embodiments, switching from a song recording recommendation interface to a song recording interface displaying recommended songs includes: setting the position of a display view of the first song recording auxiliary information in the song recording recommendation interface to be the same as the position of a display view of the second song recording auxiliary information in the song recording interface.

[0090] As an example, in order to achieve seamless switching between the song recommendation interface and the song recording interface, the terminal can set the display view of the first song recording auxiliary information in the song recommendation interface to be the same as the display view of the second song recording auxiliary information in the song recording interface during the process of the terminal's display interface switching from the song recommendation interface to the song recording interface displaying the recommended songs, so as to keep the view layout of the song recording auxiliary information in the song recommendation interface and the song recording auxiliary information in the song recording interface consistent, thereby reducing the view jump when switching pages.

[0091] In this embodiment, the display view of the first song recording auxiliary information is set to have the same position in the song recording recommendation interface as the display view of the second song recording auxiliary information in the song recording interface. This can reduce view jumps when the interfaces are switched, and achieve seamless switching between the song recording recommendation interface and the song recording interface.

[0092] In some embodiments, switching from a song recommendation interface to a song recording interface displaying recommended songs includes: in the process of switching from the song recommendation interface to the song recording interface, displaying a fade-out and fade-in animation effect of a difference view.

[0093] Among them, the difference view may refer to a view in which there is a difference between the song recording recommendation interface and the song recording interface.

[0094] As an example, when there are differences in view layout / interface elements, etc. between the song recommendation interface and the song recording interface, when the display interface of the terminal switches from the song recommendation interface to the song recording interface displaying the recommended songs, the terminal can compare the song recommendation interface with the song recording interface, determine the view that is different between the song recommendation interface and the song recording interface as the difference view, and set the fade-out and fade-in animation effect of the difference view, so as to display the fade-out and fade-in animation effect of the difference view during the process of switching from the song recommendation interface to the song recording interface.

[0095] In this embodiment, the fade-in and fade-out animation effect of the difference view between the song recording recommendation interface and the song recording interface is displayed, so that immersive song recording recommendation with seamless switching can be achieved.

[0096] In one exemplary embodiment, Figure 3 As shown, a flow chart of a method for recommending song recording based on on-terminal inference singing voice is provided. When a user starts to preview a recommended song through the song recording recommendation interface, the terminal can download the accompaniment resources of the recommended song (such as song accompaniment) and the resources required for inference singing voice (such as singing inference resources) in parallel. The terminal can synchronously parse the accompaniment while downloading the accompaniment. When the audio metadata of the accompaniment (such as duration, sampling rate, bit rate, etc.) is obtained, the terminal can indicate the song accompaniment as a playable state and keep downloading the song accompaniment until the download of the song accompaniment is completed. The terminal can generate an inferred singing voice for a recommended song based on singing inference resources. Specifically, the inferred singing voice can be composed of several segments to be played. The terminal can first determine the duration of the segment to be played (such as the first frame) in the inferred singing voice, and then calculate the amount of data required to infer the segment to be played based on the duration of the segment to be played. When the resources required for the inferred singing voice (such as a singing synthesis model, a timbre model, etc.) have all obtained the data required to infer the segment to be played, the terminal can determine that the resources required for the inferred singing voice have downloaded the data corresponding to the segment to be played. At this time, the terminal can generate an inferred singing voice (such as pure human voice) for the recommended song based on the resources required for the inferred singing voice.

[0097] When the terminal obtains the audio metadata of the accompaniment and the inference of the to-be-played segment is completed, the terminal can perform algorithmic processing and mixed playback of the inference singing voice and the song accompaniment through the preview engine, wherein the preview engine may refer to a comprehensive work playback system responsible for accompaniment data decoding and effect processing, mixed playback of human voice and accompaniment, etc. The terminal can score the inference singing voice from multiple angles while mixing the inference singing voice and the song accompaniment, and determine the achievement display information of the inference singing voice (such as achievement dynamic effects, etc.) in combination with the multi-angle scoring of each sentence in the inference singing voice. The terminal can obtain the pitch information of the inference singing voice while mixing the inference singing voice and the song accompaniment, and analyze the difference / matching degree between the pitch information of the inference singing voice and the pitch information of the recommended song (which can be sent to the terminal by the background server) by comparing the pitch information of the inference singing voice with the pitch information of the recommended song (original song), and determine the pitch information, which may include MIDI (Musical Instrument Digital Interface) effects. When an error occurs in any of the above steps, the terminal can generate corresponding guidance information and display the guidance information to the user, so as to guide the user to handle the error. In the specific implementation, when the terminal obtains insufficient song accompaniment or generates insufficient inference singing, the terminal can pause playing the inference singing and display the loading state until the terminal obtains sufficient song accompaniment and generates sufficient inference singing, and the terminal automatically continues to play the inference singing.

[0098] Taking the application of the song recommendation method to the client of the song recording platform as an example, the song recommendation function corresponding to the song recommendation method is located in the "Recommendation" list of the "Song Request Station" on the homepage of the client, such as Figure 4 As shown, a schematic diagram of a song recording recommendation interface is provided. The song recording recommendation interface supports sliding up and down to switch recommended songs. In the process of the terminal displaying recommended songs to the user through the song recording recommendation interface, the terminal can play an inferred singing voice simulating the user singing the recommended song. The song recording recommendation interface can display song information such as the song name of the recommended song and the singer of the recommended song. The pitch display interface in the song recording recommendation interface can display the pitch information of the inferred singing voice in real time and perform MIDI hit coloring. The lyrics display interface in the song recording recommendation interface can display the lyrics information of the recommended song in real time.

[0099] The song recording recommendation interface also displays a song recording entrance (such as a song recording button). During the user's preview of the recommended songs, if the user needs to record the song, the user can perform a trigger operation on the song recording entrance (such as clicking a song recording button). The terminal can respond to the trigger operation on the song recording entrance by switching from the song recording recommendation interface to the song recording interface corresponding to the recommended song. The interface layout of the song recording recommendation interface can be the same or similar to the interface layout of the song recording interface, thereby providing the user with an immersive song recording experience. After the user performs a trigger operation on the song recording entrance, the terminal can perform fade-out and fade-in animation effects on the different views of the song recording recommendation interface and the song recording interface, thereby achieving immersive song recording recommendations that can be recorded by seamless switching.

[0100] While the terminal is playing the inferred singing voice of the recommended song, it preheats the song recording engine in advance. The song recording engine may refer to a comprehensive song recording system responsible for accompaniment data decoding and effect processing, vocal recording and effect processing, mixed playback of vocals and accompaniment, and work synthesis. When there is enough cached data for the inferred singing voice (such as the playback progress of the inferred singing voice meets the preset preloading conditions), the terminal can asynchronously preheat the song recording engine in advance without waiting for the accompaniment data to be downloaded. Preheating the song recording engine occupies hardware resources, and downloading the accompaniment occupies network resources. There is no preemption between the two. Since the parsing of MIDI and singing voices is time-consuming, when jumping to the song recording page, the terminal can pass the MIDI view and lyrics view in the song recording recommendation interface to the song recording interface for reuse, avoiding repeated data parsing and repeated view creation, and speeding up page switching. The layout of the core views (such as MIDI, lyrics, ratings, etc.) of the song recording recommendation interface can be kept consistent with the song recording interface to reduce view jumps when switching pages. Figure 5 As shown, a schematic diagram of a song recording interface is provided. The song recording recommendation interface can also display the score information of the inferred singing voice. The song recording interface can also display the score information of the inferred singing voice. The song recording interface can display a tuning button for adjusting the sound effect. The user can click the tuning button, and then the terminal can display the sound effect parameter setting interface. The user can modify the various sound effect parameters in the sound effect parameter setting interface through the operation terminal to implement the sound effect adjustment operation. The song recording recommendation interface can also display a tuning button for adjusting the sound effect. The user can modify the sound effect parameters of the inferred singing voice in the song recording recommendation interface. Figure 6 As shown, a schematic diagram of another song recording interface is provided. The terminal can also determine the achievement display information (such as achievement animation, etc.) based on the real human voice of the user in the actual song recording process, and display the achievement display information in the achievement display area. The song recording interface can also include an achievement display area. The terminal can determine the achievement display information (such as achievement animation, etc.) based on the inferred singing voice and display it in the achievement display area in the song recording interface.

[0101] Since the recorded song recommendation interface supports sliding up and down to switch recommended songs, in order to increase the start speed of the recommended songs, the terminal can analyze and infer whether the playback progress of the song meets the preset preloading conditions and determine whether to preload, thereby realizing the preloading mechanism, such as Figure 7 As shown, a schematic diagram of a preloading mechanism is provided. The terminal can generate the inferred inference song based on the resources required for the downloaded inference song. Starting from the current playback position (such as the playback progress) of the inference song at the current moment, when the existing cache (such as the accompaniment and the inference song) meets the preset threshold, the terminal notifies the preloading pool to preload. When the existing cache does not meet the minimum cache threshold, the preloading pool is notified to stop preloading. All thresholds involved are issued by the background and can be dynamically configured. For example: when the time difference between the duration of the inferred inference song (such as the generation progress) and the current playback position of the inference song is greater than or equal to the preset time difference threshold (such as the time difference between the preset minimum cache line and the current playback position of the inference song), the terminal can determine that the terminal has generated enough playback cache data, that is, the playback progress of the inference song meets the preset preloading conditions, and the terminal can automatically preload (such as generate) the inference song of other works (such as another recommended song) in the preloading pool. The inference song of other works can also be composed of several clips to be played. The terminal can first determine the recommended song. The terminal calculates the duration of the segment to be played (such as the first frame) in the reasoning song, and then calculates the amount of data required for the segment to be played based on the duration of the segment to be played. When the resources required for the reasoning song (such as the singing synthesis model, etc.) are obtained (such as downloaded) to infer the data required for the segment to be played, the terminal can determine that the resources required for the reasoning song have downloaded the data corresponding to the segment to be played. At this time, the terminal can generate the reasoning song (such as pure human voice) of the recommended song in combination with the resources required for the reasoning song. Other works can be used as data to be loaded (such as recorded as to be loaded 1, to be loaded 2, ..., to be loaded N). The preload pool can preload the reasoning songs of 6 recommended songs by default. Since preloading requires system memory, if the system memory conditions are not met, the number of preloads will be dynamically reduced.

[0102] The preload pool can use the LRU (Least Recently Used) algorithm to manage the cache, thereby achieving secondary cache access, such as Figure 8As shown, a flowchart of secondary cache access is provided, and the inference singing voice of the eliminated recommended song is transferred from memory storage (such as a preset storage area) to file storage. The next time the inference singing voice of the eliminated recommended song is needed, it can be read from the corresponding file to avoid re-inference singing voice. If the work to be preloaded already has a file cache, the preload pool can map the preload task to the corresponding cache file, and reuse the cache file data during playback. When the terminal plays the inference singing voice of the recommended song, it can preferentially query the memory cache data corresponding to the inference singing voice of the recommended song from the preload pool. If the memory cache data corresponding to the inference singing voice of the recommended song is queried, the cache data in the preload pool is used. If the memory cache data corresponding to the inference singing voice of the recommended song does not exist in the preload pool, then query whether there is a file cache in the file storage manager. If the memory cache data corresponding to the inference singing voice of the recommended song is queried, the cache data in the file storage manager is used. If neither of them has the cache data corresponding to the inference singing voice of the recommended song, the terminal needs to completely infer the inference singing voice of the recommended song.

[0103] In this embodiment, in the song recording recommendation page, the singing voice generated by real-time inference on the client based on the user's timbre model is played, and the singing effect of the user in the song recording process is simulated at the same time, and the inference-generated singing voice is scored, the score achievement is displayed, MIDI coloring, etc., to maintain the same experience as the effect in the real song recording scene. The song recording recommendation interface supports users to adjust the sound effect (sound effect, up and down pitch, etc.) configuration in real time, and the inference-generated singing voice can take effect in real time, and all song recording configurations can be brought to the song recording page with one click. When clicking to jump to transcribe a song, the user can achieve seamless switching without feeling, providing users with an immersive recommendation and seamless song recording experience, which can increase the display form of song recording recommendation content, expand the gameplay of song recording recommendation content, increase the exposure of AI singing (large model inference to generate user singing) capabilities, and let users perceive the effect of their own recorded songs in advance (timbre, sound effect, score, achievement effect, etc.), enhance users' interest in song recording, and increase the conversion rate of recommended song recording.

[0104] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0105] Based on the same inventive concept, the embodiment of the present application also provides a song recommendation device for implementing the song recommendation method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more song recommendation device embodiments provided below can refer to the limitations of the song recommendation method above, and will not be repeated here.

[0106] In an exemplary embodiment, Fig. 9 As shown, a song recommendation device is provided, including: a playing module 902, a display module 904 and a switching module 906, wherein:

[0107] The playing module 902 is used to play the inference singing voice simulating the user singing the recommended song when displaying the recommended song to the user through the recorded song recommendation interface.

[0108] Display module 904 is used to display the first song recording auxiliary information and the song recording entrance on the song recording recommendation interface during the playing of the inferred song; wherein the first song recording auxiliary information includes the lyrics information of the recommended song and / or the pitch evaluation information corresponding to the inferred song.

[0109] The switching module 906 is used to respond to the triggering operation of the song recording entrance, switch from the song recording recommendation interface to the song recording interface displaying the recommended songs, and display the second song recording auxiliary information and song recording operation controls on the song recording interface; the second song recording auxiliary information includes the lyrics information of the recommended song and / or the pitch evaluation information corresponding to the user's current singing voice.

[0110] In one of the exemplary embodiments, the above-mentioned device also includes a sound effect adjustment module, which is specifically used to respond to the sound effect adjustment operation on the song recording recommendation interface, adjust the sound effect of the inferred singing according to the sound effect adjustment operation, and record the sound effect configuration of the inferred singing; switch from the song recording recommendation interface to the song recording interface displaying the recommended song, including: according to the sound effect configuration of the inferred singing, synchronously set the song recording configuration in the process of recording the recommended song.

[0111] In one of the exemplary embodiments, the above-mentioned device also includes a preloading module, which is specifically used to play the generated inference singing of the recommended song when the generation progress of the inference singing of the recommended song meets the preset progress; when the playback progress of the inference singing meets the preset preloading condition, generate an inference singing for simulating the user singing another recommended song; the other recommended song is the next recommended song in the user's song recommendation sequence; wherein the preset preloading condition includes that the time difference between the playback progress of the inference singing and the generation progress of the inference singing is greater than or equal to a preset time difference threshold.

[0112] In one of the exemplary embodiments, the preloading module is specifically used to cache the generated audio data of the inference singing of the recommended song to a preset storage area; in response to the generation progress of the inference singing of the recommended song meeting the preset progress, the audio data of the inference singing is searched from the preset storage area, and the audio data of the inference singing is played.

[0113] In one of the exemplary embodiments, the preloading module is specifically used to obtain the song accompaniment resources and singing inference resources of the recommended song when the audio data of the inferred singing cannot be found in the preset storage area; the singing inference resources are used to generate the inferred singing; when the audio metadata of the song accompaniment resources are parsed and the inferred singing is generated according to the singing inference resources, the song accompaniment of the recommended song is generated according to the audio metadata, and the inferred singing and the song accompaniment are played.

[0114] In one of the exemplary embodiments, the preloading module is specifically used to execute the task corresponding to the current frame through the first thread for the recommended song: obtaining the accompaniment data of the song accompaniment resource of the recommended song in the current frame, and parsing the accompaniment data of the song accompaniment resource of the recommended song in the current frame to generate the accompaniment audio signal of the current frame; executing the task corresponding to the current frame through the second thread: obtaining the human voice data of the singing inference resource in the current frame, and inferring the human voice data of the singing inference resource in the current frame to generate the human voice audio signal of the current frame; wherein: the current frame includes the first frame and subsequent frames, and the first frame task of the first thread is executed synchronously with the first frame task of the second thread.

[0115] In one of the exemplary embodiments, the preloading module is specifically used to execute the first frame playback task through the main process: when it is monitored that the accompaniment audio signal of the first frame and the human voice audio signal of the first frame are both generated, the preview engine is started, and the accompaniment audio signal of the first frame and the human voice audio signal of the first frame are synthesized by the preview engine to obtain the song audio signal of the first frame and the song audio signal of the first frame is played; in the process of executing the first frame playback task in the main process, the subsequent frame tasks of the first thread are synchronously executed to generate the accompaniment audio signal of the subsequent frame and the subsequent frame tasks of the second thread are synchronously executed to generate the human voice audio signal of the subsequent frame; the subsequent frame playback tasks are executed through the main process: whenever it is monitored that the accompaniment audio signal and the human voice audio signal of the same subsequent frame are both generated, the accompaniment audio signal and the human voice audio signal of the same subsequent frame are synthesized by the preview engine to obtain the song audio signal of the same subsequent frame and the song audio signal of the same subsequent frame is played, until the same subsequent frame is the end frame of the recommended song.

[0116] In one of the exemplary embodiments, the above-mentioned device also includes an engine preheating module, which is specifically used to start a song recording engine in response to the generation progress of the inferred singing of the recommended song meeting a preset progress during the process of playing the inferred singing of the recommended song; the song recording engine is used to record the singing sung by the user.

[0117] In one of the exemplary embodiments, the above-mentioned device also includes a view rendering module, which is specifically used to obtain view resources of the display view of the first song recording auxiliary information; the view resources are reused rendering resources for generating the display view of the second song recording auxiliary information; according to the view resources, the song recording interface is rendered and displayed.

[0118] In one of the exemplary embodiments, the switching module 906 is specifically used to set the position of the display view of the first song recording auxiliary information in the song recording recommendation interface to be the same as the position of the display view of the second song recording auxiliary information in the song recording interface.

[0119] In one of the exemplary embodiments, the switching module 906 is specifically used to display a fade-out and fade-in animation effect of a difference view during the process of switching from the song recommendation interface to the song recording interface; the difference view is a view where there is a difference between the song recommendation interface and the song recording interface.

[0120] Each module in the above-mentioned song recommendation device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each module.

[0121] In an exemplary embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as shown in FIG. Fig.10As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, near field communication (NFC) or other technologies. When the computer program is executed by the processor, a method for recording song recommendations is implemented. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse.

[0122] Those skilled in the art will understand that Fig.10 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0123] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program. In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and the computer program implements the steps in the above-mentioned method embodiments when executed by the processor. In one embodiment, a computer program product is provided, including a computer program, and the computer program implements the steps in the above-mentioned method embodiments when executed by the processor.

[0124] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0125] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0126] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0127] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be construed as limiting the scope of the present application. It should be noted that, for a person of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A song recommendation method, characterized in that: The method comprises: In the process of presenting the recommended songs to the user through the recorded song recommendation interface, playing the inference singing voice simulating the user singing the recommended songs; During the playing of the inferred song, the first song recording auxiliary information and the song recording entrance are displayed on the song recording recommendation interface; wherein the first song recording auxiliary information includes lyrics information of the recommended song and / or pitch evaluation information corresponding to the inferred song; In response to the triggering operation of the song recording entrance, the song recording recommendation interface switches to a song recording interface displaying the recommended song, and the second song recording auxiliary information and song recording operation controls are displayed on the song recording interface; the second song recording auxiliary information includes the lyrics information of the recommended song and / or the pitch evaluation information corresponding to the singing voice currently sung by the user.

2. The method according to claim 1, characterized in that The method further comprises: In response to a sound effect adjustment operation on the song recording recommendation interface, adjusting the sound effect of the inferred singing voice according to the sound effect adjustment operation, and recording the sound effect configuration of the inferred singing voice; The step of switching from the song recording recommendation interface to the song recording interface that displays the recommended songs includes: According to the sound effect configuration of the inferred song, the song recording configuration during the recording of the recommended song is synchronously set.

3. The method according to claim 1, characterized in that The method further comprises: When the generation progress of the inferred singing voice of the recommended song meets the preset progress, playing the generated inferred singing voice of the recommended song; When the playing progress of the inferred song meets the preset preloading condition, generating an inferred song for simulating the user singing another recommended song; the other recommended song is the next recommended song of the recommended song in the song recommendation sequence of the user; Among them, the preset preloading condition includes that the time difference between the playback progress of the inference song and the generation progress of the inference song is greater than or equal to a preset time difference threshold.

4. The method according to claim 3, characterized in that The playing is used to simulate the inference singing voice of the user singing the recommended song, including: Cache the generated audio data of the inference singing voice of the recommended song in a preset storage area; In response to the generation progress of the inferred singing voice of the recommended song satisfying the preset progress, the audio data of the inferred singing voice is searched from the preset storage area, and the audio data of the inferred singing voice is played.

5. The method according to claim 4, characterized in that The playing is used to simulate the inference singing voice of the user singing the recommended song, including: In the case that the audio data of the inferred singing voice cannot be found in the preset storage area, obtaining the song accompaniment resource and the singing voice inference resource of the recommended song; the singing voice inference resource is used to generate the inferred singing voice; When the audio metadata of the song accompaniment resource is parsed and the inferred singing voice is generated according to the singing voice inference resource, the song accompaniment of the recommended song is generated according to the audio metadata, and the inferred singing voice and the song accompaniment are played.

6. The method according to claim 5, characterized in that The generating the inferred singing voice according to the singing voice inference resource comprises: For the recommended song, the task corresponding to the current frame is executed by the first thread: the accompaniment data of the song accompaniment resource of the recommended song in the current frame is obtained, and the accompaniment data of the song accompaniment resource of the recommended song in the current frame is parsed to generate an accompaniment audio signal of the current frame; Execute the task corresponding to the current frame through the second thread: obtain the vocal data of the singing inference resource in the current frame, and infer the vocal data of the singing inference resource in the current frame to generate the vocal audio signal of the current frame; wherein: the current frame includes the first frame and subsequent frames, and the first frame task of the first thread is executed synchronously with the first frame task of the second thread.

7. The method according to claim 6, characterized in that After the step of executing the task corresponding to the current frame by the second thread, the method further includes: The first frame playing task is executed through the main process: when it is monitored that the accompaniment audio signal of the first frame and the human voice audio signal of the first frame are both generated, the preview engine is started, the accompaniment audio signal of the first frame and the human voice audio signal of the first frame are synthesized by the preview engine to obtain the song audio signal of the first frame, and the song audio signal of the first frame is played; During the execution of the first frame playing task by the main process, the subsequent frame tasks of the first thread are synchronously executed to generate the accompaniment audio signals of the subsequent frames, and the subsequent frame tasks of the second thread are synchronously executed to generate the human voice audio signals of the subsequent frames; The subsequent frame playback task is executed through the main process: whenever it is monitored that the accompaniment audio signal and the vocal audio signal of the same subsequent frame are generated, the accompaniment audio signal and the vocal audio signal of the same subsequent frame are synthesized by the preview engine to obtain the song audio signal of the same subsequent frame and play the song audio signal of the same subsequent frame until the same subsequent frame is the end frame of the recommended song.

8. The method according to claim 4, characterized in that During the process of playing the inferred singing voice of the recommended song, in response to the generation progress of the inferred singing voice of the recommended song meeting the preset progress, a song recording engine is started; the song recording engine is used to record the singing voice sung by the user.

9. The method according to claim 1, characterized in that: The step of switching from the song recording recommendation interface to the song recording interface that displays the recommended songs includes: Obtaining a view resource of a display view of the first song recording auxiliary information; the view resource is a reused rendering resource for generating a display view of the second song recording auxiliary information; According to the view resources, the song recording interface is rendered and displayed.

10. The method according to claim 1, characterized in that The step of switching from the song recording recommendation interface to the song recording interface that displays the recommended songs includes: The position of the display view of the first song recording auxiliary information in the song recording recommendation interface is set to be the same as the position of the display view of the second song recording auxiliary information in the song recording interface.

11. The method according to claim 1, characterized in that: The step of switching from the song recording recommendation interface to the song recording interface that displays the recommended songs includes: During the process of switching from the song recommendation interface to the song recording interface, a fade-in and fade-out animation effect of a difference view is displayed; the difference view is a view in which there is a difference between the song recommendation interface and the song recording interface.

12. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 11 are implemented.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 11 are implemented.

14. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 11 are implemented.

Images