Audio processing method and device, equipment and storage medium

Abstract

The invention discloses an audio processing method and device, equipment and a storage medium, and relates to the technical field of computers. The method comprises the following steps: playing a first song; under the condition that the singing following mode of the first song is started, the processed audio of the first song is obtained, the processed audio is obtained by executing audio processing on the original audio of the first song, and the audio processing is used for reducing the volume of human voice in the original audio of the first song; playing the processed first song according to the processed audio; collecting an audio signal of a first song sung by a user through a microphone to obtain a recorded audio; and according to the recorded audio, the processed audio and the original audio, generating a follow-singing audio of the first song, the follow-singing audio of the first song comprising an original audio clip of the first time period and a follow-singing audio clip of the second time period. According to the method and the device, the step that the user follows the song to follow the song is simplified, the use convenience of the song following function is improved, and the song following effect of the song following audio is improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to an audio processing method, device, equipment and storage medium. Background Art

[0002] As people's living standards improve, the usage rate of music applications is getting higher and higher, and with the continuous development of karaoke functions, people continue to use the karaoke functions in music applications to sing songs along with accompaniment.

[0003] In the related art, the user actively searches for the accompaniment in the music application and connects the microphone, then uses the karaoke function to play the song accompaniment, and the user sings along with the song accompaniment. After the user finishes singing, the karaoke function mixes and merges the song accompaniment with the recorded human voice, and uploads the merged audio file to the server for storage. The user can obtain the merged audio file on the client of the music application, and can also share the merged audio file with others so that more people can hear their singing.

[0004] However, the above steps of singing along with the song accompaniment are rather complicated, resulting in more singing restrictions. Summary of the invention

[0005] The embodiments of the present application provide an audio processing method, apparatus, device and storage medium. The technical solutions provided by the embodiments of the present application are as follows:

[0006] According to one aspect of an embodiment of the present application, there is provided an audio processing method, the method comprising:

[0007] Play the first song;

[0008] When the sing-along mode is turned on for the first song, obtaining processed audio of the first song, wherein the processed audio is obtained by performing audio processing on the original audio of the first song, and the audio processing is used to reduce the volume of human voice in the original audio of the first song;

[0009] Playing a processed first song according to the processed audio, wherein the volume of a human voice in the processed first song is smaller than the volume of a human voice in the first song;

[0010] Collecting an audio signal of the user singing the first song through a microphone to obtain recorded audio;

[0011] The sing-along audio of the first song is generated according to the recorded audio, the processed audio and the original audio, wherein the sing-along audio of the first song includes an original audio segment of a first time period and a sing-along audio segment of a second time period, wherein the second time period is a time period when the sing-along mode is on, and the first time period is all or part of the complete time period of the first song except the second time period.

[0012] According to one aspect of an embodiment of the present application, there is provided an audio processing device, the device comprising:

[0013] A first playing module, used for playing a first song;

[0014] an audio processing module, for obtaining processed audio of the first song when the sing-along mode is turned on for the first song, wherein the processed audio is obtained by performing audio processing on the original sound audio of the first song, and the audio processing is used to reduce the volume of human voice in the original sound audio of the first song;

[0015] A second playing module is used to play the processed first song according to the processed audio, wherein the volume of the human voice in the processed first song is smaller than the volume of the human voice in the first song;

[0016] An audio recording module, used to collect an audio signal of the user singing the first song through a microphone to obtain recorded audio;

[0017] An audio generation module is used to generate sing-along audio for the first song based on the recorded audio, the processed audio and the original audio, wherein the sing-along audio for the first song includes an original audio segment of a first time period and a sing-along audio segment of a second time period, wherein the second time period is a time period when the sing-along mode is on, and the first time period is all or part of the complete time period of the first song except the second time period.

[0018] According to one aspect of an embodiment of the present application, a computer device is provided, the computer device comprising a processor and a memory, the memory storing a computer program, the computer program being loaded and executed by the processor to implement the above-mentioned audio processing method.

[0019] According to one aspect of an embodiment of the present application, a computer-readable storage medium is provided, in which a computer program is stored. The computer program is loaded and executed by a processor to implement the above-mentioned audio processing method.

[0020] According to one aspect of an embodiment of the present application, a computer program product is provided. The computer program product includes a computer program. The computer program is loaded and executed by a processor to implement the above-mentioned audio processing method.

[0021] The technical solution provided in the embodiments of the present application can bring the following beneficial effects:

[0022] By turning on the sing-along mode of the first song when the first song is in the playing state, the user can start singing along with the first song at any time, and the user can end singing along with the first song at any time by turning off the sing-along mode of the first song, and generate the sing-along audio of the first song. The steps of actively searching for the accompaniment for singing along in the related art are avoided, and the singing-along form of ending the accompaniment playback is avoided, the steps of the user singing along with the song are simplified, the convenience of using the singing-along function is improved, and the singing-along experience of the user is improved. Moreover, when the sing-along mode is turned on by the first song, the processed first song is played, so that the volume of the human voice in the processed first song is less than the volume of the human voice in the first song, so that the singing-along audio clip can create a singing effect dominated by the user's voice, avoid the human voice volume in the original audio being too high and causing the user's voice to be covered, and improve the singing-along effect of the singing-along audio. Moreover, in addition to the singing-along audio clip of the second period, the generated singing-along audio of the first song also includes the original audio clip of the first period, so that the playback effect of the singing-along audio clip is more natural, and the singing-along effect of the singing-along audio is further improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 It is a schematic diagram of an implementation environment of a solution provided by an embodiment of the present application;

[0024] Figure 2 is a flowchart of an audio processing method provided by an embodiment of the present application;

[0025] Figure 3 is a schematic diagram of the collection of recorded audio provided by an embodiment of the present application;

[0026] Figure 4 is a schematic diagram of a method for determining a first time period provided by an embodiment of the present application;

[0027] Figure 5 is a schematic diagram of an implementation process of sing-along audio provided by an embodiment of the present application;

[0028] Figure 6 is a schematic diagram of an audio processing process when the recorded audio contains blank audio provided by an embodiment of the present application;

[0029] Figure 7 is a block diagram of an audio processing device provided by an embodiment of the present application;

[0030] Figure 8 It is a structural block diagram of a computer device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0031] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0032] Please refer to Figure 1 , which shows a schematic diagram of a solution implementation environment provided by an embodiment of the present application. The solution implementation environment can be implemented as an audio processing system. The solution implementation environment may include: a terminal device 10 and a server 20.

[0033] The number of terminal devices 10 can be one or more. The terminal device 10 can be an electronic device such as a mobile phone, a tablet computer, a laptop computer, a desktop computer, a game console, an e-book reader, a multimedia player, a wearable device, an intelligent voice interaction device, a smart home appliance, a vehicle terminal, an aircraft, etc.

[0034] The terminal device 10 may be installed with a client of a music application, and the music application has a karaoke function, and the user can play songs in the music application and sing along with the songs. After the user finishes singing, a sing-along audio including the song accompaniment and recorded vocals is generated. Optionally, the music application may be an application that needs to be downloaded and installed, or may be a click-to-use application, which is not limited in this application.

[0035] The server 20 is used to provide background services for the client of the music application installed and running in the terminal device 10. For example, the server 20 can be the background server of the above-mentioned music application. The server 20 can be an independent physical server, or a server cluster composed of multiple servers, or a cloud computing service center. Optionally, the server 20 provides background services for the music applications in multiple terminal devices 10 at the same time. The terminal device 10 and the server 20 can communicate with each other through the network.

[0036] In an embodiment of the present application, a user plays a first song in a music application, and when the sing-along mode is turned on for the first song, the processed audio of the first song is obtained, and the processed first song is played according to the processed audio, wherein the volume of the human voice in the processed first song is less than the volume of the human voice in the first song. The audio signal of the user singing the first song is collected through a microphone to obtain recorded audio, and when the sing-along mode is turned off for the first song, the first song is continued to be played from the time when the sing-along mode is turned off according to the original audio of the first song. Then, when the sing-along mode is turned on again for the first song, the recorded audio is obtained to obtain at least one recorded audio, and the sing-along audio of the first song is generated according to at least one recorded audio, the processed audio and the original audio. The sing-along audio of the first song includes the original audio segment of the first time period and the sing-along audio segment of the second time period, wherein the second time period is the time period when the sing-along mode is turned on, and the first time period is all or part of the time period in the complete time period of the first song except the second time period.

[0037] Please refer to Figure 2 , which shows a flow chart of an audio processing method provided by an embodiment of the present application. The execution subject of each step of the method may be a terminal device. The method may include at least one of the following steps 210 to 250:

[0038] Step 210, play the first song.

[0039] The first song is any song sung by any singer, and the original audio of the first song contains human voice, which means that the first song is not a pure music song obtained only by mechanical technology. Optionally, the first song can be a song in which the singer sings the lyrics or hums the melody.

[0040] Exemplarily, a user searches for a first song in a music application, and a search interface for the first song is displayed. Then, the user clicks on the first song in the search interface for the first song, and a play interface for the first song is displayed, in which the first song is played. In response to the click operation on the first song, the client of the music application obtains the original sound audio of the first song, and plays the first song according to the original sound audio of the first song.

[0041] The original audio of the first song refers to the original audio uploaded by the copyright owner of the first song to the client of the music application, and is the audio that has not been processed by anyone other than the copyright owner.

[0042] Step 220, when the sing-along mode is turned on for the first song, the processed audio of the first song is obtained, where the processed audio is obtained by performing audio processing on the original audio of the first song, and the audio processing is used to reduce the volume of the human voice in the original audio of the first song.

[0043] Generally, the sing-along mode refers to a song playing mode in which the audio signal of the user's singing is collected by the microphone when the user sings according to the song being played. When the sing-along mode is turned on for the first song, the microphone of the terminal device is turned on, and the microphone of the terminal device is used to collect the audio signal of the first song sung by the user. In the embodiment of the present application, in addition to the above-mentioned function of turning on the microphone of the terminal device, the sing-along mode also has the function of playing the processed first song.

[0044] When the sing-along mode is turned on for the first song, the client turns on the microphone of the terminal device, obtains the processed audio of the first song, and plays the processed first song according to the processed audio.

[0045] The processed audio is audio obtained by performing audio processing on the original audio of the first song, and the audio processing is used to reduce the volume of the human voice in the original audio of the first song. Therefore, the volume of the human voice in the processed audio is less than the volume of the human voice in the original audio of the first song. The processed audio is audio obtained by performing audio processing on the original audio of the first song in advance, that is, the client pre-stores the processed audio of the first song, and when the client detects that the first song starts the sing-along mode, it directly obtains the processed audio of the first song. The steps for the client to generate the processed audio can refer to the following embodiment and are not introduced here.

[0046] Step 230, playing the processed first song according to the processed audio, wherein the volume of the human voice in the processed first song is lower than the volume of the human voice in the first song.

[0047] After the processed audio of the first song is acquired, the processed first song is played according to the processed audio. Since the volume of the human voice in the processed audio is lower than the volume of the human voice in the original audio of the first song, the volume of the human voice in the processed first song is lower than the volume of the human voice in the first song.

[0048] When the sing-along mode is turned on for the first song, the client obtains the start time of the sing-along mode for the first song, and switches the playback of the processed first song from the start time of the sing-along mode.

[0049] Step 240, collecting the audio signal of the user singing the first song through a microphone to obtain recorded audio.

[0050] When the sing-along mode is turned on for the first song, the client turns on the microphone of the terminal device, and the client can collect the audio signal of the user singing the first song through the microphone to obtain the recorded audio. The recorded audio refers to the audio signal of the user singing the first song collected when the sing-along mode is turned on for the first song. When the sing-along mode is turned off for the first song, even if the user is still singing the first song, the recorded audio will not include the audio signal of the first song sung when the sing-along mode is turned off for the first song.

[0051] In some embodiments, when the first song starts the sing-along mode, the microphone collects the audio signal of the user singing the first song. If the first song is always in the sing-along mode before the first song ends, the recorded audio refers to the audio signal of the user singing the first song collected by the microphone during the entire period of the first song from the start time of the sing-along mode to the end time of the first song. Figure 3 As shown in Figure (1).

[0052] In some embodiments, when the sing-along mode is turned on for the first song, the microphone collects the audio signal of the user singing the first song. If the sing-along mode is turned off for the first song before the first song is finished playing, the recorded audio refers to the audio signal of the user singing the first song collected by the microphone during the entire period of the first song from the time the sing-along mode is turned on to the time the sing-along mode is turned off. Figure 3 As shown in Figure (2).

[0053] In some embodiments, during the complete period of the first song, the first song can start the sing-along mode multiple times, and turn off the sing-along mode multiple times, then a plurality of recorded audios can be obtained correspondingly, each recorded audio is the audio signal of the user singing the first song collected by the microphone from the start time of the sing-along mode to the most recent turn-off time of the sing-along mode during the complete period of the first song, which can be referred to as Figure 3 As shown in Figure (3) in the figure. Recorded audio 1 is the audio signal of the user singing the first song collected by the microphone from the first time the sing-along mode is turned on to the first time the sing-along mode is turned off. Recorded audio 2 is the audio signal of the user singing the first song collected by the microphone from the second time the sing-along mode is turned on to the second time the sing-along mode is turned off. Recorded audio 3 is the audio signal of the user singing the first song collected by the microphone from the third time the sing-along mode is turned on to the third time the sing-along mode is turned off. In some embodiments, the last recorded audio may also be the audio signal of the user singing the first song collected by the microphone during the complete period of the first song from the last time the sing-along mode is turned on to the end of the first song.

[0054] In some embodiments, when the sing-along mode is turned off for the first song, the first song continues to be played from the moment the sing-along mode is turned off based on the original audio of the first song.

[0055] That is, when the sing-along mode is turned on for the first song, the client plays the first song with a lower voice volume, and when the sing-along mode is turned off for the first song, the client plays the first song with a normal voice volume. Figure 3 As shown, in the complete period of the first song, Figure 3 The first song is played during the time period corresponding to the white part in the Figure 3 The black part in the figure corresponds to the time period during which the first song after processing is played.

[0056] By turning off the sing-along mode when the first song is played and continuing to play the first song, the user can hear the original audio of the first song in time, avoiding continuing to play the processed audio with reduced human voice volume which affects the user's listening experience. The user can judge the timing to turn on the sing-along mode and sing along based on the original audio, thereby improving the user's sing-along experience.

[0057] Step 250, generates sing-along audio for the first song based on the recorded audio, processed audio and original audio, wherein the sing-along audio for the first song includes an original audio segment of a first time period and a sing-along audio segment of a second time period, wherein the second time period is a time period when the sing-along mode is on, and the first time period is all or part of the complete time period of the first song except the second time period.

[0058] The sing-along audio of the first song includes an original audio segment of a first time period and a sing-along audio segment of a second time period, wherein the original audio segment of the first time period refers to the original audio segment of the original audio of the first song that is in the first time period, and the sing-along audio segment of the second time period is a sing-along audio segment obtained based on the processed audio segment of the recorded audio and the processed audio that is in the second time period. The sing-along audio segment is an audio segment obtained by mixing the recorded audio and the processed audio segment of the acquisition time period of the recorded audio. The specific steps for generating the sing-along audio of the first song can refer to the following embodiment and will not be introduced here.

[0059] The second time period is the time period when the sing-along mode is turned on, which can also be understood as the second time period being the time period for recording audio. The first time period is all or part of the time period in the complete time period of the first song except the second time period, that is, the first time period is all or part of the time period in the complete time period of the first song except the time period for recording audio.

[0060] The number of recorded audios can be one or more. If the number of recorded audios is one, the second time period is the collection time period of the recorded audio. If the number of recorded audios is multiple, the second time period includes the collection time periods corresponding to the multiple recorded audios.

[0061] In some embodiments, the first time period is all time periods in the complete time period of the first song except the second time period. Then the complete time period of the first song includes the first time period and the second time period, and the audio duration of the sing-along audio of the first song is the song duration of the first song.

[0062] If the number of recorded audios is one, the first time period includes all time periods before the second time period and all time periods after the second time period in the complete time period of the first song. If the start time of the second time period is the start time of the first song, the first time period includes all time periods after the second time period in the complete time period of the first song. If the end time of the second time period is the end time of the first song, the first time period includes all time periods before the second time period in the complete time period of the first song.

[0063] If there are multiple recorded audios, the first time period includes all time periods before the collection time period of the first recorded audio in the complete time period of the first song, all time periods after the collection time period with the last recorded audio, and all time periods between the collection time periods of adjacent recorded audios.

[0064] In some embodiments, the first time period is a partial time period of a complete time period of the first song excluding the second time period.

[0065] If the number of recorded audio is one, the first time period is a partial time period in the complete time period of the first song except the acquisition time period of the recorded audio. For the duration of the first time period, this application is not limited. Optionally, the duration of the first time period can be a fixed duration preset, for example, the duration of the first time period is 5s. Optionally, the duration of the first time period can also be determined according to the duration of the second time period. Exemplarily, when the duration of the second time period is greater than or equal to the first duration, the duration of the first time period is determined as the second duration, and when the duration of the second time period is less than the first duration, the duration of the first time period is determined as the third duration. For the specific values ​​of the first duration, the second duration and the third duration, this application is not limited, for example, the first duration is 30s, the second duration is 5s, and the third duration is 3s. When the duration of the second time period is greater than or equal to 30s, the duration of the first time period is determined to be 5s, and when the duration of the second time period is less than 30s, the duration of the first time period is determined to be 3s.

[0066] If the number of recorded audio is multiple, the first time period includes multiple third time periods, each recorded audio corresponds to a third time period, the third time period is a partial time period in the complete time period of the first song except for the collection time period of multiple recorded audios, and the third time period is a partial time period corresponding to the recorded audio in the first time period. For the duration of the third time period, this application does not limit it. Optionally, the duration of the third time period can be a fixed duration set in advance, for example, the duration of the third time period is 5s. Optionally, the duration of the third time period can also be determined according to the collection duration of the recorded audio corresponding to the third time period. Exemplarily, when the collection duration of the recorded audio corresponding to the third time period is greater than or equal to the first duration, the duration of the third time period is determined as the second duration, and when the collection duration of the recorded audio corresponding to the third time period is less than the first duration, the duration of the third time period is determined as the third duration. For the specific values ​​of the first duration, the second duration and the third duration, this application does not limit it, for example, the first duration is 30s, the second duration is 5s, and the third duration is 3s. When the collection time of the recorded audio corresponding to the third time period is greater than or equal to 30s, the length of the third time period is determined to be 5s. When the collection time of the recorded audio corresponding to the third time period is less than 30s, the length of the third time period is determined to be 3s.

[0067] Optionally, the first time period is a partial time period in the complete time period of the first song that is located before the acquisition time period of the recorded audio. If the number of recorded audios is one, the first time period is a partial time period in the complete time period of the first song that is located before the acquisition time period of the recorded audio. If the number of recorded audios is multiple, the first time period includes multiple third time periods, each of which is a partial time period in the complete time period of the first song that is located before the acquisition time period of the recorded audio corresponding to the third time period.

[0068] Optionally, the first time period is a partial time period in the complete time period of the first song that is located after the acquisition time period of the recorded audio. If the number of recorded audios is one, the first time period is a partial time period in the complete time period of the first song that is located after the acquisition time period of the recorded audio. If the number of recorded audios is multiple, the first time period includes multiple third time periods, each of which is a partial time period in the complete time period of the first song that is located after the acquisition time period of the recorded audio corresponding to the third time period.

[0069] Optionally, the first time period includes a partial time period before the acquisition time period of the recorded audio and a partial time period after the acquisition time period of the recorded audio in the complete time period of the first song. If the number of recorded audios is one, the first time period includes a partial time period before the acquisition time period of the recorded audio and a partial time period after the acquisition time period of the recorded audio in the complete time period of the first song. If the number of recorded audios is multiple, the first time period includes multiple third time periods, each third time period includes a partial time period before the acquisition time period of the recorded audio corresponding to the third time period and a partial time period after the acquisition time period of the recorded audio corresponding to the third time period in the complete time period of the first song.

[0070] Figure 4 A schematic diagram showing a method for determining the first time period is shown. For at least one recorded audio collected, it is necessary to determine whether the duration or quantity of at least one recorded audio reaches a threshold. When the duration or quantity of at least one recorded audio reaches the threshold, the first time period is determined as all time periods in the complete time period of the first song except the second time period. When the duration or quantity of at least one recorded audio does not reach the threshold, the first time period is determined as a partial time period in the complete time period of the first song except the second time period.

[0071] In some embodiments, when the duration of the second time period is greater than or equal to the fourth time period, the first time period is determined as all time periods in the complete time period of the first song except the second time period. When the duration of the second time period is less than the fourth time period, the first time period is determined as part of the complete time period of the first song except the second time period. If the number of recorded audios is one, then when the duration of the recorded audio is greater than or equal to the fourth time period, the first time period is determined as all time periods in the complete time period of the first song except the second time period, and when the duration of the recorded audio is less than the fourth time period, the first time period is determined as all time periods in the complete time period of the first song except the second time period. If the number of recorded audios is multiple, then when the total duration of the multiple recorded audios is greater than or equal to the fourth time period, the first time period is determined as all time periods in the complete time period of the first song except the second time period, and when the total duration of the multiple recorded audios is less than the fourth time period, the first time period is determined as all time periods in the complete time period of the first song except the second time period. For the specific value of the fourth time period, this application is not limited. Exemplarily, the fourth time period can be 1 minute.

[0072] In some embodiments, when the amount of recorded audio is greater than or equal to the first value, the first time period is determined as the entire time period of the first song except the second time period. When the amount of recorded audio is less than the first value, the first time period is determined as a partial time period of the first song except the second time period. The specific value of the first value is not limited in this application. For example, the first value can be 3.

[0073] By setting the first time period to all or part of the complete time period of the first song except the second time period, the generated sing-along audio can be the entire song length of the first song or the length of a song segment in the first song, thereby generating sing-along audio that meets user needs based on the user's sing-along situation.

[0074] The technical solution provided by the embodiment of the present application, by turning on the sing-along mode of the first song when the first song is in the playing state, allows the user to start singing along with the first song at any time, and the user can end singing along with the first song at any time by turning off the sing-along mode of the first song, and generates the sing-along audio of the first song. The steps of actively searching for the accompaniment for singing along in the related art are avoided, and the singing-along form of ending the accompaniment playback is avoided, the steps of the user singing along with the song are simplified, the convenience of using the singing-along function is improved, and the singing-along experience of the user is improved. Moreover, when the singing-along mode is turned on by the first song, the processed first song is played, so that the human voice volume in the processed first song is smaller than the human voice volume in the first song, so that the singing-along audio clip can create a singing effect dominated by the user's voice, avoid the human voice volume in the original audio being too high and causing the user's voice to be covered, and improve the singing-along effect of the singing-along audio. Moreover, in addition to the singing-along audio clip of the second period, the generated singing-along audio of the first song also includes the original audio clip of the first period, so that the playback effect of the singing-along audio clip is more natural, and the singing-along effect of the singing-along audio is further improved.

[0075] Figure 5A schematic diagram of the implementation process of the sing-along audio is shown. First, the first song is played in a music application. When the first song is in the playing state, it is determined whether the sing-along mode is turned on for the first song. When the sing-along mode is not turned on for the first song, the first song continues to be played. When the sing-along mode is turned on for the first song, the processed first song is played, and the volume of the human voice in the processed first song is less than the volume of the human voice in the first song. The audio signal of the user singing the first song is collected through a microphone to obtain recorded audio, and then it is determined whether the sing-along mode is turned off for the first song. When the sing-along mode is not turned off for the first song, the processed first song continues to be played, and the audio signal of the user singing is continued to be collected. When the sing-along mode is turned off for the first song, the first song is switched to be played from the time when the sing-along mode is turned off, that is, the volume of the human voice in the first song is restored. Then it is determined whether the first song ends playing. When the first song ends playing, the sing-along audio of the first song is generated according to the recorded audio, the processed audio and the original audio. When the first song has not finished playing, determine whether the first song has turned on the sing-along mode again, and when the first song has turned on the sing-along mode again, obtain the next recorded audio until the first song has finished playing, and generate the sing-along audio of the first song based on at least one recorded audio, processed audio and original audio.

[0076] In some embodiments, in response to a long press operation on a sing-along control for a first song, the sing-along mode is turned on for the first song; in response to a release operation on the sing-along control for the first song, the sing-along mode is turned off for the first song.

[0077] The first song playback interface displays a sing-along control for the first song, and the long-press operation for the sing-along control for the first song is an operation performed by the user to long-press the sing-along control for the first song. For example, if the user presses and holds the sing-along control for the first song, the client controls the first song to turn on the sing-along mode, and if the user releases the sing-along control for the first song, the client controls the first song to turn off the sing-along mode.

[0078] In the above case, the microphone of the terminal device is originally in an off state. In response to a long press operation of the sing-along control for the first song, the microphone is controlled to be turned on, and in response to a release operation of the sing-along control for the first song, the microphone is controlled to be turned off.

[0079] Therefore, in response to the long press operation of the sing-along control for the first song, the sing-along mode of the first song is controlled to be turned on, and the microphone is controlled to be turned on; in response to the release operation of the sing-along control for the first song, the sing-along mode of the first song is controlled to be turned off, and the microphone is controlled to be turned off.

[0080] By setting the sing-along control for the first song in the playback interface of the first song, the user can independently control whether to turn on the sing-along mode by long pressing the sing-along control, which provides the user with a relatively free operating space and improves the user's sing-along experience.

[0081] In some embodiments, when an audio signal of a user singing a first song is detected, the first song is controlled to start a sing-along mode; when no audio signal of a user singing the first song is detected within a first time period, the first song is controlled to turn off the sing-along mode.

[0082] In the above case, when the first song is being played, the microphone of the terminal device is always turned on. For example, in response to the play operation of the first song, the microphone is controlled to be turned on. Then the microphone can detect in real time whether the audio signal of the user singing the first song is collected. When the user starts to sing the first song, the terminal device detects the audio signal of the user singing the first song and controls the first song to start the sing-along mode.

[0083] It should be noted that the user may speak or sing the melody of other songs during the playing of the first song. After the microphone collects the user's voice, the terminal device needs to identify the collected voice, and control the first song to start the sing-along mode when it is detected that the sound collected by the microphone is the audio signal of the user singing the first song. And when the audio signal of the user singing the first song is not detected within the first time period, control the first song to turn off the sing-along mode.

[0084] This application does not limit the specific value of the first time period. Exemplarily, the first time period is 10 seconds. If no audio signal of the user singing the first song is detected within 10 seconds, the sing-along mode of the first song is controlled to be turned off.

[0085] By always turning on the microphone when the first song is playing, the user does not need to perform any operation. The audio signal of whether the user is singing the first song can be collected and detected in real time to control whether to turn on the sing-along mode. This simplifies the user's operation steps and allows the user to sing along and end singing along at any time, thereby improving the user's sing-along experience.

[0086] In some embodiments, the process of generating the processed audio of the first song includes the following steps A1 to A3.

[0087] Step A1, performing track separation on the original audio to obtain the vocal track and the accompaniment track in the original audio.

[0088] The original audio of the first song is separated by a multi-track separation model to obtain a vocal track and an accompaniment track in the original audio. The vocal track refers to the sound track of the singer singing the first song in the first song, and the accompaniment track refers to other tracks in the first song except the vocal track.

[0089] Optionally, the vocal track can be a single track, in which case the vocal track is a track obtained by collecting an audio signal of a singer singing the first song, and the vocal track only contains a track of the singer singing the first song. Optionally, the vocal track can also be multiple tracks, in which case the vocal track can be a track obtained by collecting an audio signal of a singer singing the first song, and the vocal track contains multiple tracks of the singer singing the first song. The vocal track can also be a track obtained by collecting audio signals of multiple singers singing the first song, and the vocal track contains at least one track of multiple singers singing the first song.

[0090] The accompaniment track includes at least one of the following: an instrument track, an effector track, and a harmony track. The instrument track is a track generated based on the performance of an instrument. The instrument mentioned here can be an actual instrument, such as a piano, a violin, a erhu, etc., and the instrument track is obtained based on the real-time performance of the instrument; it can also be an electronic instrument, and the instrument track is obtained based on data operation; it can also include actual instruments and electronic instruments, and the above-mentioned instrument tracks are obtained based on the tracks obtained by real-time performance and the tracks obtained by data operation. The effector track is a track generated based on various effectors, which is used to enhance and modify the timbre of other tracks. The harmony track is a track obtained by collecting human voices. The harmony track can be a track sung by the singer corresponding to the first song, or a track sung by other people, and this application does not limit this. And the above-mentioned instrument track, effector track, and harmony track can be a single track or multiple tracks, and this application does not limit this.

[0091] The original audio of the first song contains multiple tracks, and the multi-track separation (Music Source Separation, MSS) task usually uses a deep learning model to separate the audio data into different tracks, such as a vocal track and an accompaniment track. The track separation model here can be any publicly available multi-track separation model, such as a model based on a deep learning structure obtained by training with a large amount of data, and the large amount of data can reach more than 100 million samples, which is not limited in this application.

[0092] Exemplarily, the multi-track separation model can be a Wave-U-Net model, which gradually restores the audio signal by extracting and upsampling the multi-level features of the audio waveform, while retaining the independent characteristics between the human voice and the accompaniment, so as to achieve the separation of the human voice and the accompaniment, such as the separation of the human voice track and the accompaniment track. Exemplarily, the multi-track separation model can be an Open-Unmix model, which converts the audio to the frequency domain through short-time Fourier transform, uses a neural network to operate on the spectrum, predicts the amplitude spectrogram of each source (such as human voice or accompaniment), and finally reconstructs the separated audio signal through inverse short-time Fourier transform to obtain the human voice track and the accompaniment track.

[0093] Through the multi-track separation model, different tracks can be extracted from the original audio. The input of the multi-track separation model is the original audio of the first song, and the output is the vocal track and accompaniment track in the original audio.

[0094] Step A2, performing filtering processing on the human voice track through a band-stop filter to obtain a processed human voice track, wherein the filtering processing is used to reduce the volume of the human voice in the human voice track.

[0095] The band stop filter is a filter used to suppress signals in a specific frequency band. The human voice track is filtered by the band stop filter, that is, the audio signals in the human voice track that are within the stop band range of the band stop filter are suppressed by the band stop filter to obtain the processed human voice track. The processed human voice track includes audio signals with frequencies lower than the lower limit of the stop band range and audio signals with frequencies higher than the upper limit of the stop band range.

[0096] The stopband range of the band-stop filter is determined according to the frequency range of the human voice, which is not limited in this application. Exemplarily, the stopband range of the band-stop filter is 300 to 3000 Hz, and the band-stop filter is used to suppress the human voice signal in the frequency range of 300 to 3000 Hz in the human voice track.

[0097] It should be noted that the filtering process performed by the band-stop filter is used to reduce the volume of the human voice in the vocal track, and it does not necessarily completely eliminate the volume of the human voice in the vocal track. The vocal signal may have frequencies outside the stopband range of the band-stop filter in the vocal track. If the stopband range of the band-stop filter covers all frequencies that the human voice can reach, the filtering process can eliminate the volume of the human voice in the vocal track. Therefore, the processed vocal track is a vocal track with a reduced volume, not a vocal track with a eliminated volume.

[0098] Step A3, superimposing the processed vocal track and the accompaniment track to obtain processed audio.

[0099] The processed vocal track and the accompaniment track are superimposed to obtain the processed audio. Since the processed vocal track may contain a vocal signal with a frequency outside the stopband range of the band stop filter, and the accompaniment track may contain a harmony track, the processed audio may contain a vocal signal, but the volume of the vocal in the processed audio is less than the volume of the vocal in the original audio.

[0100] By performing audio processing on the original audio, the vocal volume in the vocal track is reduced so that the vocal volume in the processed audio is smaller than that in the original audio. When the processed first song is played, a singing effect dominated by the user's voice can be created, thereby avoiding the user's voice being covered up due to the excessively high vocal volume in the original audio, and helping to improve the sing-along effect of the sing-along audio.

[0101] In some embodiments, it is not necessary to perform track separation on the original sound audio, and the original sound audio can be directly filtered through a band stop filter to obtain the processed original sound audio, which is the processed audio mentioned above. In this way, the overall human voice volume in the original sound audio can be reduced.

[0102] In some embodiments, step 250 includes at least one of sub-steps 251 - 253 .

[0103] Sub-step 251, obtaining a processed audio segment of a second time period in the processed audio, and obtaining an original audio segment of a first time period in the original audio.

[0104] In some embodiments, if the number of recorded audios is one, the second time period is the collection time period of the recorded audio, and obtaining the processed audio segment of the second time period in the processed audio is to obtain the processed audio segment of the collection time period of the recorded audio in the processed audio.

[0105] In some embodiments, if there are multiple recorded audios, the second time period includes acquisition time periods corresponding to the multiple recorded audios respectively, and obtaining the processed audio segments of the second time period in the processed audio means obtaining the processed audio segments of the acquisition time periods corresponding to the multiple recorded audios respectively in the processed audio.

[0106] The volume of the human voice in the processed audio segment of the second period is less than the volume of the human voice in the original audio of the second period.

[0107] Sub-step 252, performing mixing processing on the recorded audio and the processed audio segment of the second time period to obtain the sing-along audio segment of the second time period.

[0108] In some embodiments, if the number of recorded audios is one, the recorded audio and the processed audio segment of the second period are mixed to obtain the sing-along audio segment of the second period. The sing-along audio segment of the second period is the sing-along audio segment corresponding to the recorded audio.

[0109] In some embodiments, if there are multiple recorded audios, mixing processing is performed on the multiple recorded audios and the processed audio segments of the acquisition time periods corresponding to the multiple recorded audios, respectively, to obtain the sing-along audio segments corresponding to the multiple recorded audios. For each of the multiple recorded audios, mixing processing is performed on the recorded audio and the processed audio segments of the acquisition time period corresponding to the recorded audio, to obtain the sing-along audio segments corresponding to the recorded audio. The sing-along audio segments of the second time period include the sing-along audio segments corresponding to the multiple recorded audios.

[0110] Mixing processing refers to the audio processing process of combining two audio data into an output audio stream. This process requires processing and merging each audio data to generate the final mixed audio. The two audio data mentioned here refer to the recorded audio and the processed audio during the acquisition period of the recorded audio. The generated mixed audio is the sing-along audio clip corresponding to the recorded audio. In the process of performing mixing processing on the client, it is usually necessary to control the volume, channel, time and other aspects of each audio data to ensure that the final mixing effect is natural and meets the requirements.

[0111] In some embodiments, the process of mixing includes the following steps B1 to B5.

[0112] Step B1, performing preprocessing on two audio data to obtain two preprocessed audio data, wherein the preprocessing includes at least one of the following: sampling rate conversion processing, channel adjustment processing, and audio format unification processing.

[0113] The sampling rate conversion process is used to ensure that the sampling rates of the two audio data are consistent to avoid incompatibility issues during mixing. The channel adjustment process is used to ensure that the number of channels of the two audio data is consistent, such as stereo, mono and other processing methods. The audio format unification process is used to convert the two audio data into a unified format for processing, such as WAV (Waveform Audio File Format), PCM (Pulse Code Modulation) and other formats.

[0114] Step B2, performing gain adjustment processing on the two pre-processed audio data respectively to obtain two audio data after volume adjustment.

[0115] Perform gain or attenuation processing on each pre-processed audio data to adjust the volume of each audio data to avoid excessive volume or distortion after audio superposition.

[0116] Step B3, respectively setting the channel positions for the two audio data after the volume is adjusted, to obtain the two audio data after the channel is set.

[0117] The channel position, such as the left channel or the right channel, is set for each audio stream after the volume is adjusted, so that when two audio data are mixed, some audio data can be biased to one side to provide a sense of space for the left and right channels, such as audio panning, stereo effects, etc.

[0118] Step B4, performing effect processing on the two audio data after the channel setting respectively to obtain at least two audio data after effect processing, and the effect processing includes at least one of the following: sound effect adding processing, filtering processing, and compression processing.

[0119] After the channel is set, the different audio data are processed with corresponding effects, such as reverberation, delay, equalization, etc., to ensure the sound coordination of each audio data and enhance the expressiveness and coordination of the mixed audio. If the audio data comes from different sources, each audio data may also need to be filtered, compressed, etc. to ensure that there is no frequency conflict between different audio data.

[0120] Step B5, performing track superposition on the two audio data after the effect processing according to the playing time of the two audio data after the effect processing to obtain mixed audio.

[0121] According to the play time of the two audio data after the effect processing, the play time of the two audio data is aligned to avoid the situation that different audio data are out of sync. For pre-recorded audio data and real-time generated audio data, synchronization processing may be required.

[0122] Perform track superposition on the two audio data after effect processing, that is, add the waveforms of the two audio data to generate the final mixed audio. In the superposition process, dynamic range compression may be required to avoid distortion caused by excessive volume. All audio tracks are merged into a final output audio stream and exported as a file (such as WAV, MP3, etc.) to obtain the input audio.

[0123] By mixing two audio data using the above method, the synchronization and coordination between the various audio data are effectively guaranteed, and the audio quality of the generated mixed audio is guaranteed.

[0124] Sub-step 253, splicing the original audio clip of the first time period and the sing-along audio clip of the second time period to generate the sing-along audio of the first song.

[0125] In some embodiments, if the number of recorded audios is one, the original audio segment of the first time period and the sing-along audio segment of the second time period (the sing-along audio segment corresponding to the recorded audio) are spliced ​​to generate the sing-along audio of the first song.

[0126] In some embodiments, if there are multiple recorded audios, when the first time period is all time periods in the complete time period of the first song except the second time period, the original audio segment of the first time period and the sing-along audio segments corresponding to the multiple recorded audios are spliced ​​to generate the sing-along audio of the first song.

[0127] In the case where there are multiple recorded audios, the first time period includes all time periods before the collection time period of the first recorded audio, all time periods after the collection time period with the last recorded audio, and all time periods between the collection time periods of adjacent recorded audios in the complete time period of the first song. The original audio segment of the first time period and the sing-along audio segments corresponding to the multiple recorded audios are spliced ​​according to the time period to obtain the sing-along audio of the first song.

[0128] In some embodiments, if there are multiple recorded audios, when the first time period is a partial time period of the complete time period of the first song excluding the second time period, multiple native audio clips of the third time periods and the sing-along audio clips corresponding to the multiple recorded audios are spliced ​​together to generate the sing-along audio of the first song. The third time period is a partial time period corresponding to the recorded audio in the first time period, and the sing-along audio of the first song includes the sing-along audios corresponding to the multiple recorded audios.

[0129] Each recorded audio corresponds to a third time period, and the third time period is a partial time period corresponding to the recorded audio in the first time period. Optionally, the third time period may be a partial time period in the complete time period of the first song that is before the acquisition time period of the recorded audio corresponding to the third time period, or may be a partial time period in the complete time period of the first song that is after the acquisition time period of the recorded audio corresponding to the third time period, or may include a partial time period in the complete time period of the first song that is before the acquisition time period of the recorded audio corresponding to the third time period, and a partial time period after the acquisition time period of the recorded audio corresponding to the third time period. Exemplarily, if the duration of the third time period is 5s, the third time period may be 5s before the acquisition time period of the recorded audio, or may be 5s after the acquisition time period of the recorded audio, or may include 5s before the acquisition time period of the recorded audio and 5s after the acquisition time period of the recorded audio.

[0130] For each acquired sing-along audio segment corresponding to the recorded audio, the original audio segment of the third time period corresponding to the recorded audio in the original audio segment of the first time period is acquired, and the original audio segment of the third time period corresponding to the recorded audio and the sing-along audio segment corresponding to the recorded audio are spliced ​​to generate the sing-along audio corresponding to the recorded audio. Thus, the sing-along audio of the first song is obtained according to the sing-along audios corresponding to the multiple recorded audios.

[0131] By performing mixing processing on the recorded audio and the processed audio clips of the second period, the obtained sing-along audio clip of the second period can better coordinate the relationship between the user's vocals and the accompaniment, thereby improving the audio quality of the sing-along audio clip. The original audio clip of the first period and the sing-along audio clip of the second period are spliced ​​together, making the playback effect of the sing-along audio clip more natural, thereby improving the sing-along effect of the sing-along audio.

[0132] In some embodiments, the recorded audio includes blank audio, in which no audio signal of the user singing the first song is detected.

[0133] When the sing-along mode is turned on for the first song, the user does not sing the first song, resulting in blank audio in the recorded audio, and the client does not detect the audio signal of the user singing the first song in the blank audio. Optionally, the blank audio may be located at the starting stage of the recorded audio, that is, the starting time of the blank audio is the starting time of the recorded audio. Optionally, the blank audio may also be located at the ending stage of the recorded audio, that is, the ending time of the blank audio is the ending time of the recorded audio. Optionally, the blank audio may also be located in the middle stage of the recorded audio, that is, the starting time of the blank audio is after the starting time of the recorded audio, and the ending time of the blank audio is after the ending time of the recorded audio.

[0134] Optionally, the recorded audio may include one blank audio or multiple blank audios.

[0135] Exemplarily, the user long presses the sing-along control of the first song to control the first song to start the sing-along mode. During the process of the user long pressing the sing-along control of the first song, the user does not sing the first song, resulting in blank audio in the recorded audio. Optionally, the blank audio can be located at the beginning stage of the recorded audio, at the end stage of the recorded audio, or at the middle stage of the recorded audio.

[0136] Exemplarily, a user sings a first song, and when an audio signal of the user singing the first song is detected, the first song is controlled to start a sing-along mode. During the user singing the first song, there is a period of time when the first song is not sung, resulting in blank audio in the recorded audio. Optionally, the blank audio can be located at the end stage of the recorded audio or at the middle stage of the recorded audio. If the blank audio is located at the end stage of the recorded audio, the duration of the blank audio is equal to the duration of the first time period. If the blank audio is located at the middle stage of the recorded audio, the duration of the blank audio is less than the duration of the first time period.

[0137] Step 250 includes at least one of sub-steps 254 - 257 .

[0138] Sub-step 254, segmenting the recorded audio according to the blank audio to obtain at least one recorded audio segment, wherein the recorded audio segment contains an audio signal of the user singing the first song.

[0139] If the number of blank audios is one, then when the blank audio is at the beginning of the recorded audio or at the end of the recorded audio, the recorded audio is segmented according to the blank audio to obtain one recorded audio segment. When the blank audio is in the middle of the recorded audio, the recorded audio is segmented according to the blank audio to obtain multiple recorded audio segments.

[0140] If there are multiple blank audios, the recorded audio is segmented according to the blank audios to obtain multiple recorded audio segments.

[0141] Sub-step 255, obtaining processed audio segments of the collection period corresponding to at least one recorded audio segment in the processed audio, and obtaining the original audio segment of the first period in the original audio and the original audio segment of the collection period of the blank audio.

[0142] Sub-step 256, performing mixing processing on at least one recorded audio segment and the processed audio segment of the collection period corresponding to the at least one recorded audio segment, respectively, to obtain the sing-along audio segment corresponding to the at least one recorded audio segment.

[0143] For each recorded audio segment, a mixing process is performed on the recorded audio segment and the processed audio segment of the acquisition period corresponding to the recorded audio segment to obtain a sing-along audio segment corresponding to the recorded audio segment. The steps of the mixing process can refer to the above embodiment and are not introduced here.

[0144] Sub-step 257, splicing the sing-along audio segments corresponding to at least one recorded audio segment, the original audio segment of the first time period and the original audio segment of the blank audio collection period, to generate the sing-along audio of the first song.

[0145] The sing-along audio clips corresponding to at least one recorded audio clip are spliced ​​with the original audio clips of the blank audio collection period to generate the sing-along audio clips of the second period; the sing-along audio clips of the second period are spliced ​​with the original audio clips of the first period to generate the sing-along audio of the first song.

[0146] By supplementing part of the blank audio with original audio clips when the recorded audio includes blank audio, the resulting sing-along audio is made more complete, avoiding the appearance of blank audio in the generated sing-along audio due to the presence of blank audio in the recorded audio, which affects the sing-along effect of the audio and improves the audio quality of the sing-along audio.

[0147] In some embodiments, when the total duration of the blank audio is greater than or equal to the fifth duration, step 250 executes at least one of sub-steps 254 to 257. When the total duration of the blank audio is less than the fifth duration, step 250 executes at least one of sub-steps 251 to 253. The specific value of the fifth duration is not limited in this application, and illustratively, the fifth duration is 10s.

[0148] In some embodiments, if the number of blank audios is one, then when the duration of the blank audio is greater than or equal to the sixth duration, step 250 executes at least one of sub-steps 254 to 257. If the duration of the blank audio is less than the sixth duration, step 250 executes at least one of sub-steps 251 to 253. If the number of blank audios is multiple, then when there is a blank audio whose duration is greater than or equal to the sixth duration among the multiple blank audios, step 250 executes at least one of sub-steps 254 to 257. If the duration of the multiple blank audios is less than the sixth duration, step 250 executes at least one of sub-steps 251 to 253. The specific value of the sixth duration is not limited in this application. By way of example, the sixth duration is 5s.

[0149] Figure 6A schematic diagram of an audio processing process when blank audio is included in the recorded audio is shown. After obtaining the recorded audio, it is determined whether there is blank audio in the recorded audio. If there is blank audio in the recorded audio, the recorded audio is segmented according to the blank audio to obtain at least one recorded audio segment and blank audio. For at least one recorded audio segment, a processed audio segment of a collection period corresponding to at least one recorded audio segment in the processed audio is obtained, and mixing processing is performed on at least one recorded audio segment and a processed audio segment of a collection period corresponding to at least one recorded audio segment to obtain a sing-along audio segment corresponding to at least one recorded audio segment. For blank audio, an original audio segment of the collection period of blank audio in the original audio is obtained. And an original audio segment of the first period in the original audio is obtained. The sing-along audio segment corresponding to at least one recorded audio segment is spliced ​​with the original audio segment of the first period and the original audio segment of the collection period of blank audio to generate the sing-along audio of the first song.

[0150] The following is an embodiment of the device of the present application, which can be used to execute the embodiment of the method of the present application. For details not disclosed in the embodiment of the device of the present application, please refer to the embodiment of the method of the present application.

[0151] Please refer to Figure 7 , which shows a block diagram of an audio processing device provided by an embodiment of the present application. The device has the function of implementing the above-mentioned audio processing method, and the function can be implemented by hardware, or by hardware executing corresponding software. The device can be the terminal device introduced above, or it can be set in the terminal device. Figure 7 As shown, the device 700 may include: a first playing module 710 , an audio processing module 720 , a second playing module 730 , an audio recording module 740 and an audio generating module 750 .

[0152] The first playing module 710 is used to play the first song.

[0153] The audio processing module 720 is used to obtain the processed audio of the first song when the sing-along mode is turned on for the first song. The processed audio is obtained by performing audio processing on the original audio of the first song. The audio processing is used to reduce the volume of the human voice in the original audio of the first song.

[0154] The second playing module 730 is used to play the processed first song according to the processed audio, wherein the volume of the human voice in the processed first song is smaller than the volume of the human voice in the first song.

[0155] The audio recording module 740 is used to collect the audio signal of the user singing the first song through a microphone to obtain recorded audio.

[0156] The audio generation module 750 is used to generate the sing-along audio of the first song based on the recorded audio, the processed audio and the original audio, wherein the sing-along audio of the first song includes an original audio segment of a first time period and a sing-along audio segment of a second time period, wherein the second time period is a time period when the sing-along mode is on, and the first time period is all or part of the complete time period of the first song except the second time period.

[0157] In some embodiments, the audio generation module 750 is used to:

[0158] Obtaining a processed audio segment of the second time period in the processed audio, and obtaining an original audio segment of the first time period in the original audio;

[0159] Performing mixing processing on the recorded audio and the processed audio segment of the second time period to obtain a sing-along audio segment of the second time period;

[0160] The original audio clip of the first time period and the sing-along audio clip of the second time period are spliced ​​together to generate the sing-along audio of the first song.

[0161] In some embodiments, the first time period is all time periods in the complete time period of the first song except the second time period;

[0162] or,

[0163] The first time period is a partial time period of the complete time period of the first song that is before the acquisition time period of the recorded audio;

[0164] or,

[0165] The first time period is a partial time period in the complete time period of the first song that is located after the acquisition time period of the recorded audio;

[0166] or,

[0167] The first time period includes a partial time period before the acquisition time period of the recorded audio and a partial time period after the acquisition time period of the recorded audio in the complete time period of the first song.

[0168] In some embodiments, the device further comprises a mode control module, wherein the mode control module is configured to:

[0169] In response to a long press operation on a sing-along control of the first song, controlling the first song to start the sing-along mode;

[0170] In response to a release operation of the sing-along control for the first song, the first song is controlled to turn off the sing-along mode.

[0171] In some embodiments, the mode control module is used to:

[0172] When an audio signal of a user singing the first song is detected, controlling the first song to start the sing-along mode;

[0173] In the case that no audio signal of the user singing the first song is detected within a first time period, the first song is controlled to turn off the sing-along mode.

[0174] In some embodiments, the first playback module 710 is used to:

[0175] In the case where the sing-along mode is turned off for the first song, the first song continues to be played from the time when the sing-along mode is turned off according to the original audio of the first song.

[0176] In some embodiments, the audio processing module 720 is used to:

[0177] Performing track separation on the original audio to obtain a vocal track and an accompaniment track in the original audio;

[0178] Performing filtering processing on the vocal track by using a band stop filter to obtain a processed vocal track, wherein the filtering processing is used to reduce the volume of the vocal in the vocal track;

[0179] The processed vocal track and the accompaniment track are superimposed to obtain the processed audio.

[0180] In some embodiments, the second time period includes a plurality of collection time periods corresponding to the recorded audios respectively; the audio generation module 750 is used to:

[0181] Obtain processed audio segments of the collection time periods respectively corresponding to the multiple recorded audios in the processed audio;

[0182] Performing mixing processing on the multiple recorded audios and the processed audio segments of the collection time periods respectively corresponding to the multiple recorded audios, respectively, to obtain the sing-along audio segments respectively corresponding to the multiple recorded audios;

[0183] In a case where the first time period is all time periods in the complete time period of the first song except the second time period, splicing the original audio segment of the first time period and the sing-along audio segments corresponding to the plurality of recorded audios respectively to generate the sing-along audio of the first song;

[0184] In the case that the first time period is a partial time period of the complete time period of the first song excluding the second time period, the native audio clips of multiple third time periods and the sing-along audio clips corresponding to the multiple recorded audios are spliced ​​together to generate the sing-along audio of the first song. The third time period is a partial time period corresponding to the recorded audio in the first time period, and the sing-along audio of the first song includes the sing-along audios corresponding to the multiple recorded audios.

[0185] In some embodiments, the recorded audio includes blank audio, and no audio signal of the user singing the first song is detected in the blank audio; the audio generation module 750 is used to:

[0186] Segment the recorded audio according to the blank audio to obtain at least one recorded audio segment, wherein the recorded audio segment contains an audio signal of the user singing the first song;

[0187] Obtaining processed audio segments of the collection time periods respectively corresponding to the at least one recorded audio segment in the processed audio, and obtaining original audio segments of the first time period in the original audio and original audio segments of the collection time period of the blank audio;

[0188] Performing mixing processing on the at least one recorded audio segment and the processed audio segments of the collection time period respectively corresponding to the at least one recorded audio segment, respectively, to obtain the sing-along audio segments respectively corresponding to the at least one recorded audio segment;

[0189] The sing-along audio segments corresponding to the at least one recorded audio segment are spliced ​​with the original audio segments of the first time period and the original audio segments of the blank audio collection period to generate the sing-along audio of the first song.

[0190] It should be noted that the device provided in the above embodiment, when implementing its functions, is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the content structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the device and method embodiments provided in the above embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0191] Please refer to Figure 8 , which shows a block diagram of a computer device 800 provided in one embodiment of the present application. The computer device 800 may be any electronic device with data calculation, processing and storage functions. The computer device 800 may be used to implement the audio processing method provided in the above embodiment.

[0192] Typically, the computer device 800 includes a processor 801 and a memory 802 .

[0193] The processor 801 may include one or more processing cores, such as a 4-core processor, an 8-core processor, and the like. The processor 801 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field Programmable Gate Array), and PLA (Programmable Logic Array). The processor 801 may also include a main processor and a coprocessor. The main processor is a processor for processing data in an awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in a standby state. In some embodiments, the processor 801 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 801 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0194] The memory 802 may include one or more computer-readable storage media, which may be non-transitory. The memory 802 may also include a high-speed random access memory and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 802 is used to store a computer program, which is configured to be executed by one or more processors to implement the above-mentioned audio processing method.

[0195] Those skilled in the art will understand that Figure 8 The structure shown in the figure does not constitute a limitation on the computer device 800, and the computer device 800 may include more or less components than those shown in the figure, or combine some components, or adopt a different arrangement of components.

[0196] In an exemplary embodiment, a computer-readable storage medium is also provided, in which a computer program is stored, and when the computer program is executed by a processor of a computer device, the above-mentioned audio processing method is implemented. Optionally, the above-mentioned computer-readable storage medium can be a ROM (Read-Only Memory), a RAM (Random Access Memory), a CD-ROM (Compact Disc Read-Only Memory), a magnetic tape, a floppy disk, an optical data storage device, etc.

[0197] In an exemplary embodiment, a computer program product is also provided, the computer program product comprising a computer program, the computer program being stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium, and the processor executes the computer program, so that the computer device performs the above audio processing method.

[0198] It should be understood that the "multiple" mentioned in this article refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. In addition, the step numbers described in this article only illustrate a possible execution sequence between the steps. In some other embodiments, the above steps may not be executed in the order of the numbers, such as two steps with different numbers are executed at the same time, or two steps with different numbers are executed in the opposite order to the diagram. The embodiments of the present application are not limited to this.

[0199] The above description is only an exemplary embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. An audio processing method, characterized in that: The method comprises: Play the first song; When the sing-along mode is turned on for the first song, obtaining processed audio of the first song, wherein the processed audio is obtained by performing audio processing on the original audio of the first song, and the audio processing is used to reduce the volume of human voice in the original audio of the first song; Playing a processed first song according to the processed audio, wherein the volume of a human voice in the processed first song is smaller than the volume of a human voice in the first song; Collecting an audio signal of the user singing the first song through a microphone to obtain recorded audio; The sing-along audio of the first song is generated according to the recorded audio, the processed audio and the original audio, wherein the sing-along audio of the first song includes an original audio segment of a first time period and a sing-along audio segment of a second time period, wherein the second time period is a time period when the sing-along mode is on, and the first time period is all or part of the complete time period of the first song except the second time period.

2. The method according to claim 1, characterized in that The step of generating the sing-along audio of the first song according to the recorded audio, the processed audio and the original audio comprises: Obtaining a processed audio segment of the second time period in the processed audio, and obtaining an original audio segment of the first time period in the original audio; Performing mixing processing on the recorded audio and the processed audio segment of the second time period to obtain a sing-along audio segment of the second time period; The original audio clip of the first time period and the sing-along audio clip of the second time period are spliced ​​together to generate the sing-along audio of the first song.

3. The method according to claim 2, characterized in that The first time period is all time periods in the complete time period of the first song except the second time period; or, The first time period is a partial time period of the complete time period of the first song that is before the acquisition time period of the recorded audio; or, The first time period is a partial time period in the complete time period of the first song that is located after the acquisition time period of the recorded audio; or, The first time period includes a partial time period before the acquisition time period of the recorded audio and a partial time period after the acquisition time period of the recorded audio in the complete time period of the first song.

4. The method according to claim 1, characterized in that: The method further comprises: In response to a long press operation on a sing-along control of the first song, controlling the first song to start the sing-along mode; In response to a release operation of the sing-along control for the first song, the first song is controlled to turn off the sing-along mode.

5. The method according to claim 1, characterized in that The method further comprises: When an audio signal of a user singing the first song is detected, controlling the first song to start the sing-along mode; In the case that no audio signal of the user singing the first song is detected within a first time period, the first song is controlled to turn off the sing-along mode.

6. The method according to claim 1, characterized in that After collecting the audio signal of the user singing the first song through the microphone to obtain the recorded audio, the method further includes: In the case where the sing-along mode is turned off for the first song, the first song continues to be played from the time when the sing-along mode is turned off according to the original audio of the first song.

7. The method according to claim 1, characterized in that The method further comprises: Performing track separation on the original audio to obtain a vocal track and an accompaniment track in the original audio; Performing filtering processing on the vocal track by using a band stop filter to obtain a processed vocal track, wherein the filtering processing is used to reduce the volume of the vocal in the vocal track; The processed vocal track and the accompaniment track are superimposed to obtain the processed audio.

8. The method according to claim 2, characterized in that: The second time period includes a plurality of collection time periods corresponding to the recorded audios respectively; The obtaining of the processed audio segment of the second time period in the processed audio includes: Obtain processed audio segments of the collection time periods respectively corresponding to the multiple recorded audios in the processed audio; The performing mixing processing on the recorded audio and the processed audio segment of the second time period to obtain the sing-along audio segment of the second time period includes: Performing mixing processing on the multiple recorded audios and the processed audio segments of the collection time periods respectively corresponding to the multiple recorded audios, respectively, to obtain the sing-along audio segments respectively corresponding to the multiple recorded audios; The step of splicing the original audio segment of the first time period and the sing-along audio segment of the second time period to generate the sing-along audio of the first song includes: In a case where the first time period is all time periods in the complete time period of the first song except the second time period, splicing the original audio segment of the first time period and the sing-along audio segments corresponding to the plurality of recorded audios respectively to generate the sing-along audio of the first song; In the case that the first time period is a partial time period of the complete time period of the first song excluding the second time period, the native audio clips of multiple third time periods and the sing-along audio clips corresponding to the multiple recorded audios are spliced ​​together to generate the sing-along audio of the first song. The third time period is a partial time period corresponding to the recorded audio in the first time period, and the sing-along audio of the first song includes the sing-along audios corresponding to the multiple recorded audios.

9. The method according to claim 1, characterized in that: The recorded audio includes blank audio, and no audio signal of the user singing the first song is detected in the blank audio; The step of generating the sing-along audio of the first song according to the recorded audio, the processed audio and the original audio comprises: Segment the recorded audio according to the blank audio to obtain at least one recorded audio segment, wherein the recorded audio segment contains an audio signal of the user singing the first song; Obtaining processed audio segments of the collection time periods respectively corresponding to the at least one recorded audio segment in the processed audio, and obtaining original audio segments of the first time period in the original audio and original audio segments of the blank audio in the collection time period; Performing mixing processing on the at least one recorded audio segment and the processed audio segments of the collection time period respectively corresponding to the at least one recorded audio segment, respectively, to obtain the sing-along audio segments respectively corresponding to the at least one recorded audio segment; The sing-along audio segments corresponding to the at least one recorded audio segment are spliced ​​with the original audio segments of the first time period and the original audio segments of the blank audio collection period to generate the sing-along audio of the first song.

10. An audio processing device, characterized in that: The device comprises: A first playing module, used for playing a first song; an audio processing module, for obtaining processed audio of the first song when the sing-along mode is turned on for the first song, wherein the processed audio is obtained by performing audio processing on the original sound audio of the first song, and the audio processing is used to reduce the volume of human voice in the original sound audio of the first song; A second playing module is used to play the processed first song according to the processed audio, wherein the volume of the human voice in the processed first song is smaller than the volume of the human voice in the first song; An audio recording module, used to collect an audio signal of the user singing the first song through a microphone to obtain recorded audio; An audio generation module is used to generate sing-along audio for the first song based on the recorded audio, the processed audio and the original audio, wherein the sing-along audio for the first song includes an original audio segment of a first time period and a sing-along audio segment of a second time period, wherein the second time period is a time period when the sing-along mode is on, and the first time period is all or part of the complete time period of the first song except the second time period.

11. A computer device, characterized in that: The computer device comprises a processor and a memory, wherein a computer program is stored in the memory, and the computer program is loaded and executed by the processor to implement the audio processing method according to any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is loaded and executed by a processor to implement the audio processing method according to any one of claims 1 to 9.

13. A computer program product, characterized in that The computer program product comprises a computer program, which is loaded and executed by a processor to implement the audio processing method according to any one of claims 1 to 9.

Images