Song recording methods, electronic devices and computer-readable storage media

Abstract

This application discloses a song recording method, electronic device, and computer-readable storage medium. The method includes: after logging into a song application with a user account, in response to a recording command for a target song triggered in the song application, playing audio of the target song containing the original vocals; during the playback of the target song, acquiring input audio through an audio input device; extracting target dry audio from the input audio that matches the user account's voiceprint features; and synthesizing the accompaniment audio of the target song and the target dry audio to obtain the recorded audio. In the process of recording the target song, this application extracts target dry audio that matches the user's voiceprint features and the musical melody of the target song from the input audio acquired by the audio input device, eliminating the original vocals of the target song from the input audio. During song synthesis, the accompaniment audio of the target song is combined with the target dry audio, eliminating the original vocals from the final synthesized audio and improving the user experience.

Description

Technical Field

[0001] This application relates to the field of music technology, and more specifically, to a song recording method, electronic device, and computer-readable storage medium. Background Technology

[0002] In related technologies, song recording software supports playing accompaniment containing the original singer's vocals during the recording process. However, when the microphone captures the singer's vocals, it also captures the original singer's vocals. When mixing and synthesizing the song, it is impossible to remove the original singer's vocals from the accompaniment, resulting in the final synthesized audio containing the original singer's vocals.

[0003] Therefore, how to support the playback of accompaniment containing the original vocals during the song recording process while eliminating the original vocals from the final synthesized audio is a technical problem that needs to be solved by those skilled in the art. Summary of the Invention

[0004] The purpose of this application is to provide a song recording method, electronic device, storage medium, and computer program product that, during the song recording process, supports playing an accompaniment containing the original vocals while removing the original vocals from the final synthesized audio.

[0005] To achieve the above objectives, the first aspect of this application provides a song recording method, comprising:

[0006] After a user logs into the music application with their account, in response to a recording command for a target song triggered in the music application, the audio of the target song containing the original singer's vocals is played.

[0007] During the playback of the target song, input audio is acquired through an audio input device;

[0008] Extract the target dry audio that matches the voiceprint features of the user account from the input audio;

[0009] The accompaniment audio of the target song is combined with the target dry audio to obtain the recorded audio.

[0010] This also includes:

[0011] Obtain the baseline dry audio of the user account; wherein, the baseline dry audio includes a cappella dry audio and / or reading dry audio, and the a cappella dry audio includes a cappella dry audio at different pitches;

[0012] The reference dry audio is input into the audio feature extraction model to extract the voiceprint features of the user account.

[0013] Extracting target dry audio that matches the voiceprint features of the user account from the input audio includes:

[0014] Extract all dry audio from the input audio, and match the target dry audio that matches the voiceprint characteristics of the user account from all the dry audio.

[0015] Determine whether the target dry audio file matches the musical melody of the target song;

[0016] If so, proceed to the step of synthesizing the accompaniment audio of the target song and the target dry audio to obtain the recorded audio.

[0017] The step of determining whether the target dry audio conforms to the musical melody of the target song includes:

[0018] At least one audio matching method is used to determine whether the target dry audio matches the musical melody of the target song; wherein, the audio matching method includes an audio fingerprint matching method, a baseband MIDI matching method, and a lyrics matching method;

[0019] If multiple audio matching methods are used, the determination of whether the target dry audio matches the musical melody of the target song is made by comprehensively considering the judgment results of the multiple audio matching methods.

[0020] The method of determining whether the target dry audio matches the musical melody of the target song using an audio fingerprint matching method includes:

[0021] Extract the target audio fingerprint of the target dry audio;

[0022] Compare the target audio fingerprint with the standard audio fingerprint of the target song, and determine whether the similarity between the target audio fingerprint and the standard audio fingerprint of the target song is greater than or equal to a preset value;

[0023] If so, then determine whether the target dry audio matches the musical melody of the target song.

[0024] The method of determining whether the target dry audio matches the musical melody of the target song using a fundamental frequency MIDI matching method includes:

[0025] Extract the target fundamental frequency MIDI of the target dry audio;

[0026] Compare the target baseband MIDI with the standard target baseband MIDI of the target song, and determine whether the similarity between the target baseband MIDI and the standard target baseband MIDI of the target song is greater than or equal to a preset value;

[0027] If so, then determine whether the target dry audio matches the musical melody of the target song.

[0028] The method of determining whether the target dry audio matches the musical melody of the target song using lyric matching includes:

[0029] Extract the target lyrics from the target dry audio file;

[0030] Compare the target lyrics with the standard lyrics of the target song, and determine whether the similarity between the target lyrics and the standard lyrics of the target song is greater than or equal to a preset value;

[0031] If so, then determine whether the target dry audio matches the musical melody of the target song.

[0032] The step of determining whether the target dry audio matches the musical melody of the target song further includes:

[0033] If the target dry audio matches the musical melody of the target song, then denoising and de-reverberation operations are performed on the target dry audio to obtain the dry audio to be synthesized.

[0034] Accordingly, the audio obtained by synthesizing the accompaniment audio of the target song and the target dry audio includes:

[0035] The accompaniment audio of the target song and the dry audio to be synthesized are combined to obtain the recorded audio.

[0036] To achieve the above objectives, a second aspect of this application provides an electronic device, comprising:

[0037] Memory, used to store computer programs;

[0038] A processor is used to execute the computer program to implement the steps of the song recording method described above.

[0039] To achieve the above objectives, a third aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the song recording method described above.

[0040] To achieve the above objectives, a fourth aspect of this application provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the song recording method described above.

[0041] As can be seen from the above scheme, the song recording method provided in this application includes: after a user account logs into a song application, in response to a recording command for a target song triggered by the song application, playing the audio of the target song containing the original vocals; during the playback of the target song, acquiring input audio through an audio input device; extracting target dry audio that matches the voiceprint features of the user account from the input audio; and synthesizing the accompaniment audio of the target song and the target dry audio to obtain the recorded audio.

[0042] The song recording method provided in this application pre-obtains the voiceprint characteristics of a user account. During the recording process of the target song, it extracts the target dry audio that matches the voiceprint characteristics and the musical melody of the target song from the input audio collected by the audio input device, thus removing the original vocals of the target song from the input audio. During song synthesis, the accompaniment audio of the target song is combined with the target dry audio, eliminating the original vocals from the final synthesized audio and improving the user experience. This application also discloses an electronic device, a computer-readable storage medium, and a computer program product, which can achieve the same technical effects.

[0043] It should be understood that the above general description and the following detailed description are merely exemplary and do not limit this application. Attached Figure Description

[0044] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. The drawings are used to provide a further understanding of this disclosure and constitute a part of the specification. They are used together with the following detailed description to explain this disclosure, but do not constitute a limitation of this disclosure. In the drawings:

[0045] Figure 1 A flowchart of a song playback software in related technologies;

[0046] Figure 2 A flowchart of a recording software in related technologies;

[0047] Figure 3 An architecture diagram of a song recording system provided in this application embodiment;

[0048] Figure 4 A flowchart illustrating a song recording method provided in this application embodiment;

[0049] Figure 5A flowchart illustrating another song recording method provided in this application embodiment;

[0050] Figure 6 A preprocessing flowchart is provided in one application embodiment of this application.

[0051] Figure 7 A flowchart illustrating a song recording process is provided in one application embodiment of this application.

[0052] Figure 8 This is a structural diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0053] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0054] Music software in related technologies mainly falls into two categories: software focused on song playback and software focused on recording / vocalizing. The workflow of song playback software is as follows: Figure 1 As shown, the song file is downloaded from the server or CDK (Content Delivery Network) acceleration node to the music player software client. The client then plays the complete music locally, including both the original vocals and instrumental tracks. The recording software process is as follows... Figure 2 As shown, the backing track file is downloaded from the server or CDK acceleration node to the recording software client. The client plays the backing track locally, guiding the user to sing along, and simultaneously captures the user's vocals from the microphone, combining them with the backing track to create a vocal blend. Although the recording software also supports playing the original vocals during recording, the microphone captures both the user's vocals and the original vocals. Therefore, during the song mixing process, the original vocals in the backing track cannot be removed, resulting in the final synthesized audio containing the original vocals.

[0055] Therefore, in this application, the voiceprint characteristics of the user account are obtained in advance. During the recording of the target song, the target dry audio that matches the voiceprint characteristics and the musical melody of the target song is extracted from the input audio collected by the audio input device. The original vocals of the target song in the input audio are removed. During the song synthesis, the accompaniment audio of the target song is synthesized with the target dry audio, thereby eliminating the original vocals in the final synthesized audio and improving the user experience.

[0056] To facilitate understanding of the song recording method provided in this application, the system used is described below. See [link to relevant documentation]. Figure 3 It shows an architecture diagram of a song recording system provided in an embodiment of this application, such as Figure 3 As shown, it includes server 10 and terminal 20.

[0057] Terminal 20 can be a fixed terminal, mobile terminal, IoT device, or wearable device. Fixed terminals include PCs (Personal Computers), mobile terminals include mobile phones, IoT devices include smart speakers, smart TVs, smart air conditioners, and smart car devices, and wearable devices include smartwatches, smart bracelets, and head-mounted devices. Terminal 20 includes an audio input device and an audio output device. The audio input device is used to collect the reference dry audio of the user account, which may include a cappella dry audio or reading dry audio. The reference dry audio is used to generate the voiceprint features of the user account. Terminal 20 can generate the voiceprint features of the user account after collecting the reference dry audio, or it can send the voiceprint features of the user account to server 10, where server 10 generates voiceprint features based on the reference dry audio of the user account. The audio output device is used to play the audio of the target song containing the original vocals when a recording command for the target song is received. This audio is downloaded and stored locally by terminal 20 from server 10. The audio input device is also used to collect input audio during the recording process.

[0058] Furthermore, terminal 20 or server 10 is used to extract the target dry audio corresponding to the target song from the input audio based on the user account's voiceprint features, and synthesize the accompaniment audio of the target song and the target dry audio to obtain the recorded audio. Server 10 is also used to store the recorded audio, which can be output through the audio output device of terminal 20.

[0059] This application discloses a song recording method that, during the song recording process, supports playing an accompaniment containing the original vocals while removing the original vocals from the final synthesized audio.

[0060] See Figure 4 The flowchart of a song recording method provided in this application embodiment is as follows: Figure 4 As shown, it includes:

[0061] S101: After logging into the song application with a user account, in response to the recording command of the target song triggered by the song application, play the audio of the target song containing the original singer's voice;

[0062] In practice, users log in to the music application with their user account, select the target song they want to record in the music application, trigger the recording command for the target song, and start playing the audio of the target song containing the original singer's vocals through the audio output device, so that the user can hear the original singer of the target song and guide the user to record the target song.

[0063] S102: During the playback of the target song, input audio is acquired through the audio input device;

[0064] In practice, while playing the target song, the user records their singing through an audio input device. This audio input device is a device used to capture sound signals and convert them into electrical signals, such as a microphone. It can be a built-in microphone in the electronic device running the song application, or an external microphone or wireless microphone connected to that electronic device. The input audio captured by the audio input device includes not only the user's singing but also the original singer's voice.

[0065] S103: Extract the target dry audio that matches the voiceprint features of the user account from the input audio;

[0066] In practice, since the input audio collected by the audio input device includes not only the user's singing voice but also the original singer's voice, in this step, the target dry audio that matches the user's voiceprint characteristics and the musical melody of the target song is extracted from the input audio, and the original singer's voice of the target song is removed from the input audio.

[0067] As a feasible implementation method, this embodiment further includes: acquiring the voiceprint features of the user account. In a specific implementation, the user's voiceprint features are acquired in advance. Voiceprint features are used to describe the unique attributes of an individual's voice, including pitch, timbre, rhythm, etc. By analyzing the voiceprint features, a digital model representing the user's voice can be created for identifying and processing the user's voice.

[0068] As a feasible implementation, obtaining the voiceprint features of the user account includes: obtaining a baseline dry audio file of the user account; wherein the baseline dry audio file includes a cappella dry audio files and / or reading dry audio files, and the a cappella dry audio files include a cappella dry audio files at different pitches; inputting the baseline dry audio file into an audio feature extraction model to extract the voiceprint features of the user account. The audio feature extraction model can be of various types, including traditional machine learning models and deep learning models, such as the Wav2Vec model, which can be obtained through pre-training. The main function of the audio feature extraction model is to extract voiceprint features from the baseline dry audio file that reflect the essential characteristics of the sound and distinguish it from other dry audio files, which may include the frequency, energy, pitch, rhythm, etc. of the audio. The process of audio feature extraction models to extract voiceprint features typically includes the following steps: preprocessing, including normalization, pre-emphasis, windowing, and framing, to eliminate the magnitude differences between different sample sounds and extract their short-term characteristics; extracting sound features, such as MFCC (Mel-Frequency Cepstral Coefficients) features and LPC (Linear Predictive Coding) features; and using deep learning models, such as Wav2Vec, to directly extract features from the original audio. These features contain complex information about the sound and are suitable for tasks such as voiceprint recognition.

[0069] In practice, after the song recording software starts, it prompts the user whether to enable the automatic song synthesis function. If enabled, the user is prompted to input the reference dry audio, which can include the dry audio of a cappella singing, dry audio covering different pitches of high, middle and low, and dry audio of text reading. The reference dry audio is then input into the audio feature extraction model to extract the user's voiceprint features.

[0070] As a feasible implementation method, extracting target dry audio that matches the voiceprint features of the user account from the input audio includes: extracting all dry audio from the input audio, matching the target dry audio that matches the voiceprint features of the user account among all dry audio; determining whether the target dry audio matches the musical melody of the target song; if so, proceeding to the step of synthesizing the accompaniment audio of the target song and the target dry audio to obtain the recorded audio.

[0071] In practice, stereo echo cancellation or voice separation technology is used to extract all dry audio from the input audio. Then, all dry audio is matched against the user's voiceprint features. The similarity between the extracted dry audio and the user's voiceprint features is calculated. Dry audio with a similarity greater than a preset value is selected as candidate dry audio. The candidate dry audio with the highest similarity is then selected as the target dry audio, which represents the user's voice. The system then determines whether the target dry audio matches the melody and rhythm of the target song. If they match, it means the target dry audio is the user singing the target song; otherwise, it means the user is not singing the target song.

[0072] As a feasible implementation method, determining whether the target dry audio conforms to the musical melody of the target song includes: using at least one audio matching method to determine whether the target dry audio conforms to the musical melody of the target song; wherein, the audio matching method includes an audio fingerprint matching method, a fundamental frequency MIDI matching method, and a lyrics matching method; if multiple audio matching methods are used, the determination of whether the target dry audio conforms to the musical melody of the target song is made comprehensively based on the determination results of the multiple audio matching methods.

[0073] In practice, one or more audio matching methods are used to determine whether the target dry audio matches the melody of the target song. These methods may include audio fingerprint matching, MIDI (Musical Instrument Digital Interface) matching, and lyrics matching. If multiple methods are used, the results from all methods are considered to determine whether the target dry audio matches the melody. For example, if an odd number of methods are used, a majority voting mechanism can be employed. If a majority vote determines that the target dry audio matches the melody, then it is considered to match the melody. Alternatively, the results of each matching method can be digitally represented: 1 for a match and 0 for a non-match. Each method is then assigned a weight, and the results are weighted and averaged. If the weighted average exceeds a pre-set threshold, the target dry audio is considered to match the melody; otherwise, it is considered not to match.

[0074] As a feasible implementation method, an audio fingerprint matching method is used to determine whether the target dry audio conforms to the musical melody of the target song, including: extracting the target audio fingerprint of the target dry audio; comparing the target audio fingerprint with the standard audio fingerprint of the target song, and determining whether the similarity between the target audio fingerprint and the standard audio fingerprint of the target song is greater than or equal to a preset value; if so, determining whether the target dry audio conforms to the musical melody of the target song.

[0075] In practice, the first step is to extract the target audio fingerprint from the target dry audio file. An audio fingerprint is a digital identifier representing the unique characteristics of an audio file, generated by analyzing its features, and used to identify and compare audio content. Then, the target audio fingerprint is compared with the pre-stored standard audio fingerprint of the target song. By calculating the similarity between the two, it can be determined whether the target dry audio file matches the melody of the target song. If the similarity is greater than or equal to a preset value, the target dry audio file is determined to match the musical melody of the target song; otherwise, it is determined that the target dry audio file does not match the musical melody of the target song.

[0076] As a feasible implementation method, a fundamental frequency MIDI matching method is used to determine whether the target dry audio audio conforms to the musical melody of the target song, including: extracting the target fundamental frequency MIDI of the target dry audio audio; comparing the target fundamental frequency MIDI with the standard target fundamental frequency MIDI of the target song, and determining whether the similarity between the target fundamental frequency MIDI and the standard target fundamental frequency MIDI of the target song is greater than or equal to a preset value; if so, determining whether the target dry audio audio conforms to the musical melody of the target song.

[0077] In practice, the target dry audio is first analyzed to extract its fundamental frequency information and converted into MIDI data, i.e., the target fundamental frequency MIDI, which represents the pitch and rhythm of the target dry audio. Then, the target fundamental frequency MIDI is compared with the pre-stored standard fundamental frequency MIDI of the target song. By calculating the similarity between the two, it can be determined whether the target dry audio matches the melody of the target song. If the similarity is greater than or equal to a preset value, the target dry audio is determined to match the musical melody of the target song; otherwise, the target dry audio is determined not to match the musical melody of the target song.

[0078] As a feasible implementation method, a lyrics matching method is used to determine whether the target dry audio conforms to the musical melody of the target song, including: extracting the target lyrics of the target dry audio; comparing the target lyrics with the standard lyrics of the target song, and determining whether the similarity between the target lyrics and the standard lyrics of the target song is greater than or equal to a preset value; if so, determining whether the target dry audio conforms to the musical melody of the target song.

[0079] In practice, the target lyrics are first extracted from the target dry audio file. Then, the target lyrics are compared with the pre-stored standard lyrics of the target song. The similarity between the two is calculated to determine if the target dry audio file matches the melody of the target song. If the similarity is greater than or equal to a preset value, the target dry audio file is determined to match the musical melody of the target song; otherwise, it is determined that the target dry audio file does not match the musical melody of the target song.

[0080] In a preferred embodiment, after determining whether the target dry audio conforms to the musical melody of the target song, the method further includes: if the target dry audio conforms to the musical melody of the target song, then performing denoising and dereverberation operations on the target dry audio to obtain the dry audio to be synthesized. In a specific implementation, denoising and dereverberation operations are performed on the target dry audio to filter background noise and other noise, resulting in clean dry audio, i.e., the dry audio to be synthesized.

[0081] S104: Synthesize the accompaniment audio of the target song with the target dry audio to obtain the recorded audio.

[0082] In this step, the extracted user-recorded target dry audio is combined with the accompaniment audio of the target song to complete the recording of the target song, resulting in the recorded audio. If denoising and dereverb operations are performed on the target dry audio to obtain the dry audio to be synthesized, then in this step, the accompaniment audio of the target song and the dry audio to be synthesized are combined to obtain the recorded audio. After recording is complete, the user can be prompted to choose whether to save and publish the recorded audio, or it can be automatically saved and published according to the user's preset settings.

[0083] The song recording method provided in this application pre-obtains the voiceprint characteristics of the user account. During the recording of the target song, the target dry audio that matches the voiceprint characteristics and the musical melody of the target song is extracted from the input audio collected by the audio input device. The original vocals of the target song in the input audio are removed. During song synthesis, the accompaniment audio of the target song is synthesized with the target dry audio, thereby eliminating the original vocals in the final synthesized audio and improving the user experience.

[0084] This application discloses a song recording method. Compared with the above embodiments, this embodiment further explains and optimizes the technical solution. Specifically:

[0085] See Figure 5 The flowchart of another song recording method provided in this application embodiment is as follows: Figure 5 As shown, it includes:

[0086] S201: Obtain the baseline dry audio of the user account; wherein, the baseline dry audio includes a cappella dry audio and / or reading dry audio, and the a cappella dry audio includes a cappella dry audio at different pitches;

[0087] S202: Input the reference dry audio into the audio feature extraction model to extract the voiceprint features of the user account;

[0088] In practice, after the song recording software is started, it acquires the user's input baseline dry audio, which may include dry audio during a cappella singing, dry audio covering different pitches, and dry audio of text reading. The baseline dry audio is then input into the audio feature extraction model to extract the user's voiceprint features.

[0089] S203: After logging into the song application with a user account, in response to a recording command for a target song triggered by the song application, play the audio of the target song containing the original singer's vocals;

[0090] In practice, after the user selects the target song to be recorded, the audio output device starts playing the audio of the target song, which includes the original vocals. It supports playing the target music through external speakers, external audio equipment, etc., so that the user can hear the original vocals of the target song and guide the user to record the target song.

[0091] S204: During the playback of the target song, input audio is acquired through the audio input device;

[0092] S205: Extract all dry audio from the input audio, and match the target dry audio that conforms to the voiceprint features among all dry audio;

[0093] In practice, stereo echo cancellation technology or human voice separation technology is used to extract all dry audio from the input audio. Then, all dry audio is matched with the user's voiceprint features, and the dry audio with the highest matching degree is selected as the target dry audio.

[0094] S206: Extract the target audio fingerprint of the target dry audio, compare the target audio fingerprint with the standard audio fingerprint of the target song, and determine whether the similarity between the target audio fingerprint and the standard audio fingerprint of the target song is greater than or equal to a preset value; if so, proceed to S207.

[0095] In practice, the first step is to extract the target audio fingerprint from the target dry audio file. An audio fingerprint is a digital identifier representing the unique characteristics of an audio file, generated by analyzing its features, and used to identify and compare audio content. Then, the target audio fingerprint is compared with the pre-stored standard audio fingerprint of the target song. By calculating the similarity between the two, it can be determined whether the target dry audio file matches the melody of the target song. If the similarity is greater than or equal to a preset value, the target dry audio file is determined to match the musical melody of the target song; otherwise, it is determined that the target dry audio file does not match the musical melody of the target song.

[0096] S207: Perform denoising and dereverberation operations on the target dry audio to obtain the dry audio to be synthesized;

[0097] In practice, denoising and de-reverberation operations are performed on the target dry audio to filter out background noise and other noise, resulting in clean dry audio, which is the dry audio to be synthesized.

[0098] S208: The accompaniment audio of the target song and the dry audio to be synthesized are combined to obtain the recorded audio.

[0099] Therefore, this embodiment can support playing target music through external speakers, external audio devices, etc. During music playback, it automatically monitors and identifies the target dry audio that matches the melody of the target music in the user's account, and performs noise reduction and de-reverberation operations on it to improve the quality of the target dry audio. The target dry audio and the accompaniment audio are then combined to create a musical work. The synthesized musical work does not include the original vocals and has a better user experience.

[0100] The following describes an application embodiment provided by this application, with the preprocessing flow as follows: Figure 6 As shown, the process includes the following steps: application startup, user registration; determining whether automatic recording and synthesis are enabled; prompting the user to input dry audio, recording the audio input; using dry audio pre-training to build a user voice model; and ending pre-processing.

[0101] The song recording process is as follows Figure 7 As shown, it includes the following steps:

[0102] Step 1: The application launches and plays music;

[0103] Step 2: Monitor the audio to determine if there is any dry audio input; if so, proceed to Step 3.

[0104] Step 3: Detect the dry audio and determine if it matches the pre-trained audio model; if so, proceed to Step 4.

[0105] Step 4: Audio fingerprint detection to determine if it is similar to the music being played; if so, proceed to Step 5.

[0106] Step 5: Eliminate background noise using a pre-trained model, retaining only the target user's clean audio.

[0107] Step 6: Extract the instrumental track without vocals and combine it with the user's dry vocals;

[0108] Step 7: Prompt the user to choose whether to save and publish the composite work;

[0109] Step 8: Save, publish, or cancel to finish.

[0110] The following describes a song recording apparatus provided in an embodiment of this application. The song recording apparatus described below can be referred to in conjunction with the song recording method described above. The song recording apparatus includes:

[0111] The playback module is used to play the audio of the target song containing the original vocals in response to a recording command of the target song triggered by the song application after the user's account logs in to the song application.

[0112] The first acquisition module is used to acquire input audio through an audio input device during the playback of the target song;

[0113] An extraction module is used to extract target dry audio that matches the voiceprint features of the user account from the input audio;

[0114] The synthesis module is used to synthesize the accompaniment audio of the target song and the target dry audio to obtain the recorded audio.

[0115] The song recording device provided in this application embodiment obtains the voiceprint characteristics of the user account in advance. During the recording of the target song, the target dry audio that matches the voiceprint characteristics and the musical melody of the target song is extracted from the input audio collected by the audio input device. The original vocals of the target song in the input audio are removed. During the song synthesis, the accompaniment audio of the target song is synthesized with the target dry audio, thereby eliminating the original vocals in the final synthesized audio and improving the user experience.

[0116] Based on the above embodiments, as a preferred embodiment, it further includes:

[0117] The second acquisition module is used to acquire the voiceprint features of the user account.

[0118] Based on the above embodiments, as a preferred implementation, the second acquisition module is specifically used to: acquire the baseline dry audio of the user account; wherein, the baseline dry audio includes a cappella dry audio and / or reading dry audio, and the a cappella dry audio includes a cappella dry audio at different pitches; input the baseline dry audio into an audio feature extraction model to extract the voiceprint features of the user account.

[0119] Based on the above embodiments, as a preferred implementation, the extraction module is specifically used for: extracting all dry audio from the input audio, matching a target dry audio that matches the voiceprint characteristics of the user account among all the dry audio; determining whether the target dry audio matches the musical melody of the target song; if so, proceeding to the step of synthesizing the accompaniment audio of the target song and the target dry audio to obtain the recorded audio.

[0120] Based on the above embodiments, as a preferred implementation, the extraction module is specifically used to: determine whether the target dry audio conforms to the musical melody of the target song using at least one audio matching method; wherein, the audio matching method includes an audio fingerprint matching method, a baseband MIDI matching method, and a lyrics matching method; if multiple audio matching methods are used, the determination of whether the target dry audio conforms to the musical melody of the target song is made comprehensively based on the determination results of the multiple audio matching methods.

[0121] Based on the above embodiments, as a preferred embodiment, the extraction module is specifically used to: extract the target audio fingerprint of the target dry audio; compare the target audio fingerprint with the standard audio fingerprint of the target song, and determine whether the similarity between the target audio fingerprint and the standard audio fingerprint of the target song is greater than or equal to a preset value; if so, determine whether the target dry audio conforms to the musical melody of the target song.

[0122] Based on the above embodiments, as a preferred embodiment, the extraction module is specifically used to: extract the target fundamental frequency MIDI of the target dry audio; compare the target fundamental frequency MIDI with the standard target fundamental frequency MIDI of the target song, and determine whether the similarity between the target fundamental frequency MIDI and the standard target fundamental frequency MIDI of the target song is greater than or equal to a preset value; if so, determine whether the target dry audio conforms to the musical melody of the target song.

[0123] Based on the above embodiments, as a preferred embodiment, the extraction module is specifically used to: extract the target lyrics of the target dry audio; compare the target lyrics with the standard lyrics of the target song, and determine whether the similarity between the target lyrics and the standard lyrics of the target song is greater than or equal to a preset value; if so, determine whether the target dry audio conforms to the musical melody of the target song.

[0124] Based on the above embodiments, as a preferred embodiment, it further includes:

[0125] The operation module is used to perform noise reduction and de-reverberation operations on the target dry audio to obtain the dry audio to be synthesized;

[0126] Accordingly, the synthesis module is specifically used to: synthesize the accompaniment audio of the target song and the dry audio to be synthesized to obtain the recorded audio.

[0127] This application also provides an electronic device, see [link to document]. Figure 8 The present application provides a structural diagram of an electronic device, such as... Figure 8 As shown, it may include a processor 11 and a memory 12.

[0128] The processor 11 may include one or more processing cores, such as a quad-core processor or an octa-core processor. The processor 11 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 11 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, the processor 11 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, the processor 11 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0129] The memory 12 may include one or more computer-readable storage media, which may be non-transitory. The memory 12 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In this embodiment, the memory 12 is used to store at least the following computer program 121, which, after being loaded and executed by the processor 11, is capable of implementing the relevant steps in the song recording method executed by the electronic device side as disclosed in any of the foregoing embodiments. In addition, the resources stored in the memory 12 may also include an operating system 122 and data 123, etc., and the storage method may be temporary storage or permanent storage. The operating system 122 may include Windows, Unix, Linux, etc.

[0130] In some embodiments, the electronic device may further include a display screen 13, an input / output interface 14, a communication interface 15, a sensor 16, a power supply 17, and a communication bus 18.

[0131] certainly, Figure 8 The structure of the electronic device shown does not constitute a limitation on the electronic device in the embodiments of this application. In practical applications, the electronic device may include more than [other components]. Figure 8 More or fewer components as shown, or combinations of certain components.

[0132] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided, which, when executed by a processor, implement the steps of the song recording method performed by the electronic device of any of the above embodiments.

[0133] In another exemplary embodiment, a computer program product including a computer program is also provided, which, when executed by a processor, implements the steps of the song recording method performed by the electronic device of any of the above embodiments.

[0134] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from the principles of this application, and these improvements and modifications also fall within the protection scope of the claims of this application.

[0135] It should also be noted that, in this specification, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

Claims

1. A song recording method, characterized in that, include: After a user logs into the music application with their account, in response to a recording command for a target song triggered in the music application, the audio of the target song containing the original singer's vocals is played. During the playback of the target song, input audio is acquired through an audio input device; Extract the target dry audio that matches the voiceprint features of the user account from the input audio; The accompaniment audio of the target song is combined with the target dry audio to obtain the recorded audio. Extracting target dry audio that matches the voiceprint features of the user account from the input audio includes: Extract all dry audio from the input audio, and match the target dry audio that matches the voiceprint characteristics of the user account from all the dry audio. Determine whether the target dry audio file matches the musical melody of the target song; If so, proceed to the step of synthesizing the accompaniment audio of the target song and the target dry audio to obtain the recorded audio.

2. The song recording method according to claim 1, characterized in that, Also includes: Obtain the baseline dry audio of the user account; wherein, the baseline dry audio includes a cappella dry audio and / or reading dry audio, and the a cappella dry audio includes a cappella dry audio at different pitches; The reference dry audio is input into the audio feature extraction model to extract the voiceprint features of the user account.

3. The song recording method according to claim 1, characterized in that, The step of determining whether the target dry audio conforms to the musical melody of the target song includes: At least one audio matching method is used to determine whether the target dry audio matches the musical melody of the target song; wherein, the audio matching method includes an audio fingerprint matching method, a baseband MIDI matching method, and a lyrics matching method; If multiple audio matching methods are used, the determination of whether the target dry audio matches the musical melody of the target song is made by comprehensively considering the judgment results of the multiple audio matching methods.

4. The song recording method according to claim 3, characterized in that, Determining whether the target dry audio matches the musical melody of the target song using an audio fingerprint matching method includes: Extract the target audio fingerprint of the target dry audio; Compare the target audio fingerprint with the standard audio fingerprint of the target song, and determine whether the similarity between the target audio fingerprint and the standard audio fingerprint of the target song is greater than or equal to a preset value; If so, then determine whether the target dry audio matches the musical melody of the target song.

5. The song recording method according to claim 3, characterized in that, Determining whether the target dry audio matches the musical melody of the target song using a fundamental frequency MIDI matching method includes: Extract the target fundamental frequency MIDI of the target dry audio; Compare the target baseband MIDI with the standard target baseband MIDI of the target song, and determine whether the similarity between the target baseband MIDI and the standard target baseband MIDI of the target song is greater than or equal to a preset value; If so, then determine whether the target dry audio matches the musical melody of the target song.

6. The song recording method according to claim 3, characterized in that, Determining whether the target dry audio matches the musical melody of the target song using a lyrics matching method includes: Extract the target lyrics from the target dry audio file; Compare the target lyrics with the standard lyrics of the target song, and determine whether the similarity between the target lyrics and the standard lyrics of the target song is greater than or equal to a preset value; If so, then determine whether the target dry audio matches the musical melody of the target song.

7. The song recording method according to claim 1, characterized in that, After determining whether the target dry audio matches the musical melody of the target song, the method further includes: If the target dry audio matches the musical melody of the target song, then denoising and de-reverberation operations are performed on the target dry audio to obtain the dry audio to be synthesized. Accordingly, the audio obtained by synthesizing the accompaniment audio of the target song and the target dry audio includes: The accompaniment audio of the target song and the dry audio to be synthesized are combined to obtain the recorded audio.

8. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of the song recording method as described in any one of claims 1 to 7 when executing the computer program.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed, implements the steps of the song recording method as described in any one of claims 1 to 7.

Images