Singing voice extraction method and device, electronic equipment and medium

Abstract

The invention relates to the technical field of audio processing, and discloses a singing voice extraction method and device, electronic equipment and a medium, and the method comprises the steps: selecting a harmony signal matched with a singing voice signal of each song from a harmony library corresponding to each song; and mixing the main singing voice signal, the harmony signal and the accompaniment corresponding to each song to generate a song signal. And training the music separation model by using the set number of song signals and the matched singing human voice lyric texts to obtain a trained music separation model. And analyzing the target song signal and the matched target singing voice lyric text by using the trained music separation model, and extracting the target singing voice signal. The training data including the voices, harmony and accompaniment of the main singing are constructed, and the distinguishing of the voices and harmony of the main singing by the music separation model is realized. The singing voice lyric text participates in the training of the music separation model, and the singing voice can be effectively extracted from the song signal.

Description

Technical Field

[0001] The present application relates to the technical field of audio processing, and in particular to a method, device, electronic device and medium for extracting a lead singer's voice. Background Art

[0002] Currently, commonly used music separation methods include frequency domain-based methods and time domain-based methods. The frequency domain-based method converts the music signal into the frequency domain, separates it into frequency domain multi-track signals, and then converts it into time domain multi-track signals. The time domain-based method directly separates the time domain music signal into time domain multi-track signals without time-frequency domain conversion. The above-mentioned music separation methods are usually used to separate vocals and accompaniment, or vocals and musical instruments. Their characteristics are that there are significant differences in timbre and listening experience between vocals and accompaniment, or between vocals and musical instruments.

[0003] However, the harmony vocals in most songs are pronounced vocals and have no significant difference from the lead vocals. Therefore, conventional music separation algorithms cannot effectively extract the lead vocals from songs.

[0004] In summary, in the process of implementing this application, the inventor discovered that there is at least a problem in the prior art that the lead singer's vocals cannot be effectively extracted from a song. Summary of the invention

[0005] In view of this, the purpose of this application is to provide a method, device, electronic device and medium for extracting the lead singer's voice, which can effectively extract the lead singer's voice signal from the song. The specific scheme is as follows:

[0006] In a first aspect, the present application discloses a method for extracting a lead singer's voice, comprising:

[0007] Selecting a harmony signal matching the lead vocal signal of each song from a harmony library corresponding to each song;

[0008] The lead vocal signal, harmony signal and accompaniment corresponding to each song are mixed to generate a song signal;

[0009] The music separation model is trained using a set number of song signals and their matching lead singer lyrics texts to obtain a trained music separation model;

[0010] When the target song signal is obtained, the trained music separation model is used to analyze the target song signal and its matching target lead singer's vocal lyrics text to extract the target lead singer's vocal signal.

[0011] Optionally, the harmony library includes a synchronous harmony signal and an interlaced harmony signal; and with respect to the construction process of the harmony library, the method includes:

[0012] Performing pitch-shifting processing on the lead singer's vocal signal of the same song to obtain a synchronous harmony signal of the same song segment as the lead singer's vocal;

[0013] For the same song, a vocal signal is randomly selected from other vocal signals different from the lead singer's vocals for simulation, so as to obtain an interlaced harmony signal of a different song section from the lead singer's vocals.

[0014] Optionally, the lead vocal signal, harmony signal and accompaniment corresponding to each song are mixed to generate a song signal including:

[0015] Superimposing the lead vocal signal and harmony signal corresponding to each song according to a set first signal-to-noise ratio range to obtain the vocal signal corresponding to each song;

[0016] The human voice signal corresponding to each song and its matching accompaniment are superimposed according to a set second signal-to-noise ratio range to obtain a song signal corresponding to each song.

[0017] Optionally, the lead vocal signal, harmony signal and accompaniment corresponding to each song are mixed to generate a song signal including:

[0018] The lead vocal signal, harmony signal and accompaniment corresponding to each song are mixed according to the set signal-to-noise ratio range to obtain the song signal corresponding to each song.

[0019] Optionally, the music separation model is trained using a set number of song signals and their matching lead singer lyrics texts to obtain a trained music separation model, including:

[0020] Encoding the song signal by using an audio editor to obtain the song audio features;

[0021] Use a text editor to encode the lead singer's lyrics text to obtain text features;

[0022] The song audio features and the matching text features are combined and input into a music separation model to output a lead singer vocal estimation signal;

[0023] Based on the lead singer vocal estimation signal and the lead singer vocal signal, the model parameters of the music separation model are updated to obtain a trained music separation model.

[0024] Optionally, using a trained music separation model to analyze the target song signal and its matching target lead singer's vocal lyrics text to extract the target lead singer's vocal signal includes:

[0025] Encoding the target song signal using the audio editor in the trained music separation model to obtain the target song audio features;

[0026] Encode the target lead singer's vocal lyrics text using the text editor in the trained music separation model to obtain target text features;

[0027] The target song audio features and the target text features are combined and input into a trained music separation model for analysis to output a target lead singer vocal signal.

[0028] Optionally, after analyzing the target song signal and the target lead singer's vocal lyrics text matched thereto by using the trained music separation model to extract the target lead singer's vocal signal, the method further includes:

[0029] Receive an actual lead singer vocal signal input by a user;

[0030] When the similarity between the target lead singer vocal signal and the actual lead singer vocal signal is less than a preset threshold, the model parameters of the trained music separation model are adjusted using the target lead singer vocal signal and the actual lead singer vocal signal.

[0031] In a second aspect, the present application discloses a device for extracting a lead singer's vocals, comprising a selection unit, a mixing unit, a training unit, and an analysis unit;

[0032] The selection unit is used to select a harmony signal matching the lead singer's vocal signal of each song from the harmony library corresponding to each song;

[0033] The mixing unit is used to mix the lead vocal signal, harmony signal and accompaniment corresponding to each song to generate a song signal;

[0034] The training unit is used to train the music separation model using a set number of song signals and their matching lead singer lyrics texts to obtain a trained music separation model;

[0035] The analysis unit is used to analyze the target song signal and its matching target lead singer's vocal lyrics text using a trained music separation model when a target song signal is acquired, so as to extract the target lead singer's vocal signal.

[0036] Optionally, the harmony library includes a synchronous harmony signal and an interlaced harmony signal; with respect to the construction process of the harmony library, the device includes a pitch shifting unit and a simulation unit;

[0037] The pitch-shifting unit is used to perform pitch-shifting processing on the vocal signal of the lead singer of the same song, so as to obtain a synchronous harmony signal of the same song segment sung by the lead singer;

[0038] The simulation unit is used to randomly select a vocal signal from other vocal signals different from the lead singer's vocals for simulation for the same song, so as to obtain an interlaced harmony signal of a song section different from that sung by the lead singer's vocals.

[0039] Optionally, the mixing unit includes a first superposition subunit and a second superposition subunit;

[0040] The first superposition subunit is used to superimpose the lead vocal signal and the harmony signal corresponding to each song according to a set first signal-to-noise ratio range to obtain the vocal signal corresponding to each song;

[0041] The second superposition subunit is used to superimpose the human voice signal corresponding to each song and its matching accompaniment according to a set second signal-to-noise ratio range to obtain the song signal corresponding to each song.

[0042] Optionally, the mixing unit is used to mix the lead singer vocal signal, harmony signal and accompaniment corresponding to each song according to a set signal-to-noise ratio range to obtain a song signal corresponding to each song.

[0043] Optionally, the training unit includes an audio encoding subunit, a text encoding subunit, a merging subunit and an updating subunit;

[0044] The audio encoding subunit is used to encode the song signal using an audio editor to obtain the song audio features;

[0045] The text encoding subunit is used to encode the lead singer's vocal lyrics text using a text editor to obtain text features;

[0046] The merging subunit is used to merge the song audio features and the matching text features and input them into the music separation model to output a lead singer vocal estimation signal;

[0047] The updating subunit is used to update the model parameters of the music separation model based on the lead singer vocal estimation signal and the lead singer vocal signal to obtain a trained music separation model.

[0048] Optionally, the analysis unit is used to encode the target song signal using the audio editor in the trained music separation model to obtain the target song audio features; encode the target lead singer vocal lyrics text using the text editor in the trained music separation model to obtain the target text features; merge the target song audio features and the target text features and input them into the trained music separation model for analysis to output the target lead singer vocal signal.

[0049] Optionally, it also includes a receiving unit and an adjusting unit;

[0050] The receiving unit is used to receive an actual lead singer vocal signal input by a user;

[0051] The adjustment unit is used to adjust the model parameters of the trained music separation model using the target lead singer vocal signal and the actual lead singer vocal signal when the similarity between the target lead singer vocal signal and the actual lead singer vocal signal is less than a preset threshold.

[0052] In a third aspect, the present application discloses an electronic device, comprising:

[0053] Memory, used to store computer programs;

[0054] The processor is used to execute the computer program to implement the aforementioned method for extracting the lead singer's voice.

[0055] In a fourth aspect, the present application discloses a computer-readable storage medium for storing a computer program, which, when executed by a processor, implements the aforementioned method for extracting lead singer's vocals.

[0056] In a fifth aspect, the present application discloses a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the aforementioned method for extracting lead singer's voice.

[0057] In the present application, a harmony signal that matches the lead vocal signal of each song is selected from the harmony library corresponding to each song; the lead vocal signal, harmony signal and accompaniment corresponding to each song are mixed to generate a song signal. Considering that the lead vocal and harmony are easily confused, but the correspondence between the lead vocal and the lyrics is closer than that between the harmony, the music separation model can be trained using a set number of song signals and their matching lead vocal lyrics texts to obtain a trained music separation model. When the target song signal is obtained, the target song signal and its matching target lead vocal lyrics text are analyzed using the trained music separation model to extract the target lead vocal signal. In this technical solution, training data containing lead vocals, harmony and accompaniment is effectively constructed, which can realize the distinction between the lead vocal and harmony by the music separation model. The lead singer's lyrics text is also used to participate in the training of the music separation model, which effectively reduces the probability of confusion when the music separation model identifies the lead singer and the harmony, so that the trained music separation model can effectively extract the lead singer's vocals from the song signal. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0059] Figure 1 A physical framework applicable to a method for extracting lead singer's voice provided in this application;

[0060] Figure 2 A flow chart of a method for extracting lead singer's voice provided in this application;

[0061] Figure 3 A schematic diagram of a training sample generation process provided in this application;

[0062] Figure 4 A schematic diagram of a training process of a music separation model provided in this application;

[0063] Figure 5 A schematic diagram of the structure of a lead singer's vocal extraction device provided in this application;

[0064] Figure 6 A structural diagram of an electronic device provided for this application. DETAILED DESCRIPTION

[0065] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0066] Figure 1 A physical framework applicable to the method for extracting the vocals of the lead singer of the present application is shown. The physical framework includes: a terminal and a server. The terminal runs a platform that supports running audio, and the platform can be any one of an audio player program or applet (a program that relies on a host program to run), an audio player web page, a video player program or applet, and a video player web page. The terminal is connected to the server via a wireless network or a wired network. The terminal generally refers to one or more terminals. For example, the above-mentioned terminal may be only one, or the above-mentioned terminal may be more than a dozen or hundreds, or more. The embodiment of the present application is only described by taking the terminal as an example. The types of terminals include: at least one of a smart phone, a tablet computer, an e-book reader, an MP3 player, an MP4 player, a laptop portable computer, and a desktop computer. The embodiment of the present application does not limit the number and type of terminals.

[0067] Figure 2 This is a flow chart of a method for extracting the lead singer's voice provided in an embodiment of the present application. Figure 2 As shown, the method for extracting the lead singer's voice can be executed by a server or a terminal, and includes the following steps:

[0068] S201: Selecting a harmony signal that matches the lead singer's vocal signal of each song from a harmony library corresponding to each song.

[0069] Lead singer vocal extraction refers to extracting the lead singer vocal signal from the song signal, that is, filtering out the accompaniment signal, harmony signal, etc. in the song. In an embodiment of the present application, a music separation model can be used to extract the lead singer vocal signal. Among them, the music separation model can select a sound source separation model (demucs) or a frequency domain-based separation model (Band-Split Recurrent Neural Network, BSRNN), etc., which is not limited here.

[0070] In order to enable the music separation model to better distinguish between the lead vocals and the harmony, and thus effectively extract the lead vocals, the harmony can be added to the training samples used in the model training stage.

[0071] Currently, training samples containing lead vocals and harmony are relatively rare, so in the embodiment of the present application, a harmony library can be constructed. A harmony library corresponding to each song can be constructed. The construction method of the harmony library corresponding to each song is similar. Next, a song will be taken as an example to introduce the construction of the harmony library of the song. There can be two types of harmony signals contained in the harmony library, one is a synchronous harmony signal, and the other is an interlaced harmony signal.

[0072] For the construction of synchronous harmony signal, the vocal signal of the lead singer of the same song can be subjected to pitch-shifting processing to obtain the synchronous harmony signal of the same song segment as the lead singer.

[0073] The pitch shifting processing of vocal signals is a relatively mature technology. In practical applications, the existing pitch shifter can be used to perform pitch shifting processing on the lead singer's vocal signal to obtain a synchronous harmony signal.

[0074] For the construction of the interlaced harmony signal, other vocal signals different from the lead singer's vocals can be collected in advance. For the same song, a vocal signal can be randomly selected from other vocal signals different from the lead singer's vocals for simulation to obtain an interlaced harmony signal of a different song segment from the lead singer's vocals.

[0075] In practical applications, one or more vocal signals may be selected from other vocal signals for random mixing to obtain interleaved harmony.

[0076] S202: Mix the lead vocal signal, harmony signal and accompaniment corresponding to each song to generate a song signal.

[0077] In addition to vocals, a song also includes accompaniment. Therefore, after obtaining the harmony signal that matches the lead singer's vocal signal, the accompaniment that matches the song can be added to obtain the song signal.

[0078] In the embodiment of the present application, there are two ways to mix the lead singer vocal signal, the harmony signal and the accompaniment. The first way is to first mix the lead singer vocal signal with the harmony signal to obtain a vocal signal, and then add the accompaniment. The second way is to mix the lead singer vocal signal, the harmony signal and the accompaniment at the same time to obtain a song signal.

[0079] In order to obtain more song signals, for the same song, different signal-to-noise ratios can be used to mix the lead vocal signal, harmony signal and accompaniment. That is, there can be multiple song signals corresponding to the same song, and these multiple song signals can be obtained in the first way or in the second way. It is also possible to obtain part of the song signals in the first way and the other part of the song signals in the second way.

[0080] For the first method, the lead singer's vocal signal and harmony signal corresponding to each song can be superimposed according to the set first signal-to-noise ratio range to obtain the vocal signal corresponding to each song; the vocal signal corresponding to each song and its matching accompaniment can be superimposed according to the set second signal-to-noise ratio range to obtain the song signal corresponding to each song.

[0081] The first signal-to-noise ratio range and the second signal-to-noise ratio range may be the same or different, which is not limited here. For example, the first signal-to-noise ratio range may be set to [-15, +15], and the second signal-to-noise ratio range may be set to [-20, +20].

[0082] For the second method, the lead vocal signal, harmony signal and accompaniment corresponding to each song can be mixed according to a set signal-to-noise ratio range to obtain the song signal corresponding to each song.

[0083] S203: Train the music separation model using a set number of song signals and their matching lead singer lyrics texts to obtain a trained music separation model.

[0084] The training samples may include a large number of song signals and their matching lead vocal lyrics. The value of the set number can be flexibly set according to the training requirements of the model and is not limited here.

[0085] Figure 3 A schematic diagram of a training sample generation process provided in an embodiment of the present application. Figure 3 A song is taken as an example. The harmony signal includes a synchronous harmony signal and an interlaced harmony signal. The lead vocal signal can be an audio signal derived from vocal 1, the synchronous harmony signal can be an audio signal derived from vocal 1 after pitch change, and the interlaced harmony signal can be an audio signal derived from a random mixture of vocal 2. It should be noted that vocal 2 is the vocals of a single person or multiple people other than vocal 1, and they are collectively referred to as vocal 2. When selecting the harmony signal, you can randomly select from the synchronous harmony signal and the interlaced harmony signal.

[0086] Figure 3 The song signal can be mixed with the lead singer vocal signal, harmony signal and accompaniment according to the first method described above. The selected harmony signal and the lead singer vocal signal are superimposed according to the set first signal-to-noise ratio range to obtain a vocal signal containing the lead singer vocal signal and the harmony signal. The vocal signal and its matching accompaniment are superimposed according to the set second signal-to-noise ratio range to obtain a song signal. The training sample can include a paired song signal and a lead singer vocal lyrics text.

[0087] In the embodiment of the present application, in order to ensure that the music separation model can simultaneously analyze the two types of information, the song signal and the lead singer's vocal lyrics text, it is necessary to convert the song signal and the lead singer's vocal lyrics text into feature vectors of the same dimension. In practical applications, the song signal can be encoded using an audio editor to obtain the song audio features; the lead singer's vocal lyrics text can be encoded using a text editor to obtain text features.

[0088] The song audio features and text features belong to feature vectors of the same dimension. After obtaining the song audio features and text features, the song audio features and their matching text features can be merged and input into the music separation model to output a lead singer vocal estimation signal. Based on the lead singer vocal estimation signal and the lead singer vocal signal, the model parameters of the music separation model are updated to obtain a trained music separation model.

[0089] The updating of model parameters mainly relies on the lead singer's vocal estimation signal and the lead singer's vocal signal to calculate the loss function, and the loss function is used for back propagation to update the model parameters.

[0090] Figure 4A schematic diagram of the training process of a music separation model provided in an embodiment of the present application, wherein a song signal is input into an audio editor to extract the audio features of the song; and a text of the lead singer's lyrics matching the song signal is input into a text editor to extract the text features. The song audio features and their matching text features are combined and input into the music separation model to output a lead singer's vocal estimation signal. The loss function is calculated through the lead singer's vocal estimation signal and the lead singer's vocal signal, and the model parameters are updated by back propagation. Among them, the audio editor and the text editor can choose a self-supervised model (Closed-Loop Attention-based Pretraining, CLAP) that connects text and audio, etc.

[0091] S204: When the target song signal is obtained, the target song signal and its matching target lead singer's vocal lyrics text are analyzed using the trained music separation model to extract the target lead singer's vocal signal.

[0092] When the target song signal is obtained, the target song signal can be encoded using the audio editor in the trained music separation model to obtain the target song audio features. The target lead singer vocal lyrics text can be encoded using the text editor in the trained music separation model to obtain the target text features. The target song audio features and target text features are combined and input into the trained music separation model for analysis to obtain the target lead singer vocal signal.

[0093] In order to further improve the accuracy of the music separation model in extracting the lead singer's vocal signal, the model parameters of the music separation model can be fine-tuned using the actual lead singer's vocal signal corresponding to the target song signal.

[0094] In actual applications, the user can input the actual lead singer vocal signal of the target song signal into the terminal through the human-computer interaction interface. The terminal can receive the actual lead singer vocal signal input by the user. The actual lead singer vocal signal is transmitted to the server for analysis. When the similarity between the target lead singer vocal signal and the actual lead singer vocal signal is less than a preset threshold, it means that there is room for adjustment of the model parameters of the music separation model. At this time, the server can use the target lead singer vocal signal and the actual lead singer vocal signal to adjust the model parameters of the trained music separation model.

[0095] In the present application, a harmony signal that matches the lead vocal signal of each song is selected from the harmony library corresponding to each song; the lead vocal signal, harmony signal and accompaniment corresponding to each song are mixed to generate a song signal. Considering that the lead vocal and harmony are easily confused, but the correspondence between the lead vocal and the lyrics is closer than that between the harmony, the music separation model can be trained using a set number of song signals and their matching lead vocal lyrics texts to obtain a trained music separation model. When the target song signal is obtained, the target song signal and its matching target lead vocal lyrics text are analyzed using the trained music separation model to extract the target lead vocal signal. In this technical solution, training data containing lead vocals, harmony and accompaniment is effectively constructed, which can realize the distinction between the lead vocal and harmony by the music separation model. The lead singer's lyrics text is also used to participate in the training of the music separation model, which effectively reduces the probability of confusion when the music separation model identifies the lead singer and the harmony, so that the trained music separation model can effectively extract the lead singer's vocals from the song signal.

[0096] See also Figure 5 As shown, the present application also discloses a schematic diagram of a structure of a lead singer's vocal extraction device, which includes a selection unit 51, a mixing unit 52, a training unit 53 and an analysis unit 54;

[0097] A selection unit 51 is used to select a harmony signal matching the lead singer's vocal signal of each song from a harmony library corresponding to each song;

[0098] A mixing unit 52, for mixing the lead vocal signal, harmony signal and accompaniment corresponding to each song to generate a song signal;

[0099] The training unit 53 is used to train the music separation model using a set number of song signals and their matching lead singer lyrics texts to obtain a trained music separation model;

[0100] The analysis unit 54 is used to analyze the target song signal and its matching target lead singer's vocal lyrics text using the trained music separation model when the target song signal is acquired, so as to extract the target lead singer's vocal signal.

[0101] In some embodiments, the harmony library includes a synchronous harmony signal and an interlaced harmony signal; for the construction process of the harmony library, the device includes a pitch shifting unit and a simulation unit;

[0102] A pitch shifting unit is used to perform pitch shifting processing on the vocal signal of the lead singer of the same song to obtain a synchronous harmony signal of the same song segment as the lead singer's vocal;

[0103] The simulation unit is used to randomly select a vocal signal from other vocal signals different from the lead singer's vocals for simulation for the same song, so as to obtain an interlaced harmony signal of a different song section sung by the lead singer's vocals.

[0104] In some embodiments, the mixing unit includes a first superposition subunit and a second superposition subunit;

[0105] A first superposition subunit is used to superimpose the lead vocal signal and the harmony signal corresponding to each song according to a set first signal-to-noise ratio range to obtain the vocal signal corresponding to each song;

[0106] The second superposition subunit is used to superimpose the human voice signal corresponding to each song and its matching accompaniment according to a set second signal-to-noise ratio range to obtain the song signal corresponding to each song.

[0107] In some embodiments, the mixing unit is used to mix the lead vocal signal, harmony signal and accompaniment corresponding to each song according to a set signal-to-noise ratio range to obtain a song signal corresponding to each song.

[0108] In some embodiments, the training unit includes an audio encoding subunit, a text encoding subunit, a merging subunit, and an updating subunit;

[0109] An audio encoding subunit, used for encoding the song signal using an audio editor to obtain the song audio features;

[0110] The text encoding subunit is used to encode the lead singer's vocal lyrics text using a text editor to obtain text features;

[0111] A merging subunit, used for merging the song audio features and the matching text features and inputting them into the music separation model to output a lead singer vocal estimation signal;

[0112] The updating subunit is used to update the model parameters of the music separation model based on the lead singer vocal estimation signal and the lead singer vocal signal to obtain a trained music separation model.

[0113] In some embodiments, the analysis unit is used to encode the target song signal using the audio editor in the trained music separation model to obtain the target song audio features; encode the target lead singer vocal lyrics text using the text editor in the trained music separation model to obtain the target text features; merge the target song audio features and the target text features and input them into the trained music separation model for analysis to output the target lead singer vocal signal.

[0114] In some embodiments, it further includes a receiving unit and an adjusting unit;

[0115] A receiving unit, used to receive an actual lead singer vocal signal input by a user;

[0116] The adjustment unit is used to adjust the model parameters of the trained music separation model using the target lead singer vocal signal and the actual lead singer vocal signal when the similarity between the target lead singer vocal signal and the actual lead singer vocal signal is less than a preset threshold.

[0117] Figure 5 The description of the features in the corresponding embodiments can be found in Figure 2 The relevant descriptions of the corresponding embodiments will not be repeated here one by one.

[0118] In the present application, a harmony signal that matches the lead vocal signal of each song is selected from the harmony library corresponding to each song; the lead vocal signal, harmony signal and accompaniment corresponding to each song are mixed to generate a song signal. Considering that the lead vocal and harmony are easily confused, but the correspondence between the lead vocal and the lyrics is closer than that between the harmony, the music separation model can be trained using a set number of song signals and their matching lead vocal lyrics texts to obtain a trained music separation model. When the target song signal is obtained, the target song signal and its matching target lead vocal lyrics text are analyzed using the trained music separation model to extract the target lead vocal signal. In this technical solution, training data containing lead vocals, harmony and accompaniment is effectively constructed, which can realize the distinction between the lead vocal and harmony by the music separation model. The lead singer's lyrics text is also used to participate in the training of the music separation model, which effectively reduces the probability of confusion when the music separation model identifies the lead singer and the harmony, so that the trained music separation model can effectively extract the lead singer's vocals from the song signal.

[0119] Furthermore, an embodiment of the present application also provides an electronic device. Figure 6 This is a structural diagram of an electronic device according to an exemplary embodiment. The content in the diagram cannot be regarded as any limitation on the scope of use of the present application.

[0120] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device may specifically include: at least one processor 61, at least one memory 62, a power supply 63, a communication interface 64, an input / output interface 65, and a communication bus 66. The memory 62 is used to store a computer program, which is loaded and executed by the processor 61 to implement the relevant steps in the method for extracting the lead singer's voice disclosed in any of the aforementioned embodiments. In addition, the electronic device in this embodiment may specifically be a server.

[0121] In this embodiment, the power supply 63 is used to provide working voltage for each hardware device on the electronic device; the communication interface 64 can create a data transmission channel between the electronic device and the external device, and the communication protocol it follows is any communication protocol that can be applied to the technical solution of the present application, and is not specifically limited here; the input and output interface 65 is used to obtain external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs and is not specifically limited here.

[0122] In addition, the memory 62, as a carrier for storing resources, can be a read-only memory, a random access memory, a disk or an optical disk, etc. The resources stored thereon may include an operating system 621, a computer program 622 and data 623, etc. The storage method can be temporary storage or permanent storage.

[0123] The operating system 621 is used to manage and control the hardware devices and computer programs 622 on the electronic device to realize the operation and processing of the massive video data 623 in the memory 62 by the processor 61, which can be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program that can be used to complete the method for extracting the lead singer's voice performed by the electronic device disclosed in any of the aforementioned embodiments, the computer program 622 can further include a computer program that can be used to complete other specific tasks. The data 623 can include various video data collected by the electronic device.

[0124] Furthermore, an embodiment of the present application also discloses a storage medium, in which a computer program is stored. When the computer program is loaded and executed by a processor, the steps of the method for extracting the lead singer's voice disclosed in any of the aforementioned embodiments are implemented.

[0125] The embodiment of the present application also discloses a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the aforementioned method for extracting lead singer's voice.

[0126] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0127] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0128] The above is a detailed introduction to the method, device, electronic device and medium for extracting the lead singer's voice provided by the present application. This article uses specific examples to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A method for extracting a lead singer's voice, characterized in that: include: Selecting a harmony signal matching the lead vocal signal of each song from a harmony library corresponding to each song; The lead vocal signal, harmony signal and accompaniment corresponding to each song are mixed to generate a song signal; The music separation model is trained using a set number of song signals and their matching lead singer lyrics texts to obtain a trained music separation model; When the target song signal is obtained, the trained music separation model is used to analyze the target song signal and its matching target lead singer's vocal lyrics text to extract the target lead singer's vocal signal.

2. The method for extracting the lead singer's voice according to claim 1, characterized in that: The harmony library includes synchronous harmony signals and interlaced harmony signals; Regarding the construction process of the harmony library, the method includes: Performing pitch-shifting processing on the lead singer's vocal signal of the same song to obtain a synchronous harmony signal of the same song segment as the lead singer's vocal; For the same song, a vocal signal is randomly selected from other vocal signals different from the lead singer's vocals for simulation, so as to obtain an interlaced harmony signal of a different song section from the lead singer's vocals.

3. The method for extracting the lead singer's voice according to claim 1, characterized in that: The lead vocal signal, harmony signal and accompaniment corresponding to each song are mixed to generate song signals including: Superimposing the lead vocal signal and harmony signal corresponding to each song according to a set first signal-to-noise ratio range to obtain the vocal signal corresponding to each song; The human voice signal corresponding to each song and its matching accompaniment are superimposed according to a set second signal-to-noise ratio range to obtain a song signal corresponding to each song.

4. The method for extracting the lead singer's voice according to claim 1, characterized in that: The lead vocal signal, harmony signal and accompaniment corresponding to each song are mixed to generate song signals including: The lead vocal signal, harmony signal and accompaniment corresponding to each song are mixed according to the set signal-to-noise ratio range to obtain the song signal corresponding to each song.

5. The method for extracting the lead singer's voice according to claim 1, characterized in that: The music separation model is trained using a set number of song signals and their matching lead singer lyrics texts to obtain a trained music separation model including: Encoding the song signal by using an audio editor to obtain the song audio features; Using a text editor to encode the lead singer's lyrics text to obtain text features; The song audio features and the matching text features are combined and input into a music separation model to output a lead singer vocal estimation signal; Based on the lead singer vocal estimation signal and the lead singer vocal signal, the model parameters of the music separation model are updated to obtain a trained music separation model.

6. The method for extracting the lead singer's voice according to claim 5, characterized in that: The target song signal and its matching target lead singer lyrics text are analyzed using the trained music separation model to extract the target lead singer vocal signal including: Encoding the target song signal using the audio editor in the trained music separation model to obtain the target song audio features; Encode the target lead singer's vocal lyrics text using the text editor in the trained music separation model to obtain target text features; The target song audio features and the target text features are combined and input into the trained music separation model for analysis to output a target lead singer vocal signal.

7. The method for extracting the lead singer's voice according to claim 1, characterized in that: After analyzing the target song signal and the target lead singer's vocal lyrics text matched thereto by using the trained music separation model to extract the target lead singer's vocal signal, the method further includes: Receive an actual lead singer vocal signal input by a user; When the similarity between the target lead singer vocal signal and the actual lead singer vocal signal is less than a preset threshold, the model parameters of the trained music separation model are adjusted using the target lead singer vocal signal and the actual lead singer vocal signal.

8. A device for extracting a lead singer's voice, characterized in that: It includes selection unit, mixing unit, training unit and analysis unit; The selection unit is used to select a harmony signal matching the lead singer's vocal signal of each song from the harmony library corresponding to each song; The mixing unit is used to mix the lead vocal signal, harmony signal and accompaniment corresponding to each song to generate a song signal; The training unit is used to train the music separation model using a set number of song signals and their matching lead singer lyrics texts to obtain a trained music separation model; The analysis unit is used to analyze the target song signal and its matching target lead singer's vocal lyrics text using a trained music separation model when a target song signal is acquired, so as to extract the target lead singer's vocal signal.

9. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, configured to execute the computer program to implement the method for extracting the lead singer's voice as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: Used to store a computer program, which, when executed by a processor, implements the method for extracting the lead singer's voice as described in any one of claims 1 to 7.

Images