Human voice separation method, device, equipment and medium

Abstract

The invention provides a human voice separation method and device, equipment and a medium, and the method comprises the steps: obtaining a mixed human voice signal, and obtaining the voiceprint features of each human voice signal contained in the mixed human voice signal; and carrying out human voice separation on the mixed human voice signal based on the acoustic characteristics of the mixed human voice signal and the voiceprint characteristics of each single human voice signal to obtain a plurality of single human voice signals. According to the human voice separation method and device, the equipment and the medium provided by the invention, the precision of human voice separation is improved by introducing the voiceprint features of the human voice signal of a single person. Through combination of voiceprint features, even under the condition that sound frequency distribution is overlapped, clear and independent single-person voice signals can be obtained in a complex scene.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method, device, equipment and medium for separating human voice. Background Art

[0002] The human voice separation technology aims to separate individual human voices from mixed human voice signals and has been widely used in speech recognition, audio editing, and intelligent assistants. With the development of multimedia and entertainment applications, users' demand for accurate separation of each human voice continues to grow. The separation of individual human voices not only improves the user experience, but also provides technical support for sentiment analysis and individual audio feature analysis, making it of great research value in the field of audio processing.

[0003] Existing human voice separation technologies mainly include frequency domain analysis, independent component analysis, and traditional machine learning methods. Frequency domain analysis is difficult to effectively separate different human voices when there is a lot of spectrum overlap; independent component analysis relies on the independence assumption of signal sources, but performs poorly for actual complex mixed human voice signals; traditional machine learning methods usually rely on manually designed features and are difficult to deal with complex features in audio signals. These methods have limited accuracy in practical applications and are difficult to meet the current high-quality human voice separation needs. Summary of the invention

[0004] The present invention provides a human voice separation method, device, equipment and medium, which are used to solve the defect of limited human voice separation accuracy in the related art.

[0005] The present invention provides a method for separating human voice, comprising the following steps: Acquire a mixed human voice signal, and acquire voiceprint features of each single human voice signal contained in the mixed human voice signal; Based on the acoustic features of the mixed vocal signal and the voiceprint features of the individual vocal signals, the mixed vocal signal is subjected to vocal separation to obtain a plurality of individual vocal signals.

[0006] According to a method for separating human voices provided by the present invention, obtaining a mixed human voice signal comprises: Acquire chorus audio signal; The vocal accompaniment is separated from the chorus audio signal to obtain a mixed vocal signal in the chorus audio signal.

[0007] According to a method for separating human voices from accompaniment provided by the present invention, the step of separating the human voice from the chorus audio signal to obtain a mixed human voice signal in the chorus audio signal comprises: The acoustic features of the chorus audio signal are segmented into sub-bands, and the vocal accompaniment is separated based on the sub-band features obtained by the segmentation to obtain a mixed vocal signal in the chorus audio signal.

[0008] According to a method for separating human voices provided by the present invention, the separation of human voices and accompaniment is implemented based on a human voice and accompaniment separation model, and the training steps of the human voice and accompaniment separation model include: Synthesizing a sample chorus audio signal based on the sample mixed vocal signal and the sample accompaniment signal; Based on the initial vocal accompaniment separation model, the sample chorus audio signal is subjected to vocal accompaniment separation to obtain a test mixed vocal signal and a test accompaniment signal; Based on the sample mixed vocal signal and the test mixed vocal signal, the sample accompaniment signal and the test accompaniment signal, the initial vocal accompaniment separation model is iterated on parameters to obtain the vocal accompaniment separation model.

[0009] According to a method for separating human voices provided by the present invention, the human voice separation is implemented based on a mixed human voice separation model, and the mixed human voice separation model is jointly trained with the human voice accompaniment separation model; The training step further comprises: synthesizing the sample mixed vocal signal based on the sample vocal signal; Based on the initial mixed vocal separation model, performing vocal separation on the test mixed vocal signal to obtain a test single vocal signal; Based on the sample vocal signal and the test single vocal signal, performing parameter iteration on the initial mixed vocal separation model to obtain the mixed vocal separation model; The method of performing parameter iteration on the initial vocal and accompaniment separation model based on the sample mixed vocal signal and the test mixed vocal signal, the sample accompaniment signal and the test accompaniment signal to obtain the vocal and accompaniment separation model comprises: Based on the sample vocal signal and the test single vocal signal, the sample mixed vocal signal and the test mixed vocal signal, the sample accompaniment signal and the test accompaniment signal, the initial vocal accompaniment separation model is iterated on parameters to obtain the vocal accompaniment separation model.

[0010] According to a method for voice separation provided by the present invention, the step of obtaining the voiceprint features of each single voice signal contained in the mixed voice signal includes: Determine the person identification corresponding to each single person voice signal contained in the mixed voice signal, and obtain the voiceprint feature of the person identification.

[0011] According to a method for voice separation provided by the present invention, the step of obtaining the voiceprint features of each single voice signal contained in the mixed voice signal includes: A single-person singing signal segment is segmented from the mixed vocal signal, and the voiceprint feature is extracted from the single-person singing signal segment.

[0012] The present invention also provides a human voice separation device, comprising the following modules: A voiceprint feature acquisition unit, which acquires a mixed human voice signal and acquires the voiceprint features of each single human voice signal contained in the mixed human voice signal; The human voice separation unit performs human voice separation on the mixed human voice signal based on the acoustic features of the mixed human voice signal and the voiceprint features of the individual human voice signals to obtain multiple individual human voice signals.

[0013] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, any of the above-mentioned human voice separation methods is implemented.

[0014] The present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the human voice separation method as described above is implemented.

[0015] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the human voice separation method described above is implemented.

[0016] The human voice separation method, device, equipment and medium provided by the present invention improve the accuracy of human voice separation by introducing the voiceprint features of a single human voice signal. By combining the voiceprint features, a clear and independent single human voice signal can be obtained in complex scenes even when the sound frequency distribution overlaps. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the present invention or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 This is one of the flow charts of the human voice separation method provided by the present invention.

[0019] Figure 2 This is the second flow chart of the human voice separation method provided by the present invention.

[0020] Figure 3 It is a structural schematic diagram of the human voice separation device provided by the present invention.

[0021] Figure 4 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0023] Voice separation technology is becoming increasingly important in the field of audio processing. It is dedicated to separating a single voice signal from a mixed voice signal, that is, accurately extracting multiple overlapping sound source signals (such as the individual voices of multiple people singing together). This technology has become an important foundation in fields such as speech recognition, audio editing, and intelligent voice assistants. Especially in multimedia and entertainment application scenarios, users have an increasing demand for refined single voice separation. Accurate separation of single voices not only improves the user's immersive experience, but also provides a wider range of application possibilities for emotional analysis and independent analysis of character audio features, making it of great research value in audio signal processing.

[0024] At present, the methods for achieving single-person voice separation mainly include frequency domain analysis, independent component analysis and traditional machine learning technology. Frequency domain analysis usually extracts the spectral features of different voices through short-time Fourier transform and non-negative matrix decomposition, but due to the overlap of multiple sound signals in frequency, especially in the low and medium frequency bands, it is difficult for frequency domain methods to effectively distinguish each single voice. Independent component analysis relies on the statistical independence of signals to separate sound sources, but the complex mixed characteristics of music and vocal signals often do not meet this assumption, and the separation results are greatly affected by initial conditions and noise. Traditional machine learning methods rely on artificially designed features, but this method has limited generalization ability when facing complex nonlinear audio signals and it is difficult to accurately separate multiple voices. In general, the existing methods show certain limitations when processing diverse mixed audio signals and fail to fully meet the needs of accurate and efficient single-person voice separation.

[0025] In view of the above problems, an embodiment of the present invention provides a method for separating human voice. Figure 1 FIG. 1 is one of the flow charts of the human voice separation method provided by the present invention. Figure 1 As shown, the method includes: Step 110: obtaining a mixed human voice signal, and obtaining voiceprint features of each single human voice signal contained in the mixed human voice signal.

[0026] Specifically, a mixed vocal signal refers to a pure vocal audio signal containing multiple single vocals, which may appear in multi-person conversations, multi-person speeches, or chorus scenes. This signal contains only vocals and does not contain accompaniment signals or background sound signals. Therefore, when separating different vocals, you only need to focus on distinguishing the acoustic features of each single vocal. By capturing the mixed vocal signal, you can obtain a complete audio signal containing all single vocals, which provides a basis for subsequent separation operations.

[0027] In addition to obtaining the mixed vocal signal, it is also necessary to obtain the voiceprint features of each individual vocal signal in the mixed vocal signal. Voiceprint features refer to the unique characteristics of a single vocal signal, including timbre, frequency, resonance peaks, etc. These features are mainly determined by the speaker's physiological structure and vocal habits, and can help effectively identify and separate each person's individual vocal signal. Extracting these features provides a clear basis for distinguishing vocal separation, so that different individual voices in the mixed vocal signal can be accurately separated.

[0028] Step 120: Based on the acoustic features of the mixed vocal signal and the voiceprint features of each single vocal signal, perform vocal separation on the mixed vocal signal to obtain a plurality of single vocal signals.

[0029] Specifically, the acoustic features of a mixed vocal signal include the frequency distribution, energy changes, and time-frequency spectrum information of multiple individual vocal signals, which to a certain extent reveal the basic properties of each individual vocal signal. By analyzing these acoustic features, we can preliminarily identify and distinguish multiple individual vocal signals in a mixed signal.

[0030] However, it is often difficult to accurately separate each single voice in a complex scene based solely on the acoustic features of a mixed vocal signal. At this point, the separation effect can be further enhanced by introducing the voiceprint features of a single voice. The addition of voiceprint features makes the vocal separation process not only rely on the acoustic features of the mixed vocal signal, but also can make use of the subtle differences unique to each single voice for more refined distinctions. Based on the overall outline provided by the acoustic features, the voiceprint features are used to distinguish details, thereby ensuring that each single vocal signal can be completely and clearly extracted during the vocal separation process. Through the comprehensive use of this dual feature, the vocal separation method can show significant advantages in dealing with the problem of overlapping multiple sound sources, greatly improving the accuracy and authenticity of the separation results, so that the final single vocal signal is closer to the true performance of the original voice.

[0031] The human voice separation method provided by the present invention improves the accuracy of human voice separation by introducing the voiceprint features of a single human voice signal. By combining the voiceprint features, a clear and independent single human voice signal can be obtained in complex scenes even when the sound frequency distribution overlaps.

[0032] Based on the above embodiment, the step of obtaining a mixed human voice signal includes: Acquire chorus audio signal; The vocal accompaniment is separated from the chorus audio signal to obtain a mixed vocal signal in the chorus audio signal.

[0033] Specifically, choral audio signals usually include mixed audio of multiple vocals and accompaniment. Choral audio signals may come from a variety of scenarios, such as choral works in a recording studio, live recordings of concerts, or multi-person choruses. In these scenarios, vocals and accompaniment are often intertwined to form complex mixed audio. Acquiring choral audio signals not only provides basic data for subsequent vocal separation, but also ensures that the processed data covers the typical characteristics of a variety of vocal and accompaniment combinations.

[0034] On this basis, the vocal accompaniment separation is performed on the chorus audio signal to obtain the mixed vocal signal. The process of vocal accompaniment separation is to separate the vocal part from the accompaniment part in the mixed vocal signal. This operation can effectively remove the interference of the accompaniment, so that subsequent processing can focus on vocal separation. Through vocal accompaniment separation, the mixed signal containing multiple vocals can be clearly extracted, laying the foundation for further vocal separation. This step provides a clean vocal audio signal for the subsequent vocal separation, making the vocal separation effect of the chorus audio signal more accurate and clear.

[0035] Based on the above embodiment, the step of separating the vocal accompaniment from the chorus audio signal to obtain a mixed vocal signal in the chorus audio signal includes: The acoustic features of the chorus audio signal are segmented into sub-bands, and the vocal accompaniment is separated based on the sub-band features obtained by the segmentation to obtain a mixed vocal signal in the chorus audio signal.

[0036] Sub-band segmentation refers to dividing the chorus audio signal into several sub-bands in frequency, so as to perform more detailed analysis and processing on different frequency bands. When performing sub-band segmentation, you can choose uniform segmentation, that is, divide the frequency range of the audio signal into several sub-bands at equal intervals, so as to facilitate the subsequent unified processing; this method is relatively simple in calculation and has strong applicability.

[0037] At the same time, a more reasonable segmentation strategy can also be implemented based on the auditory characteristics of the human ear. The human ear is more sensitive to low-frequency signals, but relatively insensitive to high-frequency signals. Therefore, in the segmentation strategy based on the auditory characteristics of the human ear, the low-frequency part is usually divided more finely, while the high-frequency part can be segmented using a wider frequency band. This segmentation method can more effectively capture the human voice characteristics in the low frequency, making the retention of the human voice more complete and accurate during the separation process.

[0038] In addition to the above methods, adaptive sub-band segmentation can also be used. This method dynamically adjusts the sub-band width and flexibly divides the frequency band according to the acoustic characteristics of the audio signal, which helps to accurately capture the characteristics of the human voice in complex signals. At the same time, the multi-scale segmentation of the wavelet transform utilizes the decomposition of different frequency scales to identify subtle changes in the audio signal in each frequency band. The entropy value segmentation method divides the sub-band according to the amount of information in the frequency band, and finely segments the frequency band with a higher amount of information, which is suitable for identifying and extracting key information in the human voice. The embodiment of the present invention does not specifically limit the sub-band segmentation method.

[0039] After completing the sub-band segmentation, each sub-band has a specific frequency range and contains the acoustic characteristics of the chorus audio signal in this frequency band. Next, the vocal accompaniment is separated based on the characteristics of each sub-band. By analyzing the acoustic characteristics in each sub-band, the difference between the vocals and the accompaniment can be identified. For example, the vocals usually show a higher energy concentration in the low-frequency and mid-frequency bands, while the accompaniment may be more prominent in other frequency bands. Taking advantage of this difference in frequency band distribution, the vocal and accompaniment signals can be processed separately in each sub-band, and the vocal part can be retained by filtering out or reducing the influence of the accompaniment component.

[0040] In this process, the accompaniment part in each sub-band is gradually stripped away, leaving only the acoustic features of the human voice. Finally, the separated vocal signals in each sub-band are recombined to form a complete mixed vocal signal. This separated signal contains all the audio components of the human voice, laying the foundation for the subsequent separation of the single vocal. Through this separation method based on sub-band features, the human voice and accompaniment can be efficiently separated in complex chorus audio signals, which improves the accuracy of the separation results and ensures that the final mixed vocal signal can better support the subsequent extraction of single vocals.

[0041] Based on the above embodiment, the vocal accompaniment separation is implemented based on a vocal accompaniment separation model, and the training steps of the vocal accompaniment separation model include: Synthesizing a sample chorus audio signal based on the sample mixed vocal signal and the sample accompaniment signal; Based on the initial vocal accompaniment separation model, the sample chorus audio signal is subjected to vocal accompaniment separation to obtain a test mixed vocal signal and a test accompaniment signal; Based on the sample mixed vocal signal and the test mixed vocal signal, the sample accompaniment signal and the test accompaniment signal, the initial vocal accompaniment separation model is iterated on parameters to obtain the vocal accompaniment separation model.

[0042] Specifically, the sample chorus audio signal is the training set of the vocal accompaniment separation model, which consists of a sample mixed vocal signal and a sample accompaniment signal. The sample mixed vocal signal contains the superposition of multiple vocals, while the sample accompaniment signal consists only of the accompaniment audio. During the synthesis process, these vocal and accompaniment signals are linearly superimposed to simulate a real chorus scene, thereby generating a set of sample chorus audio signals with mixed characteristics of multiple vocals and accompaniment. These chorus audio signals contain the overlap between different vocals and their fusion with the accompaniment, providing rich data for the vocal accompaniment separation model to learn and adapt to complex situations in practical applications.

[0043] Next, the synthesized sample chorus audio signal is separated using the initial vocal accompaniment separation model to generate a test mixed vocal signal and a test accompaniment signal. At this time, the initial vocal accompaniment separation model is in an unoptimized state, and the parameters in it are all initialization parameters. The initial vocal accompaniment separation model is designed to perform preliminary vocal accompaniment separation on the sample chorus audio signal to produce test results. This separation operation tentatively divides the chorus signal into two parts, vocals and accompaniment, and obtains a test mixed vocal signal and a test accompaniment signal. This process not only produces a separation output, but also provides a data reference for the subsequent optimization of model parameters.

[0044] Subsequently, the effect of the initial vocal accompaniment separation model is evaluated by comparing the differences between the sample mixed vocal signal and the test mixed vocal signal, and between the sample accompaniment signal and the test accompaniment signal. According to the deviation between the two, the parameters of the initial vocal accompaniment separation model are iterated and adjusted. Specifically, the initial vocal accompaniment separation model will gradually adjust its own weights and parameter configurations based on error feedback, so that the test mixed vocal signal and the test accompaniment signal can more accurately approach the sample mixed vocal signal and the sample accompaniment signal. The parameter iteration process will retrain the model after each update to ensure that each round of adjustment can effectively improve the accuracy of vocal accompaniment separation.

[0045] This iterative process usually goes through multiple rounds until the separation performance of the vocal accompaniment separation model reaches the expected standard. By gradually optimizing the model parameters, the model can more accurately identify and separate the vocals and accompaniment parts when processing complex chorus audio signals, and finally form a high-precision vocal accompaniment separation model. The optimized model has stronger adaptability and robustness, and can achieve efficient vocal accompaniment separation in different chorus scenarios, providing a pure and accurate mixed vocal signal for subsequent single vocal separation, laying a good foundation for vocal separation.

[0046] Based on the above embodiment, the vocal separation is implemented based on a mixed vocal separation model, and the mixed vocal separation model is jointly trained with the vocal accompaniment separation model; The training step further comprises: synthesizing the sample mixed vocal signal based on the sample vocal signal; Based on the initial mixed vocal separation model, performing vocal separation on the test mixed vocal signal to obtain a test single vocal signal; Based on the sample vocal signal and the test single vocal signal, performing parameter iteration on the initial mixed vocal separation model to obtain the mixed vocal separation model; The method of performing parameter iteration on the initial vocal and accompaniment separation model based on the sample mixed vocal signal and the test mixed vocal signal, the sample accompaniment signal and the test accompaniment signal to obtain the vocal and accompaniment separation model comprises: Based on the sample vocal signal and the test single vocal signal, the sample mixed vocal signal and the test mixed vocal signal, the sample accompaniment signal and the test accompaniment signal, the initial vocal accompaniment separation model is iterated on parameters to obtain the vocal accompaniment separation model.

[0047] Specifically, the training of the mixed voice separation model first requires a sample mixed voice signal. The sample mixed voice signal includes multiple independent voice signals of a single person, namely the sample voice signal. By linearly superimposing the sample voice signals, a sample mixed voice signal containing multiple voices can be generated. The sample mixed voice signal simulates the overlapping scene of multiple sound sources in actual applications, and provides complex multi-voice mixed data for model learning. The sample mixed voice signal becomes the core data source for the subsequent training of the mixed voice separation model.

[0048] The initial mixed vocal separation model will separate the test mixed vocal signal to obtain a test single vocal signal. The initial mixed vocal separation model has only been preliminarily trained at this stage to try to separate the mixed vocal signal into separate single vocal signals to form a set of test single vocal signals. This preliminary separation result provides test data for the mixed vocal separation model, so that the model parameters can be optimized later by feedback error.

[0049] Next, based on the comparison between the sample vocal signal and the test single vocal signal, the parameters of the initial mixed vocal separation model are iterated. By calculating the error between the test single vocal signal and the sample vocal signal, the initial mixed vocal separation model gradually adjusts the parameters according to the error feedback to gradually approach the target separation effect. After multiple iterations, the initial mixed vocal separation model can perform vocal separation more accurately, forming a more accurate mixed vocal separation model.

[0050] While completing the parameter adjustment of the mixed vocal separation model, the parameters of the initial vocal accompaniment separation model can also be iterated based on the difference between the sample vocal signal and the test single vocal signal, that is, the mixed vocal separation model can be jointly trained with the vocal accompaniment separation model. In the process of joint training, compared with the method of independently training the vocal accompaniment separation model, the difference is that the parameters of the initial vocal accompaniment separation model are iteratively optimized based on the difference between the sample mixed vocal signal and the test mixed vocal signal, and the difference between the sample accompaniment signal and the test accompaniment signal. At this time, the output result of the mixed vocal separation model (that is, the test single vocal signal) is also included in the reference range of the parameter optimization of the vocal accompaniment separation model, which means that the vocal accompaniment separation model not only relies on the separation results of the test accompaniment signal and the test mixed vocal signal during training, but also refers to the separation effect of the mixed vocal separation model.

[0051] This joint training method allows the two models to cooperate with each other, thereby improving the overall separation accuracy. Specifically, the optimization of the vocal accompaniment separation model will not only improve the separation of accompaniment and mixed vocals, but also improve the understanding of vocal characteristics, so that the mixed vocal separation model will perform more accurately in subsequent vocal separation. By jointly optimizing the separation of vocals and accompaniment, the two models continuously adjust to each other during the training process to form a synergistic effect. Ultimately, the jointly trained model can demonstrate higher separation accuracy and adaptability in practical applications.

[0052] Based on the above embodiment, the step of obtaining the voiceprint features of each single voice signal contained in the mixed voice signal includes: Determine the person identification corresponding to each single person voice signal contained in the mixed voice signal, and obtain the voiceprint feature of the person identification.

[0053] When obtaining the voiceprint features of each single voice signal in a mixed voice signal, the person identifier of each single voice signal can be determined first. These identifiers are usually assigned in advance based on known person identity information (such as name) during audio recording. Through these pre-defined person identifiers, it can be ensured that each single voice signal in the mixed signal accurately corresponds to a specific individual, which is convenient for subsequent separation and identification.

[0054] Next, based on the determined person identification, the corresponding voiceprint features are matched directly from the voiceprint library. In this way, not only can the voiceprint features of each single voice signal be accurately located in the mixed signal, but the accuracy and clarity of the separation can also be further improved by matching the voiceprint features, so that each single person's voice can be accurately identified and extracted in the complex mixed signal. In addition, since the person identification and voiceprint features have been assigned and stored before recording, the voiceprint features no longer require complex extraction operations, thereby greatly improving processing efficiency. This method makes it possible to separate and extract single voices more quickly and efficiently in complex audio scenarios with multiple voices.

[0055] Based on the above embodiment, the step of obtaining the voiceprint features of each single voice signal contained in the mixed voice signal includes: A single-person singing signal segment is segmented from the mixed vocal signal, and the voiceprint feature is extracted from the single-person singing signal segment.

[0056] Specifically, in an audio signal of a multi-person chorus, each person usually sings alone in a certain period of time, forming an independent audio segment, namely a "single-person singing signal segment". By analyzing the time-frequency spectrum characteristics of the mixed vocal signal, the single-person singing signal segments corresponding to each single voice can be identified and cut out. The cutting process requires accurate positioning of the start and end time points of each single voice in order to extract clean, independent single-person singing signal segments.

[0057] Next, extract the voiceprint features from the cut single-person singing signal segment. Once an independent single-person singing signal segment is obtained, the voiceprint features can be extracted based on the audio segment. These voiceprint features can reflect the unique voice characteristics of an individual, ensuring that each person's voice can be accurately identified and distinguished during the subsequent separation process. By directly extracting voiceprint features from independent single-person singing signal segments, interference in mixed audio can be effectively avoided, and the purity of voiceprint data and the accuracy of extraction can be improved. This method not only improves the extraction accuracy of voiceprint features, but also lays a stable feature foundation for subsequent vocal separation.

[0058] Based on the above embodiments, Figure 2 FIG. 2 is a flow chart of the method for voice separation provided by the present invention. Figure 2 As shown in the figure, the input chorus audio signal contains a mixed vocal signal and an accompaniment signal. In order to separate the mixed vocal signal, the first step is to separate the mixed vocal signal from the accompaniment signal. This step is completed by the vocal accompaniment separation model. The design of the vocal accompaniment separation model is usually based on deep learning. Through a large number of sample training, the model can identify the characteristics of the mixed vocal signal and accompaniment signal in a complex chorus scene, thereby achieving an efficient separation effect.

[0059] After obtaining the pure mixed vocal signal, it is input into the mixed vocal separation model. The mixed vocal separation model can distinguish different single voices based on the voiceprint features of each single vocal signal. Voiceprint features are unique sound features of human voice, such as timbre, frequency, and resonance peaks. These features are determined by the speaker's physiological structure and vocal habits and have strong distinguishing capabilities. By obtaining the voiceprint features of each single voice, the mixed vocal separation model can more accurately extract each person's voice during separation, thereby ensuring that each person's voice can be clearly presented in the output single vocal signal.

[0060] The acquisition of voiceprint features of single-person vocal signals is particularly important. The extraction of voiceprint features provides support for subsequent separation, so that the mixed vocal separation model can more accurately identify and separate each person's voice during execution. The addition of voiceprint features makes the separation process not only rely on the acoustic features of the mixed vocal signal, but also can make more detailed distinctions by utilizing the subtle differences unique to each single voice. Through the overall outline provided by the acoustic features, the voiceprint features are used to distinguish the details, thereby ensuring that each single vocal signal can be completely and clearly extracted during the vocal separation process. The comprehensive use of this dual feature enables the vocal separation method to show significant advantages in dealing with the overlap of multiple sound sources, effectively improving the accuracy and authenticity of the separation results. Finally, through the processing of the mixed vocal separation model, multiple independent single-person vocal signals are obtained.

[0061] The human voice separation device provided by the present invention is described below. The human voice separation device described below and the human voice separation method described above can be referenced to each other.

[0062] Figure 3 is a schematic diagram of the structure of the emotion assessment device provided by the present invention, such as Figure 3 As shown, the device comprises: The voiceprint feature acquisition unit 310 acquires a mixed human voice signal, and acquires the voiceprint features of each single human voice signal contained in the mixed human voice signal; The human voice separation unit 320 performs human voice separation on the mixed human voice signal based on the acoustic features of the mixed human voice signal and the voiceprint features of the individual human voice signals to obtain multiple individual human voice signals.

[0063] The human voice separation device provided by the present invention improves the accuracy of human voice separation by introducing the voiceprint features of a single human voice signal. By combining the voiceprint features, a clear and independent single human voice signal can be obtained in complex scenes even when the sound frequency distribution overlaps.

[0064] Based on any of the above embodiments, the voiceprint feature acquisition unit is specifically used to: Determine the person identification corresponding to each single person voice signal contained in the mixed voice signal, and obtain the voiceprint feature of the person identification.

[0065] A single-person singing signal segment is segmented from the mixed vocal signal, and the voiceprint feature is extracted from the single-person singing signal segment.

[0066] Based on any of the above embodiments, the human voice separation unit is specifically used for: Acquire chorus audio signal; The vocal accompaniment is separated from the chorus audio signal to obtain a mixed vocal signal in the chorus audio signal.

[0067] The acoustic features of the chorus audio signal are segmented into sub-bands, and the vocal accompaniment is separated based on the sub-band features obtained by the segmentation to obtain a mixed vocal signal in the chorus audio signal.

[0068] Synthesizing a sample chorus audio signal based on the sample mixed vocal signal and the sample accompaniment signal; Based on the initial vocal accompaniment separation model, the sample chorus audio signal is subjected to vocal accompaniment separation to obtain a test mixed vocal signal and a test accompaniment signal; Based on the sample mixed vocal signal and the test mixed vocal signal, the sample accompaniment signal and the test accompaniment signal, the initial vocal accompaniment separation model is iterated on parameters to obtain the vocal accompaniment separation model.

[0069] The training step further comprises: synthesizing the sample mixed vocal signal based on the sample vocal signal; Based on the initial mixed vocal separation model, performing vocal separation on the test mixed vocal signal to obtain a test single vocal signal; Based on the sample vocal signal and the test single vocal signal, performing parameter iteration on the initial mixed vocal separation model to obtain the mixed vocal separation model; Based on the sample vocal signal and the test single vocal signal, the sample mixed vocal signal and the test mixed vocal signal, the sample accompaniment signal and the test accompaniment signal, the initial vocal accompaniment separation model is iterated on parameters to obtain the vocal accompaniment separation model.

[0070] Determine the person identification corresponding to each single person voice signal contained in the mixed voice signal, and obtain the voiceprint feature of the person identification.

[0071] Figure 4 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 4As shown, the electronic device may include: a processor 410, a communication interface 420, a memory 430 and a communication bus 440, wherein the processor 410, the communication interface 420, and the memory 430 communicate with each other through the communication bus 440. The processor 410 may call the logic instructions in the memory 430 to execute the human voice separation method, which includes: Acquire a mixed human voice signal, and acquire voiceprint features of each single human voice signal contained in the mixed human voice signal; Based on the acoustic features of the mixed vocal signal and the voiceprint features of the individual vocal signals, the mixed vocal signal is subjected to vocal separation to obtain a plurality of individual vocal signals.

[0072] In addition, the logic instructions in the above-mentioned memory 430 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the relevant technology or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0073] On the other hand, the present invention further provides a computer program product, the computer program product comprising a computer program, the computer program can be stored on a non-transitory computer-readable storage medium, and when the computer program is executed by a processor, the computer can perform the human voice separation method provided by the above methods, the method comprising: Acquire a mixed human voice signal, and acquire voiceprint features of each single human voice signal contained in the mixed human voice signal; Based on the acoustic features of the mixed vocal signal and the voiceprint features of the individual vocal signals, the mixed vocal signal is subjected to vocal separation to obtain a plurality of individual vocal signals.

[0074] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to perform the human voice separation method provided by the above methods, the method comprising: Acquire a mixed human voice signal, and acquire voiceprint features of each single human voice signal contained in the mixed human voice signal; Based on the acoustic features of the mixed vocal signal and the voiceprint features of the individual vocal signals, the mixed vocal signal is subjected to vocal separation to obtain a plurality of individual vocal signals.

[0075] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0076] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiment.

[0077] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for separating human voice, characterized in that: include: Acquire a mixed human voice signal, and acquire voiceprint features of each single human voice signal contained in the mixed human voice signal; Based on the acoustic features of the mixed vocal signal and the voiceprint features of the individual vocal signals, the mixed vocal signal is subjected to vocal separation to obtain a plurality of individual vocal signals.

2. The method for separating human voice according to claim 1, characterized in that: The step of obtaining a mixed vocal signal comprises: Acquire chorus audio signal; The vocal accompaniment is separated from the chorus audio signal to obtain a mixed vocal signal in the chorus audio signal.

3. The method for separating human voice according to claim 2, characterized in that: The step of separating the vocal accompaniment from the chorus audio signal to obtain a mixed vocal signal in the chorus audio signal includes: The acoustic features of the chorus audio signal are segmented into sub-bands, and the vocal accompaniment is separated based on the sub-band features obtained by the segmentation to obtain a mixed vocal signal in the chorus audio signal.

4. The method for separating human voice according to claim 2, characterized in that: The vocal accompaniment separation is implemented based on a vocal accompaniment separation model, and the training steps of the vocal accompaniment separation model include: Synthesizing a sample chorus audio signal based on the sample mixed vocal signal and the sample accompaniment signal; Based on the initial vocal accompaniment separation model, the sample chorus audio signal is subjected to vocal accompaniment separation to obtain a test mixed vocal signal and a test accompaniment signal; Based on the sample mixed vocal signal and the test mixed vocal signal, the sample accompaniment signal and the test accompaniment signal, the initial vocal accompaniment separation model is iterated on parameters to obtain the vocal accompaniment separation model.

5. The method for separating human voice according to claim 4, characterized in that: The vocal separation is achieved based on a mixed vocal separation model, which is obtained by joint training with the vocal accompaniment separation model; The training step further comprises: synthesizing the sample mixed vocal signal based on the sample vocal signal; Based on the initial mixed vocal separation model, performing vocal separation on the test mixed vocal signal to obtain a test single vocal signal; Based on the sample vocal signal and the test single vocal signal, performing parameter iteration on the initial mixed vocal separation model to obtain the mixed vocal separation model; The method of performing parameter iteration on the initial vocal and accompaniment separation model based on the sample mixed vocal signal and the test mixed vocal signal, the sample accompaniment signal and the test accompaniment signal to obtain the vocal and accompaniment separation model comprises: Based on the sample vocal signal and the test single vocal signal, the sample mixed vocal signal and the test mixed vocal signal, the sample accompaniment signal and the test accompaniment signal, the initial vocal accompaniment separation model is iterated on parameters to obtain the vocal accompaniment separation model.

6. The method for separating human voice according to any one of claims 1 to 5, characterized in that: The obtaining of the voiceprint features of each single voice signal contained in the mixed voice signal includes: Determine the person identification corresponding to each single person voice signal contained in the mixed voice signal, and obtain the voiceprint feature of the person identification.

7. The method for separating human voice according to any one of claims 1 to 5, characterized in that: The obtaining of the voiceprint features of each single voice signal contained in the mixed voice signal includes: A single-person singing signal segment is segmented from the mixed vocal signal, and the voiceprint feature is extracted from the single-person singing signal segment.

8. A human voice separation device, characterized in that: include: A voiceprint feature acquisition unit, which acquires a mixed human voice signal and acquires the voiceprint features of each single human voice signal contained in the mixed human voice signal; The human voice separation unit performs human voice separation on the mixed human voice signal based on the acoustic features of the mixed human voice signal and the voiceprint features of the individual human voice signals to obtain multiple individual human voice signals.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the human voice separation method according to any one of claims 1 to 7 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the human voice separation method according to any one of claims 1 to 7 is implemented.

11. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the human voice separation method according to any one of claims 1 to 7 is implemented.

Images