Human voice separation method and device, equipment and storage medium
By performing segmentation processing and vocalprint feature extraction for the voice to be processed, combined with the query and merging operation of the voiceprint index library, the problem of multi-person voice separation is solved, and efficient voice separation effect is achieved.
Patent Information
- Application Number
- CN202411709043.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-25
- Publication Date
- 2025-03-14
AI Technical Summary
The difficulty in effectively separating the voice of multiple people speaking has led to difficulties in voice processing in finance and other fields.
By processing the pending voice segmentation, the voiceprint features of each voice segment are extracted, and the pre-configured voiceprint index library is called to determine the voiceprint index number corresponding to each voiceprint feature, and the voice segments with the same index number are merged to achieve separation of multi-person voice.
It realizes effective separation of multi-person voice, improves the accuracy and efficiency of voice processing, and is suitable for voice processing needs in finance and other fields.
Smart Images
Figure CN119626249A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the fields of speech processing and finance, and in particular to a method, device, equipment and storage medium for separating human voice. Background Art
[0002] In various fields, it is necessary to separate the human voice from the speech in many scenarios. For example, in the scenarios of voiceprint registration for telephone customers in various industries such as banking and securities, there is often more than one customer speaking on the phone, and the voices of multiple people need to be separated. For another example, in the multi-person conference speech recognition task, since the recorded speech contains multiple people speaking in turn, the human voice in the recorded speech also needs to be separated. At present, how to effectively separate the voices of different people in a speech with multiple people speaking has become an urgent problem to be solved. Summary of the invention
[0003] The present application provides a method, device, equipment and storage medium for separating human voices, aiming to separate human voices of multiple people.
[0004] To achieve the above object, the present application provides a method for separating human voice, the method comprising:
[0005] Segmenting the speech to be processed to obtain speech segments, wherein the speech to be processed includes voices of multiple speakers;
[0006] Extracting voiceprints from each of the speech segments to obtain voiceprint features corresponding to each of the speech segments;
[0007] Calling a preconfigured voiceprint index library to determine the voiceprint index number corresponding to each of the voiceprint features, wherein the voiceprint index number is mapped one-to-one with the voiceprint feature in the voiceprint index library;
[0008] The voice segments with voiceprint features corresponding to the same voiceprint index number are merged to obtain the voice of each speaker.
[0009] In addition, to achieve the above-mentioned purpose, the present application also provides a human voice separation device, the human voice separation device comprising:
[0010] A speech processing module, used for segmenting the speech to be processed to obtain speech segments, wherein the speech to be processed includes the voices of multiple speakers;
[0011] A voiceprint extraction module, used to extract voiceprints from each of the voice segments to obtain voiceprint features corresponding to each of the voice segments;
[0012] A voiceprint query module, used to call a pre-configured voiceprint index library to determine the voiceprint index number corresponding to each voiceprint feature, in which the voiceprint index number is mapped one-to-one with the voiceprint feature;
[0013] The voice separation module is used to merge the voice segments with voiceprint features corresponding to the same voiceprint index number to obtain the voice of each speaker.
[0014] In addition, to achieve the above-mentioned purpose, the present application also provides a computer device, the computer device comprising a memory and a processor;
[0015] The memory is used to store computer programs;
[0016] The processor is used to execute the computer program and implement the above-mentioned human voice separation method when executing the computer program.
[0017] In addition, to achieve the above-mentioned purpose, the present application also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned human voice separation method are implemented.
[0018] The present application discloses a method, apparatus, device and storage medium for voice separation, which obtains voice segments by segmenting the speech to be processed, wherein the speech to be processed includes the voices of multiple speakers, performs voiceprint extraction on each speech segment, obtains the voiceprint feature corresponding to each speech segment, then calls a preconfigured voiceprint index library, determines the voiceprint index number corresponding to each voiceprint feature, maps the voiceprint index number to the voiceprint feature in the voiceprint index library one by one, merges the speech segments with the voiceprint feature corresponding to the same voiceprint index number, obtains the voice of each speaker, and thus realizes voice separation of multiple speakers. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0020] Figure 1 is a schematic flow chart of the steps of a method for separating human voices provided in an embodiment of the present application;
[0021] Figure 2 This is a schematic flow chart of the steps of segmenting the speech to be processed and obtaining speech segments provided by an embodiment of the present application;
[0022] Figure 3This is a schematic flow chart of the steps of extracting voiceprint from each speech segment to obtain the voiceprint features corresponding to each speech segment, provided in an embodiment of the present application;
[0023] Figure 4 This is another schematic flow chart of the steps of extracting voiceprint from each speech segment to obtain the voiceprint features corresponding to each speech segment provided by an embodiment of the present application;
[0024] Figure 5 is a schematic flow chart of the steps of another method for separating human voices provided in an embodiment of the present application;
[0025] Figure 6 It is a schematic diagram of a process of generating a voiceprint index library provided in an embodiment of the present application;
[0026] Figure 7 It is a schematic diagram of a flow chart of separating the voices of multiple speakers provided in an embodiment of the present application;
[0027] Figure 8 is a schematic block diagram of a human voice separation device provided in an embodiment of the present application;
[0028] Fig. 9 It is a schematic block diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0029] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0030] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.
[0031] It should be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.
[0032] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.
[0033] The embodiments of the present application provide a method, apparatus, device and storage medium for separating human voices of multiple people.
[0034] See also Figure 1 , Figure 1 1 is a flow chart of a method for voice separation provided in an embodiment of the present application. The method can be applied to a computer device, and the present application does not limit the application scenario of the method. The following takes the application of the method for voice separation in a computer device as an example to introduce the method for voice separation in detail.
[0035] like Figure 1 As shown, the human voice separation method specifically includes steps S101 to S104.
[0036] S101, segmenting the speech to be processed to obtain speech segments, wherein the speech to be processed includes voices of multiple speakers.
[0037] Taking the scenario of voiceprint registration for bank customers in the financial field as an example, the voice of the customer's call is obtained, such as recording the phone call with the customer, obtaining the customer's voice, and using the voice as the voice to be processed. The voice to be processed includes the voices of multiple speakers. For example, the voice to be processed includes the voices of multiple people such as customers and customer service.
[0038] In order to separate the human voice from the speech to be processed, the speech to be processed is first segmented to obtain multiple speech segments. Exemplarily, the speech to be processed is segmented according to a set speech duration to obtain multiple speech segments.
[0039] In some embodiments, Figure 2 As shown, step S101 may include sub-step S1011 and sub-step S1012.
[0040] S1011, performing voice recognition on the speech to be processed, and marking the speech to be processed with a sound type label according to a preset duration, wherein the sound type label includes a voice label and a noise label;
[0041] S1012: Connect consecutive speech segments of the preset duration corresponding to the same sound type label to obtain the speech segments.
[0042] Exemplarily, a preset duration is pre-set, for example, the preset duration is pre-set to 30ms. It should be noted that the preset duration can be flexibly set according to actual conditions, and is not specifically limited in this application.
[0043] After obtaining the speech to be processed, human voice recognition is performed on the speech to be processed. For example, the speech to be processed can be processed by human voice recognition through WebRTC (Web Real-Time Communications) VAD (Voice Activity Detection), and the speech to be processed is marked with a corresponding sound type label according to a preset duration according to the human voice recognition result, so as to obtain multiple speech segments of preset duration marked with corresponding sound type labels. Among them, the sound type label includes a human voice label, a noise label, etc.
[0044] For example, assuming that the speech to be processed is 30 seconds long and the preset duration is 30 ms, the 30 seconds of speech to be processed is segmented and marked with corresponding voice labels or noise labels every 30 ms. In this way, 1000 30 ms speech segments marked with corresponding voice labels or noise labels are obtained.
[0045] Afterwards, the continuous speech segments of preset duration corresponding to the same vocal label or noise label are spliced to obtain the spliced speech segments. For example, for 1000 30ms speech segments marked with corresponding vocal labels or noise labels, the continuous 30ms speech segments of the same vocal label or noise label are spliced to obtain the spliced speech segments.
[0046] S102: extract voiceprint from each of the speech segments to obtain voiceprint features corresponding to each of the speech segments.
[0047] After obtaining each speech segment, voiceprint extraction is performed on each speech segment to obtain the voiceprint feature corresponding to each speech segment. Exemplarily, voiceprint extraction can be performed on the speech segment using a corresponding algorithm model to obtain the voiceprint feature corresponding to each speech segment.
[0048] In some embodiments, Figure 3 As shown, step S102 may include sub-step S1021 and sub-step S1022.
[0049] S1021, deleting the speech segment corresponding to the noise label, and retaining the speech segment corresponding to the voice label;
[0050] S1022: Perform voiceprint extraction on the speech segment corresponding to the human voice tag to obtain the voiceprint feature of the speech segment corresponding to the human voice tag.
[0051] The obtained speech segments include speech segments corresponding to noise labels and speech segments corresponding to human voice labels. The speech segments corresponding to noise labels are invalid interference information. The speech segments corresponding to noise labels are deleted, and only the speech segments corresponding to human voice labels are retained.
[0052] After that, it is only necessary to extract the voiceprint of the speech segment corresponding to the retained vocal label to obtain the voiceprint features of the speech segment corresponding to the vocal label.
[0053] In some embodiments, Figure 4 As shown, step S102 may include sub-step S1023 and sub-step S1024.
[0054] S1023, obtaining the Mel-frequency cepstral coefficient MFCC feature value corresponding to each of the speech segments;
[0055] S1024: Perform voiceprint extraction on the MFCC feature values to obtain the voiceprint features corresponding to each of the speech segments.
[0056] For each speech segment, speech recognition processing is performed to obtain the MFCC (Mel-scale Frequency Cepstral Coefficients) feature value corresponding to each speech segment. Exemplarily, the parameter values for MFCC feature value extraction of the batch script are pre-configured, for example, the parameter values are configured using the empirical values generally used in speech recognition, and then the MFCC feature value extraction is performed through the configured batch script to obtain the MFCC feature value corresponding to each speech segment. After that, the MFCC feature value corresponding to each speech segment is subjected to voiceprint extraction processing to obtain the voiceprint features corresponding to each speech segment.
[0057] In some embodiments, performing voiceprint extraction on the MFCC feature values to obtain the voiceprint features corresponding to each of the speech segments includes:
[0058] Input the MFCC feature value into a trained vector feature extraction model, and output the multidimensional vector feature corresponding to the MFCC feature value;
[0059] The multi-dimensional vector features are input into a trained dimensionality reduction model for dimensionality reduction processing to obtain low-dimensional vector features, and the low-dimensional vector features are determined as the voiceprint features.
[0060] Exemplarily, vector feature extraction models and dimensionality reduction models are modeled by algorithms such as CNN (Convolutional Neural Networks) and SVM (Support Vector Machine). The vector feature extraction model and dimensionality reduction model are trained by a large amount of human voice sample data to obtain trained vector feature extraction models and dimensionality reduction models. Among them, the vector feature extraction model includes but is not limited to the I-Vector model. The vector feature extraction model can be used to extract multidimensional vector features corresponding to MFCC eigenvalues. For example, the I-Vector model can be used to extract I-Vector features corresponding to MFCC eigenvalues. The dimensionality reduction model can be used to reduce the dimension of multidimensional vector features, for example, to reduce the dimension of I-Vector features.
[0061] After obtaining the MFCC feature values corresponding to each speech segment, the MFCC feature values corresponding to each speech segment are input into a trained vector feature extraction model, and the multi-dimensional vector features corresponding to each MFCC feature value are output. For example, the MFCC feature values corresponding to each speech segment are input into a trained I-Vector model, and the multi-dimensional I-Vector features corresponding to each MFCC feature value are output.
[0062] Afterwards, the obtained multi-dimensional vector features are input into the trained dimensionality reduction model, and the multi-dimensional vector features are subjected to dimensionality reduction processing to obtain corresponding low-dimensional vector features, and the obtained low-dimensional vector features are determined as the voiceprint features corresponding to the corresponding speech segment. For example, the multi-dimensional I-Vector features are input into the trained dimensionality reduction model to reduce the dimensionality of the multi-dimensional I-Vector features, obtain the corresponding low-dimensional I-Vector features, and determine the low-dimensional I-Vector features as the voiceprint features corresponding to the corresponding speech segment.
[0063] In some embodiments, Figure 5 As shown, step S105 to step S107 may be included before step S101.
[0064] S105, obtaining multiple human voices, wherein each human voice contains number identification information;
[0065] S106, extracting voiceprints from each of the human voices to obtain voiceprint features corresponding to each of the human voices;
[0066] S107: Use the number identification information corresponding to each of the human voices as a voiceprint index number, and associate the voiceprint feature corresponding to each of the human voices with the voiceprint index number to generate the voiceprint index library.
[0067] In order to separate the human voice, a voiceprint index library is pre-configured. In the voiceprint index library, the voiceprint index number is mapped one-to-one with the voiceprint feature, that is, each voiceprint feature in the voiceprint index library corresponds to a unique voiceprint index number.
[0068] First, a batch of human voices is obtained, wherein each human voice contains number identification information, and the number identification information includes but is not limited to an ID identification. Each human voice obtained is a relatively clean original human voice.
[0069] After obtaining these human voices, corresponding voice processing is performed on these human voices. Exemplarily, these human voices are subjected to unified format conversion, duration interception and other processing. For example, the format of the human voice is converted into WAV format, such as the sampling rate is 8000Hz, and the compression format is uncompressed. And based on the VAD (Voice Activity Detection) algorithm, the silent part in the human voice is removed, and then the corresponding duration of the processed human voice is intercepted, such as intercepting the first 6s of the processed human voice, and saving it.
[0070] Then, voiceprint extraction is performed on the processed human voice to obtain voiceprint features corresponding to each human voice. Exemplarily, the MFCC feature values corresponding to each processed human voice are obtained, and then the MFCC feature values corresponding to each processed human voice are input into a trained vector feature extraction model, and the multi-dimensional vector features corresponding to the MFCC feature values of each processed human voice are output. For example, Figure 6 As shown, the MFCC feature values corresponding to each processed human voice are input into the trained I-Vector model, and the multi-dimensional I-Vector features corresponding to the MFCC feature values of each processed human voice are output.
[0071] After that, the multi-dimensional vector features corresponding to the MFCC feature values of the processed human voice are input into the trained dimensionality reduction model for dimensionality reduction processing to obtain the corresponding low-dimensional vector features, and the obtained low-dimensional vector features are determined as the voiceprint features corresponding to the corresponding human voice. Figure 6 As shown, the multi-dimensional I-Vector features corresponding to the MFCC feature values of each processed human voice are input into the trained dimensionality reduction model, the dimension of the multi-dimensional I-Vector features is reduced, the corresponding low-dimensional I-Vector features are obtained, and the low-dimensional I-Vector features are determined as the voiceprint features corresponding to the corresponding human voice.
[0072] Based on the voiceprint features corresponding to each human voice, and the number identification information corresponding to each human voice, such as the ID identification, the number identification information is used as the voiceprint index number, the voiceprint features corresponding to each human voice are associated with the voiceprint index number, and a voiceprint index library is generated. The voiceprint index library is used for subsequent voice separation of multiple people. For example, the ID identification corresponding to each human voice is used as the voiceprint index number, and the low-dimensional I-Vector feature corresponding to each human voice is associated with the ID identification to generate a voiceprint index library.
[0073] S103, calling a pre-configured voiceprint index library to determine the voiceprint index number corresponding to each voiceprint feature, wherein in the voiceprint index library, the voiceprint index number is mapped one-to-one with the voiceprint feature.
[0074] After obtaining the voiceprint features of each voice segment corresponding to the speech to be processed, the voiceprint index library is called, and the voiceprint index library is searched based on the voiceprint features of each voice segment, and the voiceprint index number associated with the voiceprint features that match the voiceprint features of each voice segment in the voiceprint index library is determined as the voiceprint index number corresponding to the voiceprint features of each voice segment. It can be understood that since the same person may speak more than one voice segment, it is very likely that the voiceprint features of two or more voice segments correspond to the same voiceprint index number, that is, one voiceprint index number corresponds to multiple voice segments.
[0075] Moreover, for the noise fragments, when searching in the voiceprint index library, the voiceprint index number associated with the matching voiceprint feature will not be retrieved, so the noise fragments are excluded.
[0076] Exemplarily, multiple voice segments corresponding to the same voiceprint index number may be grouped together to obtain a voice segment group corresponding to the voiceprint index number. For example, the voice segment group corresponding to each id identifier is determined.
[0077] S104: merge the voiceprint features of the voice segments corresponding to the same voiceprint index number to obtain the voice of each speaker.
[0078] After determining the voiceprint index number corresponding to the voiceprint feature of each voice segment, the voice segments with voiceprint features corresponding to the same voiceprint index number are merged. Each merged voice segment corresponds to the same speaker's voice, thereby achieving the separation of the voices of multiple speakers in the speech to be processed.
[0079] In the scenario of voiceprint registration for bank customers in the financial field, after voice separation is performed in the above manner, subsequent voiceprint registration operations for bank customers can be performed.
[0080] For example, Figure 7As shown, the process of separating the voices of multiple speakers is as follows:
[0081] The first step is to obtain a speech to be processed, where the speech to be processed includes the voices of multiple speakers;
[0082] The second step is to process the speech to be processed based on WebRtcVad, label the segments with human voice or noise, and segment the speech to be processed;
[0083] The third step is to splice the continuous human voice or noise segments together to obtain individual speech segments;
[0084] The fourth step is to search and query the voiceprint index library for each voice segment in turn to determine the voiceprint index number corresponding to each voice segment, such as an ID identifier;
[0085] Step 5: concatenate the voice clips corresponding to the same ID to obtain the result of voice separation;
[0086] Step 6: Return the result.
[0087] Through the pre-configured voiceprint index library, the voiceprint index library can be used for search during the voice separation process. Even at a large data scale, matching voiceprint features can be quickly found to achieve voice separation. Moreover, the voiceprint index library is used to quickly search for human voices and filter out noise, retaining only the required human voices, which also improves the accuracy of voice separation.
[0088] In some embodiments, after generating the voiceprint index library, the following steps are included:
[0089] A new human voice is obtained, and the voiceprint index library is updated according to the new human voice.
[0090] By continuously adding new human voices, the new human voices also contain number identification information, such as ID identification, and the new human voices are also relatively clean original human voices.
[0091] The same processing is performed on the new human voice to obtain the voiceprint features corresponding to the new human voice, the number identification information corresponding to the new human voice is used as the voiceprint index number, the voiceprint features corresponding to the new human voice are associated with the voiceprint index number, and saved to the voiceprint index library. The original voiceprint index library is expanded and updated to improve the voiceprint index library, thereby further improving the accuracy of voice separation.
[0092] In the above embodiment, the speech to be processed is segmented to obtain speech segments, wherein the speech to be processed includes voices of multiple speakers, voiceprint extraction is performed on each speech segment, and voiceprint features corresponding to each speech segment are obtained. Then, a preconfigured voiceprint index library is called to determine the voiceprint index number corresponding to each voiceprint feature. In the voiceprint index library, the voiceprint index number is mapped one-to-one with the voiceprint feature, and the speech segments with voiceprint features corresponding to the same voiceprint index number are merged to obtain the voice of each speaker, thereby realizing voice separation of multiple speakers.
[0093] See also Figure 8 , Figure 8 It is a schematic block diagram of a human voice separation device provided in an embodiment of the present application. The human voice separation device can be configured in a computer device to execute the aforementioned human voice separation method.
[0094] like Figure 7 As shown, the human voice separation device 1000 includes: a speech processing module 1001, a voiceprint extraction module 1002, a voiceprint query module 1003, and a human voice separation module 1004.
[0095] The speech processing module 1001 is used to process the speech to be processed in segments to obtain speech segments, wherein the speech to be processed includes the voices of multiple speakers;
[0096] A voiceprint extraction module 1002 is used to extract voiceprints from each of the speech segments to obtain voiceprint features corresponding to each of the speech segments;
[0097] The voiceprint query module 1003 is used to call a pre-configured voiceprint index library to determine the voiceprint index number corresponding to each voiceprint feature. In the voiceprint index library, the voiceprint index number is mapped one-to-one with the voiceprint feature.
[0098] The voice separation module 1004 is used to merge the speech segments with voiceprint features corresponding to the same voiceprint index number to obtain the voice of each speaker.
[0099] In one embodiment, the image acquisition module 1001 is further used for:
[0100] Performing voice recognition on the speech to be processed, and marking the speech to be processed with a sound type label according to a preset duration, wherein the sound type label includes a voice label and a noise label;
[0101] The continuous speech segments of the preset duration corresponding to the same sound type label are spliced to obtain the speech fragment.
[0102] In one embodiment, the voiceprint extraction module 1002 is further used to:
[0103] The speech segment corresponding to the noise label is deleted, and the speech segment corresponding to the human voice label is retained;
[0104] Voiceprint extraction is performed on the speech segment corresponding to the human voice tag to obtain the voiceprint feature of the speech segment corresponding to the human voice tag.
[0105] In one embodiment, the voiceprint extraction module 1002 is further used to:
[0106] Obtain the Mel-frequency cepstral coefficient MFCC feature value corresponding to each of the speech segments;
[0107] Perform voiceprint extraction on the MFCC feature values to obtain the voiceprint features corresponding to each of the speech segments.
[0108] In one embodiment, the voiceprint extraction module 1002 is further used to:
[0109] Input the MFCC feature value into a trained vector feature extraction model, and output the multidimensional vector feature corresponding to the MFCC feature value;
[0110] The multi-dimensional vector features are input into a trained dimensionality reduction model for dimensionality reduction processing to obtain low-dimensional vector features, and the low-dimensional vector features are determined as the voiceprint features.
[0111] In one embodiment, the human voice separation device 1000 further includes a sample acquisition module and an index library generation module, wherein:
[0112] The sample acquisition module is used to acquire multiple human voices, wherein each human voice contains number identification information;
[0113] The voiceprint extraction module 1002 is further used to extract the voiceprint of each of the human voices to obtain the voiceprint features corresponding to each of the human voices;
[0114] The index library generation module is used to use the number identification information corresponding to each of the human voices as the voiceprint index number, and associate the voiceprint features corresponding to each of the human voices with the voiceprint index number to generate the voiceprint index library.
[0115] In one embodiment, the human voice separation device 1000 further includes:
[0116] The index library updating module is used to obtain new human voice and update the voiceprint index library according to the new human voice.
[0117] Among them, each module in the above-mentioned human voice separation device 1000 corresponds to each step in the above-mentioned human voice separation method embodiment, and its function and implementation process will not be repeated here one by one.
[0118] The method and apparatus of the present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.
[0119] Exemplarily, the above method and apparatus may be implemented in the form of a computer program. The computer program may be implemented in the form of a computer program. Fig. 9 Runs on the computer device shown.
[0120] See also Fig. 9 , Fig. 9 It is a schematic block diagram of the structure of a computer device provided in an embodiment of the present application.
[0121] See also Fig. 9 The computer device includes a processor and a memory connected via a system bus, wherein the memory may include a non-volatile storage medium and an internal memory.
[0122] The processor is used to provide computing and control capabilities and support the operation of the entire computer equipment.
[0123] The internal memory provides an environment for the operation of the computer program in the non-volatile storage medium. When the computer program is executed by the processor, the processor can execute any human voice separation method.
[0124] It should be understood that the processor may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.
[0125] In one embodiment, the processor is used to run a computer program stored in the memory to implement the following steps:
[0126] Segmenting the speech to be processed to obtain speech segments, wherein the speech to be processed includes voices of multiple speakers;
[0127] Extracting voiceprints from each of the speech segments to obtain voiceprint features corresponding to each of the speech segments;
[0128] Calling a preconfigured voiceprint index library to determine the voiceprint index number corresponding to each of the voiceprint features, wherein the voiceprint index number is mapped one-to-one with the voiceprint feature in the voiceprint index library;
[0129] The speech segments with voiceprint features corresponding to the same voiceprint index number are merged to obtain the voice of each speaker.
[0130] In one embodiment, when the processor implements the segmentation processing of the speech to be processed to obtain the speech segments, it is used to implement:
[0131] Performing voice recognition on the speech to be processed, and marking the speech to be processed with a sound type label according to a preset duration, wherein the sound type label includes a voice label and a noise label;
[0132] The continuous speech segments of the preset duration corresponding to the same sound type label are spliced to obtain the speech fragment.
[0133] In one embodiment, when the processor extracts the voiceprint from each of the voice segments to obtain the voiceprint features corresponding to each of the voice segments, it is configured to implement:
[0134] The speech segment corresponding to the noise label is deleted, and the speech segment corresponding to the human voice label is retained;
[0135] Voiceprint extraction is performed on the speech segment corresponding to the human voice tag to obtain the voiceprint feature of the speech segment corresponding to the human voice tag.
[0136] In one embodiment, when the processor extracts the voiceprint from each of the voice segments to obtain the voiceprint features corresponding to each of the voice segments, it is configured to implement:
[0137] Obtain the Mel-frequency cepstral coefficient MFCC feature value corresponding to each of the speech segments;
[0138] Perform voiceprint extraction on the MFCC feature values to obtain the voiceprint features corresponding to each of the speech segments.
[0139] In one embodiment, when the processor implements the voiceprint extraction of the MFCC feature value to obtain the voiceprint feature corresponding to each of the voice segments, it is used to implement:
[0140] Input the MFCC feature value into a trained vector feature extraction model, and output the multidimensional vector feature corresponding to the MFCC feature value;
[0141] The multi-dimensional vector features are input into a trained dimensionality reduction model for dimensionality reduction processing to obtain low-dimensional vector features, and the low-dimensional vector features are determined as the voiceprint features.
[0142] In one embodiment, before implementing the segmentation processing of the to-be-processed speech to obtain speech segments, the processor is used to implement:
[0143] Acquire multiple human voices, wherein each of the human voices contains serial number identification information;
[0144] Extracting voiceprints from each of the human voices to obtain voiceprint features corresponding to each of the human voices;
[0145] The number identification information corresponding to each of the human voices is used as a voiceprint index number, and the voiceprint features corresponding to each of the human voices are associated with the voiceprint index number to generate the voiceprint index library.
[0146] In one embodiment, after generating the voiceprint index library, the processor is configured to:
[0147] A new human voice is obtained, and the voiceprint index library is updated according to the new human voice.
[0148] An embodiment of the present application also provides a computer-readable storage medium.
[0149] The computer-readable storage medium of the present application stores a computer program, and when the computer program is executed by a processor, the steps of the human voice separation method as described above are implemented.
[0150] The computer-readable storage medium may be an internal storage unit of the human voice separation device or computer device described in the aforementioned embodiment, such as a hard disk or memory of the human voice separation device or computer device. The computer-readable storage medium may also be an external storage device of the human voice separation device or computer device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital card (Secure Digital Card, SD Card), a flash card (Flash Card), etc., equipped on the human voice separation device or computer device.
[0151] Furthermore, the computer-readable storage medium may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function, etc.; the data storage area may store data created according to the use of the blockchain node, etc.
[0152] The blockchain referred to in this application is a new application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, encryption algorithm, etc. Blockchain is essentially a decentralized database, a string of data blocks generated by cryptographic methods. Each data block contains a batch of network transaction information, which is used to verify the validity of its information (anti-counterfeiting) and generate the next block. Blockchain can include the underlying blockchain platform, platform product service layer, and application service layer.
[0153] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.
[0154] The above description is only a specific implementation mode of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or substitutions within the technical scope disclosed in the present application, and these modifications or substitutions should be included in the protection scope of the present application.
Claims
1. A method for separating human voice, characterized in that: The human voice separation method comprises: Segmenting the speech to be processed to obtain speech segments, wherein the speech to be processed includes voices of multiple speakers; Extracting voiceprints from each of the speech segments to obtain voiceprint features corresponding to each of the speech segments; Calling a preconfigured voiceprint index library to determine the voiceprint index number corresponding to each of the voiceprint features, wherein the voiceprint index number is mapped one-to-one with the voiceprint feature in the voiceprint index library; The voice segments with voiceprint features corresponding to the same voiceprint index number are merged to obtain the voice of each speaker.
2. The method for separating human voice according to claim 1, wherein: The step of segmenting the speech to be processed to obtain speech segments includes: Performing voice recognition on the speech to be processed, and marking the speech to be processed with a sound type label according to a preset duration, wherein the sound type label includes a voice label and a noise label; The continuous speech segments of the preset duration corresponding to the same sound type label are spliced to obtain the speech fragment.
3. The method for separating human voice according to claim 2, wherein: The extracting voiceprint from each of the voice segments to obtain the voiceprint features corresponding to each of the voice segments includes: The speech segment corresponding to the noise label is deleted, and the speech segment corresponding to the human voice label is retained; Voiceprint extraction is performed on the speech segment corresponding to the human voice tag to obtain the voiceprint feature of the speech segment corresponding to the human voice tag.
4. The method for separating human voice according to claim 1, wherein: The extracting voiceprint from each of the voice segments to obtain the voiceprint features corresponding to each of the voice segments includes: Obtain the Mel-frequency cepstral coefficient MFCC feature value corresponding to each of the speech segments; Perform voiceprint extraction on the MFCC feature values to obtain the voiceprint features corresponding to each of the speech segments.
5. The method for separating human voice according to claim 4, characterized in that: The extracting voiceprint from the MFCC feature value to obtain the voiceprint feature corresponding to each speech segment includes: Input the MFCC feature value into a trained vector feature extraction model, and output the multidimensional vector feature corresponding to the MFCC feature value; The multi-dimensional vector features are input into a trained dimensionality reduction model for dimensionality reduction processing to obtain low-dimensional vector features, and the low-dimensional vector features are determined as the voiceprint features.
6. The method for separating human voice according to any one of claims 1 to 5, characterized in that: The method of segmenting the speech to be processed to obtain speech segments includes: Acquire multiple human voices, wherein each of the human voices contains serial number identification information; Extracting voiceprints from each of the human voices to obtain voiceprint features corresponding to each of the human voices; The number identification information corresponding to each of the human voices is used as a voiceprint index number, and the voiceprint features corresponding to each of the human voices are associated with the voiceprint index number to generate the voiceprint index library.
7. The method for separating human voice according to claim 6, characterized in that: After generating the voiceprint index library, the method includes: A new human voice is obtained, and the voiceprint index library is updated according to the new human voice.
8. A human voice separation device, characterized in that: The human voice separation device comprises: A speech processing module, used for segmenting the speech to be processed to obtain speech segments, wherein the speech to be processed includes the voices of multiple speakers; A voiceprint extraction module, used to extract voiceprints from each of the voice segments to obtain voiceprint features corresponding to each of the voice segments; A voiceprint query module, used to call a pre-configured voiceprint index library to determine the voiceprint index number corresponding to each voiceprint feature, in which the voiceprint index number is mapped one-to-one with the voiceprint feature; The voice separation module is used to merge the voice segments with voiceprint features corresponding to the same voiceprint index number to obtain the voice of each speaker.
9. A computer device, characterized in that: The computer device includes a memory and a processor; The memory is used to store computer programs; The processor is used to execute the computer program and implement the human voice separation method according to any one of claims 1 to 7 when executing the computer program.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the human voice separation method according to any one of claims 1 to 7 are implemented.