Speaker segmentation method, system, terminal and storage medium

Through standard deviation processing and pre-training model analysis, combined with multiple speech processing technologies, the problem of low accuracy of speaker segmentation in the existing technology is solved, and the accuracy and user experience of speech segmentation are improved.

CN115457973BActive Publication Date: 2024-11-05BEIJING UNISOUND INFORMATION TECH CO LTD
View PDF 7 Cites 0 Cited by

Patent Information

Application Number
CN202211085775.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-09-06
Publication Date
2024-11-05
Estimated Expiration
2042-09-06

AI Technical Summary

Technical Problem

The existing speaker segmentation method has low accuracy, resulting in poor overall voice segmentation performance and reduces the user experience.

Method used

By obtaining the feature vector of the speech data to be segmented, performing standard deviation processing and combining it with the initial zero vector, a vector of speakers is generated, inputting the pre-trained model for analysis, determining the number of speakers, and combining speech segmentation, filtering, pre-emphasis, and segmentation Frame, windowing, endpoint detection and time-frequency mask optimization improve the accuracy of speaker segmentation.

Benefits of technology

It effectively eliminates unit and scale differences between feature vectors, improves the accuracy of speaker segmentation, automatically determines the number of speakers, and enhances the performance of voice segmentation.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115457973B_ABST
    Figure CN115457973B_ABST
Patent Text Reader

Abstract

The present invention provides a speaker segmentation method, system, terminal and storage medium, the method comprising: obtaining speech data to be segmented, extracting feature vectors in the speech data to be segmented; performing standard deviation processing on the feature vectors to obtain standard deviation vectors, and combining the standard deviation vectors with an initial zero vector to obtain a speaker number vector; inputting the speaker number vector into a pre-trained speaker analysis model to perform speaker analysis to obtain a speaker number vector, and determining the number of speakers in the speech data to be segmented according to the speaker number vector. The present invention effectively eliminates the influence of unit and scale differences between feature vectors by performing standard deviation processing on feature vectors, thereby improving the accuracy of feature vectors, and effectively enhances the feature of the standard deviation vector by combining the standard deviation vector with an initial zero vector to obtain a speaker number vector, thereby improving the accuracy of speaker segmentation.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of speech processing technology, and in particular to a speaker segmentation method, system, terminal and storage medium. Background Art

[0002] Speaker segmentation technology mainly solves the problem of "who spoke and when". The industry generally uses solutions that combine multiple modules such as voice time segmentation, speaker positioning based on voiceprints, and clustering. Since the joint optimization of multiple modules is relatively complex and cannot handle problems such as sound aliasing well, an overall optimization solution for end-to-end neural networks has emerged.

[0003] Compared with the technical solution of jointly optimizing multiple different modules, the end-to-end speaker segmentation task uses a single-module neural network model to implement the speaker segmentation task, and the multi-classification at the frame level can handle the problem of multi-person aliasing very well. The network has two objective functions: one is to distinguish whether there are several people speaking in each small segment of time (generally referred to as a frame); the other is to determine "which people" are in each small segment of time; by improving the accuracy of these two goals, the performance of speaker segmentation is improved;

[0004] When using existing speaker segmentation methods, the accuracy of speaker segmentation is low, resulting in poor overall speech segmentation performance and reducing the user experience. Summary of the invention

[0005] The purpose of the embodiments of the present invention is to provide a speaker segmentation method, system, terminal and storage medium, aiming to solve the problem of low accuracy of existing speaker segmentation methods.

[0006] The embodiment of the present invention is implemented as follows: a method for speaker segmentation, the method comprising:

[0007] Acquire speech data to be segmented, and extract feature vectors from the speech data to be segmented;

[0008] Performing standard deviation processing on the feature vector to obtain a standard deviation vector, and combining the standard deviation vector with an initial zero vector to obtain a speaker number vector;

[0009] The speaker number vector is input into a pre-trained speaker analysis model to perform speaker analysis to obtain a speaker number vector, and the number of speakers in the speech data to be segmented is determined according to the speaker number vector.

[0010] Furthermore, determining the number of speakers in the speech data to be segmented according to the speaker number vector includes:

[0011] The value in the speaker number vector is matched with a preset value, and the number of successful matches is set as the speaker number.

[0012] Furthermore, before obtaining the voice data to be segmented, the method further includes:

[0013] Performing speech segmentation on the speech data to be segmented to obtain segmented speech, and extracting speech features of the segmented speech respectively;

[0014] The feature similarities between the speech features of the different segmented speech are calculated respectively, and the segmented speech is speech combined according to the feature similarities.

[0015] Furthermore, after determining the number of speakers in the speech data to be segmented according to the speaker number vector, the method further includes:

[0016] Respectively obtaining the phase-sensitive mask of each speaker in the speech data to be segmented, and obtaining the speech spectrum of the speech data to be segmented;

[0017] Determining the speech spectrum of each speaker respectively according to the phase sensitivity mask of each speaker and the speech spectrum of the speech data to be segmented;

[0018] According to the speech spectrum of each speaker, the speech of each speaker in the speech data to be segmented is determined respectively.

[0019] Furthermore, after obtaining the voice data to be segmented, the method further includes:

[0020] Performing filtering processing on the speech data to be segmented, and performing pre-emphasis processing on the speech data to be segmented after the filtering processing;

[0021] The speech data to be segmented after the pre-emphasis processing is subjected to frame processing, and the speech data to be segmented after the frame processing is subjected to speech windowing processing.

[0022] Furthermore, after performing voice windowing processing on the voice data to be segmented after the frame processing, the method further includes:

[0023] Endpoint detection is performed on the speech data to be segmented after the speech windowing process, and denoising is performed on the speech data to be segmented according to the endpoint detection result.

[0024] Furthermore, after determining the spoken voice of each speaker according to the speech spectrum of each speaker, the method further includes:

[0025] The time-frequency masks of the spoken voices are respectively obtained, and the spoken voices are voice optimized according to the time-frequency masks.

[0026] Another object of an embodiment of the present invention is to provide a speaker segmentation system, the system comprising:

[0027] A feature extraction module, used to obtain the speech data to be segmented and extract feature vectors from the speech data to be segmented;

[0028] A vector combination module, used for performing standard deviation processing on the feature vector to obtain a standard deviation vector, and combining the standard deviation vector with an initial zero vector to obtain a speaker number vector;

[0029] The speaker number output module is used to input the speaker number vector into the pre-trained speaker analysis model to perform speaker analysis to obtain the speaker number vector, and determine the number of speakers in the speech data to be segmented according to the speaker number vector.

[0030] Another object of an embodiment of the present invention is to provide a terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.

[0031] Another object of an embodiment of the present invention is to provide a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0032] The embodiments of the present invention perform standard deviation processing on feature vectors, thereby effectively eliminating the influence of unit and scale differences between feature vectors and improving the accuracy of feature vectors. The standard deviation vector is combined with an initial zero vector to obtain a speaker number vector, which effectively enhances the features of the standard deviation vector and improves the accuracy of speaker segmentation. The speaker number vector can be automatically obtained by inputting the speaker number vector into a pre-trained speaker analysis model for speaker analysis. Based on the speaker number vector, the number of speakers in the speech data to be segmented can be effectively determined. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 is a flow chart of a speaker segmentation method provided by a first embodiment of the present invention;

[0034] Figure 2 is a schematic diagram of specific implementation steps of the speaker segmentation method provided by the first embodiment of the present invention;

[0035] Figure 3 is a flow chart of a speaker segmentation method provided by a second embodiment of the present invention;

[0036] Figure 4is a schematic diagram of the structure of a speaker segmentation system provided by a third embodiment of the present invention;

[0037] Figure 5 It is a schematic diagram of the structure of a terminal device provided in the fourth embodiment of the present invention. DETAILED DESCRIPTION

[0038] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0039] In order to illustrate the technical solution of the present invention, a specific embodiment is provided below for illustration.

[0040] Embodiment 1

[0041] See also Figure 1 to Figure 2 , is a flow chart of a speaker segmentation method provided by a first embodiment of the present invention. The speaker segmentation method can be applied to any terminal device or system. The speaker segmentation method comprises the steps of:

[0042] Step S10, obtaining speech data to be segmented, and extracting feature vectors from the speech data to be segmented;

[0043] The feature vector may be extracted by a feature extraction network, and the feature extraction network may be set according to requirements. For example, the feature extraction network may be set as a transformer network or a conformer network, etc.

[0044] Optionally, in this step, before obtaining the voice data to be segmented, the step further includes:

[0045] Performing speech segmentation on the speech data to be segmented to obtain segmented speech, and extracting speech features of the segmented speech respectively;

[0046] The speech data to be segmented is segmented by a preset time length to obtain the segmented speech, and each segmented speech is input into a feature extraction network for feature extraction to obtain the speech feature, and the preset time length can be set according to demand;

[0047] The feature similarities between the speech features of the different segmented speech are calculated respectively, and the segmented speech is speech combined according to the feature similarities.

[0048] Furthermore, after obtaining the voice data to be segmented, the method further includes:

[0049] The voice data to be segmented is subjected to filtering processing, and the voice data to be segmented after filtering processing is subjected to pre-emphasis processing; wherein, by filtering processing the voice data to be segmented, the voice data to be segmented is effectively subjected to the effect of filtering noise;

[0050] The speech data to be segmented after the pre-emphasis processing is framed, and the speech data to be segmented after the framed processing is subjected to speech windowing processing; wherein, by performing frame processing on the speech data to be segmented after the pre-emphasis processing, the accuracy of the speech windowing processing on the speech data to be segmented is effectively improved.

[0051] Furthermore, after performing voice windowing processing on the voice data to be segmented after the frame processing, the method further includes:

[0052] Endpoint detection is performed on the speech data to be segmented after speech windowing processing, and denoising is performed on the speech data to be segmented according to the endpoint detection result; wherein, by performing endpoint detection on the speech data to be segmented after speech windowing processing, the silent data in the speech data to be segmented can be effectively determined based on the endpoint detection result, and by deleting the silent data in the speech data to be segmented, the denoising effect is effectively achieved on the speech data to be segmented, thereby improving the accuracy of the speech data to be segmented.

[0053] Step S20, performing standard deviation processing on the feature vector to obtain a standard deviation vector, and combining the standard deviation vector with an initial zero vector to obtain a speaker number vector;

[0054] Among them, by performing standard deviation processing on the feature vector, the influence of the unit and scale differences between the feature vectors is effectively eliminated, and the accuracy of the feature vector is improved. By combining the standard deviation vector with the initial zero vector, the speaker number vector is obtained, which effectively enhances the features of the standard deviation vector and improves the accuracy of speaker segmentation. Optionally, in the process of performing standard deviation processing on the feature vector, a layer of DNN network is added for dimensional adaptation to further improve the accuracy of the standard deviation vector. Specifically, in this step, the LSTM network is used as a decoder to combine the standard deviation vector and the initial zero vector (Zero Vector) to output the speaker number vector.

[0055] Step S30, inputting the speaker number vector into the pre-trained speaker analysis model to perform speaker analysis to obtain the speaker number vector, and determining the number of speakers in the speech data to be segmented according to the speaker number vector;

[0056] Among them, by inputting the speaker number vector into the pre-trained speaker analysis model for speaker analysis, the speaker number vector can be automatically obtained, and based on the speaker number vector, the number of speakers in the speech data to be segmented can be effectively determined;

[0057] Optionally, in this step, determining the number of speakers in the speech data to be segmented according to the speaker number vector includes:

[0058] The value in the speaker number vector is matched with a preset value, and the number of successful matches is set to the speaker number, wherein the preset value can be set as required, and the dimension of the vector set to be greater than a threshold (generally 0.5) indicates that there is a speaker, and the judgment is terminated when it is detected that the dimension is less than the threshold. Finally, the number of 1s in the speaker number vector is the number of speakers, for example (1,1,1,0) and (1,1,0,0) are vectors of 3 and 2 speaker numbers.

[0059] In this embodiment, by performing standard deviation processing on the feature vector, the influence of the unit and scale differences between the feature vectors is effectively eliminated, and the accuracy of the feature vector is improved. By combining the standard deviation vector with the initial zero vector, a speaker number vector is obtained, which effectively enhances the features of the standard deviation vector and improves the accuracy of speaker segmentation. By inputting the speaker number vector into a pre-trained speaker analysis model for speaker analysis, the speaker number vector can be automatically obtained. Based on the speaker number vector, the number of speakers in the speech data to be segmented can be effectively determined.

[0060] Embodiment 2

[0061] See also Figure 3 , is a flow chart of a speaker segmentation method provided by a second embodiment of the present invention, and this embodiment is used to further refine the steps after step S30 in the first embodiment, including the steps of:

[0062] Step S40, respectively obtaining the phase sensitive mask of each speaker in the speech data to be segmented, and obtaining the speech spectrum of the speech data to be segmented;

[0063] Among them, the phase sensitive mask (PSM) is the ideal amplitude mask (IAM) multiplied by the cosine similarity between the clean speech and the noisy speech;

[0064] Step S50, determining the speech spectrum of each speaker respectively according to the phase sensitivity mask of each speaker and the speech spectrum of the speech data to be segmented;

[0065] Among them, the speech spectrum of each speaker can be effectively determined respectively through the phase sensitive mask of each speaker and the speech spectrum of the speech data to be segmented, and based on the speech spectrum of each speaker, the accuracy of the subsequent speech of each speaker in the speech data to be segmented is improved;

[0066] Step S60, determining the spoken voice of each speaker in the speech data to be segmented according to the speech spectrum of each speaker;

[0067] Optionally, in this step, after determining the spoken voice of each speaker in the voice data to be segmented based on the speech spectrum of each speaker, the step further includes: obtaining a time-frequency mask of each spoken voice, and performing speech optimization on the spoken voice based on the time-frequency mask; wherein a deep learning model is trained based on the time-frequency mask, and the spoken voice is speech optimized based on the trained deep learning model, thereby improving the accuracy of the spoken voice.

[0068] In this embodiment, the speech spectrum of each speaker can be effectively determined respectively through the phase-sensitive mask of each speaker and the speech spectrum of the speech data to be segmented. Based on the speech spectrum of each speaker, the accuracy of the subsequent speaking speech of each speaker in the speech data to be segmented is improved. Based on the speech spectrum of each speaker, the speaking speech of each speaker in the speech data to be segmented can be automatically determined, thereby facilitating the segmentation of each speaking speech in the speech data to be segmented.

[0069] Embodiment 3

[0070] See also Figure 4 , is a schematic diagram of the structure of a speaker segmentation system 100 provided in the third embodiment of the present invention, comprising: a feature extraction module 10, a vector combination module 11 and a speaker number output module 12, wherein:

[0071] The feature extraction module 10 is used to obtain the speech data to be segmented and extract the feature vector in the speech data to be segmented. The feature vector can be extracted by using a feature extraction network, which can be set according to the requirements. For example, the feature extraction network can be set as a transformer network or a conformer network.

[0072] Optionally, the feature extraction module 10 is further used to: perform speech segmentation on the speech data to be segmented to obtain segmented speech, and respectively extract speech features of the segmented speech; wherein, the speech data to be segmented is segmented by a preset time length to obtain the segmented speech, and each segmented speech is input into a feature extraction network for feature extraction to obtain the speech feature, and the preset time length can be set according to demand;

[0073] The feature similarities between the speech features of the different segmented speech are calculated respectively, and the segmented speech is speech combined according to the feature similarities.

[0074] Furthermore, the feature extraction module 10 is also used to: filter the speech data to be segmented, and perform pre-emphasis processing on the speech data to be segmented after filtering; wherein, by filtering the speech data to be segmented, the speech data to be segmented is effectively filtered to filter noise;

[0075] The speech data to be segmented after the pre-emphasis processing is framed, and the speech data to be segmented after the framed processing is subjected to speech windowing processing; wherein, by performing frame processing on the speech data to be segmented after the pre-emphasis processing, the accuracy of the speech windowing processing on the speech data to be segmented is effectively improved.

[0076] Furthermore, the feature extraction module 10 is also used to: perform endpoint detection on the speech data to be segmented after the speech windowing processing, and denoise the speech data to be segmented according to the endpoint detection result; wherein, by performing endpoint detection on the speech data to be segmented after the speech windowing processing, the silent data in the speech data to be segmented can be effectively determined based on the endpoint detection result, and by deleting the silent data in the speech data to be segmented, the speech data to be segmented is effectively denoised, thereby improving the accuracy of the speech data to be segmented.

[0077] The vector combination module 11 is used to perform standard deviation processing on the feature vector to obtain a standard deviation vector, and combine the standard deviation vector with an initial zero vector to obtain a speaker number vector. Among them, by performing standard deviation processing on the feature vector, the influence of unit and scale differences between feature vectors is effectively eliminated, and the accuracy of the feature vector is improved. By combining the standard deviation vector with the initial zero vector to obtain the speaker number vector, the standard deviation vector is effectively enhanced, and the accuracy of speaker segmentation is improved. Optionally, in the process of performing standard deviation processing on the feature vector, a layer of DNN network is added for dimensional adaptation to further improve the accuracy of the standard deviation vector; specifically, in this step, the LSTM network is used as a decoder to combine the standard deviation vector and the initial zero vector (Zero Vector) to output the speaker number vector.

[0078] The speaker number output module 12 is used to input the speaker number vector into the pre-trained speaker analysis model for speaker analysis to obtain the speaker number vector, and determine the number of speakers in the speech data to be segmented according to the speaker number vector. By inputting the speaker number vector into the pre-trained speaker analysis model for speaker analysis, the speaker number vector can be automatically obtained, and based on the speaker number vector, the number of speakers in the speech data to be segmented can be effectively determined.

[0079] Optionally, the speaker number output module 12 is further used to: match the value in the speaker number vector with a preset value, and set the number of successful matches to the speaker number; wherein the preset value can be set as required, and the dimension of the vector set to be greater than a threshold (generally 0.5) indicates that there is a speaker, and the judgment is terminated when it is detected that the dimension is less than the threshold. Finally, the number of 1s in the speaker number vector is the number of speakers, for example (1,1,1,0) and (1,1,0,0) are vectors of 3 and 2 speaker numbers.

[0080] Furthermore, the speaker number output module 12 is further used to: respectively obtain the phase sensitive mask of each speaker in the speech data to be segmented, and obtain the speech spectrum of the speech data to be segmented;

[0081] Determining the speech spectrum of each speaker respectively according to the phase sensitivity mask of each speaker and the speech spectrum of the speech data to be segmented;

[0082] According to the speech spectrum of each speaker, the speech of each speaker in the speech data to be segmented is determined respectively.

[0083] Furthermore, the speaker number output module 12 is further configured to respectively obtain a time-frequency mask of each spoken voice, and perform voice optimization on the spoken voice according to the time-frequency mask.

[0084] In this embodiment, by performing standard deviation processing on the feature vector, the influence of the unit and scale differences between the feature vectors is effectively eliminated, and the accuracy of the feature vector is improved. By combining the standard deviation vector with the initial zero vector, a speaker number vector is obtained, which effectively enhances the features of the standard deviation vector and improves the accuracy of speaker segmentation. By inputting the speaker number vector into a pre-trained speaker analysis model for speaker analysis, the speaker number vector can be automatically obtained. Based on the speaker number vector, the number of speakers in the speech data to be segmented can be effectively determined.

[0085] Embodiment 4

[0086] Figure 5 2 is a block diagram of a terminal device 2 provided in the fourth embodiment of the present application. Figure 5 As shown, the terminal device 2 of this embodiment includes: a processor 20, a memory 21, and a computer program 22 stored in the memory 21 and executable on the processor 20, such as a program of a speaker segmentation method. When the processor 20 executes the computer program 22, the steps in each embodiment of the above-mentioned speaker segmentation method are implemented.

[0087] Exemplarily, the computer program 22 may be divided into one or more modules, which are stored in the memory 21 and executed by the processor 20 to complete the present application. The one or more modules may be a series of computer program instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program 22 in the terminal device 2. The terminal device may include, but is not limited to, a processor 20 and a memory 21.

[0088] The processor 20 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0089] The memory 21 may be an internal storage unit of the terminal device 2, such as a hard disk or memory of the terminal device 2. The memory 21 may also be an external storage device of the terminal device 2, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the terminal device 2. Further, the memory 21 may also include both an internal storage unit and an external storage device of the terminal device 2. The memory 21 is used to store the computer program and other programs and data required by the terminal device. The memory 21 may also be used to temporarily store data that has been output or is to be output.

[0090] In addition, each functional module in each embodiment of the present application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of software functional unit.

[0091] If the integrated module is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Among them, the computer-readable storage medium can be non-volatile or volatile. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable storage medium may include: any entity or device capable of carrying computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electrical carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in computer-readable storage media can be appropriately increased or decreased according to the requirements of legislation and patent practices in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practices, computer-readable storage media do not include electrical carrier signals and telecommunication signals.

[0092] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A speaker segmentation method, characterized in that: The method comprises: Acquire speech data to be segmented, and extract feature vectors from the speech data to be segmented; Performing standard deviation processing on the feature vector to obtain a standard deviation vector, and using an LSTM network as a decoder to combine the standard deviation vector and an initial zero vector to output a speaker number vector; The speaker number vector is input into a pre-trained speaker analysis model to perform speaker analysis to obtain a speaker number vector, and the number of speakers in the speech data to be segmented is determined according to the speaker number vector.

2. The speaker segmentation method according to claim 1, characterized in that: The step of determining the number of speakers in the speech data to be segmented according to the speaker number vector includes: The value in the speaker number vector is matched with a preset value, and the number of successful matches is set as the speaker number.

3. The speaker segmentation method according to claim 1, characterized in that: Before obtaining the voice data to be segmented, the method further includes: Performing speech segmentation on the speech data to be segmented to obtain segmented speech, and extracting speech features of the segmented speech respectively; The feature similarities between the speech features of the different segmented speech are calculated respectively, and the segmented speech is speech combined according to the feature similarities.

4. The speaker segmentation method according to claim 1, wherein: After determining the number of speakers in the speech data to be segmented according to the speaker number vector, the method further includes: Respectively obtaining the phase-sensitive mask of each speaker in the speech data to be segmented, and obtaining the speech spectrum of the speech data to be segmented; Determining the speech spectrum of each speaker respectively according to the phase sensitivity mask of each speaker and the speech spectrum of the speech data to be segmented; According to the speech spectrum of each speaker, the speech of each speaker in the speech data to be segmented is determined respectively.

5. The speaker segmentation method according to claim 1, characterized in that: After obtaining the voice data to be segmented, the method further includes: Performing filtering processing on the speech data to be segmented, and performing pre-emphasis processing on the speech data to be segmented after the filtering processing; The speech data to be segmented after the pre-emphasis processing is subjected to frame processing, and the speech data to be segmented after the frame processing is subjected to speech windowing processing.

6. The speaker segmentation method according to claim 5, characterized in that: After performing voice windowing processing on the voice data to be segmented after the frame processing, the method further comprises: Endpoint detection is performed on the speech data to be segmented after the speech windowing process, and denoising is performed on the speech data to be segmented according to the endpoint detection result.

7. The speaker segmentation method according to claim 4, characterized in that: After determining the spoken voice of each speaker according to the speech spectrum of each speaker, the method further comprises: The time-frequency masks of the spoken voices are respectively obtained, and the spoken voices are voice optimized according to the time-frequency masks.

8. A speaker segmentation system, characterized in that: The system comprises: A feature extraction module, used to obtain the speech data to be segmented and extract feature vectors from the speech data to be segmented; A vector combination module is used to perform standard deviation processing on the feature vector to obtain a standard deviation vector, and use an LSTM network as a decoder to combine the standard deviation vector and an initial zero vector to output a speaker number vector; The speaker number output module is used to input the speaker number vector into the pre-trained speaker analysis model to perform speaker analysis to obtain the speaker number vector, and determine the number of speakers in the speech data to be segmented according to the speaker number vector.

9. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Citation Information

Patent Citations

  • Method for judging number of speakers

    CN107393527A

  • Voice model training method, speaker identification method, apparatus and device, and medium

    CN108777146A

  • Voice enhancing method and device, model training method and device, and computer device

    CN110176243A

  • Speaker vector regularization method and device, electronic equipment and storage medium

    CN111462762A

  • Speaker recognition system and method based on composite acoustic features and low-rank decomposition TDNN

    CN112331216A