Speaker segmentation and clustering method, device, storage medium and electronic device

Abstract

The present invention discloses a speaker segmentation and clustering method, device, storage medium and electronic device. The speaker segmentation and clustering method includes: using a speech recognition engine to segment input speech, which can more accurately distinguish speech and non-speech, improve the problem of high error rate of using VAD to segment speech, thereby improving the accuracy of the subsequent clustering module, thereby reducing the error rate of the entire speaker segmentation system, so as to at least solve the technical problem of low accuracy of speaker segmentation and clustering in the prior art.

Description

Technical Field

[0001] The present invention relates to the field of speech processing, and in particular to a speaker segmentation and clustering method, device, storage medium and electronic device. Background Art

[0002] Speaker Diarization is a technology that distinguishes the voices of different speakers according to their identities, solving the problem of "who spoke and when".

[0003] At present, the mainstream speaker segmentation system process is: input speech -> front-end processing -> voice endpoint detection (VAD / SAD) to obtain speech segments -> extract speaker embedding information from the detected speech segments -> cluster the embeddings to obtain the final clustering results -> post-processing (optional).

[0004] The high error rate of voice endpoint detection (VAD) leads to a high error rate in the final clustering, including miss error, false alarm, confusion, etc. If there is no pause between different speakers during a conversation, that is, there are at least two speakers in a segment, then the clustering result of the segment is definitely wrong. In the case of strong noise, VAD can easily detect noise as speech, which results in a false alarm error. If the energy of the speech is very low, VAD may not be able to detect the speech, which results in a miss error.

[0005] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0006] The embodiments of the present invention provide a speaker segmentation and clustering method, device, storage medium and electronic device to at least solve the technical problem of low accuracy of speaker segmentation and clustering in the prior art.

[0007] According to one aspect of an embodiment of the present invention, a speaker segmentation and clustering method is provided, comprising: performing speech recognition on a target speech using a speech recognition engine to obtain a recognized text result and a start time and an end time of each word in the text; finding a demarcation point between speech and non-speech based on the start time and the end time, and determining speech segments included in the target speech; inputting the speech segments into a speech clustering model, clustering each speech segment, obtaining local clustered speech segments and identification information of each local clustered speech segment, wherein the speech clustering model is obtained by pre-training speech with speaker annotations, each local clustered speech segment corresponds to an independent speaker, and the identification information is information for distinguishing different speakers; inputting the local clustered speech segments and the identification information into a global clustering model to obtain speaker classification clustered speech segments.

[0008] Optionally, the step of inputting the local clustered speech segments and the identification information into a global clustering model to obtain speaker classification clustered speech segments comprises: inputting the local clustered speech segments into a global clustering model to extract speech features of the local clustered speech segments; calculating the similarity between each local clustered speech segment based on the speech features, wherein if two segments belong to the same class, their similarity is set to infinity, and if two segments belong to different classes, their similarity is set to infinitesimal; and using the opposite of the similarity as a loss function to perform global clustering to obtain the speaker classification clustered speech segments.

[0009] Optionally, the method further comprises: obtaining an original speech segment of the target speaker; and performing noise reduction processing on the original speech segment to obtain the target speech before using a speech recognition engine to perform speech recognition on the target speech to obtain a recognized text result and the start time and end time of each word in the text.

[0010] Optionally, after inputting the local clustered speech segments and the identification information into the global clustering model to obtain the speaker classification clustered speech segments, the method further includes: performing speech recognition according to the speaker classification clustered speech segments to obtain the target speech text.

[0011] According to one aspect of an embodiment of the present invention, a speaker segmentation and clustering device is provided, comprising: a first speech recognition unit, used to perform speech recognition on a target speech using a speech recognition engine, and obtain a recognized text result and a start time and an end time of each word in the text; a determination unit, used to find a demarcation point between speech and non-speech based on the start time and the end time, and determine the speech segments included in the target speech; a local clustering unit, used to input the speech segments into a speech clustering model, cluster each speech segment, and obtain local clustered speech segments and identification information of each local clustered speech segment, wherein the speech clustering model is obtained by pre-training speech with speaker annotations, each local clustered speech segment corresponds to an independent speaker, and the identification information is information for distinguishing different speakers; and a global clustering unit, used to input the local clustered speech segments and the identification information into a global clustering model, and obtain speaker classification clustered speech segments.

[0012] Optionally, the global clustering unit includes: an extraction module, used to input the local clustering speech segments into the global clustering model to extract the speech features of the local clustering speech segments; a calculation module, used to calculate the similarity between each local clustering speech segment based on the speech features, wherein if two segments belong to the same class, their similarity is set to infinity, and if two segments belong to different classes, their similarity is set to infinitesimal; a global clustering module, used to use the opposite of the similarity as a loss function to perform global clustering to obtain the speaker classification clustered speech segments.

[0013] Optionally, the device also includes: an acquisition unit, used to perform speech recognition on the target speech using the speech recognition engine, obtain the recognized text result and the start time and end time of each word in the text, and obtain the original speech segment of the target speaker; a processing unit, used to perform noise reduction processing on the original speech segment to obtain the target speech.

[0014] Optionally, the device also includes: a second speech recognition unit, which is used to input the local clustered speech segments and the identification information into the global clustering model to obtain the speaker classification clustered speech segments, and then perform speech recognition based on the speaker classification clustered speech segments to obtain the target speech text.

[0015] According to a first aspect of an embodiment of the present application, a computer-readable storage medium is provided, characterized in that a computer program is stored in the storage medium, wherein the computer program is configured to execute the above-mentioned speaker segmentation and clustering method when running.

[0016] According to a first aspect of an embodiment of the present application, an electronic device is provided, comprising a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the above-mentioned speaker segmentation clustering method.

[0017] In an embodiment of the present invention, a speech recognition engine is used to perform speech recognition on a target speech to obtain a recognized text result and the start time and end time of each word in the text; based on the start time and the end time, a demarcation point between speech and non-speech is found to determine the speech segments included in the target speech; the speech segments are input into a speech clustering model, each speech segment is clustered, and local clustered speech segments and identification information of each local clustered speech segment are obtained, wherein the speech clustering model is obtained by pre-training speech annotated by a speaker, each local clustered speech segment corresponds to an independent speaker, and the identification information is information for distinguishing different speakers; the local clustered speech segments and the identification information are input into a global clustering model to obtain speaker classification clustered speech segments. In this embodiment, the use of a speech recognition engine to segment the input speech can more accurately distinguish speech and non-speech, improve the problem of high error rate of using VAD to segment speech, thereby improving the accuracy of the subsequent clustering module, thereby reducing the error rate of the entire speaker segmentation system, so as to at least solve the technical problem of low accuracy of speaker segmentation clustering in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0019] Figure 1 is a hardware structure block diagram of a mobile terminal according to an optional speaker segmentation and clustering method of an embodiment of the present invention;

[0020] Figure 2 is a flow chart of an optional speaker segmentation and clustering method according to an embodiment of the present invention;

[0021] Figure 3 is a schematic diagram of an optional voice segment according to an embodiment of the present invention;

[0022] Figure 4 is a schematic diagram of an optional local clustering result of speech segments according to an embodiment of the present invention;

[0023] Figure 5 is a schematic diagram of an optional global clustering result of speech segments according to an embodiment of the present invention;

[0024] Figure 6is a schematic diagram of an optional deep neural network voiceprint model structure according to an embodiment of the present invention;

[0025] Figure 7 4 is a diagram of an optional speaker segmentation and clustering device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0026] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0027] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a sequence of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0028] The speaker segmentation and clustering method provided in the embodiment of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Taking running on a mobile terminal as an example, Figure 1 FIG. 1 is a hardware structure block diagram of a mobile terminal of a speaker segmentation and clustering method according to an embodiment of the present invention. Figure 1 As shown, the mobile terminal 10 may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data. Optionally, the mobile terminal may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is for illustration only and does not limit the structure of the mobile terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0029] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the speaker segmentation clustering method in the embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the mobile terminal 10 via a network. Examples of the above network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0030] This embodiment also provides a speaker segmentation and clustering method. Figure 2 is a flow chart of a speaker segmentation and clustering method according to an embodiment of the present invention. Figure 2 As shown, the speaker segmentation clustering method process includes the following steps:

[0031] Step S202: Use a speech recognition engine to perform speech recognition on the target speech to obtain a recognized text result and a start time and an end time of each word in the text.

[0032] Step S204, finding the demarcation point between speech and non-speech based on the start time and the end time, and determining the speech segments included in the target speech.

[0033] Step S206, input the speech segments into the speech clustering model, cluster each speech segment, obtain local clustered speech segments and identification information of each local clustered speech segment, wherein the speech clustering model is obtained by pre-training using speaker-annotated speech, each local clustered speech segment corresponds to an independent speaker, and the identification information is information used to distinguish different speakers.

[0034] Step S208: input the local clustered speech segments and identification information into the global clustering model to obtain speaker-classified clustered speech segments.

[0035] In this embodiment, in step S204, the start time and end time of each word obtained in step S202 are used to find the demarcation point between speech and non-speech. In this way, individual speech segments are obtained. Figure 3 As shown, a schematic diagram of a speech segment, wherein: Figure 3 Among them, 5s-13s, 23s-33s, 43s-50s, and 65s-75s have recognition results, which are speech segments. 0s-5s, 13s-23s, 33s-43s, and 50s-65s are non-speech segments.

[0036] In step S204, the function of using VAD to perform speech endpoint detection is substituted for the traditional speaker segmentation system. Since the performance of the ASR engine is far superior to that of the VAD, speech segments with much higher accuracy than that of the VAD can be obtained here.

[0037] In step S206, the speech segments are input into the speech clustering model, and each speech segment is clustered to obtain a clustering result for each segment. Figure 3 The results of local clustering for the speech clip shown are as follows Figure 4 As shown, it is a schematic diagram of the local clustering results of speech fragments.

[0038] The speech segments from 5s to 13s are marked with two speakers, and the corresponding speaker IDs in the time stamp are: S1, S2, S1;

[0039] The speech segment from 23s to 33s is marked with three speakers, and the corresponding speaker IDs in the time stamp are: S1, S2, S3, S2;

[0040] Three speakers are marked in the speech segment from 43s to 50s, and the corresponding speaker IDs in the time stamp are: S1, S2, S1, S2, S1;

[0041] Three speakers are marked in the speech segment between 65s and 75s, and the corresponding speaker IDs in the time stamp are: S1, S2, S1, S3, S2.

[0042] The speaker IDs marked in each of the above segments are numbered independently. This way, smaller speech segments are obtained, each of which is a separate speaker. The following refers to the segment as a locally clustered speech segment. The so-called local clustering means that a speaker ID has been given by E2E-SD, but this ID is not global.

[0043] In this embodiment, the speaker IDs of the local cluster segments are uniformly numbered, such as Figure 5 As shown, a schematic diagram of the global clustering results of speech fragments.

[0044] Through the embodiment provided by the present application, a speech recognition engine is used to perform speech recognition on the target speech to obtain the recognized text result and the start time and end time of each word in the text; based on the start time and the end time, the demarcation point between speech and non-speech is found to determine the speech segments included in the target speech; the speech segments are input into the speech clustering model, each speech segment is clustered, and local clustered speech segments and identification information of each local clustered speech segment are obtained, wherein the speech clustering model is obtained by pre-training speech annotated by the speaker, each local clustered speech segment corresponds to an independent speaker, and the identification information is information used to distinguish different speakers; the local clustered speech segments and the identification information are input into the global clustering model to obtain speaker classification clustered speech segments. In this embodiment, the use of the speech recognition engine to segment the input speech can more accurately distinguish speech and non-speech, improve the problem of high error rate of using VAD to segment speech, thereby improving the accuracy of the subsequent clustering module, thereby reducing the error rate of the entire speaker segmentation system, so as to at least solve the technical problem of low accuracy of speaker segmentation clustering in the prior art.

[0045] Optionally, the local clustered speech segments and identification information are input into a global clustering model to obtain speaker classification clustered speech segments, including: inputting the local clustered speech segments into a global clustering model to extract speech features of the local clustered speech segments; calculating the similarity between each local clustered speech segment based on the speech features, wherein if two segments belong to the same category, their similarity is set to infinity, and if two segments belong to different categories, their similarity is set to infinitesimal; using the opposite of the similarity as a loss function, performing global clustering to obtain speaker classification clustered speech segments.

[0046] Optionally, a speech recognition engine is used to perform speech recognition on the target speech to obtain a recognized text result and the start time and end time of each word in the text. The method also includes: obtaining an original speech segment of the target speaker; and performing noise reduction processing on the original speech segment to obtain the target speech.

[0047] Optionally, after the local clustered speech segments and identification information are input into the global clustering model to obtain the speaker classification clustered speech segments, the method further includes: performing speech recognition according to the speaker classification clustered speech segments to obtain the target speech text.

[0048] As an optional embodiment, the present application also provides a speaker segmentation method.

[0049] Step 1: Use the speech recognition engine to perform speech recognition on the speech and obtain the recognized text results as well as the start and end time of each word in the text.

[0050] Step 2: Find the timing of the speech.

[0051] The implementation process is: use the start time and end time of each word obtained in step 1 to find the dividing point between speech and non-speech. In this way, individual speech segments are obtained. Figure 3 As shown: 5s-13s, 23s-33s, 43s-50s, 65s-75s have recognition results, that is, voice segments. 0s-5s, 13s-23s, 33s-43s, 50s-65s are non-voice segments.

[0052] In step 2, the function of using VAD to perform speech endpoint detection is substituted for the traditional speaker segmentation system. Because the performance of the ASR engine is far superior to that of VAD, speech clips with much higher accuracy than VAD can be obtained here.

[0053] Step 3: Send the speech segments obtained in step 2 to the end-to-end speaker log system (hereinafter referred to as: E2E-SD. E2E-SD is a model that is trained in advance using speaker-labeled speech and can perform speaker clustering) to obtain the clustering results of each segment. Figure 3 , the results are as follows Figure 4 As shown, the schematic diagram of the local clustering results of speech fragments:

[0054] The speech segments from 5s to 13s are marked with two speakers, and the corresponding speaker IDs in the time stamp are: S1, S2, S1;

[0055] The speech segment from 23s to 33s is marked with three speakers, and the corresponding speaker IDs in the time stamp are: S1, S2, S3, S2;

[0056] Three speakers are marked in the speech segment from 43s to 50s, and the corresponding speaker IDs in the time stamp are: S1, S2, S1, S2, S1;

[0057] Three speakers are marked in the speech segment between 65s and 75s, and the corresponding speaker IDs in the time stamp are: S1, S2, S1, S3, S2.

[0058] The speaker IDs marked in each of the above segments are numbered independently. This way, smaller speech segments are obtained, each of which is a separate speaker. The following refers to the segment as a locally clustered speech segment. The so-called local clustering means that a speaker ID has been given by E2E-SD, but this ID is not global.

[0059] Step 4: Unify the speaker IDs of the local clustered segments obtained in step 3, such as Figure 5As shown by the arrow in the middle, the speech segment from 5s to 13s is marked with two speakers, and the corresponding speaker IDs in the time sample are: S1, S2, S1;

[0060] Three speakers are marked in the speech segment from 23s to 33s, and the corresponding speaker IDs in the time stamp are: S11, S12, S13, S12;

[0061] Three speakers are marked in the speech segment from 43s to 50s, and the corresponding speaker IDs in the time stamp are: S21, S22, S21, S22, S21;

[0062] Three speakers are marked in the speech segment between 65s and 75s, and the corresponding speaker IDs in the time sample are: S41, S42, S41, S43, S42.

[0063] Step 5: Send the speech segments from step 4 into the clustering system based on hierarchical clustering for global clustering. The process is as follows:

[0064] Step 5.1, extract the speaker embedding information of each segment. Use the TDNN-based deep neural network voiceprint extractor x-vector, such as Figure 7 As shown in the figure, the schematic diagram of the neural network voiceprint model structure is used to extract the embedding information of the speech fragment.

[0065] Step 5.2, use embedding to calculate the similarity between fragments (PLDA score).

[0066] Step 5.3, modify the similarity between the fragments according to the local clustering results. If the two fragments belong to the same category in step 4, then set their similarity to infinity. If the two fragments belong to different categories in step 4, then set their similarity to infinitesimal.

[0067] Step 5.4, global clustering: Use the opposite of the similarity calculated in 5.2 and 5.3 as the cost and perform AHC clustering, so that all speech segments are grouped uniformly.

[0068] Through the embodiments provided in this application, the speech recognition engine is used to segment the input speech, which can more accurately distinguish speech and non-speech, and improve the high error rate of speech segmentation using VAD, thereby improving the accuracy of the subsequent clustering module, thereby reducing the error rate of the entire speaker segmentation system. And use E2E-sd to perform a preliminary speaker clustering on the speech segments determined by the speech recognition engine. In the process of global clustering, the use of the preliminary clustering results will greatly reduce the clustering confusion error. Tested on an internal data set, the confusion error is from 26.5%->12%.

[0069] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0070] In this embodiment, a speaker segmentation and clustering device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0071] Figure 7 is a structural block diagram of a speaker segmentation and clustering device according to an embodiment of the present invention. Figure 7 As shown, the speaker segmentation and clustering device includes:

[0072] The first speech recognition unit 71 is used to perform speech recognition on the target speech using a speech recognition engine to obtain a recognized text result and a start time and an end time of each word in the text.

[0073] The determination unit 73 is used to find the demarcation point between speech and non-speech based on the start time and the end time, and determine the speech segment included in the target speech.

[0074] The local clustering unit 75 is used to input the speech segments into the speech clustering model, cluster each speech segment, and obtain the local clustered speech segments and the identification information of each local clustered speech segment, wherein the speech clustering model is obtained by pre-training the speech with speaker annotations, each local clustered speech segment corresponds to an independent speaker, and the identification information is the information used to distinguish different speakers.

[0075] The global clustering unit 77 is used to input the local clustered speech segments and identification information into the global clustering model to obtain speaker-classified clustered speech segments.

[0076] Through the embodiments provided by the present application, the first speech recognition unit 71 uses a speech recognition engine to perform speech recognition on the target speech, and obtains the recognized text result and the start time and end time of each word in the text; the determination unit 73 finds the dividing point between speech and non-speech based on the start time and the end time, and determines the speech segments included in the target speech; the local clustering unit 75 inputs the speech segments into the speech clustering model, clusters each speech segment, and obtains local clustered speech segments and identification information of each local clustered speech segment, wherein the speech clustering model is obtained by pre-training speech annotated by the speaker, each local clustered speech segment corresponds to an independent speaker, and the identification information is information used to distinguish different speakers; the global clustering unit 77 inputs the local clustered speech segments and the identification information into the global clustering model to obtain speaker classification clustered speech segments. In this embodiment, the use of a speech recognition engine to segment the input speech can more accurately distinguish between speech and non-speech, thereby improving the high error rate of speech segmentation using VAD, thereby improving the accuracy of the subsequent clustering module, thereby reducing the error rate of the entire speaker segmentation system, and at least solving the technical problem of low speaker segmentation clustering accuracy in the prior art.

[0077] Optionally, the above-mentioned global clustering unit 77 may include: an extraction module, used to input the local clustering speech segments into the global clustering model to extract the speech features of the local clustering speech segments; a calculation module, used to calculate the similarity between each local clustering speech segment based on the speech features, wherein if two segments belong to the same class, their similarity is set to infinity, and if two segments belong to different classes, their similarity is set to infinitesimal; a global clustering module, used to use the opposite of the similarity as the loss function to perform global clustering to obtain speaker classification clustered speech segments.

[0078] Optionally, the above-mentioned device may also include: an acquisition unit, used to use a speech recognition engine to perform speech recognition on the target speech, obtain the recognized text result and the start time and end time of each word in the text, and obtain the original speech segment of the target speaker; a processing unit, used to perform noise reduction processing on the original speech segment to obtain the target speech.

[0079] Optionally, the above-mentioned device may also include: a second speech recognition unit, which is used to input the local clustered speech segments and identification information into the global clustering model, obtain the speaker classification clustered speech segments, and then perform speech recognition according to the speaker classification clustered speech segments to obtain the target speech text.

[0080] It should be noted that the above modules can be implemented by software or hardware. For the latter, it can be implemented in the following ways, but not limited to: the above modules are all located in the same processor; or the above modules are located in different processors in any combination.

[0081] An embodiment of the present invention further provides a storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when running.

[0082] Optionally, in this embodiment, the storage medium may be configured to store a computer program for performing the following steps:

[0083] S1, using a speech recognition engine to perform speech recognition on the target speech, and obtain the recognized text result and the start time and end time of each word in the text;

[0084] S2, finding the dividing point between speech and non-speech based on the start time and the end time, and determining the speech segment included in the target speech;

[0085] S3, inputting the speech segments into the speech clustering model, clustering each speech segment, obtaining local clustered speech segments and identification information of each local clustered speech segment, wherein the speech clustering model is obtained by pre-training using speech annotated by speakers, each local clustered speech segment corresponds to an independent speaker, and the identification information is information used to distinguish different speakers;

[0086] S4, inputting the local clustered speech segments and identification information into the global clustering model to obtain speaker classification clustered speech segments.

[0087] Optionally, in this embodiment, the above-mentioned storage medium may include but is not limited to: a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and other media that can store computer programs.

[0088] An embodiment of the present invention further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0089] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0090] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program:

[0091] S1, using a speech recognition engine to perform speech recognition on the target speech, and obtain the recognized text result and the start time and end time of each word in the text;

[0092] S2, finding the dividing point between speech and non-speech based on the start time and the end time, and determining the speech segment included in the target speech;

[0093] S3, inputting the speech segments into the speech clustering model, clustering each speech segment, obtaining local clustered speech segments and identification information of each local clustered speech segment, wherein the speech clustering model is obtained by pre-training using speech annotated by speakers, each local clustered speech segment corresponds to an independent speaker, and the identification information is information used to distinguish different speakers;

[0094] S4, inputting the local clustered speech segments and identification information into the global clustering model to obtain speaker classification clustered speech segments.

[0095] Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be described in detail here.

[0096] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, and optionally, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0097] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the principles of the present invention shall be included in the protection scope of the present invention.

Claims

1. A speaker segmentation and clustering method, characterized in that: include: Using a speech recognition engine to perform speech recognition on the target speech, obtaining a recognized text result and a start time and an end time of each word in the text; Finding a demarcation point between speech and non-speech based on the start time and the end time, and determining a speech segment included in the target speech; Inputting the speech segments into a speech clustering model, clustering each speech segment, and obtaining local clustered speech segments and identification information of each local clustered speech segment, wherein the speech clustering model is obtained by pre-training speech labeled by speakers, each local clustered speech segment corresponds to an independent speaker, and the identification information is information used to distinguish different speakers; Inputting the local clustered speech segments and the identification information into a global clustering model to obtain speaker classification clustered speech segments; The step of inputting the local clustered speech segments and the identification information into a global clustering model to obtain speaker-classified clustered speech segments comprises: Inputting the local clustered speech segments into a global clustering model to extract speech features of the local clustered speech segments; Calculate the similarity between each local clustered speech segment based on the speech feature, wherein if two segments belong to the same category, their similarity is set to infinity, and if two segments belong to different categories, their similarity is set to infinitesimal; The inverse of the similarity is used as a loss function to perform global clustering to obtain the speaker classification clustering speech segments.

2. The method according to claim 1, characterized in that: The method further comprises: performing speech recognition on the target speech using a speech recognition engine to obtain a recognized text result and a start time and an end time of each word in the text; Obtain the original speech clip of the target speaker; The original speech segment is subjected to noise reduction processing to obtain the target speech.

3. The method according to claim 1, characterized in that After inputting the local clustered speech segments and the identification information into the global clustering model to obtain the speaker classification clustered speech segments, the method further includes: The speech segments are classified and clustered according to the speakers, and speech recognition is performed to obtain target speech text.

4. A speaker segmentation and clustering device, characterized in that: include: A first speech recognition unit, configured to perform speech recognition on a target speech using a speech recognition engine, and obtain a recognized text result and a start time and an end time of each word in the text; A determination unit, configured to find a demarcation point between speech and non-speech based on the start time and the end time, and determine a speech segment included in the target speech; A local clustering unit, used for inputting the speech segments into a speech clustering model, clustering each speech segment, and obtaining local clustered speech segments and identification information of each local clustered speech segment, wherein the speech clustering model is obtained by pre-training speech labeled by speakers, each local clustered speech segment corresponds to an independent speaker, and the identification information is information for distinguishing different speakers; A global clustering unit, used for inputting the local clustered speech segments and the identification information into a global clustering model to obtain speaker-classified clustered speech segments; Wherein, the global clustering unit comprises: An extraction module, used for inputting the local clustered speech segments into a global clustering model to extract speech features of the local clustered speech segments; A calculation module, used for calculating the similarity between each local clustered speech segment based on the speech feature, wherein if two segments belong to the same category, their similarity is set to infinity, and if two segments belong to different categories, their similarity is set to infinitesimal; The global clustering module is used to use the opposite number of the similarity as a loss function to perform global clustering to obtain the speaker classification clustering speech segments.

5. The device according to claim 4, characterized in that The device also includes: An acquisition unit, configured to perform speech recognition on the target speech using the speech recognition engine, obtain the recognized text result and the start time and end time of each word in the text, and acquire the original speech segment of the target speaker; The processing unit is used to perform noise reduction processing on the original speech segment to obtain the target speech.

6. The device according to claim 4, characterized in that The device also includes: The second speech recognition unit is used for inputting the local clustered speech segments and the identification information into the global clustering model to obtain the speaker classification clustered speech segments, and then performing speech recognition according to the speaker classification clustered speech segments to obtain the target speech text.

7. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, wherein the computer program is configured to execute the method according to any one of claims 1 to 3 when executed.

8. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to perform the method according to any one of claims 1 to 3.

Images