Voiceprint clustering method, device, storage medium and electronic device

Abstract

The present invention discloses a voiceprint clustering method, device, storage medium and electronic device. The voiceprint clustering method comprises: obtaining the first identification information of a user with a known target voiceprint; extracting the voiceprint information of each voice segment, using a voiceprint model to calculate the similarity between two voiceprints; correcting the similarity by using the known target voiceprint; clustering each voice segment based on a loss function to obtain a target voice segment; determining the target voice segment whose similarity between the known target voiceprint information meets a preset condition, and clustering the target voice segment with the known target voiceprint, that is, using the known information to artificially modify the similarity between them, because the distance between the grouped groups is infinite, this is equivalent to increasing the boundary between the groups, thereby indirectly reducing the probability of classification errors, and solving the technical problem in the prior art that the error rate of clustering results is high due to the short original voice and inaccurate extraction of voiceprint information.

Description

Technical Field

[0001] The present invention relates to finance-related fields, and in particular to a voiceprint clustering method, device, storage medium and electronic device. Background Art

[0002] With the development of society, the recognition of various voice information is applied in more fields. Speech recognition designs speech clustering, and speech recognition after clustering will be more accurate. Among them, the mainstream voiceprint clustering process is: input the voiceprint set to be clustered -> calculate the similarity between voiceprints -> perform blind clustering based on the similarity between voiceprints (for example: hierarchical clustering, spectral clustering, K-means, etc.).

[0003] According to the principle of blind clustering, at the beginning of clustering, each voiceprint forms a group. At this time, if some voiceprint information is greatly disturbed and is incorrectly merged with other voiceprints, this error will continue to affect the subsequent mergers. Step by step, the errors will become more and more serious, resulting in a very high clustering error rate.

[0004] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0005] The embodiments of the present invention provide a voiceprint clustering method, device, storage medium and electronic device to at least solve the problem in the prior art that the error rate of clustering results is high due to the short original speech and inaccurate extraction of voiceprint information.

[0006] According to one aspect of an embodiment of the present invention, there is provided a voiceprint clustering method, comprising: obtaining first identification information of a user of a known target voiceprint; extracting voiceprint information of each voice segment, and using a voiceprint model to calculate the similarity between each pair of voiceprints; correcting the similarity by using the known target voiceprint; clustering each voice segment based on a loss function to obtain a target voice segment; determining a target voice segment whose similarity between the known target voiceprint information meets a preset condition, and clustering the target voice segment with the known target voiceprint.

[0007] Optionally, the extracting the voiceprint information of each speech segment includes: extracting speech features from each speech segment; sending the speech features to an x-vector network, and outputting the voiceprint information of each speech segment.

[0008] Optionally, the correcting the similarity by using the known target voiceprint includes: setting the similarity of the voice segment consistent with the known voiceprint to infinity; and setting the similarity of the voice segment inconsistent with the known voiceprint to infinitesimal.

[0009] Optionally, clustering each speech segment based on a loss function to obtain a target speech segment includes: using the inverse of the plda score as the loss function, clustering the voiceprints in each speech segment set to obtain the target speech segment, wherein the target speech segment includes one or more.

[0010] Optionally, the method further includes: when the target voice segment includes multiple ones, determining second identification information of each target voice segment: determining the second identification information consistent with the first identification information as the target voice segment of the known target voiceprint cluster.

[0011] According to another aspect of an embodiment of the present invention, a voiceprint clustering device is also provided, including: an acquisition unit, used to acquire first identification information of a user of a known target voiceprint; a calculation unit, used to extract voiceprint information of each voice segment, and use a voiceprint model to calculate the similarity between each voiceprint; a correction unit, used to correct the similarity through the known target voiceprint; a first clustering unit, used to cluster each voice segment based on a loss function to obtain a target voice segment; a second clustering unit, used to determine a target voice segment whose similarity between the known target voiceprint information meets a preset condition, and cluster the target voice segment with the known target voiceprint.

[0012] Optionally, the computing unit includes: an extraction module for extracting speech features from each speech segment; and an output module for feeding the speech features into an x-vector network to output voiceprint information of each speech segment.

[0013] Optionally, the correction unit includes: a first correction module, used to set the similarity of the voice segment consistent with the known voiceprint to infinity; and a second correction module, used to set the similarity of the voice segment inconsistent with the known voiceprint to infinitesimal.

[0014] Optionally, the first clustering unit includes: a clustering module, used to use the inverse of the plda score as the loss function, cluster the voiceprints in each speech segment set to obtain the target speech segment, wherein the target speech segment includes one or more.

[0015] Optionally, the device also includes: a first determination module, used to determine the second identification information of each target voice segment when the target voice segment includes multiple ones; a second determination module, used to determine the second identification information consistent with the first identification information as the target voice segment of the known target voiceprint clustering.

[0016] According to a first aspect of an embodiment of the present application, a computer-readable storage medium is provided, characterized in that a computer program is stored in the storage medium, wherein the computer program is configured to execute the above-mentioned voiceprint clustering method when running.

[0017] According to a first aspect of an embodiment of the present application, there is provided an electronic device, comprising a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the above-mentioned voiceprint clustering method.

[0018] In an embodiment of the present invention, first identification information of a user with a known target voiceprint is obtained; voiceprint information of each speech segment is extracted, and the similarity between two voiceprints is calculated using a voiceprint model; the similarity is corrected using the known target voiceprint; each speech segment is clustered based on a loss function to obtain a target speech segment; a target speech segment whose similarity between the known target voiceprint information meets a preset condition is determined, and the target speech segment is clustered with the known target voiceprint, that is, the similarity between them is artificially modified using the known information, so that in the subsequent clustering process, voiceprints with infinitesimal similarity will definitely be clustered together first, and voiceprints with infinite similarity will definitely not be clustered together, and the distance between different known speakers is infinite. Since the distance between the grouped groups is infinite, this is equivalent to increasing the boundary between groups, thereby indirectly reducing the probability of classification errors, and solving the technical problem in the prior art that the error rate of clustering results is high due to the short original speech and inaccurate extraction of voiceprint information. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0020] Figure 1 is a hardware structure block diagram of a mobile terminal according to an optional voiceprint clustering method of an embodiment of the present invention;

[0021] Figure 2 is a flow chart of an optional voiceprint clustering method according to an embodiment of the present invention;

[0022] Figure 3 4 is a diagram of an optional voiceprint clustering device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0023] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0024] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a sequence of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0025] The voiceprint clustering method embodiment provided in the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Taking running on a mobile terminal as an example, Figure 1 FIG. 1 is a hardware structure block diagram of a mobile terminal of a voiceprint clustering method according to an embodiment of the present invention. Figure 1 As shown, the mobile terminal 10 may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data. Optionally, the mobile terminal may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is for illustration only and does not limit the structure of the mobile terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.

[0026] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the voiceprint clustering method in the embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories can be connected to the mobile terminal 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0027] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the mobile terminal 10. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0028] This embodiment also provides a voiceprint clustering method. Figure 2 is a flow chart of a voiceprint clustering method according to an embodiment of the present invention. Figure 2 As shown, the voiceprint clustering method process includes the following steps:

[0029] Step S202: Acquire first identification information of a user whose target voiceprint is known.

[0030] Step S204: extract the voiceprint information of each speech segment, and use the voiceprint model to calculate the similarity between each pair of voiceprints.

[0031] Step S206, correcting the similarity using the known target voiceprint.

[0032] Step S208: cluster each speech segment based on the loss function to obtain a target speech segment.

[0033] Step S210, determining target speech segments whose similarity with known target voiceprint information meets a preset condition, and clustering the target speech segments with the known target voiceprints.

[0034] In this embodiment, the voiceprint clustering method may include but is not limited to being applied to various speech recognition scenarios.

[0035] Through the embodiments provided by the present application, the first identification information of the user with a known target voiceprint is obtained; the voiceprint information of each voice segment is extracted, and the similarity between the two voiceprints is calculated using a voiceprint model; the similarity is corrected using the known target voiceprint; each voice segment is clustered based on a loss function to obtain a target voice segment; the target voice segment whose similarity between the known target voiceprint information meets a preset condition is determined, and the target voice segment is clustered with the known target voiceprint, that is, the similarity between them is artificially modified using the known information, so that in the subsequent clustering process, voiceprints with infinitesimal similarity will definitely be clustered together first, and voiceprints with infinite similarity will definitely not be clustered together, and the distance between different known speakers is infinite. Since the distance between the grouped groups is infinite, this is equivalent to increasing the boundary between groups, thereby indirectly reducing the probability of classification errors, and solving the technical problem in the prior art that the error rate of clustering results is high due to the short original speech and inaccurate extraction of voiceprint information.

[0036] Optionally, extracting the voiceprint information of each speech segment may include: extracting speech features from each speech segment; sending the speech features to an x-vector network, and outputting the voiceprint information of each speech segment.

[0037] Optionally, correcting the similarity by using a known target voiceprint may include: setting the similarity of a voice segment that is consistent with the known voiceprint to infinity; and setting the similarity of a voice segment that is inconsistent with the known voiceprint to infinitesimal.

[0038] Optionally, clustering each speech segment based on a loss function to obtain a target speech segment includes: using the inverse of the plda score as a loss function, clustering the voiceprints in each speech segment set to obtain a target speech segment, wherein the target speech segment includes one or more.

[0039] Optionally, the method may further include: in the case where the target voice segment includes multiple ones, determining the second identification information of each target voice segment: determining the second identification information consistent with the first identification information as the target voice segment of the known target voiceprint clustering.

[0040] As an optional embodiment, the present application also provides a voiceprint clustering method for partially known speakers, which includes the following contents.

[0041] Step 1: Input the voice clip and the speaker ID of some known voiceprints;

[0042] Step 2: Extract the voiceprint information of each speech segment.

[0043] The voiceprint extraction process is as follows: extract MFCC features from the speech segment; send the features to the x-vector network; the input of the x-vector network is the voiceprint information of the speech segment.

[0044] Step 3: Use the PLDA model to calculate the similarity between two voiceprints (PLDA score)

[0045] Step 4: Correct the similarity in step 3 according to the input known voiceprint speaker information. If the two voiceprints belong to the same speaker in the known information, then set the similarity to infinity; if the two voiceprints belong to different speakers in the known information, then set the similarity to infinitesimal.

[0046] Step 5: Use the opposite of the PLDA score after the rest in step 4 as the cost of AHC clustering, perform AHC clustering on the voiceprints, and each voiceprint gets a group ID. According to the principle of AHC clustering, the clustering result will not destroy the relationship between the known speakers of the voiceprints. At the same time, it increases the probability of correct clustering of other voiceprints.

[0047] Step 6: Remap the ID in step 5 so that the final ID is consistent with the known speaker ID entered in step 1.

[0048] In this embodiment, known information is used to artificially modify the similarity between them, so that in the subsequent clustering process, voiceprints with infinitesimal similarity will definitely be clustered together first, and those with infinite similarity will definitely not be clustered together, and the distance between different speakers is infinite; compared with ordinary hierarchical clustering, this method is equivalent to the starting point of clustering not only having isolated voiceprints, but also having voiceprint groups that are definitely correctly grouped. These voiceprint groups can more accurately attract voiceprints in the same group as themselves; at the same time, since the distance between the grouped groups is infinite, this is equivalent to increasing the boundaries between groups. This indirectly reduces the probability of classification errors.

[0049] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0050] In this embodiment, a voiceprint clustering device is also provided, which is used to implement the above embodiments and preferred implementation modes, and will not be repeated here. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0051] Figure 3 is a structural block diagram of a voiceprint clustering device according to an embodiment of the present invention. Figure 3 As shown, the voiceprint clustering device includes:

[0052] The acquisition unit 301 is configured to acquire first identification information of a user whose target voiceprint is known.

[0053] The calculation unit 303 is used to extract the voiceprint information of each speech segment and calculate the similarity between two voiceprints using the voiceprint model.

[0054] The correction unit 305 is used to correct the similarity using the known target voiceprint.

[0055] The first clustering unit 307 is used to cluster each speech segment based on the loss function to obtain a target speech segment.

[0056] The second clustering unit 309 is used to determine target speech segments whose similarity between known target voiceprint information meets a preset condition, and cluster the target speech segments with the known target voiceprints.

[0057] Through the embodiment provided by the present application, the acquisition unit 301 acquires the first identification information of the user of the known target voiceprint; the calculation unit 303 extracts the voiceprint information of each voice segment, and uses the voiceprint model to calculate the similarity between the two voiceprints; the correction unit 305 corrects the similarity by the known target voiceprint; the first clustering unit 307 clusters each voice segment based on the loss function to obtain the target voice segment; the second clustering unit 309 determines the target voice segment whose similarity between the known target voiceprint information meets the preset conditions, and clusters the target voice segment with the known target voiceprint. That is, by using the length information of the original voice of the voiceprint, the voiceprint of the longer voice is clustered first, and then the voiceprint of the shorter voice is classified, which solves the technical problem in the prior art that the error rate of the clustering result is high due to the short original voice and the inaccurate extraction of voiceprint information.

[0058] Optionally, the calculation unit 303 may include: an extraction module for extracting speech features from each speech segment; and an output module for sending the speech features to an x-vector network to output voiceprint information of each speech segment.

[0059] Optionally, the correction unit 305 may include: a first correction module, used to set the similarity of the voice segment consistent with the known voiceprint to infinity; and a second correction module, used to set the similarity of the voice segment inconsistent with the known voiceprint to infinitesimal.

[0060] Optionally, the first clustering unit 307 may include: a clustering module, configured to cluster the voiceprints in each speech segment set using the inverse of the plda score as a loss function to obtain a target speech segment, wherein the target speech segment includes one or more.

[0061] Optionally, the above-mentioned device may also include: a first determination module, used to determine the second identification information of each target voice segment when the target voice segment includes multiple ones; a second determination module, used to determine the second identification information consistent with the first identification information as the target voice segment of the known target voiceprint cluster.

[0062] It should be noted that the above modules can be implemented by software or hardware. For the latter, it can be implemented in the following ways, but not limited to: the above modules are all located in the same processor; or the above modules are located in different processors in any combination.

[0063] An embodiment of the present invention further provides a storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when running.

[0064] Optionally, in this embodiment, the storage medium may be configured to store a computer program for performing the following steps:

[0065] S1, obtaining first identification information of a user with a known target voiceprint;

[0066] S2, extract the voiceprint information of each speech segment, use the voiceprint model to calculate the similarity between two voiceprints; and correct the similarity by using the known target voiceprint;

[0067] S3, clustering each speech segment based on the loss function to obtain the target speech segment;

[0068] S4, determining target speech segments whose similarity with known target voiceprint information meets a preset condition, and clustering the target speech segments with the known target voiceprints.

[0069] Optionally, in this embodiment, the above-mentioned storage medium may include but is not limited to: a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and other media that can store computer programs.

[0070] An embodiment of the present invention further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0071] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0072] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program:

[0073] S1, obtaining first identification information of a user with a known target voiceprint;

[0074] S2, extract the voiceprint information of each speech segment, use the voiceprint model to calculate the similarity between two voiceprints; and correct the similarity by using the known target voiceprint;

[0075] S3, clustering each speech segment based on the loss function to obtain the target speech segment;

[0076] S4, determining target speech segments whose similarity with known target voiceprint information meets a preset condition, and clustering the target speech segments with the known target voiceprints.

[0077] Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be described in detail here.

[0078] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, and optionally, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0079] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the principles of the present invention shall be included in the protection scope of the present invention.

Claims

1. A voiceprint clustering method, characterized in that: include: Acquire first identification information of a user whose target voiceprint is known; Extract the voiceprint information of each speech segment, and use the voiceprint model to calculate the similarity between two voiceprints; Correcting the similarity by using the known target voiceprint; The correcting the similarity by using the known target voiceprint includes: Setting the similarity of the voice segment consistent with the known voiceprint to infinitesimal; Setting the similarity of the voice segment that is inconsistent with the known voiceprint to infinity; Clustering each of the speech segments based on the loss function to obtain a target speech segment; Determine a target voice segment whose similarity between the known target voiceprint information meets a preset condition, and cluster the target voice segment with the known target voiceprint; In the case where the target speech segment includes multiple segments, second identification information of each target speech segment is determined: The second identification information consistent with the first identification information is determined as the target speech segment of the known target voiceprint clustering.

2. The method according to claim 1, characterized in that: The extracting of voiceprint information of each voice segment includes: Extracting speech features from each speech segment; The speech features are fed into an x-vector network, and the voiceprint information of each speech segment is output.

3. The method according to claim 1, characterized in that The step of clustering each of the speech segments based on a loss function to obtain a target speech segment comprises: The inverse of the plda score is used as the loss function to cluster the voiceprints in each speech segment set to obtain the target speech segment, wherein the target speech segment includes one or more.

4. A voiceprint clustering device, characterized in that: include: An acquiring unit, configured to acquire first identification information of a user whose target voiceprint is known; A calculation unit, used to extract the voiceprint information of each speech segment, and use the voiceprint model to calculate the similarity between two voiceprints; A correction unit, configured to correct the similarity using the known target voiceprint; The correction unit includes: a first correction module, used to set the similarity of the voice segment consistent with the known voiceprint to infinitesimal; a second correction module, used to set the similarity of the voice segment inconsistent with the known voiceprint to infinite; A first clustering unit, configured to cluster each of the speech segments based on a loss function to obtain a target speech segment; A second clustering unit is used to determine a target speech segment whose similarity between the known target voiceprint information meets a preset condition, and cluster the target speech segment with the known target voiceprint; The first determining module is configured to determine the second identification information of each target speech segment when the target speech segment includes multiple target speech segments: The second determining module is used to determine the second identification information consistent with the first identification information as the target speech segment of the known target voiceprint clustering.

5. The device according to claim 4, characterized in that The computing unit comprises: An extraction module, used for extracting speech features from each speech segment; The output module is used to send the speech features into the x-vector network and output the voiceprint information of each speech segment.

6. The device according to claim 4, characterized in that The first clustering unit comprises: A clustering module is used to cluster the voiceprints in each speech segment set using the inverse of the plda score as the loss function to obtain the target speech segment, wherein the target speech segment includes one or more.

7. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, wherein the computer program is configured to execute the method according to any one of claims 1 to 3 when executed.

8. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to perform the method according to any one of claims 1 to 3.

Images