Voiceprint clustering method, voiceprint recognition method, device and electronic equipment

Abstract

The present application provides a voiceprint clustering method, a voiceprint recognition method, a device and an electronic device. The voiceprint clustering method includes: performing a transposition operation on a speech voiceprint feature vector to obtain a speech voiceprint feature vector after the transposition operation; performing an inner product operation on the speech voiceprint feature vector and the speech voiceprint feature vector after the transposition operation to obtain a cross-similarity matrix, wherein the cross-similarity matrix includes multiple rows of first feature vectors; traversing the first feature vectors and calculating the similarity between a target feature vector in multiple rows of first feature vectors and other rows of first feature vectors; clustering the target feature vector and the second feature vector to obtain a target speech voiceprint feature vector. In this way, the accuracy of using the target speech voiceprint feature vector for speaker search or training is improved.

Description

Technical Field

[0001] The present application relates to the technical field of voiceprint recognition, and in particular to a voiceprint clustering method, a voiceprint recognition method, a device and an electronic device. Background Art

[0002] Voiceprint information has gradually become an important identity information of a person. Therefore, the recognition of voiceprint information is becoming more and more important, and in daily life, the recognition of voiceprint information has been more and more widely used. However, in actual use, the voiceprint information is usually mixed with the voices of other speakers, which leads to poor accuracy in speaker search or training using the above voiceprint information. Summary of the invention

[0003] The embodiments of the present application provide a voiceprint clustering method, a voiceprint recognition method, a device and an electronic device.

[0004] In a first aspect, an embodiment of the present application provides a voiceprint clustering method, including:

[0005] Performing a transposition operation on the speech voiceprint feature vector to obtain a speech voiceprint feature vector after the transposition operation;

[0006] Performing an inner product operation on the speech voiceprint feature vector and the speech voiceprint feature vector after the transposition operation to obtain a cross-similarity matrix, wherein the cross-similarity matrix includes a plurality of rows of first feature vectors;

[0007] Traversing the first feature vector, and calculating the similarity between the target feature vector in multiple rows of the first feature vector and the first feature vector in other rows;

[0008] The target feature vector and the second feature vector are clustered to obtain a target speech voiceprint feature vector, wherein the second feature vector is a vector with the greatest similarity to the target feature vector among the first feature vectors in other rows, and the target feature vector and the second feature vector correspond to the same speaker.

[0009] It can be seen that in this embodiment, the target feature vector and the second feature vector belonging to the same speaker can be clustered to obtain a target speech voiceprint feature vector, thereby reducing the interference of other speakers in the target speech voiceprint feature vector, thereby improving the accuracy of speaker search or training using the target speech voiceprint feature vector.

[0010] In addition, since the second eigenvector is the vector with the greatest similarity with the target eigenvector among the first eigenvectors in other rows, that is, the second eigenvector and the target eigenvector have the most overlapping content, the two are clustered to minimize the loss of vector information due to clustering. At the same time, the number of vectors can be reduced, vector storage space can be saved, and the number of comparisons in subsequent voiceprint recognition can be reduced to improve the efficiency of voiceprint recognition.

[0011] In a second aspect, an embodiment of the present application provides a voiceprint recognition method, the method comprising:

[0012] Inputting the voice file to be recognized into the voiceprint registration network to obtain a first voiceprint feature vector;

[0013] The first speech voiceprint feature vector is input into the voiceprint recognition model for recognition processing, and a recognition result is output; wherein the recognition processing is specifically implemented by: comparing the first language voiceprint feature vector with all target speech voiceprint feature vectors associated with the voiceprint recognition model; determining the recognition result according to the feature vector comparison result; the target speech voiceprint feature vector is determined according to the voiceprint clustering method described in the first aspect.

[0014] It can be seen that in this embodiment, since the target speech voiceprint feature vector is obtained through voiceprint clustering, the interference of other speakers has been reduced, and voiceprint recognition using the target speech voiceprint feature vector can improve recognition accuracy.

[0015] In a third aspect, the present application also provides a voiceprint clustering device, including:

[0016] A transposition module, used for performing a transposition operation on the speech voiceprint feature vector to obtain the speech voiceprint feature vector after the transposition operation;

[0017] An inner product module, used for performing an inner product operation on the speech voiceprint feature vector and the speech voiceprint feature vector after the transposition operation to obtain a cross-similarity matrix, wherein the cross-similarity matrix includes a plurality of rows of first feature vectors;

[0018] A calculation module, used for traversing the first feature vectors, and calculating the similarity between the target feature vectors in a plurality of rows of the first feature vectors and the first feature vectors in other rows;

[0019] A clustering module is used to cluster the target feature vector and the second feature vector to obtain a target speech voiceprint feature vector, wherein the second feature vector is the vector with the greatest similarity with the target feature vector among the first feature vectors in other rows, and the target feature vector and the second feature vector correspond to the same speaker.

[0020] In a fourth aspect, an embodiment of the present application further provides a voiceprint recognition device, the voiceprint recognition device comprising:

[0021] A voiceprint registration module, used for inputting a voice file to be recognized into a voiceprint registration network to obtain a first voice voiceprint feature vector;

[0022] A recognition module is used to input the first speech voiceprint feature vector into a voiceprint recognition model for recognition processing and output a recognition result; wherein the recognition processing is specifically implemented by: comparing the first language voiceprint feature vector with all target speech voiceprint feature vectors associated with the voiceprint recognition model; determining the recognition result according to the feature vector comparison result; the target speech voiceprint feature vector is determined according to the voiceprint clustering method described in the first aspect.

[0023] In a fifth aspect, an embodiment of the present application further provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps in the above-mentioned voiceprint clustering method or the above-mentioned voiceprint recognition method when executing the computer program.

[0024] In a sixth aspect, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps in the above-mentioned voiceprint clustering method or the above-mentioned voiceprint recognition method are implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0026] Figure 1 is a flow chart of a voiceprint clustering method provided in an embodiment of the present application;

[0027] Figure 2 is a flow chart of another voiceprint recognition method provided by an embodiment of the present application;

[0028] Figure 3 It is a structural schematic diagram of a voiceprint clustering device provided in an embodiment of the present application;

[0029] Figure 4 It is a structural schematic diagram of a voiceprint recognition device provided in an embodiment of the present application;

[0030] Figure 5It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0031] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0032] See also Figure 1 , Figure 1 is a flow chart of the voiceprint clustering method provided in the embodiment of the present application. Figure 1 As shown, the method is applied to an electronic device and comprises the following steps:

[0033] Step 101: perform a transposition operation on the speech voiceprint feature vector to obtain the speech voiceprint feature vector after the transposition operation.

[0034] Among them, the speech voiceprint feature vector can also be called a voiceprint feature vector, and the method of obtaining the speech voiceprint feature vector is not limited here. For example, the speech voiceprint feature vector can be directly obtained from the sample pool, that is, the speech voiceprint vector is a speech voiceprint feature vector pre-existing in the sample pool; in addition, the speech voiceprint feature vector can also be extracted from the audio.

[0035] As an optional implementation, before performing a transposition operation on the speech voiceprint feature vector to obtain the speech voiceprint feature vector after the transposition operation, the method further includes:

[0036] Input N recordings into the speaker segmentation model and output M speech segments, each of which corresponds to only one speaker;

[0037] Merging the M speech segments according to the corresponding speakers to obtain L speech files, each speech file corresponding to only one speaker;

[0038] Input the target voice file among the L voice files into the voiceprint registration network to obtain the voice voiceprint feature vector, the target voice file corresponds to the voice voiceprint feature vector one by one, and M, N and L are all positive integers.

[0039] The specific structure of the voiceprint registration network is not limited here. For example, the voiceprint registration network can be a residual network (Resnet) voiceprint recognition network, or a time-delay neural network (TDNN) voiceprint recognition network, thereby enhancing the diversity and flexibility of the voiceprint registration network.

[0040] The specific type of recording is not limited here. For example, the recording may refer to an audio file recorded by a user during a call with customer service, or a call recording automatically recorded by a server.

[0041] Among them, the structure of the speaker segmentation model is not limited here. The speaker segmentation model is mainly used to segment the recording into multiple speech segments according to the identities of the speakers in the recording, and each speech segment can correspond to only one speaker, and the above-mentioned speech segments can be divided according to different time periods.

[0042] For example: if user A and user B have a continuous conversation between the first moment and the second moment, and between the third moment and the fourth moment of the recording, the recording can be divided into a first voice segment, a second voice segment, a third voice segment and a fourth voice segment, wherein the first voice segment can be a voice segment of user A between the first moment and the second moment, the second voice segment can be a voice segment of user B between the first moment and the second moment, the third voice segment can be a voice segment of user A between the third moment and the fourth moment, and the fourth voice segment can be a voice segment of user B between the third moment and the fourth moment. It can be seen that the first voice segment, the second voice segment, the third voice segment and the fourth voice segment each correspond to only one speaker, and the voice segments can be divided according to time.

[0043] In addition, after obtaining the above multiple voice segments, the voice segments corresponding to the same speaker can be merged to obtain multiple voice files. For example, the above first voice segment and the third voice segment can be merged, and the second voice segment and the fourth voice segment can be merged.

[0044] The target voice file obtained after merging is input into the voiceprint registration network for voiceprint feature extraction, so as to obtain the corresponding voiceprint feature vector.

[0045] The target voice file may be one voice file or at least two voice files in the voice files. When the target voice files are at least two voice files, the voice files may be sequentially input into the voiceprint registration network to extract the voiceprint feature vector, and then sequentially perform steps 101 to 104 of this embodiment. Of course, at least two voice files may also be synthesized and then input into the voiceprint registration network to extract the voiceprint feature vector to obtain the voice voiceprint feature vector. The specific method is not limited here.

[0046] In this embodiment, the speaker segmentation is performed by inputting the recording into the speaker segmentation model to obtain multiple speech segments, and then the speech segments corresponding to the same speaker are merged to obtain a speech file, and the speech voiceprint feature vector is extracted based on the speech file. In this way, the speech segments can be segmented and reorganized, so that the speaker in the speech file obtained after the reorganization and merging is unique, that is, the interference of other speakers is reduced, thereby improving the cleanliness of the speech voiceprint feature vector obtained according to the speech file. When the subsequent voiceprint clustering is performed, the cleanliness of the target speech voiceprint feature vector is also higher, and the fewer the number of speakers, the higher the cleanliness of the speech voiceprint feature.

[0047] It should be noted that the above cleanliness can also be referred to as purity.

[0048] In one embodiment, the model to be trained includes a first linear layer, a first coding layer, a target network layer, a cross entropy loss network and a supervised clustering loss network; the first coding layer is connected to the first linear layer, the first coding layer is connected to the cross entropy loss network through the target network layer, and the first coding layer is connected to the supervised clustering loss network; the training steps of the above speaker segmentation model may include:

[0049] The audio features are input into the model to be trained for training; wherein, in the Nth training process, the following steps are performed:

[0050] The first linear layer is used to perform feature mapping on the audio features to obtain a fourth feature vector; the first encoding layer is used to perform attention mechanism fitting on the fourth feature vector to obtain the fifth feature vector; the target network layer and the cross entropy loss network are used to obtain a cross entropy loss value according to the fifth feature vector; the supervised clustering loss network is used to obtain a clustering loss value according to the fifth feature vector;

[0051] When the sum of the cross entropy loss value and the clustering loss value is less than a first preset value, or the weighted sum of the cross entropy loss value and the clustering loss value is less than a second preset value, the model to be trained after the Nth training can be determined as a speaker segmentation model.

[0052] It should be noted that the target network layer can process the input fifth feature vector and output a speech voiceprint feature vector, so that the speaker segmentation result can be determined based on the above speech voiceprint feature vector.

[0053] As an optional implementation, the target network includes a second encoding layer and a second linear layer connected in sequence;

[0054] The second encoding layer is used to perform attention mechanism fitting on the input fifth eigenvector and output a sixth eigenvector;

[0055] The second linear layer is used to perform feature mapping on the input sixth eigenvector and output the target eigenvector.

[0056] In this implementation, since the target network layer includes a second coding layer and a second linear layer, the vector can be further feature mapped and the attention mechanism can be fitted through the second coding layer and the second linear layer, thereby further improving the accuracy of the final cross entropy loss value; in addition, since the target network layer only includes one second coding layer and one second linear layer, the number of network layers of the target network layer is reduced, saving the computing resources of the target network layer.

[0057] As an optional implementation, the target network layer includes: M second coding layers and M second linear layers, where M is a positive integer;

[0058] If M is greater than 1, a second linear layer connection is provided between any two adjacent second coding layers;

[0059] If the M is equal to 1, the output of the second coding layer is the input of the second linear layer.

[0060] The second coding layer may perform the same operation as the first coding layer, and the second linear layer may perform the same operation as the first linear layer.

[0061] Among them, the structure connected to the first encoding layer may be the second encoding layer, and the structure connected to the cross entropy loss network may be the second linear layer.

[0062] In this implementation, since the target network layer includes M second encoding layers and M second linear layers, the vector can be further feature mapped and the attention mechanism can be fitted, thereby further improving the accuracy of the final cross entropy loss value.

[0063] As another optional implementation, the target network layer only includes: a second coding layer and a second linear layer. In this way, since the target network layer only includes a second coding layer and a second linear layer, the number of network layers of the target network layer is reduced, saving computing resources of the target network layer.

[0064] As an optional implementation, before inputting the target voice file among the L voice files into the voiceprint registration network to obtain the voice voiceprint feature vector, the method further includes:

[0065] The target voice file is determined according to the speaker types of the L voice files.

[0066] The speaker type of the target voice file is the main speaker.

[0067] The speaker type of each voice file can be obtained, so that the speaker corresponding to the target voice file is determined according to the speaker category of the voice file.

[0068] It should be noted that each voice file corresponds to only one speaker, and each voice segment of each voice file obtained by merging also corresponds to only one speaker.

[0069] For example, when L is 7, the speaker types are 0, 0, 0, 1, 0, 0, and 0 respectively, indicating that except for the speaker corresponding to the fourth voice file, which is a non-main speaker, the speakers corresponding to other voice files are all main speakers, that is, the voice files except the fourth voice file can be merged.

[0070] In this implementation, the target voice file is determined by the speaker type of the voice file, that is, the voice file can be preliminarily screened first, without the need to extract voice features for each voice file, thus reducing the workload and improving the efficiency of voiceprint clustering.

[0071] As an optional application scenario, the recording generally includes the voices of the main customer service and the user, and at the same time, the voices of other customer service are also mixed in. The main customer service and the user can be called the main speakers, and other customer service can be called non-main speakers. It is obvious that the main customer service and the user, that is, the main speaker mentioned above, generally speak the most in the recording. Therefore, the speaker category that corresponds to the most in the voice file is generally the main speaker, and the voice file corresponding to the main speaker can be determined as the target voice file for subsequent voiceprint clustering, and stored in the database via the target voice voiceprint feature vector to facilitate subsequent comparison.

[0072] As an optional implementation, performing a transposition operation on the speech voiceprint feature vector to obtain the speech voiceprint feature vector after the transposition operation includes:

[0073] Normalize the voiceprint feature vector;

[0074] The speech voiceprint feature vector after the normalization operation is transposed to obtain the speech voiceprint feature vector after the transposition operation.

[0075] The normalization process is mainly to process different types of feature vectors, so that the feature vectors of different types are all converted into feature vectors of the same type to facilitate subsequent transposition operations.

[0076] In this embodiment, by normalizing the speech voiceprint feature vector, the type of the speech voiceprint feature vector after the normalization operation can be made more unified, thereby facilitating subsequent transposition, reducing the occurrence of transposition failure, and thereby improving the efficiency of obtaining the cross-similarity matrix.

[0077] Step 102: performing an inner product operation on the speech voiceprint feature vector and the speech voiceprint feature vector after the transposition operation to obtain a cross-similarity matrix, wherein the cross-similarity matrix includes a plurality of rows of first feature vectors.

[0078] It should be noted that, compared with the method of clustering a speech voiceprint feature vector with other speech voiceprint feature vectors, it is necessary to compare other speech voiceprint feature vectors one by one to find other speech voiceprint feature vectors that may belong to the same speaker as the speech voiceprint feature vector, in the embodiment of the present application, an inner product operation is performed on the speech voiceprint feature vector and the speech voiceprint feature after the transposition operation to obtain a cross-similarity matrix, and then referring to the similarity between the target feature vector and the first feature vector of other rows in the cross-similarity matrix as described later, the rate of obtaining the similarity can be greatly improved; at the same time, since the target feature vector is clustered with the second feature vector, and the second feature vector is the vector with the greatest similarity with the target feature vector among the first feature vectors of other rows, that is, the target feature vector and the second feature vector are most likely to be from the same speaker, thereby improving the accuracy and efficiency of clustering.

[0079] In addition, if each first eigenvector has a corresponding score (ie, the score referred to below), using the score to obtain the similarity can further improve the speed of obtaining the similarity.

[0080] In addition, the target feature vector and the second feature vector are clustered to minimize the loss of vector information due to clustering. At the same time, the number of vectors can be reduced, vector storage space can be saved, and the number of comparisons in subsequent voiceprint recognition can be reduced to improve the efficiency of voiceprint recognition.

[0081] Among them, each first feature vector can correspond to a voice print feature vector of a voice file, that is to say: different first feature vectors can correspond to the same speaker or different speakers. When different first feature vectors correspond to the same speaker, different first feature vectors can correspond to different time periods or different scenes of the speaker.

[0082] The speech voiceprint feature vector refers to the speech voiceprint feature vector before the transposition operation.

[0083] Step 103: traverse the first feature vectors, and calculate the similarity between the target feature vectors in multiple rows of the first feature vectors and the first feature vectors in other rows.

[0084] Among them, each row of the first eigenvector can be called a class, and calculating the similarity between the target feature vector in the first eigenvector and other first eigenvectors can also be called calculating the distance between the target feature vector and the first eigenvectors in other rows, and the distance can be used to characterize the similarity between the target feature vector and the first eigenvectors in other rows.

[0085] It should be noted that the number of first feature vectors included in each row of first feature vectors may be at least one, and the target feature vector may refer to at least one first feature vector included in each row of first feature vectors. That is to say: when the number of first feature vectors included in a row of first feature vectors is one, the target feature vector refers to one of the first feature vectors included in the row of first feature vectors (for example, the first feature vector of the row, the last feature vector of the row, not limited here); when the number of first feature vectors included in a row of first feature vectors is multiple, the target feature vector may refer to at least some of the multiple first feature vectors included in the row of first feature vectors (for example, the first feature vector of the row, the last feature vector of the row, the first two feature vectors of the row, the last two feature vectors of the row, all feature vectors of the row, feature vectors other than the last feature vector of the row, feature vectors other than the first feature vector of the row, not limited here).

[0086] In the process of traversing the first feature vectors, each first feature vector can be determined as a target feature vector in turn, that is to say, the number of the above-mentioned target feature vectors can match the number of the first feature vectors. In this way, when each first feature vector is determined as a target feature vector, each step in this embodiment can be executed.

[0087] The calculation of the similarity between the target feature vector and the first feature vector of other rows can be described as follows:

[0088] The target feature vector and the first feature vector can be converted into corresponding numerical values ​​respectively, and then the similarity can be calculated according to the magnitude of the numerical values; for example, the similarity can be equal to the ratio between the numerical value corresponding to the target feature vector and the numerical value corresponding to the first feature vector. When the numerical value corresponding to the target feature vector is 80 and the numerical value corresponding to the first feature vector is 100, the similarity between the two can be 0.8.

[0089] In addition, the scores corresponding to the target feature vector and the first feature vector may be directly obtained, and then the similarity is calculated based on the scores. It should be noted that the scores may be an estimated value of the similarity between each first feature vector and other first feature vectors. Of course, the scores may also be a value obtained by quantifying each first feature vector according to a certain indicator.

[0090] For example: when the number of first eigenvectors is 3, the 3 first eigenvectors can be quantized into numerical values ​​according to data such as length or dimension. If the dimension of the first first eigenvector is 3, the score of the first first eigenvector can be rated as 30. If the dimension of the second first eigenvector is 4, the score of the second first eigenvector can be rated as 40. If the dimension of the third first eigenvector is 5, the score of the third first eigenvector can be rated as 50.

[0091] As an optional implementation manner, the calculating the similarity between the target feature vector in the plurality of rows of the first feature vectors and the first feature vectors in other rows includes:

[0092] The similarity between the target feature vector and the first feature vectors in other rows is calculated according to the score of the target feature vector and the scores of the first feature vectors in other rows.

[0093] The score of the first eigenvector may be a score obtained by scoring according to parameters such as the eigenvalue of each first eigenvector.

[0094] Of course, the above score can also be the score calculated by matching each first eigenvector with all the first eigenvectors in other rows one by one, that is, the above score can be the average, weighted average, sum or weighted sum obtained after matching the first eigenvector with all the first eigenvectors in other rows one by one. No specific limitation is made here.

[0095] The calculation of similarity based on scores may refer to the above description of calculating similarity based on numerical values, that is, the similarity is determined based on the ratio between the two, and the details will not be repeated here.

[0096] In addition, when the score of the target feature vector is compared with the score of the first feature vector of other rows, the score corresponding to the target feature vector and the score corresponding to the first feature vector can be multiplied by the corresponding weights to obtain their respective products, and then the similarity is determined according to the ratio of the products.

[0097] The above weights may be the same or different, and the weights may be determined based on the distance between the target feature vector and the first feature vector. For example, when the target feature vector is the first feature vector in the first row, when calculating the similarity between the target feature vector and the first feature vector in the second row, the weights may be 50% respectively; when calculating the similarity between the target feature vector and the first feature vector in the third row, the weight of the target feature vector may be 40%, and the weight of the first feature vector in the third row may be 60%. The above is merely an exemplary description, and the specific method for determining the weights is not limited herein.

[0098] In this embodiment, since the cross-similarity matrix also includes scores corresponding to the first eigenvectors one by one, the similarity between the target eigenvector and the first eigenvector can be calculated by obtaining the scores corresponding to the first eigenvectors, thereby improving the calculation efficiency and accuracy of the similarity.

[0099] Step 104: cluster the target feature vector and the second feature vector to obtain a target speech voiceprint feature vector, wherein the second feature vector is the vector with the greatest similarity to the target feature vector among the first feature vectors in other rows, and the target feature vector and the second feature vector correspond to the same speaker.

[0100] Among them, the specific method of determining the second eigenvector is not limited here. For example, the second eigenvector can be the vector with the greatest similarity to the target eigenvector. When the similarity is calculated by score, the second eigenvector can also be the vector with the largest score among the first eigenvectors in other rows, or the second eigenvector can be the vector with the smallest score difference with the target eigenvector among the first eigenvectors in other rows.

[0101] As an optional implementation manner, clustering the target feature vector and the second feature vector to obtain the target speech voiceprint feature vector includes:

[0102] When the difference between the score corresponding to the target feature vector and the score corresponding to the second feature vector is greater than a preset threshold, clustering the target feature vector and the second feature vector to obtain a target speech voiceprint feature vector;

[0103] When the difference between the score corresponding to the target feature vector and the score corresponding to the second feature vector is less than or equal to a preset threshold, clustering the target feature vector and the second feature vector is stopped.

[0104] Among them, when the difference between the score corresponding to the target feature vector and the score corresponding to the second feature vector is less than a preset threshold, it means that the similarity between the target feature vector and the second feature vector is high, and it can be considered that they belong to the same speaker, that is, the two can be merged; conversely, when the difference between the score corresponding to the target feature vector and the score corresponding to the second feature vector is greater than or equal to the preset threshold, it means that the similarity between the target feature vector and the second feature vector is low, and it can be considered that they belong to different speakers, so the two do not need to be merged, thereby ensuring the accuracy of vector merging.

[0105] In this embodiment, by judging whether the difference between the score corresponding to the target feature vector and the score corresponding to the second feature vector is less than a preset threshold, it is decided whether to merge the target feature vector with the second feature vector, thereby improving the accuracy of the merging of the target feature vector and the second feature vector and reducing the occurrence of merging the target feature vector and the second feature vector that do not belong to the same speaker.

[0106] In this embodiment, through steps 101 to 104, the target feature vector and the second feature vector belonging to the same speaker can be clustered to obtain the target speech voiceprint feature vector, reducing the interference of other speakers in the target speech voiceprint feature vector, thereby improving the accuracy of speaker search or training using the target speech voiceprint feature vector. In addition, since the second feature vector is the vector with the greatest similarity with the target feature vector among the first feature vectors in other rows, that is, the second feature vector and the target feature vector have the most overlapping content, the two are clustered to minimize the occurrence of the phenomenon of vector information loss caused by clustering. At the same time, the number of vectors can be reduced, the vector storage space can be saved, and the number of comparisons in subsequent voiceprint recognition can be increased to improve the efficiency of voiceprint recognition.

[0107] In addition, the target feature vector and the second feature vector belonging to the same speaker can be clustered to obtain a target speech voiceprint feature vector, and the above target speech voiceprint feature vector can be stored in a database, reducing the number of vectors belonging to the same speaker, thereby reducing the storage space occupied by the above vectors. The same speaker can be clustered into a target speech voiceprint feature vector, which can make it more convenient to compare the target speech voiceprint feature vector with other voiceprints in the future.

[0108] The present application is illustrated below by taking a specific embodiment as an example.

[0109] Step 1: First, L2 normalize the N*M dimensional speech feature vector, and then perform a transposition operation to obtain a transposed feature vector of M*N dimension. Perform an inner product operation on the feature vector before transposition and the vector after transposition to obtain a cross-similarity matrix, and the dimension of the cross-similarity matrix is ​​N*N.

[0110] Step 2: The first eigenvector of each row in the N*N dimensional cross-similarity matrix corresponds to a corresponding score, and the above score can be the similarity score result of the first eigenvector to all other first eigenvectors, or the above score can be a score obtained by scoring various parameters of each first eigenvector, and the above parameters include parameters such as the value of the eigenvalue.

[0111] Step 3: Calculate the distance between each two first eigenvectors (the ratio can be calculated based on the corresponding scores. The larger the ratio, the larger the distance, and the smaller the ratio, the smaller the distance. The above distance can also be called similarity), and merge the two first eigenvectors with the closest distance (that is, clustering);

[0112] Step 4: Repeat step 3 until the distance between the two closest first feature vectors to be merged is less than a preset threshold, and then stop clustering.

[0113] See also Figure 2 , Figure 2 is a flow chart of a voiceprint recognition method provided by an embodiment of the present application. Figure 2 As shown, the following steps are included:

[0114] Step 201: input the voice file to be recognized into the voiceprint registration network to obtain a first voiceprint feature vector.

[0115] The voiceprint registration network may extract the voiceprint of the first voice file, thereby obtaining a first voiceprint feature vector.

[0116] The first speech voiceprint feature vector may be referred to as a vector to be compared or a voiceprint vector to be compared.

[0117] Step 202: input the first speech voiceprint feature vector into the voiceprint recognition model for recognition processing, and output the recognition result; wherein, the specific implementation method of the recognition processing is: compare the first language voiceprint feature vector with all target speech voiceprint feature vectors associated with the voiceprint recognition model; determine the recognition result according to the feature vector comparison result; the target speech voiceprint feature vector is determined according to the above-mentioned voiceprint clustering method.

[0118] Among them, the first voiceprint feature vector is compared with all target voiceprint feature vectors associated with the voiceprint recognition model, and the similarity between the first voiceprint feature vector and all target voiceprint feature vectors in the voiceprint recognition model can be calculated, for example, cosine similarity can be used.

[0119] Among them, all the target speech voiceprint feature vectors associated with the voiceprint recognition model can be stored in the database corresponding to the voiceprint recognition model, and can also be directly stored in the voiceprint recognition model.

[0120] The step of determining the recognition result according to the feature vector comparison result includes: determining the recognition result according to a vector having the highest similarity to the first language voiceprint feature vector.

[0121] The recognition result is the user associated with the vector with the highest similarity. For example, if the user associated with the vector with the highest similarity is A, then the recognition result is user A.

[0122] Optionally, before step 201, the method further includes: inputting the recording into a speaker segmentation model to obtain the first voice file;

[0123] The speaker segmentation model includes a first linear layer, a first encoding layer, a target network and an aggregation network; the first linear layer is used to perform feature mapping on the input recording and output a first target feature vector; the first encoding layer is used to perform attention mechanism fitting on the input first target feature vector and output a second target feature vector; the target network layer is used to process the input second target feature vector to obtain M speech segments; and is used to obtain the speech segment corresponding to the first speaker from the M speech segments and merge them to obtain a first speech file, wherein each speech segment corresponds to only one speaker and M is a positive integer.

[0124] The voiceprint segmentation process and merging process of the recording can refer to the corresponding description in the above embodiment, and will not be described in detail here.

[0125] It should be noted that the target network layer in the embodiment of the present application can directly process the input second feature vector and output the speech voiceprint feature vector (i.e., M speech segments). At the same time, the target network layer can also merge the speech segments corresponding to the same speaker in the above M speech segments, for example: merge the speech segments belonging to the first speaker to obtain the first voice file. In this way, the segmentation results of different speakers (i.e., different voice files corresponding to different speakers) are obtained.

[0126] Among them, in an optional scenario, the first voice file may refer to the seat in the recording (also referred to as customer service). When the customer service is complained by the customer, but the identity of the customer service is unknown, voiceprint segmentation can be performed from the recording and the segmented voice segments can be merged to obtain the first voice file, so as to facilitate the subsequent identity identification of the customer service.

[0127] Wherein, the voiceprint recognition model is connected to the speaker segmentation model through the voiceprint registration network.

[0128] In this embodiment, since the identity information of all target voiceprint feature vectors associated with the voiceprint recognition model is determined information, by comparing the first voiceprint feature vector with the target voiceprint feature vector to determine the second voiceprint feature vector, the identity information corresponding to the second voiceprint feature vector can be determined, so that the identity information of the first speaker corresponding to the first voice file can be accurately and quickly determined. In addition, since the first voice file is a voice segment corresponding to the first speaker of M voice segments, that is to say, the first voice file only corresponds to the first speaker, so that the first voiceprint feature vector corresponding to the first voice file is compared with the target voiceprint feature vector, the accuracy of the comparison result can be improved, thereby further improving the accuracy of the determination result of the identity information of the first speaker corresponding to the first voice file.

[0129] It should be noted that, when applied to the above-mentioned determination of the identity of the customer service, this embodiment can accurately and quickly determine the specific identity of the customer service, that is, complete the determination of the identity of the above-mentioned customer service being complained about.

[0130] As an optional implementation manner, comparing the first language voiceprint feature vector with all target speech voiceprint feature vectors associated with the voiceprint recognition model includes:

[0131] Compare the first language voiceprint feature vector with all target voiceprint feature vectors associated with the voiceprint recognition model to determine S second voice voiceprint feature vectors, wherein the second voice voiceprint feature vector is a vector of all target voiceprint feature vectors whose similarity with the first voice voiceprint feature vector is greater than a preset threshold, and S is a positive integer;

[0132] A third voice print feature vector is determined from the S second voice print feature vectors, where the third voice feature vector is a vector with the highest similarity, or any one of the vectors.

[0133] In this embodiment, it is also possible to first determine multiple second voice voiceprint feature vectors whose similarity with the first voice voiceprint feature vector is greater than a preset threshold, and then carefully screen the multiple second voice voiceprint feature vectors to determine the third voice voiceprint feature vector. In this way, the range of the target voiceprint feature vector can be narrowed first, and then the third voice voiceprint feature vector can be carefully screened within the above-mentioned narrowed range, thereby reducing the occurrence of the third voice voiceprint feature vector being missed, that is, further improving the accuracy of the second voice voiceprint feature vector determination result.

[0134] See also Figure 3 , Figure 3 is a structural diagram of the voiceprint clustering device provided in the embodiment of the present application, which can implement the details of the voiceprint clustering method in the above embodiment and achieve the same effect. Figure 3 As shown, the voiceprint clustering device 300 includes:

[0135] The transposition module 301 is used to perform a transposition operation on the speech voiceprint feature vector to obtain the speech voiceprint feature vector after the transposition operation;

[0136] An inner product module 302 is used to perform an inner product operation on the speech voiceprint feature vector and the speech voiceprint feature vector after the transposition operation to obtain a cross-similarity matrix, wherein the cross-similarity matrix includes a plurality of rows of first feature vectors;

[0137] A calculation module 303 is used to traverse the first feature vectors and calculate the similarity between the target feature vectors in multiple rows of the first feature vectors and the first feature vectors in other rows;

[0138] The clustering module 304 is used to cluster the target feature vector and the second feature vector to obtain a target speech voiceprint feature vector, wherein the second feature vector is the vector with the greatest similarity with the target feature vector among the first feature vectors in other rows, and the target feature vector and the second feature vector correspond to the same speaker.

[0139] Optionally, the cross-similarity matrix further includes: a plurality of scores, and the scores correspond one-to-one to the first eigenvectors;

[0140] The calculation module 303 is further configured to calculate the similarity between the target feature vector and the first feature vectors in other rows according to the score of the target feature vector and the scores of the first feature vectors in other rows.

[0141] Optionally, the clustering module 304 includes:

[0142] A clustering submodule, configured to cluster the target feature vector and the second feature vector to obtain a target speech voiceprint feature vector when the difference between the score corresponding to the target feature vector and the score corresponding to the second feature vector is less than a preset threshold;

[0143] The stop clustering submodule is used to stop clustering the target feature vector and the second feature vector when the difference between the score corresponding to the target feature vector and the score corresponding to the second feature vector is greater than or equal to a preset threshold.

[0144] Optionally, the voiceprint clustering device 300 further includes:

[0145] The output module is used to input N recordings into the speaker segmentation model and output M speech segments, each of which corresponds to only one speaker;

[0146] A merging module, used for merging the M speech segments according to the corresponding speakers to obtain L speech files, each of which corresponds to only one speaker;

[0147] The input module is used to input the target voice file among the L voice files into the voiceprint registration network to obtain the voice voiceprint feature vector, the target voice file corresponds to the voice voiceprint feature vector one by one, and M, N and L are all positive integers.

[0148] Optionally, the voiceprint clustering device 300 further includes:

[0149] A determination module is used to determine the target voice file according to the speaker types of the L voice files.

[0150] The voiceprint clustering device provided in the embodiment of the present application can achieve Figure 1 In order to avoid repetition, the various processes implemented by the voiceprint clustering device in the method embodiment will not be described here.

[0151] That is to say, this embodiment can cluster the target feature vector and the second feature vector belonging to the same speaker to obtain the target speech voiceprint feature vector, thereby reducing the interference of other speakers in the target speech voiceprint feature vector, thereby improving the accuracy of speaker search or training using the target speech voiceprint feature vector.

[0152] See also Figure 4 , Figure 4 A schematic diagram of the structure of a voiceprint recognition device provided in an embodiment of the present application is shown in FIG. Figure 4 As shown, the voiceprint recognition device 400 includes:

[0153] The voiceprint registration module 401 is used to input the voice file to be recognized into the voiceprint registration network to obtain a first voice voiceprint feature vector;

[0154] The recognition module 402 is used to input the first speech voiceprint feature vector into the voiceprint recognition model for recognition processing and output a recognition result; wherein the recognition processing is specifically implemented by: comparing the first language voiceprint feature vector with all target speech voiceprint feature vectors associated with the voiceprint recognition model; determining the recognition result according to the feature vector comparison result; the target speech voiceprint feature vector is determined according to the above-mentioned voiceprint clustering method.

[0155] The voiceprint recognition device provided in the embodiment of the present application can achieve Figure 3 In order to avoid repetition, the various processes implemented by the voiceprint recognition device in the method embodiment will not be described here.

[0156] like Figure 5As shown, an embodiment of the present application further provides an electronic device, including a processor 501, a memory 502, and a computer program stored in the memory 502 and executable on the processor 501. When the computer program is executed by the processor 501, the computer program implements the various processes of the above-mentioned voiceprint clustering method embodiment or the various processes of the above-mentioned voiceprint recognition method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be described here.

[0157] The embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, each process of the above-mentioned voiceprint clustering method embodiment or each process of the above-mentioned voiceprint recognition method embodiment can be implemented, and the same technical effect can be achieved. To avoid repetition, it is not repeated here. Among them, the computer-readable storage medium is, for example, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc.

[0158] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0159] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0160] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A voiceprint clustering method, characterized in that: include: Performing a transposition operation on the speech voiceprint feature vector to obtain a speech voiceprint feature vector after the transposition operation; Performing an inner product operation on the speech voiceprint feature vector and the speech voiceprint feature vector after the transposition operation to obtain a cross-similarity matrix, wherein the cross-similarity matrix includes a plurality of rows of first feature vectors; Traversing the first feature vector, and calculating the similarity between the target feature vector in multiple rows of the first feature vector and the first feature vector in other rows; The target feature vector and the second feature vector are clustered to obtain a target speech voiceprint feature vector, wherein the second feature vector is a vector with the greatest similarity to the target feature vector among the first feature vectors in other rows, and the target feature vector and the second feature vector correspond to the same speaker.

2. The method according to claim 1, characterized in that The calculating of the similarity between the target feature vector in the plurality of rows of the first feature vectors and the first feature vectors in other rows comprises: The similarity between the target feature vector and the first feature vectors in other rows is calculated according to the score corresponding to the target feature vector and the scores of the first feature vectors in other rows.

3. The method according to claim 2, characterized in that The step of clustering the target feature vector and the second feature vector to obtain a target speech voiceprint feature vector includes: When the difference between the score corresponding to the target feature vector and the score corresponding to the second feature vector is less than a preset threshold, clustering the target feature vector and the second feature vector to obtain a target speech voiceprint feature vector; When the difference between the score corresponding to the target feature vector and the score corresponding to the second feature vector is greater than or equal to a preset threshold, clustering the target feature vector and the second feature vector is stopped.

4. The method according to claim 1, characterized in that: Before performing a transposition operation on the speech voiceprint feature vector to obtain the speech voiceprint feature vector after the transposition operation, the method further includes: Input N recordings into the speaker segmentation model and output M speech segments, each of which corresponds to only one speaker; Merging the M speech segments according to the corresponding speakers to obtain L speech files, each speech file corresponding to only one speaker; Input the target voice file among the L voice files into the voiceprint registration network to obtain the voice voiceprint feature vector, the target voice file corresponds to the voice voiceprint feature vector one by one, and M, N and L are all positive integers.

5. The method according to claim 4, characterized in that Before inputting the target voice file among the L voice files into the voiceprint registration network to obtain the voice voiceprint feature vector, the method further includes: The target voice file is determined according to the speaker types of the L voice files.

6. A voiceprint recognition method, characterized in that: The method comprises: Inputting the voice file to be recognized into the voiceprint registration network to obtain a first voiceprint feature vector; The first speech voiceprint feature vector is input into the voiceprint recognition model for recognition processing, and a recognition result is output; wherein the recognition processing is specifically implemented by: comparing the first language voiceprint feature vector with all target speech voiceprint feature vectors associated with the voiceprint recognition model; determining the recognition result according to the feature vector comparison result; the target speech voiceprint feature vector is determined according to the voiceprint clustering method according to any one of claims 1-5.

7. A voiceprint clustering device, characterized in that: include: A transposition module, used for performing a transposition operation on the speech voiceprint feature vector to obtain the speech voiceprint feature vector after the transposition operation; An inner product module, used for performing an inner product operation on the speech voiceprint feature vector and the speech voiceprint feature vector after the transposition operation to obtain a cross-similarity matrix, wherein the cross-similarity matrix includes a plurality of rows of first feature vectors; A calculation module, used for traversing the first feature vectors, and calculating the similarity between the target feature vectors in a plurality of rows of the first feature vectors and the first feature vectors in other rows; A clustering module is used to cluster the target feature vector and the second feature vector to obtain a target speech voiceprint feature vector, wherein the second feature vector is the vector with the greatest similarity with the target feature vector among the first feature vectors in other rows, and the target feature vector and the second feature vector correspond to the same speaker.

8. A voiceprint recognition device, characterized in that: include: A voiceprint registration module, used for inputting a voice file to be recognized into a voiceprint registration network to obtain a first voice voiceprint feature vector; a recognition module, configured to input the first speech voiceprint feature vector into a voiceprint recognition model for recognition processing and output a recognition result; wherein the recognition processing is specifically implemented by: comparing the first language voiceprint feature vector with all target speech voiceprint feature vectors associated with the voiceprint recognition model; The recognition result is determined according to the feature vector comparison result; the target speech voiceprint feature vector is determined according to the voiceprint clustering method according to any one of claims 1-5.

9. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the voiceprint clustering method as described in any one of claims 1 to 5 are implemented, or when the processor executes the computer program, the steps of the voiceprint recognition method as described in claim 6 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the voiceprint clustering method according to any one of claims 1 to 5, or, when executed by a processor, implements the steps of the voiceprint recognition method according to claim 6.

Images