A voiceprint contrast recognition method, device and equipment and a storage medium thereof

Abstract

The embodiment of the application belongs to the technical field of voice recognition, is applied to a voice comparison recognition scene, and relates to a voiceprint comparison recognition method, device and equipment and a storage medium thereof. The voiceprint comparison recognition method comprises the following steps: obtaining to-be-compared voice; performing voiceprint feature extraction; performing linear mapping layer mapping to obtain high-dimensional feature vector representation; inputting the high-dimensional feature vector representation into a feature representation fusion model to obtain feature comprehensive representation corresponding to the to-be-compared voice; calculating a distance value between the feature comprehensive representation corresponding to the to-be-compared voice; and identifying voice segments of the same object from the to-be-compared voice based on the distance value. Compared with direct comparison of voiceprint features, comparison based on feature comprehensive representation can reduce the calculation of comparison quantities to a certain extent and improve comparison recognition efficiency. The voiceprint comparison recognition method of the application is applied to the technical field of biometric recognition, for example, a bank or financial service voice unlocking scene, and can more safely provide fund security guarantee services for customers.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of speech recognition, and is applied to a speech comparison recognition scene, and particularly relates to a voiceprint comparison recognition method and device, equipment and a storage medium thereof. BACKGROUND

[0002] With the rapid development of the Internet, the financial industry has also developed rapidly. Traditional security encryption and decryption methods are increasingly developing in the direction of biometric features, such as face recognition, fingerprint recognition, and voiceprint recognition. At present, there has been great development and application in face recognition and fingerprint recognition, but the application range of voiceprint encryption and decryption is still relatively limited, and common applications are mainly in the field of mobile phone voice unlocking.

[0003] The existing voiceprint recognition method is to convert the speaker's voice data into specific feature information, and compare it with the identity information corresponding to each person pre-stored in the database, so as to identify the identity of the corresponding person. According to specific tasks, voiceprint recognition can be divided into semantic-related and semantic-independent voiceprint recognition. Template matching, Gaussian mixture model, and joint factor method are commonly used to realize voiceprint recognition model modeling and prediction, and deep learning is also used to realize end-to-end voiceprint feature extraction. However, the above methods have great limitations in voiceprint feature modeling. The extracted voiceprint features have great shortcomings in robustness and accuracy, and the model structure is also relatively complex. Therefore, the existing technology still has the problems of complex modeling, large comparison recognition calculation amount, and low recognition efficiency in voiceprint comparison recognition. SUMMARY

[0004] The purpose of the embodiments of the present application is to propose a voiceprint comparison recognition method, device, equipment and storage medium to solve the problems of complex modeling, large comparison recognition calculation amount, and low recognition efficiency in voiceprint comparison recognition of the existing technology.

[0005] To solve the above technical problems, the voiceprint comparison recognition method provided by the embodiments of the present application adopts the following technical solutions:

[0006] A voiceprint comparison recognition method includes the following steps:

[0007] Obtaining a voice to be compared, wherein the voice to be compared includes at least two voice segments;

[0008] Inputting the voice to be compared into a preset voiceprint feature extraction network to extract voiceprint features, and obtaining a voiceprint feature extraction result;

[0009] mapping the voiceprint feature extraction result by using a preset linear mapping layer to obtain a high-dimensional feature vector representation corresponding to the voice to be compared respectively;

[0010] inputting the high-dimensional feature vector representation into a pre-trained feature representation fusion model to obtain a feature comprehensive representation corresponding to the voice to be compared respectively, wherein the feature representation fusion model is composed of N feature classification sub-models, and N is a positive integer;

[0011] calculating the distance value between the feature comprehensive representations corresponding to the voice to be compared respectively by using a preset distance algorithm;

[0012] identifying the voice segments of the same object from the voice to be compared based on the distance value.

[0013] Further, before the step of inputting the high-dimensional feature vector representation into a pre-trained feature representation fusion model to obtain a feature comprehensive representation corresponding to the voice to be compared respectively, the method further comprises:

[0014] Step 301, obtaining voiceprint feature training data, wherein the voiceprint feature training data includes at least one voice segment;

[0015] Step 302, inputting the voiceprint feature training data into the voiceprint feature extraction network respectively for voiceprint feature extraction to obtain voiceprint feature extraction results;

[0016] Step 303, mapping the voiceprint feature extraction result by using the linear mapping layer to obtain a high-dimensional feature vector representation corresponding to the voiceprint feature training data;

[0017] Step 304, inputting the high-dimensional feature vector representation corresponding to the voiceprint feature training data into N pre-constructed feature classification sub-models to obtain voiceprint feature data extracted by the N feature classification sub-models respectively;

[0018] Step 305, collating the voiceprint feature data extracted by the N feature classification sub-models respectively to obtain a voiceprint feature collation result;

[0019] Step 306, calculating a loss value of the voiceprint feature collation result compared with the voiceprint feature extraction result according to a preset loss function;

[0020] Step 307, if the loss value exceeds a preset loss threshold, adjusting the classification extraction parameters of the N feature classification sub-models again, and repeating steps 304 to 306 until the loss value does not exceed the preset loss threshold, and then the N feature classification sub-models are pre-trained;

[0021] Step 308: If the loss value does not exceed the preset loss threshold, obtain the pre-trained N feature classification sub-models;

[0022] Step 309: Fuse the N pre-trained feature classification sub-models to obtain a preliminary fused feature representation fusion model;

[0023] Step 310: Based on the voiceprint feature data extracted by the N feature classification sub-models that have been pre-trained, set the feature extraction weights corresponding to each feature classification sub-model in the preliminary fusion feature representation fusion model until the feature extraction weights are set, and obtain the pre-trained feature representation fusion model.

[0024] Furthermore, before performing the step of inputting the high-dimensional feature vector representation corresponding to the voiceprint feature training data into N pre-constructed feature classification sub-models to obtain the voiceprint feature data extracted by the N feature classification sub-models respectively, the method further includes:

[0025] A GRL gradient inversion layer is set between the linear mapping layer and the preset feature classification extraction layer, wherein the preset feature classification extraction layer is the neural network layer in which the N feature classification sub-models are located.

[0026] Furthermore, the step of setting the feature extraction weights corresponding to each feature classification sub-model in the preliminary fused feature representation fusion model based on the voiceprint feature data extracted from the N feature classification sub-models after pre-training specifically includes:

[0027] Select any dimension from all feature dimensions as the target dimension;

[0028] Calculate the amount of feature data extracted by each feature classification sub-model on the target dimension.

[0029] Calculate the proportion of the amount of feature data extracted by each feature classification sub-model in the target dimension to the total amount of feature data extracted by all feature classification sub-models in the target dimension;

[0030] The percentage value is set as the feature extraction weight of the corresponding feature classification sub-model on the target dimension.

[0031] Furthermore, the step of inputting the high-dimensional feature vector representation into a pre-trained feature representation fusion model to obtain the comprehensive feature representations corresponding to the speech to be compared specifically includes:

[0032] Identify all high-dimensional feature vector representations corresponding to the target speech segment, wherein the target speech is any speech segment in the speech to be compared;

[0033] Obtain the dimensional weight of each dimension of the feature vector representation in the feature representation fusion model;

[0034] Perform multiplication on the feature vector representation of each dimension and its corresponding dimension weight to obtain the multiplication result;

[0035] The multiplication results corresponding to all the high-dimensional feature vector representations are obtained and used as the feature synthesis representation of the target speech segment.

[0036] Furthermore, the step of obtaining the dimensional weights corresponding to the feature vector representations of each dimension in the feature representation fusion model in all the high-dimensional feature vector representations specifically includes:

[0037] Select any dimension from all feature dimensions as the target dimension;

[0038] Identify all feature classification sub-models that extract features for the target dimension, and use them as the target sub-model;

[0039] Obtain the feature extraction weights of all target sub-models corresponding to the target dimension;

[0040] The feature extraction weights of all target sub-models corresponding to the target dimension are used as feature vectors of the target dimension to represent the corresponding dimension weights in the feature representation fusion model.

[0041] Furthermore, the step of calculating the distance values ​​between the feature representations corresponding to the speech to be compared using a preset distance algorithm specifically includes:

[0042] The cosine distance algorithm is used to compare the feature representations of all speech segments in the speech to be compared pairwise to obtain the distance value between the feature representations of all speech segments.

[0043] The step of identifying speech segments of the same object from the speech to be compared based on the distance value specifically includes:

[0044] The distance values ​​between the feature representations corresponding to all speech segments are compared with a preset distance threshold.

[0045] By comparison, distance values ​​that are less than a preset distance threshold are selected as target distance values;

[0046] Based on the target distance value, the speech segments to be clustered are selected, and pairwise clustering is performed to obtain preliminary clustering results;

[0047] Based on the preliminary clustering results, determine whether the same speech segment is clustered into different clustering results;

[0048] If they exist, the clustering continues for the different clustering results corresponding to the same speech segment until no more speech segments are clustered into different clustering results, and the clustering is completed.

[0049] If it does not exist, then the clustering is complete, and the speech segments in the same cluster represent speech segments of the same object.

[0050] To address the aforementioned technical problems, this application also provides a voiceprint comparison and recognition device, which employs the following technical solution:

[0051] A voiceprint comparison and recognition device includes:

[0052] The comparison speech acquisition module is used to acquire the comparison speech, wherein the comparison speech includes at least two speech segments;

[0053] The voiceprint feature extraction module is used to input the speech to be compared into a preset voiceprint feature extraction network to extract voiceprint features and obtain voiceprint feature extraction results.

[0054] The feature vector mapping module is used to map the voiceprint feature extraction results using a preset linear mapping layer to obtain high-dimensional feature vector representations corresponding to the speech to be compared.

[0055] The feature fusion representation module is used to input the high-dimensional feature vector representation into the pre-trained feature representation fusion model to obtain the comprehensive feature representations corresponding to the speech to be compared, wherein the feature representation fusion model is composed of N feature classification sub-models, where N is a positive integer;

[0056] The distance calculation module is used to calculate the distance between the feature comprehensive representations corresponding to the speech to be compared using a preset distance algorithm;

[0057] The same object speech recognition module is used to identify speech segments of the same object from the speech to be compared based on the distance value.

[0058] To address the aforementioned technical problems, this application also provides a computer device that employs the following technical solution:

[0059] A computer device includes a memory and a processor, wherein the memory stores computer-readable instructions, and the processor executes the computer-readable instructions to implement the steps of the voiceprint comparison and recognition method described above.

[0060] To address the aforementioned technical problems, this application also provides a computer-readable storage medium, employing the technical solution described below:

[0061] A computer-readable storage medium storing computer-readable instructions, which, when executed by a processor, implement the steps of the voiceprint comparison and recognition method described above.

[0062] Compared with the prior art, the embodiments of this application have the following main advantages:

[0063] The voiceprint comparison and recognition method described in this application involves: acquiring the speech to be compared; extracting voiceprint features; mapping them through a linear mapping layer to obtain a high-dimensional feature vector representation; inputting the high-dimensional feature vector representation into a feature representation fusion model to obtain a comprehensive feature representation corresponding to each of the speech samples; calculating the distance between the comprehensive feature representations corresponding to each of the speech samples; and identifying speech segments of the same object from the speech samples based on the distance values. Comparing through comprehensive feature representations reduces the computational workload and improves the efficiency of comparison and recognition compared to direct voiceprint feature comparison. Applying this voiceprint comparison and recognition method to the field of biometrics, such as in banking or financial services voice unlocking scenarios, can provide more secure financial security services for customers. Attached Figure Description

[0064] To more clearly illustrate the solutions in this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0065] Figure 1 This is an exemplary system architecture diagram to which this application can be applied;

[0066] Figure 2 This is a flowchart of an embodiment of the voiceprint comparison and recognition method according to this application;

[0067] Figure 3 This is a flowchart of a specific embodiment of training the feature representation fusion model in the voiceprint comparison and recognition method described in this application;

[0068] Figure 4 yes Figure 3 A flowchart of a specific embodiment of step 310 shown;

[0069] Figure 5 yes Figure 2 A flowchart of a specific embodiment of step 204 shown;

[0070] Figure 6 yes Figure 5 A flowchart of a specific embodiment of step 502 shown;

[0071] Figure 7 yes Figure 2 A flowchart of a specific embodiment of step 206 shown;

[0072] Figure 8 This is a schematic diagram of a structure of an embodiment of the voiceprint comparison and recognition device according to this application;

[0073] Figure 9 This is a schematic diagram of the structure of one embodiment of the computer device according to this application. Detailed Implementation

[0074] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein in the specification of the application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application; the terms "comprising" and "having," and any variations thereof, in the specification, claims, and foregoing drawings of this application, are intended to cover non-exclusive inclusion. The terms "first," "second," etc., in the specification, claims, or foregoing drawings of this application are used to distinguish different objects, not to describe a particular order.

[0075] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0076] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.

[0077] like Figure 1 As shown, system architecture 100 may include terminal devices 101, 102, and 103, a network 104, and a server 105. Network 104 serves as the medium for providing communication links between terminal devices 101, 102, and 103 and server 105. Network 104 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0078] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, and 103, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social media platform software, etc.

[0079] Terminal devices 101, 102, and 103 can be various electronic devices with displays and support web browsing, including but not limited to smartphones, tablets, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III), MP4 players (Moving Picture Experts Group Audio Layer IV), laptops, and desktop computers, etc.

[0080] Server 105 can be a server that provides various services, such as a backend server that supports the pages displayed on terminal devices 101, 102, and 103.

[0081] It should be noted that the voiceprint comparison and recognition method provided in this application embodiment is generally executed by a terminal device, and correspondingly, the voiceprint comparison and recognition device is generally installed in the terminal device.

[0082] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0083] Continue to refer to Figure 2 The diagram shows a flowchart of an embodiment of the voiceprint comparison and recognition method according to this application. The voiceprint comparison and recognition method includes the following steps:

[0084] Step 201: Obtain the speech to be compared, wherein the speech to be compared includes at least two speech segments.

[0085] In this embodiment, the voice to be compared includes at least two voice segments. The voiceprint comparison and recognition method described in this application can be applied to the field of biometrics, such as target voice classification and recognition in criminal investigation scenarios, or in scenarios like access control voice unlocking and mobile phone voice unlocking. In these cases, a voice segment of a known target can be used as the source domain voice segment, while a newly acquired voice segment is used as the voice segment to be compared with the source domain voice segment. This can better assist criminal investigation or access control / mobile phone unlocking systems in performing biometrics and identifying the same target. Of course, the voiceprint comparison and recognition method can also be applied to the field of financial technology, such as in voice unlocking scenarios in banks or financial institutions, where fingerprint unlocking or facial recognition unlocking can be improved to voiceprint unlocking, providing more secure financial security services for customers.

[0086] Step 202: Input the speech to be compared into a preset voiceprint feature extraction network to extract voiceprint features and obtain voiceprint feature extraction results.

[0087] In this embodiment, the preset voiceprint feature extraction network includes voiceprint feature extraction networks, components, or models composed of convolutional neural networks (CNN), recurrent neural networks (RNN), and their variants (such as LSTM and GRU) through multi-layer nonlinear transformations. These networks can map sound signals to a high-dimensional feature space to extract the speaker's voiceprint features. Specifically, examples include: i-vector voiceprint feature extraction model, x-vector voiceprint feature extraction model, ECAPA-TDNN voiceprint feature extraction model (i.e., a voiceprint feature extraction network based on a time-delay neural network (TDNN), and a voiceprint feature extraction network improved by adding a CBAM (Convolutional Block Attention Module) convolutional attention module to the ResNet50 residual network structure.

[0088] Specifically, when extracting voiceprint features, the preset voiceprint feature extraction network is used. However, when performing voice comparison and recognition, since the directly extracted voiceprint features are non-quantitative data and may contain various forms of data, such as text data and mathematical values, the extracted voiceprint features are not directly used for comparison to avoid making the comparison too complicated.

[0089] Step 203: The voiceprint feature extraction results are mapped using a preset linear mapping layer to obtain the high-dimensional feature vector representations corresponding to the voices to be compared.

[0090] By mapping the low-dimensional voiceprint feature extraction results to a high-dimensional feature vector representation, it is easier to process the subsequent execution program and facilitate subsequent comparison and recognition.

[0091] Step 204: Input the high-dimensional feature vector representation into the pre-trained feature representation fusion model to obtain the comprehensive feature representations corresponding to the speech to be compared. The feature representation fusion model is composed of N feature classification sub-models, where N is a positive integer.

[0092] Specifically, the high-dimensional feature vector representation is input into a pre-trained feature representation fusion model to obtain the feature vector representations corresponding to the voiceprint feature extraction results of each feature classification sub-model. These representations are then integrated to obtain a comprehensive feature representation. Based on this comprehensive feature representation, speech segments of the same object can be identified. Essentially, the comprehensive feature representation can be understood as a two-dimensional matrix with N feature dimensions. This matrix maps the high-dimensional feature vector representations corresponding to feature extraction at different feature dimensions by each feature classification sub-model. Subsequently, the two two-dimensional matrices are compared to identify whether the compared speech segments belong to the same object, thereby achieving voiceprint recognition.

[0093] In this embodiment, before performing the step of inputting the high-dimensional feature vector representation into the pre-trained feature representation fusion model to obtain the feature comprehensive representations corresponding to the speech to be compared, the method further includes training the feature representation fusion model.

[0094] Continue to refer to Figure 3 , Figure 3 This is a flowchart of a specific embodiment of training the feature representation fusion model in the voiceprint comparison and recognition method described in this application, including:

[0095] Step 301: Obtain voiceprint feature training data, wherein the voiceprint feature training data includes at least one speech segment;

[0096] Step 302: Input the voiceprint feature training data into the voiceprint feature extraction network to extract voiceprint features and obtain voiceprint feature extraction results;

[0097] Step 303: The linear mapping layer is used to map the voiceprint feature extraction result to obtain the high-dimensional feature vector representation corresponding to the voiceprint feature training data;

[0098] Step 304: Input the high-dimensional feature vector representation corresponding to the voiceprint feature training data into the pre-constructed N feature classification sub-models, and obtain the voiceprint feature data extracted by the N feature classification sub-models respectively;

[0099] In this embodiment, the feature classification sub-model includes a feature classifier with pre-set extraction dimensions and a feature classification learner without pre-set extraction dimensions. For example, the high-dimensional feature vector representation corresponding to the voiceprint feature training data has 10 dimensions, and the total number of feature classification sub-models is 20. Each feature classification sub-model is set to extract voiceprint features of 1 to 10 dimensions. That is, the feature classification sub-model can extract voiceprint features of 5 dimensions or 3 dimensions, without limitation. The 5 dimensions of voiceprint features can be 5 explicitly extracted feature fields or 5 categories of features to be extracted. The difference is that the feature classifier with pre-set extraction dimensions can only extract voiceprint feature data of the dimension corresponding to the set item, while the feature classification learner without pre-set extraction dimensions can freely extract features of different dimensions according to the actual situation, which is more flexible.

[0100] In this embodiment, before performing the step of inputting the high-dimensional feature vector representation corresponding to the voiceprint feature training data into N pre-constructed feature classification sub-models and obtaining the voiceprint feature data extracted by the N feature classification sub-models respectively, the method further includes: setting a GRL gradient inversion layer between the linear mapping layer and the preset feature classification extraction layer, wherein the preset feature classification extraction layer is the neural network layer where the N feature classification sub-models are located.

[0101] Specifically, the purpose of setting a GRL gradient inversion layer between the linear mapping layer and the preset feature classification extraction layer is to ensure that the N feature classification sub-models extract different and complementary features when processing feature data of the same dimension, thus avoiding repeated extraction of the same feature data of the same dimension by the N feature classification sub-models. By setting a GRL gradient inversion layer during training, the feature data extracted by all feature classification sub-models in the trained feature representation fusion model are different and complementary when actually performing voiceprint feature recognition, which avoids extracting too much duplicate data and improves voiceprint recognition efficiency.

[0102] Step 305: Extract the voiceprint feature data from the N feature classification sub-models and organize them to obtain the voiceprint feature organization result;

[0103] Step 306: Calculate the loss value of the voiceprint feature processing result compared with the voiceprint feature extraction result according to the preset loss function;

[0104] In essence, the voiceprint features extracted by each feature classification sub-model are sorted out, and the sorted results are compared with the voiceprint features extracted by the voiceprint feature extraction network to determine the loss value. Of course, the larger the loss value, the more successful the model training is; the smaller the loss value, the more successful the model training is.

[0105] Step 307: If the loss value exceeds the preset loss threshold, the classification extraction parameters of the N feature classification sub-models are readjusted, and steps 304 to 306 are repeated until the loss value does not exceed the preset loss threshold, at which point the pre-training of the N feature classification sub-models is completed.

[0106] Step 308: If the loss value does not exceed the preset loss threshold, obtain the pre-trained N feature classification sub-models;

[0107] Specifically, the closer the loss threshold is to 0, the more successful the training of the N feature classification sub-models, that is, the more accurate the voiceprint feature extraction result. Since the GRL gradient inversion layer is introduced first, the extraction results of the N feature classification sub-models are different and complementary, without repeated extraction. This achieves the extraction of all voiceprint feature data with the least amount of data extracted, avoids repeated extraction, and ensures extraction efficiency.

[0108] Step 309: Fuse the N pre-trained feature classification sub-models to obtain a preliminary fused feature representation fusion model;

[0109] Step 310: Based on the voiceprint feature data extracted by the N feature classification sub-models that have been pre-trained, set the feature extraction weights corresponding to each feature classification sub-model in the preliminary fusion feature representation fusion model until the feature extraction weights are set, and obtain the pre-trained feature representation fusion model.

[0110] Continue to refer to Figure 4 , Figure 4 yes Figure 3 A flowchart of a specific embodiment of step 310 shown includes:

[0111] Step 401: Select any dimension from all feature dimensions as the target dimension;

[0112] Step 402: Calculate the amount of feature data extracted by each feature classification sub-model on the target dimension.

[0113] Step 403: Calculate the proportion of the amount of feature data extracted by each feature classification sub-model in the target dimension to the total amount of feature data extracted by all feature classification sub-models in the target dimension;

[0114] Step 404: Set the percentage value as the feature extraction weight of the corresponding feature classification sub-model on the target dimension.

[0115] Specifically, assuming the voiceprint features correspond to 10 feature dimensions, these 10 dimensions are sequentially used as target dimensions, and steps 401 to 404 are executed to obtain the feature extraction weights of each feature classification sub-model corresponding to different feature dimensions. Based on the training results, the feature extraction weights of each feature classification sub-model corresponding to different feature dimensions are set to facilitate subsequent comparison calculations when comparing voiceprint features of the speech segments to be compared.

[0116] Continue to refer to Figure 5 , Figure 5 yes Figure 2 A flowchart of a specific embodiment of step 204 shown includes:

[0117] Step 501: Identify all high-dimensional feature vector representations corresponding to the target speech segment, wherein the target speech is any speech segment in the speech to be compared;

[0118] Step 502: Obtain the dimension weights of the feature vector representation of each dimension in the feature representation fusion model.

[0119] Continue to refer to Figure 6 , Figure 6 yes Figure 5 A flowchart of a specific embodiment of step 502 shown includes:

[0120] Step 601: Select any dimension from all feature dimensions as the target dimension;

[0121] Step 602: Identify all feature classification sub-models that extract features for the target dimension, and use them as the target sub-model;

[0122] Step 603: Obtain the feature extraction weights of all target sub-models corresponding to the target dimension;

[0123] Step 604: Extract the weights of the features of all target sub-models corresponding to the target dimension, and use them as feature vectors of the target dimension to represent the corresponding dimension weights in the feature representation fusion model.

[0124] Specifically, because a GRL gradient inversion layer is introduced when training all the feature classification sub-models, the voiceprint feature data extracted by all feature classification sub-models are different and complementary. In extreme cases, a certain feature classification sub-model may not extract a certain dimension of voiceprint features, but other feature classification sub-models may extract them. Therefore, it is necessary to identify all feature classification sub-models that extract voiceprint features in the target dimension, and the feature extraction weights of each feature classification sub-model that extracts voiceprint features in the target dimension are used as the feature vector representations of the target dimension in the feature representation fusion model. This ensures that the feature vector representation is allocated according to the feature extraction weights and avoids unscientific allocation of feature vector representations.

[0125] Step 503: Perform multiplication on the feature vector representation of each dimension and its corresponding dimension weight to obtain the multiplication result;

[0126] Step 504: Obtain the multiplication results corresponding to all the high-dimensional feature vector representations, and use them as the feature synthesis representation of the target speech segment.

[0127] Specifically, a multiplication operation is performed on the feature vector representation of each dimension and its corresponding dimension weight to obtain the multiplication result. The multiplication result corresponding to all the high-dimensional feature vector representations is obtained, which is the feature comprehensive representation of the target speech segment. In essence, it is a two-dimensional matrix multiplied by N. This matrix maps the high-dimensional feature vector representations corresponding to each feature classification sub-model when extracting features in different feature dimensions. Subsequently, the two two-dimensional matrices are compared to identify whether the compared speech segments are speech segments of the same object, thereby realizing voiceprint recognition.

[0128] Step 205: Using a preset distance algorithm, calculate the distance between the feature representations corresponding to the speech to be compared.

[0129] In this embodiment, the step of using a preset distance algorithm to calculate the distance between the feature comprehensive representations corresponding to the speech to be compared specifically includes: using a cosine distance algorithm to compare the feature comprehensive representations of all speech segments in the speech to be compared pairwise to obtain the distance between the feature comprehensive representations corresponding to all speech segments.

[0130] Specifically, the cosine distance algorithm is used to compare the feature representations of all speech segments in the speech to be compared pairwise. That is, the distance value is calculated on the two-dimensional matrix corresponding to the two speech segments to be compared. Based on the calculation result, the distance value between the feature representations corresponding to all speech segments is obtained. Theoretically speaking, the smaller the distance value, the more it indicates that the two corresponding speech segments are spoken by the same person.

[0131] Step 206: Based on the distance value, identify the speech segments of the same object from the speech to be compared.

[0132] Continue to refer to Figure 7 , Figure 7 yes Figure 2 A flowchart of a specific embodiment of step 206 shown includes:

[0133] Step 701: Compare the distance values ​​between the feature representations corresponding to all speech segments with a preset distance threshold;

[0134] Specifically, the closer the preset distance threshold is to 0, the more accurate the comparison of voice segments.

[0135] Step 702: By comparison, select distance values ​​that are less than a preset distance threshold as target distance values;

[0136] Step 703: Select the speech segments to be clustered based on the target distance value, perform pairwise clustering, and obtain preliminary clustering results;

[0137] Step 704: Based on the preliminary clustering results, determine whether the same speech segment is clustered into different clustering results;

[0138] Step 705: If they exist, continue to cluster the different clustering results corresponding to the same speech segment until no more speech segments are clustered into different clustering results, and the clustering is completed.

[0139] Step 706: If it does not exist, then the clustering is complete. Speech segments in the same cluster represent speech segments of the same object.

[0140] Clustering is performed using the distance values ​​to group corresponding speech segments together. In practical acoustic feature comparison and recognition, a distance threshold can be directly set. The distance between a newly acquired speech segment and a pre-defined source domain speech segment is calculated using the above method. If the distance value is less than the distance threshold, the newly acquired speech segment and the pre-defined source domain speech segment are considered to have been emitted by the same person, thus achieving biometric identification based on voiceprint feature comparison. Applying this voiceprint comparison and recognition method to voice unlocking scenarios in banks or financial institutions, replacing fingerprint or facial recognition unlocking with voiceprint unlocking, can provide more secure financial security services for customers.

[0141] This application involves acquiring the speech samples to be compared; extracting voiceprint features; mapping them through a linear mapping layer to obtain a high-dimensional feature vector representation; inputting the high-dimensional feature vector representation into a feature representation fusion model to obtain a comprehensive feature representation corresponding to each of the speech samples to be compared; calculating the distance between the comprehensive feature representations corresponding to each of the speech samples to be compared; and identifying speech segments of the same object from the speech samples to be compared based on the distance values. Comparing through comprehensive feature representations reduces the computational workload and improves the efficiency of comparison and recognition compared to direct voiceprint feature comparison. Applying the voiceprint comparison and recognition method of this application to the field of biometrics, such as in banking or financial services' voice unlocking scenarios, can provide more secure financial security services for customers. Voiceprint recognition technology can also be used to verify user identity in telephone banking, remote stock trading, and other services, providing security guarantees.

[0142] The embodiments of this application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) is the theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0143] Foundational technologies for artificial intelligence generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0144] In this embodiment, the following steps are taken: acquiring the speech to be compared; extracting voiceprint features; mapping through a linear mapping layer to obtain a high-dimensional feature vector representation; inputting the high-dimensional feature vector representation into a feature representation fusion model to obtain a comprehensive feature representation corresponding to each of the speech samples to be compared; calculating the distance between the comprehensive feature representations corresponding to each of the speech samples to be compared; and identifying speech segments of the same object from the speech samples to be compared based on the distance values. Comparing through comprehensive feature representations reduces the computational workload and improves comparison and recognition efficiency compared to direct voiceprint feature comparison. Applying the voiceprint comparison and recognition method of this application to the field of biometrics, such as in banking or financial services voice unlocking scenarios, can provide more secure financial security services for customers.

[0145] Further reference Figure 8 As a response to the above Figure 2 To implement the method shown, this application provides an embodiment of a voiceprint comparison and recognition device, which is similar to... Figure 2Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0146] like Figure 8 As shown, the voiceprint comparison and recognition device 800 described in this embodiment includes: a voice acquisition module 801 to be compared, a voiceprint feature extraction module 802, a feature vector mapping module 803, a feature fusion representation module 804, a distance value calculation module 805, and a same-object voice recognition module 806. Wherein:

[0147] The speech acquisition module 801 is used to acquire the speech to be compared, wherein the speech to be compared includes at least two speech segments;

[0148] The voiceprint feature extraction module 802 is used to input the speech to be compared into a preset voiceprint feature extraction network to extract voiceprint features and obtain voiceprint feature extraction results.

[0149] The feature vector mapping module 803 is used to map the voiceprint feature extraction results using a preset linear mapping layer to obtain the high-dimensional feature vector representations corresponding to the speech to be compared.

[0150] The feature fusion representation module 804 is used to input the high-dimensional feature vector representation into the pre-trained feature representation fusion model to obtain the comprehensive feature representations corresponding to the speech to be compared, wherein the feature representation fusion model is composed of N feature classification sub-models, where N is a positive integer;

[0151] The distance calculation module 805 is used to calculate the distance between the feature comprehensive representations corresponding to the speech to be compared using a preset distance algorithm;

[0152] The same object speech recognition module 806 is used to identify speech segments of the same object from the speech to be compared based on the distance value.

[0153] This application acquires the speech samples to be compared; extracts voiceprint features; maps these features through a linear mapping layer to obtain a high-dimensional feature vector representation; inputs the high-dimensional feature vector representation into a feature representation fusion model to obtain a comprehensive feature representation corresponding to each of the speech samples to be compared; calculates the distance between the comprehensive feature representations corresponding to each of the speech samples to be compared; and identifies speech segments of the same object from the speech samples to be compared based on the distance values. Comparing through comprehensive feature representations reduces the computational workload and improves the efficiency of comparison and recognition compared to direct voiceprint feature comparison. Applying the voiceprint comparison and recognition method of this application to the field of biometric technology, such as in banking or financial services voice unlocking scenarios, can provide more secure financial security services for customers.

[0154] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by instructing related hardware through computer-readable instructions. These computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the methods described above. The aforementioned storage medium can be a non-volatile storage medium such as a magnetic disk, optical disk, or read-only memory (ROM), or random access memory (RAM).

[0155] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0156] To address the aforementioned technical problems, embodiments of this application also provide a computer device. Please refer to [link / reference needed]. Figure 9 , Figure 9 This is a basic structural block diagram of the computer device in this embodiment.

[0157] The computer device 9 includes a memory 9a, a processor 9b, and a network interface 9c that are interconnected via a system bus. It should be noted that only the computer device 9 with components 9a-9c is shown in the figure; however, it should be understood that it is not required to implement all the shown components, and more or fewer components can be implemented alternatively. Those skilled in the art will understand that the computer device described here is a device capable of automatically performing numerical calculations and / or information processing according to pre-set or stored instructions. Its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), programmable gate arrays (FPGAs), digital digital processors (DSPs), embedded devices, etc.

[0158] The computer device can be a desktop computer, laptop, handheld computer, or cloud server, etc. The computer device can interact with the user via a keyboard, mouse, remote control, touchpad, or voice control.

[0159] The memory 9a includes at least one type of readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 9a may be an internal storage unit of the computer device 9, such as the hard disk or memory of the computer device 9. In other embodiments, the memory 9a may also be an external storage device of the computer device 9, such as a plug-in hard disk, smart memory card (SMC), secure digital card (SD), flash card, etc., equipped on the computer device 9. Of course, the memory 9a may also include both the internal storage unit and its external storage device of the computer device 9. In this embodiment, the memory 9a is typically used to store the operating system and various application software installed on the computer device 9, such as computer-readable instructions for a voiceprint comparison recognition method. In addition, the memory 9a can also be used to temporarily store various types of data that have been output or will be output.

[0160] In some embodiments, the processor 9b may be a central processing unit (CPU), controller, microcontroller, microprocessor, or other data processing chip. The processor 9b is typically used to control the overall operation of the computer device 9. In this embodiment, the processor 9b is used to execute computer-readable instructions stored in the memory 9a or to process data, for example, to execute computer-readable instructions for the voiceprint comparison and recognition method.

[0161] The network interface 9c may include a wireless network interface or a wired network interface, which is typically used to establish communication connections between the computer device 9 and other electronic devices.

[0162] The computer device proposed in this embodiment belongs to the field of speech recognition technology and is applied in speech comparison and recognition scenarios. This application acquires the speech to be compared; extracts voiceprint features; maps these features through a linear mapping layer to obtain a high-dimensional feature vector representation; inputs the high-dimensional feature vector representation into a feature representation fusion model to obtain a comprehensive feature representation corresponding to each of the speech samples; calculates the distance between the comprehensive feature representations corresponding to each of the speech samples; and identifies speech segments of the same object from the speech samples based on the distance values. Comparing through comprehensive feature representations reduces the computational workload and improves comparison and recognition efficiency compared to direct voiceprint feature comparison. Applying the voiceprint comparison and recognition method of this application to the field of biometrics, such as in banking or financial services voice unlocking scenarios, can provide more secure financial security services to customers.

[0163] This application also provides another embodiment, namely, providing a computer-readable storage medium storing computer-readable instructions that can be executed by a processor to cause the processor to perform the steps of the voiceprint comparison and recognition method described above.

[0164] The computer-readable storage medium proposed in this embodiment belongs to the field of speech recognition technology and is applied in speech comparison recognition scenarios. This application acquires the speech to be compared; extracts voiceprint features; maps these features through a linear mapping layer to obtain a high-dimensional feature vector representation; inputs the high-dimensional feature vector representation into a feature representation fusion model to obtain a comprehensive feature representation corresponding to each of the speech samples; calculates the distance between the comprehensive feature representations corresponding to each of the speech samples; and identifies speech segments of the same object from the speech samples based on the distance values. Comparing through comprehensive feature representations reduces the computational workload and improves comparison and recognition efficiency compared to direct voiceprint feature comparison. Applying the voiceprint comparison recognition method of this application to the field of biometrics, such as in banking or financial services voice unlocking scenarios, can provide more secure financial security services to customers.

[0165] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0166] Obviously, the embodiments described above are only some embodiments of this application, not all embodiments. The accompanying drawings show preferred embodiments of this application, but do not limit the patent scope of this application. This application can be implemented in many different forms; rather, the purpose of providing these embodiments is to provide a more thorough and comprehensive understanding of the disclosure of this application. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing specific embodiments, or make equivalent substitutions for some of the technical features. Any equivalent structures made using the content of this application's specification and drawings, directly or indirectly applied to other related technical fields, are similarly within the scope of patent protection of this application.

Claims

1. A voiceprint contrast recognition method, characterized in that, The method comprises the following steps: obtaining voice to be compared, wherein the voice to be compared comprises at least two voice segments; inputting the voice to be compared into a preset voiceprint feature extraction network to perform voiceprint feature extraction, and obtaining a voiceprint feature extraction result; mapping the voiceprint feature extraction result by using a preset linear mapping layer to obtain a high-dimensional feature vector representation corresponding to the voice to be compared; inputting the high-dimensional feature vector representation into a pre-trained feature representation fusion model to obtain a feature comprehensive representation corresponding to the voice to be compared, wherein the feature representation fusion model is composed of N feature classification sub-models, N is a positive integer, and the step of inputting the high-dimensional feature vector representation into the pre-trained feature representation fusion model to obtain the feature comprehensive representation corresponding to the voice to be compared specifically comprises: identifying all high-dimensional feature vector representations corresponding to a target voice segment, wherein the target voice is any voice segment in the voice to be compared; obtaining a dimension weight corresponding to each dimension of the high-dimensional feature vector representation in the feature representation fusion model; performing multiplication operation on each dimension of the feature vector representation and the corresponding dimension weight to obtain a multiplication result; obtaining the multiplication result corresponding to each high-dimensional feature vector representation as the feature comprehensive representation of the target voice segment; calculating distance values between the feature comprehensive representations corresponding to the voice to be compared by using a preset distance algorithm; based on the distance values, identifying voice segments of the same object from the voice to be compared.

2. The voiceprint contrast identification method of claim 1, wherein, Before the step of inputting the high-dimensional feature vector representation into the pre-trained feature representation fusion model to obtain the feature comprehensive representation corresponding to the voice to be compared, the method further comprises: step 301, obtaining voiceprint feature training data, wherein the voiceprint feature training data comprises at least one voice segment; step 302, inputting the voiceprint feature training data into the voiceprint feature extraction network to perform voiceprint feature extraction, and obtaining a voiceprint feature extraction result; step 303, mapping the voiceprint feature extraction result by using the linear mapping layer to obtain a high-dimensional feature vector representation corresponding to the voiceprint feature training data; step 304, inputting the high-dimensional feature vector representation corresponding to the voiceprint feature training data into N pre-constructed feature classification sub-models to obtain voiceprint feature data extracted by the N feature classification sub-models; step 305, arranging the voiceprint feature data extracted by the N feature classification sub-models to obtain a voiceprint feature arrangement result; step 306, calculating a loss value of the voiceprint feature arrangement result compared with the voiceprint feature extraction result according to a preset loss function; step 307, if the loss value exceeds a preset loss threshold, adjusting classification extraction parameters of the N feature classification sub-models, and repeating steps 304 to 306 until the loss value does not exceed the preset loss threshold, and the N feature classification sub-models are pre-trained. Step 308, if the loss value does not exceed the preset loss threshold, obtaining the N feature classification sub-models pre-trained; Step 309, fusing the N feature classification sub-models pre-trained to obtain a preliminary fused feature representation fusion model; Step 310, setting a feature extraction weight corresponding to each feature classification sub-model in the preliminary fused feature representation fusion model according to the voiceprint feature data extracted by the N feature classification sub-models pre-trained, until the feature extraction weight is set, and obtaining a feature representation fusion model pre-trained. 3.The voiceprint contrast identification method of claim 2, wherein, Before the step of inputting the high-dimensional feature vector representation corresponding to the voiceprint feature training data into the N feature classification sub-models pre-constructed to obtain the voiceprint feature data extracted by the N feature classification sub-models, the method further comprises: A GRL gradient reversal layer is arranged between the linear mapping layer and a preset feature classification extraction layer, wherein the preset feature classification extraction layer is a neural network layer where the N feature classification sub-models are located. 4.The voiceprint contrast identification method of claim 2, wherein, The step of setting a feature extraction weight corresponding to each feature classification sub-model in the preliminary fused feature representation fusion model according to the voiceprint feature data extracted by the N feature classification sub-models pre-trained specifically comprises: Filtering any dimension as a target dimension from all feature dimensions; Respectively counting the amount of feature data extracted by all feature classification sub-models on the target dimension; Calculating the proportion of the amount of feature data extracted by each feature classification sub-model on the target dimension in the total amount of feature data extracted by all feature classification sub-models on the target dimension; Setting the proportion as the feature extraction weight of the corresponding feature classification sub-model on the target dimension. 5.The voiceprint contrast identification method of claim 1, wherein, The step of obtaining a dimension weight corresponding to each feature vector representation in the feature representation fusion model specifically comprises: Filtering any dimension as a target dimension from all feature dimensions; Identifying all feature classification sub-models that extract features of the target dimension as target sub-models; Obtaining feature extraction weights corresponding to the target dimension of all target sub-models; Setting the feature extraction weights corresponding to the target dimension of all target sub-models as the dimension weight corresponding to the feature vector representation of the target dimension in the feature representation fusion model. 6.The voiceprint contrast identification method of claim 1, wherein, The step of calculating the distance value between the feature comprehensive representations corresponding to the to-be-compared voices by using a preset distance algorithm specifically comprises: Using a cosine distance algorithm to compare the feature comprehensive representations of all voice segments in the to-be-compared voices two by two to obtain the distance values between the feature comprehensive representations corresponding to all voice segments; The step of identifying voice segments of the same object from the to-be-compared voices based on the distance values specifically comprises: Comparing the distance values between the feature comprehensive representations corresponding to all voice segments with a preset distance threshold; Through comparison, filtering out distance values less than the preset distance threshold as target distance values; Filtering out speech segments to be clustered according to the target distance value, clustering two by two, and obtaining a preliminary clustering result; According to the preliminary clustering result, it is judged whether the same speech segment is clustered into different clustering results; If so, the different clustering results corresponding to the same speech segment are further clustered until there is no same speech segment clustered into different clustering results, and the clustering is completed; If not, the clustering is completed, and the speech segments in the same clustering cluster represent the speech segments of the same object.

7. A voiceprint contrast recognition device, characterized in that, Comprise: A speech to be compared acquisition module is used to acquire speech to be compared, wherein the speech to be compared includes at least two speech segments; A voiceprint feature extraction module is used to input the speech to be compared into a preset voiceprint feature extraction network to extract voiceprint features, and obtain a voiceprint feature extraction result; A feature vector mapping module is used to map the voiceprint feature extraction result by using a preset linear mapping layer to obtain a high-dimensional feature vector representation corresponding to the speech to be compared respectively; A feature fusion representation module is used to input the high-dimensional feature vector representation into a pre-trained feature representation fusion model to obtain a feature comprehensive representation corresponding to the speech to be compared respectively, wherein the feature representation fusion model is composed of N feature classification sub-models, N is a positive integer, and the step of inputting the high-dimensional feature vector representation into the pre-trained feature representation fusion model to obtain the feature comprehensive representation corresponding to the speech to be compared respectively specifically comprises: Identifying all high-dimensional feature vector representations corresponding to the target speech segment, wherein the target speech is any speech segment in the speech to be compared; Obtaining the dimension weight corresponding to each dimension of the high-dimensional feature vector representation in the feature representation fusion model; Performing multiplication operation on each dimension of the feature vector representation and its corresponding dimension weight to obtain a multiplication result; Obtaining the multiplication result corresponding to each high-dimensional feature vector representation as the feature comprehensive representation of the target speech segment; A distance value calculation module is used to calculate the distance value between the feature comprehensive representations corresponding to the speech to be compared by using a preset distance algorithm; A same object speech recognition module is used to identify the speech segments of the same object from the speech to be compared based on the distance value.

8. A computer device, comprising: The computer readable storage medium stores computer readable instructions, and the processor executes the computer readable instructions to realize the steps of the voiceprint comparison and recognition method in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer readable instructions, and the processor executes the computer readable instructions to realize the steps of the voiceprint comparison and recognition method in any one of claims 1 to 6.

Images