Voiceprint comparison recognition method, device and equipment and storage medium thereof
Through the combination of voiceprint feature extraction network and feature representation fusion model, the problem of complex modeling and large computing in voiceprint comparison recognition is solved, the recognition efficiency is improved, and a safer voiceprint unlocking solution is provided in financial services.
Patent Information
- Application Number
- CN202411066807.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-05
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2044-08-05
AI Technical Summary
The existing voiceprint recognition technology has problems such as complex modeling, large calculation amount and low recognition efficiency in voiceprint comparison recognition.
Voiceprint feature extraction network is used for voiceprint feature extraction, and the features are mapped into high-dimensional feature vectors through a linear mapping layer, and input them into the feature representation fusion model for comprehensive feature representation calculation, and use the preset distance algorithm to identify the voice of the same object. Fragment.
Reduce the amount of calculations, improve identification efficiency, and provide safer financial security in biometric technology fields such as banking or financial services.
Smart Images

Figure CN119091885A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of speech recognition technology, and is applied to speech comparison and recognition scenarios, and in particular to a voiceprint comparison and recognition method, device, equipment and storage medium thereof. Background Art
[0002] The rapid development of the Internet has also promoted the rapid development of the financial industry. Traditional security encryption and decryption methods are increasingly tending towards encryption and decryption based on biometrics, such as face recognition, fingerprint recognition, and voiceprint recognition. At present, there has been considerable development and application in face recognition and fingerprint recognition, but the application scope of voiceprint encryption and decryption is still relatively limited, and the most common application is voice unlocking of mobile phones.
[0003] Existing voiceprint recognition methods mostly obtain the speaker's voice data and convert it into specific feature information, and compare it with the identity information corresponding to each person pre-stored in the database to determine the identity of the corresponding person. According to the specific task, voiceprint recognition can be divided into semantically related and semantically unrelated voiceprint recognition. Template matching, Gaussian mixture model and joint factor method are often used to achieve voiceprint recognition model modeling and prediction, including end-to-end voiceprint feature extraction through deep learning. However, the above methods have great limitations in voiceprint feature modeling. The extracted voiceprint features have great deficiencies in robustness and accuracy, and their model structure is also relatively complex. Therefore, the existing technology still has the problems of complex modeling, large amount of comparative recognition calculation and low recognition efficiency in voiceprint comparative recognition. Summary of the invention
[0004] The purpose of the embodiments of the present application is to propose a voiceprint comparison and recognition method, device, equipment and storage medium thereof to solve the problems of complex modeling, large amount of comparison and recognition calculation and low recognition efficiency in the prior art for voiceprint comparison and recognition.
[0005] In order to solve the above technical problems, the embodiment of the present application provides a voiceprint comparison and recognition method, which adopts the following technical solutions:
[0006] A voiceprint comparison and recognition method comprises the following steps:
[0007] Acquire speech to be compared, wherein the speech to be compared includes at least two speech segments;
[0008] Inputting the speech to be compared into a preset voiceprint feature extraction network to extract voiceprint features, and obtaining a voiceprint feature extraction result;
[0009] Mapping the voiceprint feature extraction results using a preset linear mapping layer to obtain high-dimensional feature vector representations corresponding to the speech to be compared;
[0010] Inputting the high-dimensional feature vector representation into a pre-trained feature representation fusion model to obtain comprehensive feature representations corresponding to the speech to be compared, wherein the feature representation fusion model is composed of N feature classification sub-models, where N is a positive integer;
[0011] Using a preset distance algorithm, calculate the distance values between the feature comprehensive representations corresponding to the speech to be compared;
[0012] Based on the distance value, speech segments of the same subject are identified from the speech to be compared.
[0013] Furthermore, before executing the step of inputting the high-dimensional feature vector representation into the pre-trained feature representation fusion model to obtain the feature comprehensive representations corresponding to the speech to be compared, the method further includes:
[0014] Step 301, obtaining voiceprint feature training data, wherein the voiceprint feature training data includes at least one speech segment;
[0015] Step 302, inputting the voiceprint feature training data into the voiceprint feature extraction network to perform voiceprint feature extraction, and obtaining a voiceprint feature extraction result;
[0016] Step 303, using the linear mapping layer to map the voiceprint feature extraction result to obtain a high-dimensional feature vector representation corresponding to the voiceprint feature training data;
[0017] Step 304, inputting the high-dimensional feature vector representation corresponding to the voiceprint feature training data into N pre-built feature classification sub-models, and obtaining the N feature classification sub-models to extract voiceprint feature data respectively;
[0018] Step 305, extracting voiceprint feature data from the N feature classification sub-models respectively and arranging them to obtain voiceprint feature arrangement results;
[0019] Step 306, calculating the loss value of the voiceprint feature sorting result compared to the voiceprint feature extraction result according to a preset loss function;
[0020] Step 307: if the loss value exceeds the preset loss threshold, the classification extraction parameters of the N feature classification sub-models are readjusted, and steps 304 to 306 are repeated until the loss value does not exceed the preset loss threshold, and the pre-training of the N feature classification sub-models is completed;
[0021] Step 308: if the loss value does not exceed the preset loss threshold, the N feature classification sub-models that have been pre-trained are obtained;
[0022] Step 309, fusing the N pre-trained feature classification sub-models to obtain a preliminary fused feature representation fusion model;
[0023] Step 310, according to the voiceprint feature data extracted by the N pre-trained feature classification sub-models, the feature extraction weights corresponding to each feature classification sub-model are set in the preliminary fused feature representation fusion model until the feature extraction weights are set and the pre-trained feature representation fusion model is obtained.
[0024] Furthermore, before executing the step of inputting the high-dimensional feature vector representation corresponding to the voiceprint feature training data into the pre-built N feature classification sub-models to obtain the voiceprint feature data extracted by the N feature classification sub-models, the method further includes:
[0025] A GRL gradient reversal layer is provided between the linear mapping layer and a preset feature classification extraction layer, wherein the preset feature classification extraction layer is a neural network layer where the N feature classification sub-models are located.
[0026] Furthermore, the step of setting the feature extraction weights corresponding to each feature classification sub-model in the preliminary fused feature representation fusion model according to the voiceprint feature data extracted by the N feature classification sub-models completed by pre-training specifically includes:
[0027] Select any dimension from all feature dimensions as the target dimension;
[0028] Counting the amount of feature data extracted by all feature classification sub-models on the target dimension respectively;
[0029] Calculate the ratio of the amount of feature data extracted by each feature classification sub-model on the target dimension to the total amount of feature data extracted by all feature classification sub-models on the target dimension;
[0030] The proportion value is set as the feature extraction weight of the corresponding feature classification sub-model on the target dimension.
[0031] Furthermore, the step of inputting the high-dimensional feature vector representation into the pre-trained feature representation fusion model to obtain the feature comprehensive representations corresponding to the speech to be compared comprises:
[0032] Identify all high-dimensional feature vector representations corresponding to a target speech segment, wherein the target speech is any speech segment in the speech to be compared;
[0033] Obtaining the dimension weights corresponding to the feature vector representations of each dimension in all the high-dimensional feature vector representations in the feature representation fusion model;
[0034] Perform multiplication operation on the feature vector representation of each dimension and its corresponding dimension weight to obtain the multiplication result;
[0035] The multiplication results corresponding to all the high-dimensional feature vector representations are obtained as the comprehensive feature representation of the target speech segment.
[0036] Furthermore, the step of obtaining the dimension weights corresponding to the feature vector representations of each dimension in all the high-dimensional feature vector representations in the feature representation fusion model specifically includes:
[0037] Select any dimension from all feature dimensions as the target dimension;
[0038] Identify all feature classification sub-models that extract features of the target dimension as target sub-models;
[0039] Obtain the feature extraction weights corresponding to the target dimension for all target sub-models;
[0040] The feature extraction weights corresponding to the target dimension of all the target sub-models are used as the feature vector representation of the target dimension, and the dimensional weights corresponding to each of them in the feature representation fusion model are represented.
[0041] Furthermore, the step of using a preset distance algorithm to calculate the distance values between the feature comprehensive representations corresponding to the speech to be compared specifically includes:
[0042] Using a cosine distance algorithm, the feature comprehensive representations of all speech segments in the speech to be compared are compared in pairs, so as to obtain distance values between the feature comprehensive representations corresponding to all speech segments;
[0043] The step of identifying speech segments of the same object from the speech to be compared based on the distance value specifically includes:
[0044] Comparing the distance values between the feature comprehensive representations corresponding to all speech segments with a preset distance threshold;
[0045] By comparison, a distance value smaller than a preset distance threshold is selected as a target distance value;
[0046] The speech segments to be clustered are selected according to the target distance value, and clustered in pairs to obtain a preliminary clustering result;
[0047] According to the preliminary clustering result, determining whether the same speech segment is clustered into different clustering results;
[0048] If so, continue clustering the different clustering results corresponding to the same speech segment until the same speech segment is no longer clustered into different clustering results, and the clustering is completed;
[0049] If it does not exist, the clustering is completed, and the speech clips in the same cluster represent the speech clips of the same object.
[0050] In order to solve the above technical problems, the embodiment of the present application also provides a voiceprint comparison and recognition device, which adopts the following technical solution:
[0051] A voiceprint comparison and recognition device, comprising:
[0052] A speech acquisition module to be compared, used to acquire the speech to be compared, wherein the speech to be compared includes at least two speech segments;
[0053] A voiceprint feature extraction module, used for inputting the speech to be compared into a preset voiceprint feature extraction network to extract voiceprint features and obtain a voiceprint feature extraction result;
[0054] A feature vector mapping module, used to map the voiceprint feature extraction result using a preset linear mapping layer to obtain high-dimensional feature vector representations corresponding to the speech to be compared;
[0055] A feature fusion representation module, used for inputting the high-dimensional feature vector representation into a pre-trained feature representation fusion model to obtain the feature comprehensive representations corresponding to the speech to be compared, wherein the feature representation fusion model is composed of N feature classification sub-models, where N is a positive integer;
[0056] A distance value calculation module, used to calculate the distance values between the feature comprehensive representations corresponding to the speech to be compared respectively by using a preset distance algorithm;
[0057] The same-object speech recognition module is used to recognize speech segments of the same object from the speech to be compared based on the distance value.
[0058] In order to solve the above technical problems, the embodiment of the present application further provides a computer device, which adopts the following technical solution:
[0059] A computer device includes a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of the above-mentioned voiceprint comparison and recognition method when executing the computer-readable instructions.
[0060] In order to solve the above technical problems, the embodiment of the present application further provides a computer-readable storage medium, which adopts the following technical solution:
[0061] A computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, implement the steps of the voiceprint comparison and recognition method as described above.
[0062] Compared with the prior art, the embodiments of the present application have the following beneficial effects:
[0063] The voiceprint comparison and recognition method described in the embodiment of the present application is as follows: obtaining the speech to be compared; performing voiceprint feature extraction; obtaining a high-dimensional feature vector representation through linear mapping layer mapping; inputting the high-dimensional feature vector representation into the feature representation fusion model to obtain the feature comprehensive representations corresponding to the speech to be compared; calculating the distance value between the feature comprehensive representations corresponding to the speech to be compared; based on the distance value, identifying the voice segments of the same object from the speech to be compared. Compared with direct comparison of voiceprint features, comparison through feature comprehensive representation reduces the amount of calculation and comparison to a certain extent, and improves the comparison and recognition efficiency. The voiceprint comparison and recognition method of the present application is applied to the field of biometric identification technology, such as voice unlocking scenarios in banks or financial services, which can provide customers with more secure financial security services. BRIEF DESCRIPTION OF THE DRAWINGS
[0064] In order to more clearly illustrate the scheme in the present application, a brief introduction is given below to the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0065] Figure 1 is an exemplary system architecture diagram to which the present application may be applied;
[0066] Figure 2 is a flow chart of an embodiment of a voiceprint comparison and recognition method according to the present application;
[0067] Figure 3 It is a flowchart of a specific embodiment of training the feature representation fusion model in the voiceprint comparison and recognition method described in this application;
[0068] Figure 4 yes Figure 3 A flowchart of a specific embodiment of step 310 is shown;
[0069] Figure 5 yes Figure 2 A flowchart of a specific embodiment of step 204 is shown;
[0070] Figure 6 yes Figure 5 A flowchart of a specific embodiment of step 502 is shown;
[0071] Figure 7 yes Figure 2 A flowchart of a specific embodiment of step 206 is shown;
[0072] Figure 8 is a schematic structural diagram of an embodiment of a voiceprint comparison and recognition device according to the present application;
[0073] Fig. 9 It is a structural diagram of an embodiment of a computer device according to the present application. DETAILED DESCRIPTION
[0074] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.
[0075] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.
[0076] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.
[0077] like Figure 1 As shown, the system architecture 100 may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.
[0078] Users can use terminal devices 101, 102, 103 to interact with server 105 through network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, 103, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.
[0079] The terminal devices 101, 102, and 103 can be various electronic devices with display screens and supporting web browsing, including but not limited to smart phones, tablet computers, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III), MP4 (Moving Picture Experts Group Audio Layer IV) players, laptop computers, desktop computers, etc.
[0080] The server 105 may be a server that provides various services, such as a background server that provides support for web pages displayed on the terminal devices 101 , 102 , and 103 .
[0081] It should be noted that the voiceprint comparison and recognition method provided in the embodiments of the present application is generally executed by a terminal device, and accordingly, the voiceprint comparison and recognition device is generally arranged in the terminal device.
[0082] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.
[0083] Continue to refer Figure 2 , shows a flow chart of an embodiment of a voiceprint comparison and recognition method according to the present application. The voiceprint comparison and recognition method comprises the following steps:
[0084] Step 201: Acquire speech to be compared, wherein the speech to be compared includes at least two speech segments.
[0085] In this embodiment, the voice to be compared includes at least two voice segments. The voiceprint comparison and recognition method described in the present application can be applied to the field of biometric identification technology, for example: target voice classification and recognition in criminal investigation scenarios, or in scenarios such as access control voice unlocking and mobile phone voice unlocking. At this time, a voice segment of a clearly identified object can be used as a source domain voice segment, and the newly acquired voice segment can be used as a voice segment to be compared with the source domain voice segment, which can better assist criminal investigation or access control and mobile phone unlocking systems in biometric identification and identify the same object. Of course, the voiceprint comparison and recognition method can also be applied to the field of financial technology technology, for example: in the voice unlocking scenario in a bank or financial institution, fingerprint unlocking or face recognition unlocking is improved to voiceprint unlocking, which can provide customers with more secure financial security services.
[0086] Step 202: input the speech to be compared into a preset voiceprint feature extraction network to extract voiceprint features and obtain a voiceprint feature extraction result.
[0087] In this embodiment, the preset voiceprint feature extraction network includes a voiceprint feature extraction network, component or model formed by multi-layer nonlinear transformation of a convolutional neural network (CNN), a recurrent neural network (RNN) and its variants (such as LSTM, GRU), etc., which can map the sound signal to a high-dimensional feature space, so as to extract the voiceprint features of the speaker. Specifically, for example: an i-vector voiceprint feature extraction model, an x-vector voiceprint feature extraction model, an ECAPA-TDNN voiceprint feature extraction model, that is, a voiceprint feature extraction network based on a time delay neural network (TDNN), and a voiceprint feature extraction network improved by adding a CBAM (Convolutional Block Attention Module) convolutional attention module to the ResNet50 residual network structure.
[0088] Specifically, when performing voiceprint feature extraction, the preset voiceprint feature extraction network is used. However, when performing speech comparison and recognition, since the directly extracted voiceprint features are non-quantified data and may contain data in various forms, such as text data, mathematical values, etc., the extracted voiceprint features are not directly used for comparison to avoid complex comparison.
[0089] Step 203: Map the voiceprint feature extraction result using a preset linear mapping layer to obtain high-dimensional feature vector representations corresponding to the speech to be compared.
[0090] By mapping the low-dimensional voiceprint feature extraction results into a high-dimensional feature vector representation, it is convenient for subsequent execution of the program and for subsequent comparative identification.
[0091] Step 204, inputting the high-dimensional feature vector representation into a pre-trained feature representation fusion model to obtain comprehensive feature representations corresponding to the speech to be compared, wherein the feature representation fusion model is composed of N feature classification sub-models, where N is a positive integer.
[0092] Specifically, the high-dimensional feature vector representation is input into the pre-trained feature representation fusion model, and the feature vector representations corresponding to each feature classification sub-model when classifying and extracting the voiceprint feature extraction results are obtained, and then integrated to obtain a feature comprehensive representation, and the voice segments of the same object are identified based on the feature comprehensive representation. In essence, it can be understood that the feature comprehensive representation is a two-dimensional matrix with the number of feature dimensions multiplied by N, which maps out the high-dimensional feature vector representations corresponding to each feature classification sub-model when extracting features on different feature dimensions. Subsequently, the two two-dimensional matrices are compared to identify whether the compared voice segments are voice segments of the same object, thereby realizing voiceprint recognition.
[0093] In this embodiment, before executing the step of inputting the high-dimensional feature vector representation into a pre-trained feature representation fusion model to obtain the comprehensive feature representations corresponding to the speech to be compared, the method also includes training the feature representation fusion model.
[0094] Continue to refer Figure 3 , Figure 3 It is a flowchart of a specific embodiment of training the feature representation fusion model in the voiceprint comparison and recognition method described in this application, including:
[0095] Step 301, obtaining voiceprint feature training data, wherein the voiceprint feature training data includes at least one speech segment;
[0096] Step 302, inputting the voiceprint feature training data into the voiceprint feature extraction network to perform voiceprint feature extraction, and obtaining a voiceprint feature extraction result;
[0097] Step 303, using the linear mapping layer to map the voiceprint feature extraction result to obtain a high-dimensional feature vector representation corresponding to the voiceprint feature training data;
[0098] Step 304, inputting the high-dimensional feature vector representation corresponding to the voiceprint feature training data into N pre-built feature classification sub-models, and obtaining the N feature classification sub-models to extract voiceprint feature data respectively;
[0099] In this embodiment, the feature classification submodel includes a feature classifier with pre-set extraction dimensions, and also includes a feature classification learner without setting extraction dimensions. For example, the number of dimensions in the high-dimensional feature vector representation corresponding to the voiceprint feature training data is 10, and the total number of feature classification submodels is 20. Each feature classification submodel is set to be able to extract voiceprint features of 1 to 10 dimensions, that is, the feature classification submodel can extract voiceprint features of 5 dimensions or 3 dimensions, which is not limited here. The 5-dimensional voiceprint features can be 5 categories of feature fields that are explicitly extracted, or can be set to 5 categories of extracted features. The difference is that the feature classifier with pre-set extraction dimensions can only extract voiceprint feature data of the dimensions corresponding to the setting items, while the feature classification learner without setting extraction dimensions can freely extract features of different dimensions according to actual conditions, which is more flexible.
[0100] In this embodiment, before executing the step of inputting the high-dimensional feature vector representation corresponding to the voiceprint feature training data into N pre-built feature classification sub-models to obtain the N feature classification sub-models to extract the voiceprint feature data respectively, the method also includes: setting a GRL gradient reversal layer between the linear mapping layer and the preset feature classification extraction layer, wherein the preset feature classification extraction layer is the neural network layer where the N feature classification sub-models are located.
[0101] Specifically, the purpose of setting a GRL gradient reversal layer between the linear mapping layer and the preset feature classification extraction layer is to make the extraction results of the N feature classification sub-models different and complementary when extracting feature data of the same dimension, thereby avoiding repeated extraction of the same feature data of the same dimension by the N feature classification sub-models. By setting a GRL gradient reversal layer during training, all feature classification sub-models in the trained feature representation fusion model can extract different and complementary feature data when actually performing voiceprint feature recognition, thus avoiding the extraction of too much repeated data and improving the efficiency of voiceprint recognition.
[0102] Step 305, extracting voiceprint feature data from the N feature classification sub-models respectively and arranging them to obtain voiceprint feature arrangement results;
[0103] Step 306, calculating the loss value of the voiceprint feature sorting result compared to the voiceprint feature extraction result according to a preset loss function;
[0104] In essence, the voiceprint features extracted by each feature classification sub-model are sorted, and the sorted results are compared with the voiceprint features extracted by the voiceprint feature extraction network to determine the loss value. Of course, the larger the loss value, the more unsuccessful the model training is, and the smaller the loss value, the more successful the model training is.
[0105] Step 307: if the loss value exceeds the preset loss threshold, the classification extraction parameters of the N feature classification sub-models are readjusted, and steps 304 to 306 are repeated until the loss value does not exceed the preset loss threshold, and the pre-training of the N feature classification sub-models is completed;
[0106] Step 308: if the loss value does not exceed the preset loss threshold, the N feature classification sub-models that have been pre-trained are obtained;
[0107] Specifically, the closer the loss threshold is to 0, the more successfully the training of the N feature classification sub-models is, that is, the more accurate the voiceprint feature extraction result is. Since the GRL gradient reversal layer is introduced in advance, the extraction results of the N feature classification sub-models are different and complementary, and there is no repeated extraction, which achieves the extraction of all voiceprint feature data with the least extracted data, avoids repeated extraction, and ensures extraction efficiency.
[0108] Step 309, fusing the N pre-trained feature classification sub-models to obtain a preliminary fused feature representation fusion model;
[0109] Step 310, according to the voiceprint feature data extracted by the N pre-trained feature classification sub-models, the feature extraction weights corresponding to each feature classification sub-model are set in the preliminary fused feature representation fusion model until the feature extraction weights are set and the pre-trained feature representation fusion model is obtained.
[0110] Continue to refer Figure 4 , Figure 4 yes Figure 3 The flowchart of a specific embodiment of step 310 shown includes:
[0111] Step 401, selecting any dimension from all feature dimensions as a target dimension;
[0112] Step 402, respectively counting the amount of feature data extracted by all feature classification sub-models on the target dimension;
[0113] Step 403, calculating the proportion of the amount of feature data extracted by each feature classification sub-model on the target dimension to the total amount of feature data extracted by all feature classification sub-models on the target dimension;
[0114] Step 404: Set the proportion value as the feature extraction weight of the corresponding feature classification sub-model on the target dimension.
[0115] Specifically, assuming that there are 10 feature dimensions corresponding to the voiceprint feature, these 10 dimensions are used as target dimensions in turn, and steps 401 to 404 are performed to obtain feature extraction weights corresponding to different feature dimensions for each feature classification sub-model. The feature extraction weights corresponding to different feature dimensions for each feature classification sub-model are set according to the training results, so as to facilitate the subsequent comparison calculation when performing voiceprint feature recognition comparison on the voice clips to be compared.
[0116] Continue to refer Figure 5 , Figure 5 yes Figure 2 The flowchart of a specific embodiment of step 204 shown includes:
[0117] Step 501, identifying all high-dimensional feature vector representations corresponding to a target speech segment, wherein the target speech is any speech segment in the speech to be compared;
[0118] Step 502, obtaining the dimension weights corresponding to the feature vector representations of each dimension in all the high-dimensional feature vector representations in the feature representation fusion model;
[0119] Continue to refer Figure 6 , Figure 6 yes Figure 5 The flowchart of a specific embodiment of step 502 shown includes:
[0120] Step 601, selecting any dimension from all feature dimensions as a target dimension;
[0121] Step 602, identifying all feature classification sub-models that extract features of the target dimension as target sub-models;
[0122] Step 603, obtaining feature extraction weights corresponding to the target dimension for all target sub-models;
[0123] Step 604: The feature extraction weights corresponding to the target dimension of all target sub-models are used as the dimensional weights corresponding to the feature vector representation of the target dimension in the feature representation fusion model.
[0124] Specifically, since the GRL gradient reversal layer is introduced when training all the feature classification sub-models, the voiceprint feature data extracted by all the feature classification sub-models are different and complementary. Extreme situations may also occur, that is, a feature classification sub-model does not extract the voiceprint feature of a certain dimension, but it is extracted by other feature classification sub-models. Therefore, it is necessary to identify all the feature classification sub-models that extract voiceprint features for the target dimension, and the feature extraction weights corresponding to all the feature classification sub-models that extract voiceprint features for the dimension are used as the feature vector representation of the target dimension and the dimensional weights corresponding to them in the feature representation fusion model, thereby ensuring that the feature vector representation is allocated according to the feature extraction weights and avoiding unscientific allocation of feature vector representations.
[0125] Step 503, performing a multiplication operation on the feature vector representation of each dimension and its corresponding dimension weight to obtain a multiplication operation result;
[0126] Step 504: Obtain the multiplication results corresponding to all the high-dimensional feature vector representations as the comprehensive feature representation of the target speech segment.
[0127] Specifically, the feature vector representation of each dimension and its corresponding dimensional weight are multiplied to obtain a multiplication result, and the multiplication results corresponding to all the high-dimensional feature vector representations are obtained, that is, the comprehensive feature representation of the target voice segment, which is essentially a two-dimensional matrix of the number of feature dimensions multiplied by N. The matrix maps out the high-dimensional feature vector representations corresponding to each feature classification sub-model when performing feature extraction on different feature dimensions. Subsequently, the two two-dimensional matrices are compared to identify whether the compared voice segments are voice segments of the same object, thereby realizing voiceprint recognition.
[0128] Step 205: using a preset distance algorithm to calculate the distance values between the feature comprehensive representations corresponding to the speech to be compared.
[0129] In this embodiment, the step of using a preset distance algorithm to calculate the distance values between the feature comprehensive representations corresponding to the speech to be compared specifically includes: using a cosine distance algorithm to compare the feature comprehensive representations of all speech segments in the speech to be compared pairwise, and obtaining the distance values between the feature comprehensive representations corresponding to all speech segments.
[0130] Specifically, the cosine distance algorithm is used to compare the feature comprehensive representations of all the speech segments in the speech to be compared, that is, the distance values of the two-dimensional matrices corresponding to the two speech segments to be compared are calculated, so as to obtain the distance values between the feature comprehensive representations corresponding to all the speech segments according to the calculation results. Theoretically speaking, the smaller the distance value, the more it can explain that the two corresponding speech segments are the speech uttered by the same object.
[0131] Step 206: based on the distance value, identify speech segments of the same subject from the speech to be compared.
[0132] Continue to refer Figure 7 , Figure 7 yes Figure 2 The flowchart of a specific embodiment of step 206 shown includes:
[0133] Step 701, comparing the distance values between the feature comprehensive representations corresponding to all speech segments with a preset distance threshold;
[0134] Specifically, the closer the preset distance threshold is to 0, the more accurate the comparison of the voice segments is.
[0135] Step 702, by comparison, a distance value less than a preset distance threshold is selected as a target distance value;
[0136] Step 703, selecting speech segments to be clustered according to the target distance value, performing pairwise clustering, and obtaining a preliminary clustering result;
[0137] Step 704, judging whether the same speech segment is clustered into different clustering results according to the preliminary clustering result;
[0138] Step 705: If yes, continue clustering the different clustering results corresponding to the same speech segment until no more same speech segment is clustered into different clustering results, and clustering is completed;
[0139] Step 706: If not, the clustering is completed, and the speech segments in the same cluster represent speech segments of the same object.
[0140] Clustering is performed through the distance value, so that the same corresponding voice segments are clustered together. Of course, in actual acoustic feature comparison and recognition, the distance threshold can be directly set, and the distance value between the newly acquired voice segment and the preset source domain voice segment can be calculated by the above method. If the distance value is less than the distance threshold, it is considered that the newly acquired voice segment and the preset source domain voice segment are emitted by the same object, thereby realizing biometric recognition based on voiceprint feature comparison. The voiceprint comparison and recognition method is applied to the voice unlocking scenario in banks or financial institutions, and fingerprint unlocking or face recognition unlocking is improved to voiceprint unlocking, which can provide customers with more secure financial security services.
[0141] The present application obtains the speech to be compared; extracts the voiceprint features; obtains a high-dimensional feature vector representation through linear mapping layer mapping; inputs the high-dimensional feature vector representation into the feature representation fusion model to obtain the feature comprehensive representations corresponding to the speech to be compared; calculates the distance value between the feature comprehensive representations corresponding to the speech to be compared; and identifies the voice segments of the same object from the speech to be compared based on the distance value. Compared with direct comparison of voiceprint features, comparison through feature comprehensive representation reduces the amount of calculation and comparison to a certain extent, and improves the comparison and recognition efficiency. The voiceprint comparison and recognition method of the present application is applied to the field of biometrics, such as voice unlocking scenarios in banks or financial services, which can provide customers with more secure financial security services. Voiceprint recognition technology can also be used to confirm the identity of users in services such as telephone banking and remote stock trading to provide security.
[0142] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.
[0143] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.
[0144] In the embodiment of the present application, the speech to be compared is obtained; voiceprint features are extracted; a high-dimensional feature vector representation is obtained through linear mapping layer mapping; the high-dimensional feature vector representation is input into the feature representation fusion model to obtain the feature comprehensive representations corresponding to the speech to be compared; the distance values between the feature comprehensive representations corresponding to the speech to be compared are calculated; based on the distance values, the voice segments of the same object are identified from the speech to be compared. Compared with direct comparison of voiceprint features, comparison through feature comprehensive representation reduces the amount of calculation and comparison to a certain extent, and improves the comparison and recognition efficiency. The voiceprint comparison and recognition method of the present application is applied to the field of biometric identification technology, such as voice unlocking scenarios in banks or financial services, which can provide customers with more secure financial security services.
[0145] Further references Figure 8 , as a response to the above Figure 2 In order to realize the method shown in the figure, the present application provides an embodiment of a voiceprint comparison and recognition device, and the device embodiment is similar to Figure 2Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.
[0146] like Figure 8 As shown, the voiceprint comparison and recognition device 800 described in this embodiment includes: a voice acquisition module 801 to be compared, a voiceprint feature extraction module 802, a feature vector mapping module 803, a feature fusion representation module 804, a distance value calculation module 805 and a same-object voice recognition module 806. Among them:
[0147] The speech acquisition module 801 is used to acquire the speech to be compared, wherein the speech to be compared includes at least two speech segments;
[0148] The voiceprint feature extraction module 802 is used to input the speech to be compared into a preset voiceprint feature extraction network to extract voiceprint features and obtain a voiceprint feature extraction result;
[0149] A feature vector mapping module 803 is used to map the voiceprint feature extraction result using a preset linear mapping layer to obtain high-dimensional feature vector representations corresponding to the speech to be compared;
[0150] The feature fusion representation module 804 is used to input the high-dimensional feature vector representation into the pre-trained feature representation fusion model to obtain the feature comprehensive representations corresponding to the speech to be compared, wherein the feature representation fusion model is composed of N feature classification sub-models, where N is a positive integer;
[0151] The distance value calculation module 805 is used to calculate the distance values between the feature comprehensive representations corresponding to the speech to be compared respectively by using a preset distance algorithm;
[0152] The same-subject speech recognition module 806 is configured to recognize speech segments of the same subject from the speech to be compared based on the distance value.
[0153] The present application obtains the speech to be compared; extracts the voiceprint features; obtains a high-dimensional feature vector representation through linear mapping layer mapping; inputs the high-dimensional feature vector representation into the feature representation fusion model to obtain the feature comprehensive representations corresponding to the speech to be compared; calculates the distance values between the feature comprehensive representations corresponding to the speech to be compared; and based on the distance values, identifies the voice segments of the same object from the speech to be compared. Compared with direct comparison of voiceprint features, comparison through feature comprehensive representation reduces the amount of calculation and comparison to a certain extent, and improves the comparison and recognition efficiency. The voiceprint comparison and recognition method of the present application is applied to the field of biometrics, such as voice unlocking scenarios in banks or financial services, which can provide customers with more secure financial security services.
[0154] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through computer-readable instructions, and the computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a non-volatile storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).
[0155] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.
[0156] To solve the above technical problems, the present application also provides a computer device. Fig. 9 , Fig. 9 This is a basic structural block diagram of the computer device in this embodiment.
[0157] The computer device 9 includes a memory 9a, a processor 9b, and a network interface 9c that are interconnected through a system bus. It should be noted that the figure only shows a computer device 9 having components 9a-9c, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital processors (DSPs), embedded devices, etc.
[0158] The computer device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The computer device may interact with a user through a keyboard, a mouse, a remote controller, a touch pad, or a voice control device.
[0159] The memory 9a includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 9a can be an internal storage unit of the computer device 9, such as a hard disk or memory of the computer device 9. In other embodiments, the memory 9a can also be an external storage device of the computer device 9, such as a plug-in hard disk equipped on the computer device 9, a smart memory card (Smart Med i aCard, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. Of course, the memory 9a can also include both the internal storage unit of the computer device 9 and its external storage device. In this embodiment, the memory 9a is generally used to store the operating system and various application software installed on the computer device 9, such as a computer-readable instruction of a voiceprint comparison and recognition method. In addition, the memory 9a can also be used to temporarily store various types of data that have been output or are to be output.
[0160] The processor 9b may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips in some embodiments. The processor 9b is generally used to control the overall operation of the computer device 9. In this embodiment, the processor 9b is used to run the computer-readable instructions stored in the memory 9a or process data, such as computer-readable instructions for running the voiceprint comparison and recognition method.
[0161] The network interface 9c may include a wireless network interface or a wired network interface. The network interface 9c is generally used to establish a communication connection between the computer device 9 and other electronic devices.
[0162] The computer device proposed in this embodiment belongs to the field of speech recognition technology and is applied to speech comparison and recognition scenarios. This application obtains the speech to be compared; extracts voiceprint features; obtains a high-dimensional feature vector representation through linear mapping layer mapping; inputs the high-dimensional feature vector representation into the feature representation fusion model to obtain the feature comprehensive representations corresponding to the speech to be compared; calculates the distance value between the feature comprehensive representations corresponding to the speech to be compared; and identifies the speech segments of the same object from the speech to be compared based on the distance value. Compared with direct comparison of voiceprint features, comparison through feature comprehensive representation reduces the amount of calculation and comparison to a certain extent, and improves the comparison and recognition efficiency. The voiceprint comparison and recognition method of this application is applied to the field of biometric recognition technology, such as voice unlocking scenarios in banks or financial services, which can provide customers with more secure financial security services.
[0163] The present application also provides another implementation, namely, providing a computer-readable storage medium, wherein the computer-readable storage medium stores computer-readable instructions, and the computer-readable instructions can be executed by a processor to enable the processor to perform the steps of the voiceprint comparison and recognition method as described above.
[0164] The computer-readable storage medium proposed in this embodiment belongs to the field of speech recognition technology and is applied to speech comparison and recognition scenarios. This application obtains the speech to be compared; performs voiceprint feature extraction; obtains a high-dimensional feature vector representation through linear mapping layer mapping; inputs the high-dimensional feature vector representation into the feature representation fusion model to obtain the feature comprehensive representations corresponding to the speech to be compared; calculates the distance value between the feature comprehensive representations corresponding to the speech to be compared; and identifies the speech segments of the same object from the speech to be compared based on the distance value. Compared with direct comparison of voiceprint features, comparison through feature comprehensive representation reduces the amount of calculation and comparison to a certain extent, and improves the comparison and recognition efficiency. The voiceprint comparison and recognition method of this application is applied to the field of biometric recognition technology, such as voice unlocking scenarios in banks or financial services, which can provide customers with more secure financial security services.
[0165] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.
[0166] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to perform equivalent replacement of some of the technical features therein. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is similarly within the scope of patent protection of this application.
Claims
1. A voiceprint comparison and recognition method, characterized in that: The steps include: Acquire speech to be compared, wherein the speech to be compared includes at least two speech segments; Inputting the speech to be compared into a preset voiceprint feature extraction network to extract voiceprint features, and obtaining a voiceprint feature extraction result; Mapping the voiceprint feature extraction results using a preset linear mapping layer to obtain high-dimensional feature vector representations corresponding to the speech to be compared; Inputting the high-dimensional feature vector representation into a pre-trained feature representation fusion model to obtain comprehensive feature representations corresponding to the speech to be compared, wherein the feature representation fusion model is composed of N feature classification sub-models, where N is a positive integer; Using a preset distance algorithm, calculate the distance values between the feature comprehensive representations corresponding to the speech to be compared; Based on the distance value, speech segments of the same subject are identified from the speech to be compared.
2. The voiceprint comparison and recognition method according to claim 1, characterized in that: Before executing the step of inputting the high-dimensional feature vector representation into the pre-trained feature representation fusion model to obtain the feature comprehensive representations corresponding to the speech to be compared, the method further includes: Step 301, obtaining voiceprint feature training data, wherein the voiceprint feature training data includes at least one speech segment; Step 302, inputting the voiceprint feature training data into the voiceprint feature extraction network to perform voiceprint feature extraction, and obtaining a voiceprint feature extraction result; Step 303, using the linear mapping layer to map the voiceprint feature extraction result to obtain a high-dimensional feature vector representation corresponding to the voiceprint feature training data; Step 304, inputting the high-dimensional feature vector representation corresponding to the voiceprint feature training data into N pre-built feature classification sub-models, and obtaining the N feature classification sub-models to extract voiceprint feature data respectively; Step 305, extracting voiceprint feature data from the N feature classification sub-models respectively and arranging them to obtain voiceprint feature arrangement results; Step 306, calculating the loss value of the voiceprint feature sorting result compared to the voiceprint feature extraction result according to a preset loss function; Step 307: if the loss value exceeds the preset loss threshold, the classification extraction parameters of the N feature classification sub-models are readjusted, and steps 304 to 306 are repeated until the loss value does not exceed the preset loss threshold, and the pre-training of the N feature classification sub-models is completed; Step 308: if the loss value does not exceed the preset loss threshold, the N feature classification sub-models that have been pre-trained are obtained; Step 309, fusing the N pre-trained feature classification sub-models to obtain a preliminary fused feature representation fusion model; Step 310, according to the voiceprint feature data extracted by the N pre-trained feature classification sub-models, the feature extraction weights corresponding to each feature classification sub-model are set in the preliminary fused feature representation fusion model until the feature extraction weights are set and the pre-trained feature representation fusion model is obtained.
3. The voiceprint comparison and recognition method according to claim 2, characterized in that: Before executing the step of inputting the high-dimensional feature vector representation corresponding to the voiceprint feature training data into the pre-built N feature classification sub-models to obtain the voiceprint feature data extracted by the N feature classification sub-models, the method further includes: A GRL gradient reversal layer is provided between the linear mapping layer and a preset feature classification extraction layer, wherein the preset feature classification extraction layer is a neural network layer where the N feature classification sub-models are located.
4. The voiceprint comparison and recognition method according to claim 2, characterized in that: The step of setting the feature extraction weights corresponding to each feature classification sub-model in the preliminary fused feature representation fusion model according to the voiceprint feature data extracted by the N feature classification sub-models completed by pre-training specifically includes: Select any dimension from all feature dimensions as the target dimension; Counting the amount of feature data extracted by all feature classification sub-models on the target dimension respectively; Calculate the ratio of the amount of feature data extracted by each feature classification sub-model on the target dimension to the total amount of feature data extracted by all feature classification sub-models on the target dimension; The proportion value is set as the feature extraction weight of the corresponding feature classification sub-model on the target dimension.
5. The voiceprint comparison and recognition method according to claim 1, characterized in that: The step of inputting the high-dimensional feature vector representation into the pre-trained feature representation fusion model to obtain the feature comprehensive representations corresponding to the speech to be compared includes: Identify all high-dimensional feature vector representations corresponding to a target speech segment, wherein the target speech is any speech segment in the speech to be compared; Obtaining the dimension weights respectively corresponding to the feature vector representations of each dimension in all the high-dimensional feature vector representations in the feature representation fusion model; Perform multiplication operation on the feature vector representation of each dimension and its corresponding dimension weight to obtain the multiplication result; The multiplication results corresponding to the representations of all the high-dimensional feature vectors are obtained as a comprehensive representation of the features of the target speech segment.
6. The voiceprint comparison and recognition method according to claim 5, characterized in that: The step of obtaining the dimension weights respectively corresponding to the feature vector representations of each dimension in all the high-dimensional feature vector representations in the feature representation fusion model specifically includes: Select any dimension from all feature dimensions as the target dimension; Identify all feature classification sub-models that extract features of the target dimension as target sub-models; Obtain the feature extraction weights corresponding to the target dimension for all target sub-models; The feature extraction weights corresponding to the target dimension of all the target sub-models are used as the feature vector representation of the target dimension, and the dimensional weights corresponding to each other in the feature representation fusion model are represented.
7. The voiceprint comparison and recognition method according to claim 1, characterized in that: The step of using a preset distance algorithm to calculate the distance values between the feature comprehensive representations corresponding to the speech to be compared specifically includes: Using a cosine distance algorithm, the feature comprehensive representations of all speech segments in the speech to be compared are compared in pairs to obtain distance values between the feature comprehensive representations corresponding to all speech segments; The step of identifying speech segments of the same object from the speech to be compared based on the distance value specifically includes: Comparing the distance values between the feature comprehensive representations corresponding to all speech segments with a preset distance threshold; By comparison, a distance value smaller than a preset distance threshold is selected as a target distance value; The speech segments to be clustered are selected according to the target distance value, and clustered in pairs to obtain a preliminary clustering result; According to the preliminary clustering result, determining whether the same speech segment is clustered into different clustering results; If so, continue clustering the different clustering results corresponding to the same speech segment until the same speech segment is no longer clustered into different clustering results, and the clustering is completed; If it does not exist, the clustering is completed, and the speech clips in the same cluster represent the speech clips of the same object.
8. A voiceprint comparison and recognition device, characterized in that: include: A speech acquisition module to be compared, used to acquire the speech to be compared, wherein the speech to be compared includes at least two speech segments; A voiceprint feature extraction module, used for inputting the speech to be compared into a preset voiceprint feature extraction network to extract voiceprint features and obtain a voiceprint feature extraction result; A feature vector mapping module, used to map the voiceprint feature extraction result using a preset linear mapping layer to obtain high-dimensional feature vector representations corresponding to the speech to be compared; A feature fusion representation module, used for inputting the high-dimensional feature vector representation into a pre-trained feature representation fusion model to obtain the feature comprehensive representations corresponding to the speech to be compared, wherein the feature representation fusion model is composed of N feature classification sub-models, where N is a positive integer; A distance value calculation module, used to calculate the distance values between the feature comprehensive representations corresponding to the speech to be compared respectively by using a preset distance algorithm; The same-object speech recognition module is used to recognize speech segments of the same object from the speech to be compared based on the distance value.
9. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of the voiceprint comparison and recognition method according to any one of claims 1 to 7 when executing the computer-readable instructions.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the steps of the voiceprint comparison and recognition method according to any one of claims 1 to 7 are implemented.
Citation Information
Patent Citations
Method and device for self-updating fingerprint database based on manifold learning
CN104424276A
Underwater target classification method
CN109977724A
Identity authentication method and device based on fusion features, equipment and storage medium
CN111814128A
Training method and device, voiceprint recognition method and device, and electronic equipment
CN113409795A
Voiceprint recognition method and device
CN114495948A
