Speaker recognition method and system for real scene

Abstract

The invention relates to the technical field of speaker recognition, and discloses a speaker recognition method and system for a real scene, and the method comprises the steps: obtaining external sound information, carrying out the processing through a multi-resolution convolution kernel, extracting features in a time domain, and carrying out the feature fusion only in the time domain; after feature fusion in the time domain, features are extracted in the frequency domain by using grouping convolution, processing is carried out, and feature fusion is carried out only in the frequency domain; and after the frequency domain features are fused, performing mask processing, and finally outputting an identification result. Through a noise mask module and multi-resolution feature extraction, extraction of features of a target speaker is enhanced, and the influence of noise on an identification result is reduced. Meanwhile, in a complex noise environment, the system can effectively focus on the target speaker, and the recognition stability and reliability are improved. More comprehensive feature information is provided through multi-scale feature extraction of a time domain and a frequency domain, and fine differences of speakers can be captured.

Description

Technical Field

[0001] The present invention relates to the technical field of speaker recognition, and in particular to a speaker recognition method oriented to real scenarios. Background Art

[0002] The human auditory system is the most convenient and efficient recognition system besides vision. With the rapid development of the Internet and the Internet of Things, the use scenarios of personal identity authentication technology are becoming more and more diversified. At present, the field of identity authentication mainly adopts single or multi-modal personal identity authentication technologies such as fingerprints and faces. However, identity authentication technologies such as face recognition and fingerprint recognition not only rely on expensive professional equipment, but also can only achieve high accuracy in relatively specific environments. As a communication method that has existed since the birth of mankind, voice itself is universal. In addition, the individual differences and growth environment differences of different speakers lead to the differences and uniqueness of the voices of different speakers. Therefore, voiceprint recognition technology based on voice can realize the verification of personal identity.

[0003] In real scenarios, compared with identity authentication technologies such as face recognition and fingerprint recognition, speaker recognition technology has the following advantages: (1) High security. Voice is a dynamic feature that is more difficult to forge than static identity information. Voice can be combined with dynamic text passwords to reduce the risk of transcribing voice. (2) Strong privacy. Compared with visual information such as pictures and videos, users are more inclined to upload their own voice. (3) Low cost. Compared with fingerprint collectors, cameras and other collection devices, microphones are cheaper. In addition, speaker recognition technology can also help record voice in conference rooms, improve the accuracy of meeting records, and provide users with better voice interaction experience and safer device control experience in smart cars, smart homes and other devices.

[0004] With its wide range of application scenarios and important research value, speaker recognition technology is gradually entering every aspect of our lives and has broad development prospects and commercial value. Summary of the invention

[0005] In view of the above-mentioned problems, the present invention is proposed.

[0006] Therefore, the technical problem solved by the present invention is: the problem of noise interference and insufficient information in short-term speech recognition.

[0007] In order to solve the above technical problems, the present invention provides the following technical solutions: a speaker recognition method for real scenes, comprising: Acquire external sound information, process it through multi-resolution convolution kernels, extract features in the time domain and perform feature fusion only in the time domain; After feature fusion in the time domain, grouped convolution is used to extract features in the frequency domain, process them, and perform feature fusion only in the frequency domain; After the frequency domain features are fused, mask processing is performed and the recognition results are finally output.

[0008] As a preferred solution of the speaker recognition method for real scenes described in the present invention, the multi-resolution convolution kernel includes setting three parallel branch filters of different scales, each branch consisting of two layers of one-dimensional convolution; The first layer of one-dimensional convolution is used for primary filtering, and the second layer of one-dimensional convolution is used for dimension matching.

[0009] As a preferred solution of the speaker recognition method for real scenes described in the present invention, wherein: performing feature fusion in the time domain includes extracting features at different time resolutions through different convolution kernel sizes and step sizes, and splicing features of three different time resolutions along the channel dimension; Find the minimum time step and clip all features so that all features have the same time step.

[0010] As a preferred solution of the speaker recognition method for real scenes described in the present invention, wherein: extracting features in the frequency domain includes dividing the input frequency channel features into 4 groups to obtain four grouping subsets , , , ; The first subset is , the output is ; The second subset is ,pass The convolution kernel is processed to capture wider context information, and the output is ; The third subset is ,Will and Add, use The convolution kernel is processed to get the output ; The fourth subset is ,Will and Add, use The convolution kernel is processed to get the output ; By comparing Perform feature concatenation, input the merged feature vector into the classifier, and output the final speaker recognition result through activation and normalization processing.

[0011] As a preferred solution of the speaker recognition method for real scenes described in the present invention, the masking of the time-frequency features includes obtaining acoustic features of external sound information, predicting enhanced acoustic features, and obtaining a feature map; Apply a noise mask on the feature map to blur out irrelevant noise.

[0012] As a preferred solution of the speaker recognition method for real scenes described in the present invention, the masking of the time-frequency features further includes extracting speech-level feature vectors, calculating the mean and standard deviation, and mapping the mean and standard deviation to auxiliary context features; Auxiliary context features and acoustic features are used to generate noise masks and calculate the final feature map. The formula is expressed as: , , , in, represents the auxiliary context feature vector, represents the weight matrix, represents the transpose of the weight matrix; represents the mean vector calculated by statistical pooling, represents the standard deviation vector calculated by statistical pooling, represents the bias term, Indicated in The noise mask vector generated by the frame, represents the Sigmoid function, represents the weight matrix, represents the transpose of the weight matrix; Represents the combination of ReLU function and BN; represents another weight matrix, represents the transpose of the weight matrix; Indicates The feature vector of the frame, represents the bias term, represents the feature map after noise mask processing, represents the transformation function, Representation of feature representation; represents the generated noise mask, Indicates multiplication.

[0013] As a preferred solution of the real-scene speaker recognition method described in the present invention, the final output recognition result includes inputting the merged feature vector into a classifier, and the classifier outputs the final speaker recognition result through activation and normalization processing.

[0014] A speaker recognition system for real scenarios, wherein: The noise mask module processes noise interference, extracts robust speaker feature vectors, and focuses on the target speaker and blurs irrelevant noise by dynamically predicting the noise mask; The time domain multi-resolution encoder extracts features of different time resolutions in the time domain, sets multiple parallel branch filters, extracts features from the input speech signal through convolution kernels of different scales, and performs feature fusion; The frequency domain multi-channel encoder extracts features in the frequency domain by grouping convolution to process short-term speech information, divides the frequency domain features into multiple groups, and uses different convolution kernels for step-by-step processing and fusion; Classifier, input the processed features into the classifier for speaker recognition and output the recognition results.

[0015] A computer device comprises: a memory and a processor; the memory stores a computer program, wherein: the processor implements the steps of any one of the methods of the present invention when executing the computer program.

[0016] A computer-readable storage medium stores a computer program, wherein: when the computer program is executed by a processor, the steps of any one of the methods of the present invention are implemented.

[0017] Beneficial effects of the present invention: The speaker recognition method for real scenes provided by the present invention enhances the extraction of target speaker features and reduces the impact of noise on recognition results through noise masking modules and multi-resolution feature extraction. At the same time, in a complex noise environment, the system can effectively focus on the target speaker and improve the stability and reliability of recognition. Through multi-scale feature extraction in the time domain and frequency domain, more comprehensive feature information is provided, which helps to capture the subtle differences between speakers. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0019] Figure 1 An overall flow chart of a speaker recognition method for real scenarios provided by the first embodiment of the present invention; Figure 2 A system structure diagram of a speaker recognition method for real scenarios provided by the first embodiment of the present invention; Figure 3A MTFNM-TDNN model structure diagram of a real-scene speaker recognition method provided by the first embodiment of the present invention; Figure 4 A noise masking module of a speaker recognition method for real scenes provided by the first embodiment of the present invention; Figure 5 A time domain multi-resolution encoder for a speaker recognition method for real scenes provided by the first embodiment of the present invention; Figure 6 A ResfBlock structure diagram of a speaker recognition method for real scenarios provided by the first embodiment of the present invention; Figure 7 A frequency domain multi-channel encoder for a speaker recognition method for real scenarios is provided in the first embodiment of the present invention. DETAILED DESCRIPTION

[0020] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in the art without creative work should fall within the scope of protection of the present invention.

[0021] Example 1, reference Figure 1 , Figure 2 and Figure 3 , as an embodiment of the present invention, provides a speaker recognition method for real scenarios, comprising: S1: Obtain external sound information, process it through multi-resolution convolution kernels, and extract features in the time domain.

[0022] In view of the problem of multiple types of noise and short speech duration in real scenes, a noise masking method and short-time speech information enhancement method based on deep neural network are proposed, including: in view of the problem that there are multiple types of noise in real scenes, which affects the accuracy of the speaker recognition system, it is considered to uniformly mask different types of noise, and apply the noise masking module to the deep and shallow layers of the network respectively. When applied to the shallow layer of the network, it can process non-speech information such as background noise. When applied to the deep layer of the network, it can process other speaker interference noise such as laughter and discussion. This module solves the noise interference problem by omitting the speech and non-speech noise that interferes with the speaker. The discourse-level feature vector is extracted by statistical pooling to distinguish the target speaker from the global noise, and all frames are combined with the statistical pooling layer to assign different mask values ​​to different channels and frames.

[0023] Then, in order to address the problem of insufficient short-term speech information, we consider enhancing short-term speech information in both the time domain and the frequency domain. A multi-resolution encoder is constructed in the time domain, and a multi-channel encoder is constructed in the frequency domain to extract features of different frequencies and time resolutions, focusing on global information and local detail information. The time domain multi-resolution encoder uses convolution kernels with different step sizes to extract multi-scale convolution features. These convolution kernels extract features at different time resolutions, and then splice the features of three different time resolutions along the channel dimension to ensure that all features have the same time step. The multi-channel encoder adopts the idea of ​​grouped convolution. Compared with ordinary single-layer convolution, the output result of grouped convolution will contain a combination of different receptive field sizes, respectively obtaining information at different frequency scales.

[0024] The degradation of recognition performance caused by noise has always been a long-standing challenge for speaker recognition. For the processing of noise, previous methods usually perform denoising transformation on the speaker's feature vector or enhance the speaker's feature vector. These methods are lossy and inefficient for extracting speaker feature vectors. The present invention proposes a method for extracting robust speaker feature vectors, a noise masking module. The noise masking module enables the speaker recognition model to focus on the target speaker and blur irrelevant noise.

[0025] The method of applying denoising transformation to speaker feature vector is to transform the noisy speaker feature vector into an enhanced speaker feature vector using a statistical backend or a neural network backend. The speaker feature vector is obtained by aggregating hundreds of frame-level features into an utterance-level speaker feature vector through a statistical pooling layer (such as mean pooling). During the aggregation stage, information loss is inevitable. Therefore, post-processing the noisy speaker feature vector after the aggregation step is lossy.

[0026] Methods for filtering noise through speech enhancement models, which train models based on noise-clean speech pairs or perform unsupervised speech enhancement through generative adversarial networks. However, these methods are not specific to speaker recognition tasks, and the training data for speech enhancement and speaker feature vectors may belong to different fields, which greatly increases the cost of training.

[0027] In speaker recognition tasks, the model input usually uses filter bank band (Fbank) features and Mel frequency cepstral coefficient (MFCC) features. However, for speaker recognition tasks with short speech, since the speech time is short and contains limited information, using Fbank features and MFCC features as input will lead to a decrease in model performance.

[0028] Fbank features have some shortcomings when processing short-term speech. The short duration of short-term speech leads to less spectral information in Fbank features, which may make the features insufficient to fully characterize the characteristics of the speaker, thereby reducing recognition accuracy. Fbank features mainly reflect the spectral information of speech, but in short-term speech, due to limited timing information, it may not be able to fully capture the dynamic changes of speaker characteristics.

[0029] For MFCC features, the extraction process relies on a fixed-length time window. For short-term speech, this fixed time window may not be able to fully adapt to the changes in short-term speech, thus affecting the accuracy of the features. In addition, in the process of extracting MFCC features, the spectrum information is compressed by discrete cosine transform (DCT), which will lose some high-frequency information. For short-term speech, the already limited information is further compressed, which may lead to the loss of important speaker features.

[0030] In order to solve the problem of insufficient speech information caused by short-term speech, the present invention designs a time domain multi-resolution encoder, which sets three parallel branch filters of different scales. The convolution kernel size and step size of these branch filters are different. Each branch consists of two layers of one-dimensional convolution, such as Figure 5 As shown in Figure 1, the first layer of one-dimensional convolution is used for primary filtering, and the second layer of one-dimensional convolution is used for dimension matching to compensate for the resulting dimension difference.

[0031] The detailed settings of temporal multi-resolution coding are shown in Table 1, and TDNN is used as a frame aggregator to match the dimension with the post-order structure. The numbers in brackets are convolution parameters: they represent the number of channels, size, and stride of the convolution kernel, respectively.

[0032] Table 1 Multi-resolution encoder design table , The time domain multi-resolution encoder extracts features at different time resolutions through three different branches of convolution kernels. The three different time resolution features are spliced ​​along the channel dimension through torch.cat, the minimum time step is found, and all features are cropped to ensure that all features have the same time step.

[0033] Furthermore, in this way, the time-domain multi-resolution encoder can effectively capture the features of different time resolutions in the input speech signal, and fuse these features to increase the information of short-term speech for subsequent processing.

[0034] S2: After extracting features in the time domain, group convolution is used to extract features in the frequency domain for processing and feature fusion.

[0035] Short-term speech may contain limited frequency information, especially in the high-frequency part, where the information may be lost or insufficient. This may cause the model to be unable to capture the complete spectral characteristics of the speech signal. Secondly, the currently commonly used frequency domain feature extraction methods (such as MFCC or Fbank) may not be able to fully capture the frequency domain characteristics of the speech signal, especially in the case of complex background noise or drastic speech changes, their expression ability is limited. In addition, the current frequency domain features may not be able to fully express the characteristics of the speaker. The frequency domain information in the speech signal may have different importance in different frequency ranges, and existing methods may not be able to effectively capture these differences.

[0036] In order to solve the above problems, the present invention designs a frequency domain multi-channel encoder, which applies the idea of ​​grouped convolution to the frequency dimension. Aiming at the problem of insufficient speech information due to the short duration of short-term speech, the frequency domain multi-channel encoder based on the idea of ​​grouped convolution divides the frequency domain features into multiple groups, each of which can be regarded as features at different scales. This allows the model to capture frequency domain information at different scales more comprehensively, thereby improving the diversity and richness of features.

[0037] In addition, group convolution processes features at different granularities in the frequency domain, avoiding the risk of information loss. Each group can extract features within a specific frequency range through an independent convolution kernel, thereby retaining more frequency domain information. Finally, the features within each group can communicate and integrate information through residual connections or other methods. This communication mechanism can help effectively transfer and learn information between features, enhancing the information expression ability of short-term speech in the frequency domain.

[0038] like Figure 6 and Figure 7 As shown, the input features First, we group the original The data of dimension is divided into 4 groups dimensional data, so we get four grouping subsets , , , , the frequency channel dimension of each subset is , thus realizing the frequency grouping processing. For the first subset , apply BN and ReLU activation functions to get the output , the purpose is to improve the stability of the model training process and accelerate convergence. For the second subset , first use a The convolution kernel is processed to capture more extensive context information, and then processed by BN and ReLU to output This step enables the model to maintain local details of features while also paying attention to broader feature information. With the third subset Add as the input of the next convolution operation. Here we use a The convolution kernel is used to balance the width of the receptive field and the specificity of the features to obtain a more fine-grained feature representation. After processing, it is also processed by BN and ReLU, and the output is . Finally, With the fourth subset Add, use The convolution kernel is used to process. It aims to further extract more refined local features to further enhance the model's ability to capture details. After BN and ReLU, the final output is obtained. .

[0039] In summary, through a series of grouped convolutions and gradually integrating feature processing at different scales, the final multi-scale grouped convolution output is This is done by , , and Feature stitching.

[0040] In the experimental stage, the Equal Error Rate (EER) and the minimum detection cost function (minDCF) indicators are used to evaluate the overall performance of the speaker recognition model. The Equal Error Rate indicates the value when the False Acceptance Rate (FAR) is equal to the False Rejection Rate (FRR). The calculation formulas for FAR and FRR are shown in equations (10) and (11) respectively. When the two error values ​​are equal, the value is EER.

[0041] (10) (11) in, Indicates the number of incorrect acceptances, Indicates the total number of test speech segments of other speakers, Indicates the number of false rejections, Indicates the total number of target speaker test speech segments.

[0042] The minimum detection cost function compares the error rate taking into account the prior probability and different costs, and the calculation is shown in formula (12).

[0043] (12) in, represents the risk factor of incorrectly accepting samples, represents the prior probability of the positive example pair, represents the risk factor of falsely rejecting the sample.

[0044] Furthermore, the frequency domain multi-channel encoder not only enhances the model's ability to process features of different frequency scales, but also obtains a rich feature representation by step-by-step processing and gradual fusion of the output of each group. These features contain information at multiple scales, enabling the model to capture subtle differences in the frequency domain and effectively enhancing the information of short-term speech in the frequency domain.

[0045] S3: After extracting the time domain and frequency domain features, mask processing is performed and the recognition results are finally output.

[0046] like Figure 4 As shown in the figure, the noise mask module is used to enable the speaker recognition model to process noise and extract a robust speaker feature vector. The noise mask module is applied before the aggregation step to avoid information loss. The noise mask module extracts an auxiliary context feature vector to distinguish the characteristics of the target speaker and noise, so that the speaker recognition model can focus on the target speaker and blur irrelevant noise.

[0047] The specific design of the noise mask module is as follows: The input acoustic features are expressed as , then the selected hidden layer output feature map is (1): (1) in, represents the transformation of the hidden layer, represents the transformation before the hidden layer.

[0048] Based on previous speech enhancement methods, the first step is to predict the enhanced acoustic features, expressed as , then the output feature map of the corresponding hidden layer becomes formula (2): (2) The noise mask is to apply a ratio mask to the feature map. If a mask that satisfies the following formula (3) can be found , it is possible to obtain an effect similar to speech enhancement: (3) in, represents element-wise multiplication, The values ​​in are normalized to (0,1).

[0049] The following section will demonstrate how to estimate a suitable noise mask. To achieve this goal, the present invention utilizes the feature vectors of the target speaker and the noise, which can be expressed as The auxiliary context feature vector is obtained. Figure 3 As shown, this module predicts the noise mask frame by frame: (4) (5) (6) in, represents the Sigmoid function, Represents the combination of ReLU function and BN, is the predicted noise mask, and Indicates that you are currently in frame. Represents the feature map after noise masking, auxiliary context feature Used as a dynamic vector to control the activation threshold.

[0050] The noise mask module dynamically extracts auxiliary context feature vectors without providing additional clean speech of the target speaker as a reference. The noise mask module automatically finds the target speaker in the forward propagation.

[0051] The noise masking module identifies a speaker as the target speaker when any of the following conditions are met: (1) the speaker occupies most of the speech in the utterance; (2) the speaker's volume is significantly higher than that of other speakers. The speaker appearing in the utterance is called an interfering speaker. In the speaker recognition task, blurring the speech and non-speech noise of the interfering speaker is beneficial.

[0052] A simple way to distinguish the target speaker from the noise is to extract the utterance-level feature vector by first combining all frames with a statistical pooling layer: , , After the statistical pooling layer, the FC layer maps the mean and standard deviation vectors into auxiliary context features: , in, represents the auxiliary context feature vector, represents the weight matrix, represents the mean vector calculated by statistical pooling, represents the standard deviation vector calculated by statistical pooling, represents the bias term, Indicated in The noise mask vector generated by the frame, represents the Sigmoid function, represents the weight matrix, Represents the combination of ReLU function and BN; represents another weight matrix, Indicates The feature vector of the frame, represents the bias term, represents the feature map after noise mask processing, represents the transformation function, Representation of feature representation; represents the generated noise mask, Indicates multiplication.

[0053] On the other hand, this embodiment also provides a speaker recognition system for real scenarios, which includes: The noise mask module handles noise interference and extracts robust speaker feature vectors. It focuses on the target speaker and blurs irrelevant noise by dynamically predicting the noise mask.

[0054] The time domain multi-resolution encoder extracts features of different time resolutions in the time domain, sets multiple parallel branch filters, extracts features from the input speech signal through convolution kernels of different scales, and performs feature fusion.

[0055] The frequency domain multi-channel encoder extracts features in the frequency domain by grouped convolution to process short-term speech information, divides the frequency domain features into multiple groups, and uses different convolution kernels for step-by-step processing and fusion.

[0056] Classifier, input the processed features into the classifier for speaker recognition and output the recognition results.

[0057] If the above functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program code.

[0058] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0059] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0060] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0061] Example 2, in order to improve the accuracy of speaker recognition, the present invention designs a set of base model comparison experiments to verify the impact of different methods on speaker recognition accuracy. The purpose of this experiment is to verify whether the MTFNM⁃TDNN model proposed in the present invention is more competitive than the selected mainstream speaker recognition model ECAPA-TDNN model. The recognition effects of the ECAPA-TDNN model and the MTFNM⁃TDNN model proposed in this paper are compared on the two evaluation indicators of equal error rate and minimum detection cost function.

[0062] The experiment compares and analyzes the equal error rate and minimum cost test function of the two models, and the results are shown in Table 2. As can be seen from the table, the noise mask module and the multi-scale time-frequency channel module that adds short-term speech information in the present invention are conducive to improving the performance of the speaker recognition system in real scenarios. An offline speaker recognition system was designed on Jetson Nano, which has functions such as recording registration, recording recognition, and model selection. The experimental results are shown in Table 2.

[0063] Table 2 Performance comparison of the two models , The present invention proposes a speaker recognition method for real scenes and designs an offline speaker recognition system. The method uses a noise mask module to deal with the problem of noise diversity in real scenes causing the performance of the speaker recognition system to decline, and a multi-scale time-frequency channel module to deal with the problem of insufficient short-term speech information. In addition, the present invention implements an offline speaker recognition system, creating conditions for future model deployment.

[0064] The method of the present invention not only improves the performance of the speaker recognition system in real scenarios, but also deploys the model offline, which is suitable for speaker recognition applications in real scenarios, such as intelligent conference recording, vehicle-mounted human-computer interaction, etc. The technical solution of the present invention provides technical support for the development of speaker recognition technology in real scenarios, and has broad development prospects and commercial value.

[0065] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A speaker recognition method for real scenarios, characterized in that: include: Acquire external sound information, process it through multi-resolution convolution kernels, extract features in the time domain and perform feature fusion only in the time domain; After feature fusion in the time domain, grouped convolution is used to extract features in the frequency domain, process them, and perform feature fusion only in the frequency domain; After the frequency domain features are fused, mask processing is performed and the recognition results are finally output.

2. The real-scene speaker recognition method according to claim 1, characterized in that: The multi-resolution convolution kernel includes setting three parallel branch filters of different scales, each branch consisting of two layers of one-dimensional convolution; The first layer of one-dimensional convolution is used for primary filtering, and the second layer of one-dimensional convolution is used for dimension matching.

3. The real-scene speaker recognition method according to claim 2, characterized in that: Feature fusion in the time domain includes extracting features at different time resolutions through different convolution kernel sizes and step sizes, and splicing the features of three different time resolutions along the channel dimension; Find the minimum time step and clip all features so that all features have the same time step.

4. The real-scene speaker recognition method according to claim 3, characterized in that: Extracting features in the frequency domain includes dividing the input frequency channel features into 4 groups to obtain four grouping subsets , , , ; The first subset is , the output is ; The second subset is ,pass The convolution kernel is processed to capture wider context information, and the output is ; The third subset is ,Will and Add, use The convolution kernel is processed to get the output ; The fourth subset is ,Will and Add, use The convolution kernel is processed to get the output ; By comparing Perform feature concatenation, input the merged feature vector into the classifier, and output the final speaker recognition result through activation and normalization processing.

5. The real-scene speaker recognition method according to claim 4, characterized in that: Masking the time-frequency features includes obtaining acoustic features of external sound information, predicting enhanced acoustic features, and obtaining a feature map; Apply a noise mask on the feature map to blur out irrelevant noise.

6. The real-scene speaker recognition method according to claim 5, characterized in that: Masking the time-frequency features also includes extracting a discourse-level feature vector, calculating a mean and a standard deviation, and mapping the mean and the standard deviation to auxiliary context features; Auxiliary context features and acoustic features are used to generate noise masks and calculate the final feature map. The formula is expressed as: ， ， ， in, represents the auxiliary context feature vector, represents the weight matrix, represents the transpose of the weight matrix; represents the mean vector calculated by statistical pooling, represents the standard deviation vector calculated by statistical pooling, represents the bias term, Indicated in The noise mask vector generated by the frame, represents the Sigmoid function, represents the weight matrix, represents the transpose of the weight matrix; Represents the combination of ReLU function and BN; represents another weight matrix, represents the transpose of the weight matrix; Indicates The feature vector of the frame, represents the bias term, represents the feature map after noise mask processing, represents the transformation function, Representation of characteristic representation; represents the generated noise mask, Indicates multiplication.

7. The real-scene speaker recognition method according to claim 6, characterized in that: The final output recognition result includes inputting the combined feature vector into a classifier, and the classifier outputs the final speaker recognition result through activation and normalization processing.

8. A real-scene speaker recognition system using the method according to any one of claims 1 to 7, characterized in that: The noise mask module processes noise interference, extracts robust speaker feature vectors, and focuses on the target speaker and blurs irrelevant noise by dynamically predicting the noise mask; The time domain multi-resolution encoder extracts features of different time resolutions in the time domain, sets multiple parallel branch filters, extracts features from the input speech signal through convolution kernels of different scales, and performs feature fusion; The frequency domain multi-channel encoder extracts features in the frequency domain by grouping convolution to process short-term speech information, divides the frequency domain features into multiple groups, and uses different convolution kernels for step-by-step processing and fusion; Classifier, input the processed features into the classifier for speaker recognition and output the recognition results.

9. A computer device comprising: Memory and processor; The memory stores a computer program, characterized in that when the processor executes the computer program, the processor implements the steps of a speaker recognition method for real scenarios.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of a speaker recognition method for real scenarios are implemented.

Images