Speaker recognition method based on speaker voice micro-action

Through comparative learning between teacher network and student network, combined with ECAPA-TDNN voiceprint model and accent data, the problem of insufficient generalization ability of speakers to identify data in the existing technology is solved, and efficient identification among large-scale populations is achieved. Manual labeling requirements are improved, and the generalization performance and practicality of the model are improved.

CN118918900AActive Publication Date: 2024-11-08NANJING LONGYUAN INFORMATION TECH CO LTD +1
View PDF 8 Cites 0 Cited by

Patent Information

Application Number
CN202411154941.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-22
Publication Date
2024-11-08
Estimated Expiration
2044-08-22

AI Technical Summary

Technical Problem

The existing speaker recognition technology has shortcomings in data generalization capabilities and practicality, especially in complex data environments. Machine learning methods rely on low-level rhythm characteristics and require fine manual annotation, so they cannot effectively consider the changes in speech flow.

Method used

Using a speech stream-based speaker recognition method, through comparative learning between teacher network and student network, combined with ECAPA-TDNN voiceprint model and accent data, new features are introduced for feature aggregation and classification, and data enhancement is used to improve the generalization performance of the model. Avoid fitting in channel features and no manual annotation is required.

Benefits of technology

Improves the ability of the speaker recognition model to recognize in large-scale populations, can focus on phoneme and vocabulary-level features for longer periods, improves the generalization performance and practicality of the model, and is suitable for speaking in millions to tens of millions of people. Person verification.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118918900A_ABST
    Figure CN118918900A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of electronic digital data processing, in particular to a speaker recognition method based on speaker voice micro-actions, which comprises the following steps of: preprocessing a voice stream, extracting Fbank features, and respectively sending the Fbank features into a teacher network and a student network to obtain respective corresponding feature embedding; embedding the features obtained by the teacher network and the student network into a loss function and carrying out back propagation; performing normal iteration on the student network, and performing iteration on the teacher network through an EMA moving average method; carrying out feature aggregation and classification on voiceprint feature information extracted by the ECAPA-TDNN voiceprint model and speaker voice micro-action information obtained by accent data training, and carrying out speaker recognition; according to the method, the generalization performance of the model is improved by using methods such as data enhancement and the like, fitting in channel features is avoided, manual labeling is not needed, and new features are introduced to help the speaker verification model to obtain the ability of identifying a speaker in a larger crowd.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of electronic digital data processing, and in particular to a speaker recognition method based on speaker voice micro-movements. Background Art

[0002] At present, text-independent speaker recognition technology often relies on voiceprint recognition technology, which can extract text-independent speaker voice features from speech. One of the most typical and best-captured micro-movements of a speaker is the speaker's accent, so accent recognition can be used to achieve voice micro-movement recognition. Accent is a specific way of pronunciation in a language that is affected by many factors such as region, native language, and education level. Accent classification technology can be very helpful for tasks such as speech recognition (ASR) or voice assistants. In addition, it can also help with speaker recognition tasks and measure the degree of pronunciation standards.

[0003] In the new century of computer technology development, with the rise of machine learning and deep learning, deep neural networks such as CNN and RNN are applied to speaker verification. Starting from the initial machine learning methods such as GMM-UBM, speaker verification methods have gone through the i-vector method extracted by GMM and DNN networks and the i-vector method extracted by DNN-UBM. These methods use artificially designed features to allow learning algorithms to extract the information required for speaker verification by performing spectral processing on the signal and calculating statistics. Subsequently, the d-vector method using DNN emerged. This method extracts the frame-level features of a segment of speech and extracts the features of a segment of audio by averaging and other methods. Subsequently, the x-vector method for speaker verification was used to further aggregate the frame-level features through learnable parameters to obtain the features of a segment of speech. The above methods were eventually transformed into the ECAPA-TDNN method. This method has achieved leading results on various data sets and is widely used in practical speech recognition applications.

[0004] However, using the above methods, the data generalization ability of machine learning is very poor. The use of machine learning and methods that rely on low-level musical features require careful manual annotation and have high requirements for data integrity. These methods only point out the differences in time and frequency between different accents in phonology, and these differences can be used to guide manual classification. However, these methods do not consider situations such as sound changes in speech flow, and are not very practical for large amounts of complex data in reality. Summary of the invention

[0005] The purpose of the present invention is to provide a speaker recognition method based on the speaker's voice micro-movements. When data is widely collected, the generalization performance of the model is improved by using methods such as data enhancement to avoid fitting in channel features without the need for manual labeling. New features are introduced to help the speaker verification model acquire the ability to recognize speakers in a larger population.

[0006] To achieve the above object, the present invention provides a speaker recognition method based on speaker voice micro-movements, comprising:

[0007] After preprocessing, the speech stream extracts Fbank features and sends them to the teacher network and student network respectively to obtain their corresponding feature embeddings;

[0008] The features obtained from the teacher network and the student network are embedded into the loss function and back-propagated;

[0009] The student network is iterated normally, and the teacher network is iterated by the EMA sliding average method;

[0010] The voiceprint feature information extracted by the ECAPA-TDNN voiceprint model is aggregated and classified with the speaker's voice micro-movement information obtained through accent data training to perform speaker recognition.

[0011] The teacher network and the student network have the same structure and hyperparameters, and the difference between the teacher network and the student network lies in the different parameters in the model.

[0012] The operation process of the teacher network and the student network includes:

[0013] After segmentation and data enhancement, the speech stream is extracted and Fbank features are sent to the general spoken acoustic model and the standard Mandarin acoustic model;

[0014] The sound features extracted from the universal accent acoustic model and the standard Mandarin acoustic model are combined;

[0015] Perform feature contrast encoding on the fused sound features and map the features back to the original data space.

[0016] The universal accent acoustic model and the standard Mandarin acoustic model adopt the Uni-ASR model.

[0017] Among them, the data sets of the universal acoustic model with accent include the accented Mandarin data set, the AISHELL-1 data set and the AISHELL-2 data set. The accented Mandarin data set is Mandarin with accent collected on the Internet through crawlers, and the video subtitles or text content it carries can be used as its speech transcription. The total length of the AISHELL-1 data set and the AISHELL-2 data set is more than 1,100 hours.

[0018] The data set of the standard Mandarin acoustic model uses broadcast data collected online, which is annotated by ASR after background music separation and advertisement removal. A total of 2,100 hours of audio are collected, and more than 1,750 hours of valid data are obtained after data cleaning.

[0019] The specific steps of fusing the sound features extracted from the universal accent acoustic model and the standard Mandarin acoustic model include:

[0020] The features output by the universal accent acoustic model and the standard Mandarin acoustic model are spliced ​​in the channel dimension, and the spliced ​​features are passed through a global maximum pooling and a global average pooling, and then passed through a bottleneck layer to compress its dimension and obtain the attention parameters, which are then convolved with the original spliced ​​features.

[0021] The present invention discloses a speaker recognition method based on the speaker's voice micro-movements, which is completed by means of an accent recognition framework and a contrastive learning framework. By means of this framework, a speaker verification method integrating voice micro-movements and voiceprint features is proposed. In the case of extensive data collection, data enhancement and other methods are used to improve the generalization performance of the model, avoid fitting in channel features, and do not require manual labeling. Through the design of the model and the selection of features, the model can focus on features at the phoneme level and above the phoneme level for a longer period of time. The model of the present invention can be ultimately used for speaker verification to help the model achieve better speaker recognition results by introducing new features. The method of the present invention can help the speaker verification model play a role in the number of people ranging from millions to tens of millions. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art are briefly introduced below.

[0023] Figure 1 The present invention is a flow chart of a speaker recognition method based on speaker voice micro-movements.

[0024] Figure 2 It is an operation flow chart of the teacher network and the student network of the present invention.

[0025] Figure 3 It is a schematic diagram of the overall structure of the model of the present invention.

[0026] Figure 4 It is a schematic diagram of the feature encoder and its auxiliary model structure of the present invention.

[0027] Figure 5 It is a schematic diagram of the Uni-ASR model structure of the present invention.

[0028] Figure 6 It is a structural schematic diagram of the feature aggregation module of the present invention. DETAILED DESCRIPTION

[0029] See also Figure 1-Figure 6 The present invention provides a speaker recognition method based on speaker voice micro-movements, comprising:

[0030] After preprocessing, the S1 speech stream extracts the Fbank features and sends them to the teacher network and student network respectively to obtain their corresponding feature embeddings;

[0031] The teacher network and the student network have the same structure and hyperparameters, and the difference between the teacher network and the student network lies in the different parameters in the model. The present invention can be integrated into an end-to-end system after multiple stages of training, such as Figure 3 As shown in the figure, the left and right system branches have the same structure and hyperparameters. The difference between the two is the parameters in the model. This is a comparative learning structure. After the speech stream is segmented and data enhanced, the Fbank features are extracted and sent to the model.

[0032] The operation process of the teacher network and the student network includes:

[0033] After segmentation and data enhancement, the S11 speech stream extracts Fbank features and sends them to the spoken general acoustic model and the standard Mandarin acoustic model;

[0034] The universal acoustic model for accents and the standard Mandarin acoustic model adopt the Uni-ASR model. The universal acoustic model for accents and the standard Mandarin acoustic model adopt the same structure, and through the model with the same structure, the model can give acoustic features that are aligned in time. The ASR model does not need to distinguish different accents, and the ASR model will ignore different accents and still map some different pronunciations of the same text to the non-accent transcription of the dataset. In other words, the ASR model will not strive to explore the differences between different accents. On the contrary, accent classification needs to find the difference between the accented and the reference original accent-free. Therefore, by allowing the deep neural network to grasp the difference between the two, different accents can be better classified.

[0035] The data sets of the universal acoustic model for accents include an accented Mandarin data set, an AISHELL-1 data set, and an AISHELL-2 data set. The accented Mandarin data set is Mandarin with accents collected on the Internet through crawlers, and the video subtitles or text content it carries can be used as its speech transcription. The accented video is processed by audio extraction, downsampling, etc.; the total length of the AISHELL-1 data set and the AISHELL-2 data set is greater than 1,100 hours, and can provide high-quality speech audio and speech transcription.

[0036] The data set of the standard Mandarin acoustic model uses broadcast data collected online, which is annotated by ASR after background music separation and advertisement removal. A total of 2,100 hours of audio are collected, and more than 1,750 hours of valid data are obtained after data cleaning. The present invention hopes that this model can faithfully reflect its situation relative to standard Mandarin pronunciation, that is, it can faithfully represent the phoneme situation of the input speech, and map the accented phonemes in the dialect into different vectors.

[0037] The universal accent acoustic model and the standard Mandarin acoustic model use two acoustic models (AM) with the same structure. The AM model is Uni-ASR. The model architecture is as follows: Figure 5 As shown in the figure, two complete models are trained for standard Mandarin AM and accented AM respectively. When performing end-to-end tasks, only the encoder part is used. The complete ASR model structure is introduced below.

[0038] The architecture consists of online and offline components, both of which share a dynamic block encoder to reduce computational complexity. The online component includes a dynamic block encoder and a SCAMA decoder. The encoder uses the LC-SAN-M structure, while the decoder is mainly based on a unidirectional deep feedforward sequential memory network (DFSMN) with a multi-head attention (MHA) layer, and its attention mechanism uses the SCAMA structure. The offline component includes a strided convolution, a full sequence encoder, an optional text encoder, and a full sequence decoder. Strided convolution is a convolutional layer with a stride of 2. Both the full sequence encoder and the text encoder use the SAN-M structure, while the full sequence decoder contains DFSMN and MHA layers. The attention mechanism inside the full sequence decoder consists of an acoustic context vector and a text context vector. The acoustic context vector is the output of the offline encoder, while the text context vector is the output of the online SCAMA layer, and the two context vectors are then concatenated to produce the attention vector.

[0039] (1) SAN-M layer

[0040] This layer improves the self-attention capability through DFSMN memory blocks. SAN-M provides an effective mechanism to incorporate FSMN memory blocks into self-attention to obtain powerful local and long-term dependency modeling capabilities. The MHA layer can be expressed using the following formula:

[0041] MultiHead(Q,K,V)=[head1,...,head h ]W O (1)

[0042]

[0043] The FSMN memory block output is calculated as follows:

[0044]

[0045] where v t L represents the t-th time instance in the self-attention value. l ,L r are the forward or backward der of the FSMN memory block, respectively. ⊙ represents the Hadamard product. A DFSMN filter is added to the value inside the MHA to output the memory block. The memory content is then added to the output of the MHA, i.e.:

[0046] Y=MultiHead(X)+M (5)

[0047] (2) LC-SAN-M layer

[0048] Input sequence Divide according to the preset block size c to form The block size c is related to the encoder delay. K ={[x kc+1 ,...,x (k+1)c ]} T represents the samples in the k-th block. For each time instance in the k-th block, it can only access samples in the current and previous blocks. The calculation for X K The output of LC-SAN-M can be expressed as:

[0049]

[0050] MultiHead(X k )=[head1(k),...,head h (k)]W O (10)

[0051] Then formula (3) can be rewritten as follows:

[0052]

[0053] M=[m kc+1 ,...,m (k+1)c ] T (12)

[0054] The L represents the total number of filters in the FSMN memory block. Finally, LC-SAN-M can be written as follows:

[0055] Y k =MultiHead(X k )+M k (13)

[0056] S12 combines the sound features extracted from the universal accent acoustic model and the standard Mandarin acoustic model;

[0057] The specific steps include:

[0058] The features output by the universal accent acoustic model and the standard Mandarin acoustic model are spliced ​​in the channel dimension, and the spliced ​​features are passed through a global maximum pooling and a global average pooling, and then passed through a bottleneck layer to compress its dimension and obtain the attention parameters, which are then convolved with the original spliced ​​features.

[0059] The feature fusion module fuses the sound features extracted by the two acoustic models and provides them to the subsequent feature encoding comparison module for comparison. Obviously, there are many ways to aggregate, and the present invention chooses the aggregation method of channel attention. This method can automatically highlight the features so that the feature comparison module can more effectively capture the differences and similarities between the two, while reducing the features that do not need to be paid attention to. The features of the present invention can be described by the following formula:

[0060] First, the features of the two AM outputs are combined in the channel dimension, where A asr , Refers to the feature vectors of the output of the common accent AM and the standard Mandarin AM:

[0061]

[0062] Then Through a global maximum pooling and global average pooling, and then through a bottleneck layer, the dimension is compressed and then air-converted. After that, the attention parameter can be obtained and convolved with the original spliced ​​features. The formula for each step is as follows:

[0063]

[0064] C=ReLU(linear(ReLU(linear(c)))) (16)

[0065] CA=Sigmoid(C) (17)

[0066]

[0067] The bottleneck layer in formula (16) in this embodiment selects a compression dimension of 16, that is, the original feature is 2d emb If , the dimension after compression is

[0068] S13 performs feature comparison encoding on the fused sound features and maps the features back to the original data space.

[0069] Feature contrast coding uses a feature contrast coding module, which uses the encoder module of Transformer. This module has been proven to be a very effective way to extract information from sequence information. The present invention chooses to stack encoder modules to perform high-level extraction of accent attributes for each frame. This encoder module is composed of a multi-head attention mechanism and a feedforward neural network consisting of two layers of inverse bottleneck layers. The attention mechanism finds the similarity between the K matrix and the Q matrix, and uses the similarity score to obtain the output of the V matrix. In the experiment, the attention mechanism is applied in the form of self-attention, where the Q, K and V matrices are the same (the input of the self-attention layer). The self-attention layer finds the relationship between acoustic features at different time steps and outputs the processed features.

[0070] The present invention inserts a [cls] tag at the beginning of the sequence. For any input, the [cls] tag always appears before all actual input audio embeddings. The input sequence is processed through the Transformer architecture, and the output of each layer generates a vector representation for each token in the sequence. For sequences containing the [cls] tag, the output vector at the [cls] tag position is regarded as a comprehensive feature representation of the entire feature sequence. This is because the Transformer model can learn the representation of each position in a context-independent manner, and the [cls] tag, because it does not carry specific semantic information, can more "fairly" fuse the information of the entire feature sequence. Therefore, it can be considered that this tag can capture the main information of the entire input feature sequence. In this way, the [cls] tag, as a feature vector representing the entire feature sequence, greatly simplifies the subsequent model, so that the model does not need to consider the influence of sequence length, and can better adapt to various types of inputs.

[0071] S2 embeds the features obtained from the teacher network and the student network into the loss function and performs back propagation;

[0072] S3 The student network is iterated normally, and the teacher network is iterated by the EMA sliding average method;

[0073] The teacher network and the student network are a contrastive learning structure, and the contrastive learning framework used is a self-distillation algorithm framework, in which the output of the teacher network is used as the optimization target for optimizing the student network. Through data augmentation, the input speech is converted into a variety of different lengths and "enhanced perspectives". On this basis, the audio is divided into global perspective and local perspective according to the length of the input segment. The global perspective is fed into the teacher network, and the audio of both the global and local perspectives are fed into the student network. This design encourages the student network to learn the correspondence between local features and global features. The two networks use the same structure and hyperparameters, but due to the different input features, the weights within the model are independent of each other. The two networks consist of a backbone network and a mapping layer. The backbone network is as described above, and the classification layer includes three fully connected layers and a normalization layer. Finally, the cross entropy loss function is used to optimize the network, and the formula is as follows:

[0074]

[0075] Where H(a|b) = -a*log(b) represents the cross entropy, X l ={x l1 ,x l2} represents two long segments, X s ={x s1 ,x s2 ,x s3 ,x s4} represents four short segments, P tea and P stu Represent the probability distribution of the teacher network and the student network output respectively. Specifically, the probability distribution of the teacher network output is calculated by the softmax function, and the sharpness of the output is controlled by the temperature parameter τt:

[0076]

[0077] Similarly, the student network also has its corresponding temperature parameter τ s To avoid the model collapsing into a simple solution (e.g. all embedding vectors converge to the same), the parameters of the teacher network are updated via the exponential moving average (EMA) of the student network parameters, and sharpening and centering are applied to the output of the teacher network.

[0078] To ensure the diversity of embeddings within a batch, this paper proposes a diversity regularization term. It forces the embeddings of different utterances to be distinguished from each other, preventing simple solutions. The regularization loss is calculated as follows:

[0079]

[0080] in, and Represent the embedding of the teacher and student networks on dimension j, Var(·) represents the variance, and ∈ is a small quantity close to 0. This regularization term forces the embedding standard deviation of each dimension in each batch to be close to 1, thereby ensuring the diversity of embedding.

[0081] The redundancy elimination regularization term aims to reduce the redundancy between each embedding variable while minimizing the redundancy. It makes the covariance of each dimension of all speaker embeddings on the batch dimension approach zero, preventing information collapse, that is, the state where variables are highly correlated. The redundancy elimination regularization loss is calculated as follows:

[0082]

[0083] Here, C is the cross-correlation matrix between the global outputs of the teacher network and the student network, calculated along the batch dimension, and Cij is defined as:

[0084]

[0085] Where b indexes the batch samples and i,j indexes the embedding dimensions. This regularization term decorrelates different vector components of the embedding by forcing the off-diagonal elements across the correlation matrix to be close to 0.

[0086] S4 aggregates and classifies the voiceprint feature information extracted by the ECAPA-TDNN voiceprint model and the speaker's voice micro-movement information obtained through accent data training to perform speaker recognition;

[0087] The present invention does not adopt a new speaker recognition model. The present invention integrates the voiceprint feature information extracted by the ECAPA-TDNN voiceprint model and the speaker's voice micro-movement information obtained by accent data training into a carefully designed feature aggregation module. A speaker verification system can be obtained through the feature aggregation module and the final classification module. The feature aggregation module can also be used as an embedding module for speaker features. Obviously, such a feature aggregation mode can help the system capture the speaker's voiceprint features at a relatively short time level, and can also capture the speaker's micro-movements that are difficult to change at a relatively long time level. This can enable subsequent models to have greater confidence in dealing with speaker verification needs of millions or even tens of millions of orders of magnitude.

[0088] The speaker recognition method based on the speaker's voice micro-movement of the present invention uses a learning strategy of contrastive learning, so that the model can use a larger data set while getting rid of the requirement of fine annotation, so that the difficulty of data collection is greatly reduced. The annotation of large and small pieces of specific dialects requires preparation by professionals. The change of dialect accents in geographical space is a continuous process, and the degree of accent expression makes it a relatively difficult task to label classification labels. Contrastive learning can explore the relationship between different accents, spontaneously gather similar accents, and keep dissimilar accents away from each other. This is a great help for extracting an embedding that can accurately reflect the relationship between different accents and the degree of accent. The accent model that does not use the ASR model embedding often fits the recording channel characteristics or fits the speaker's voiceprint characteristics. Because it is much easier for the model to fit low-level features such as voiceprint features than accent features. Humans are also more likely to distinguish the difference in timbre, and it takes a period of professional learning to distinguish different dialects. This is unavoidable when a text-related model is not used. And this fitting is often the key to the poor generalization performance of the model. The introduction of the ASR model makes it possible to compare the difference between its meaning content (ASR for common accents) and the real pronunciation content (standard Mandarin ASR) at the feature level, which is the basis for linguists to judge different accents. The speaker micro-motion system used in the present invention can extract various tiny movements of the speaker when pronouncing, which is exactly what the original speaker verification model cannot do. Among hundreds of thousands or even millions of speakers, it is easy to find that there are multiple pairs of speakers with similar voiceprint features. After the introduction of new features, it can provide another perspective for speaker verification models to perform speaker verification at the scale of millions or more speakers, which can help the model distinguish between speakers with similar voices. The present invention is accomplished with the aid of a framework of accent recognition and a framework of contrastive learning. With the aid of this framework, a speaker verification method that fuses speech micro-movements and voiceprint features is proposed. When data is widely collected, methods such as data enhancement are used to improve the generalization performance of the model, avoiding fitting in channel features, and no manual labeling is required. Through the design of the model and the selection of features, the model can focus on features at the phoneme level and above the phoneme level to the vocabulary level for a longer period of time. The model of the present invention can be ultimately used for speaker verification to help the model achieve better speaker recognition results by introducing new features. The method of the present invention can help the speaker verification model play a role in the population of millions to tens of millions.

[0089] What is disclosed above is only one or more preferred embodiments of the present application, and cannot be used to limit the scope of rights of the present application. Ordinary technicians in this field can understand that all or part of the processes of implementing the above embodiments and equivalent changes made according to the claims of the present application are still within the scope covered by the present application.

Claims

1. A speaker recognition method based on speaker voice micro-movements, characterized in that: include: After preprocessing, the speech stream extracts Fbank features and sends them to the teacher network and student network respectively to obtain their corresponding feature embeddings; The features obtained from the teacher network and the student network are embedded into the loss function and back-propagated; The student network is iterated normally, and the teacher network is iterated by the EMA sliding average method; The voiceprint feature information extracted by the ECAPA-TDNN voiceprint model is aggregated and classified with the speaker's voice micro-movement information obtained through accent data training to perform speaker recognition.

2. A speaker recognition method based on speaker voice micro-movements as claimed in claim 1, characterized in that: The teacher network and the student network have the same structure and hyperparameters, and the difference between the teacher network and the student network lies in the different parameters in the model.

3. A speaker recognition method based on speaker voice micro-movements as claimed in claim 2, characterized in that: The operation process of the teacher network and the student network includes: After segmentation and data enhancement, the speech stream is extracted and Fbank features are sent to the general spoken acoustic model and the standard Mandarin acoustic model; The sound features extracted from the universal accent acoustic model and the standard Mandarin acoustic model are combined; Perform feature contrast encoding on the fused sound features and map the features back to the original data space.

4. A speaker recognition method based on speaker voice micro-movements as claimed in claim 3, characterized in that: The universal accent acoustic model and the standard Mandarin acoustic model adopt the Uni-ASR model.

5. A speaker recognition method based on speaker voice micro-movements as claimed in claim 4, characterized in that: The data sets of the universal acoustic model for accents include an accented Mandarin data set, an AISHELL-1 data set, and an AISHELL-2 data set. The accented Mandarin data set is Mandarin with accents collected on the Internet through crawlers, and the video subtitles or text content it carries can be used as its speech transcription. The total length of the AISHELL-1 data set and the AISHELL-2 data set is greater than 1,100 hours.

6. A speaker recognition method based on speaker voice micro-movements as claimed in claim 5, characterized in that: The data set of the standard Mandarin acoustic model uses broadcast data collected online, which is annotated by ASR after background music separation and advertisement removal. A total of 2,100 hours of audio are collected, and more than 1,750 hours of valid data are obtained after data cleaning.

7. A speaker recognition method based on speaker voice micro-movements as claimed in claim 6, characterized in that: The specific steps of fusing the sound features extracted from the universal accent acoustic model and the standard Mandarin acoustic model include: The features output by the universal accent acoustic model and the standard Mandarin acoustic model are spliced ​​in the channel dimension, and the spliced ​​features are passed through a global maximum pooling and a global average pooling, and then passed through a bottleneck layer to compress its dimension and obtain the attention parameters, which are then convolved with the original spliced ​​features.

Citation Information

Patent Citations

  • End-to-end accent conversion method

    CN111462769A

  • Training device of voiceprint recognition model

    CN112259104A

  • Voiceprint recognition method and device, electronic equipment and storage medium

    CN114913859A

  • Speech recognition error correction method and system based on artificial intelligence

    CN117238278A

  • Self-adaptive speech recognition method and system

    CN117558278A