Speaker recognition method, model, computer device and storage medium

Abstract

The application relates to a speaker recognition method, a model, a computer device and a storage medium. The method comprises the following steps: obtaining target voice data; extracting frame feature vectors corresponding to each voice frame from the target voice data; calculating fusion weight coefficients corresponding to each frame feature vector, wherein the fusion weight coefficients are obtained by fusing weight coefficients calculated according to at least two different attention mechanisms; calculating an embedded feature vector of the target voice data according to each frame feature vector and the fusion weight coefficient corresponding to each frame feature vector; and determining a target speaker corresponding to the target voice data according to the embedded feature vector and a pre-trained voiceprint scoring unit. The method can improve the accuracy of speaker recognition.

Description

Technical Field

[0001] This application relates to the field of voiceprint recognition technology, and in particular to a speaker recognition method, model, computer device, and storage medium. Background Technology

[0002] With the development of voiceprint recognition technology, in order to make smart devices more secure during use, more and more smart devices, such as smart vehicles, are beginning to use voiceprint recognition technology to verify user identity and ensure that only specific users can use specific smart devices.

[0003] The vocal organs used by people when speaking (tongue, teeth, larynx, lungs, nasal cavity, etc.) vary greatly in size and shape, resulting in differences in the voiceprint profiles of any two people. Therefore, under normal circumstances, voiceprint recognition technology can distinguish between different individuals. Currently, mainstream voiceprint recognition technologies include methods for extracting ivector features and methods for extracting xvector features. Among them, the xvector feature extraction method is based on deep learning. Traditional xvector feature extraction methods aggregate speech characteristics based on average statistical pooling. This method ignores the differences in the importance of features in each frame, thus leading to low accuracy in speaker recognition based on this approach. Summary of the Invention

[0004] Therefore, it is necessary to provide a speaker recognition method, model, computer device, and storage medium that can improve the accuracy of speaker recognition in response to the above-mentioned technical problems.

[0005] A speaker recognition method, the method comprising:

[0006] Acquire target speech data;

[0007] Extract the frame feature vectors corresponding to each speech frame from the target speech data;

[0008] Calculate the fusion weight coefficients corresponding to the feature vectors of each frame. The fusion weight coefficients are obtained by fusing the weight coefficients calculated by at least two different attention mechanisms.

[0009] The embedded feature vector of the target speech data is calculated based on the feature vectors of each frame and their respective fusion weight coefficients.

[0010] The target speaker corresponding to the target speech data is determined based on the embedded feature vector and the pre-trained voiceprint scoring unit.

[0011] In one embodiment, the fusion weight coefficients corresponding to the feature vectors of each frame are calculated, including:

[0012] The first weight coefficient of the feature vector of each frame is calculated based on the multi-head attention mechanism;

[0013] The second weight coefficient of the feature vector of each frame is calculated based on the self-attention mechanism;

[0014] The fusion weight coefficients corresponding to the feature vectors of each frame are obtained by averaging the first weight coefficients and the second weight coefficients.

[0015] In one embodiment, the multi-head attention mechanism includes at least one first-head attention unit, which calculates first weight coefficients for the feature vectors of each frame according to the multi-head attention mechanism, including:

[0016] Obtain the resolution information corresponding to each first-head attention unit;

[0017] The first score of each frame feature vector is calculated based on the first score calculation function and the resolution information corresponding to each first head attention unit.

[0018] The first scores are normalized to obtain the first weight coefficients corresponding to the feature vectors of each frame.

[0019] In one embodiment, calculating the second weight coefficients of the feature vectors of each frame according to the self-attention mechanism includes:

[0020] The target Q vector corresponding to the feature vector of each frame is determined by the statistical pooling method. The Q vector is pre-constructed based on each sentence in the target speech data. Different sentences correspond to different Q vectors. A sentence includes at least one speech frame.

[0021] The target K vector corresponding to the feature vector of each frame is determined based on the linear mapping function;

[0022] Calculate the second score of the feature vector for each frame based on the target Q vector, the target K vector, and the second score calculation function;

[0023] The second scores are normalized to obtain the second weight coefficients corresponding to the feature vectors of each frame.

[0024] In one embodiment, the self-attention mechanism includes at least one second-head attention unit that calculates a second score for each frame's feature vector based on the target Q-vector, the target K-vector, and a second score calculation function, including:

[0025] Obtain the resolution information corresponding to each second-head attention unit;

[0026] The second score is calculated based on the target Q vector, the target K vector, the second score calculation function, and the resolution information corresponding to each second head attention unit.

[0027] In one embodiment, extracting frame feature vectors corresponding to each speech frame from the target speech data includes:

[0028] Preprocess the target speech data to obtain the target audio segment;

[0029] Extract the filter bank feature vectors of each speech frame from the target audio segment;

[0030] By using at least one residual block to perform neural network operations on the filter bank feature vectors of each speech frame, the frame feature vectors corresponding to each speech frame output by at least one residual block are obtained.

[0031] In one embodiment, the method further includes:

[0032] Obtain sample voice data and user tags for each of the multiple users;

[0033] Each attention mechanism is trained using multiple sample speech data to obtain the sample embedded feature vector corresponding to each sample speech data.

[0034] The embedded feature vectors of each sample are registered as the voiceprint features of the corresponding user in the voiceprint database.

[0035] The voiceprint scoring unit is trained based on the embedded feature vectors of each sample and user labels.

[0036] A speaker recognition model for speaker recognition, the model comprising:

[0037] The input layer is used to acquire the target speech data;

[0038] The residual network layer is used to extract the frame feature vectors corresponding to each speech frame from the target speech data.

[0039] The hybrid pooling layer is used to calculate the fusion weight coefficients corresponding to the feature vectors of each frame. The fusion weight coefficients are obtained by fusing the weight coefficients calculated by at least two different attention mechanisms.

[0040] The embedded extraction layer is used to calculate the embedded feature vector of the target speech data based on the feature vectors of each frame and their respective fusion weight coefficients.

[0041] The output layer is used to determine the target speaker corresponding to the target speech data based on the embedded feature vector and the pre-trained speaker scoring unit.

[0042] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the speaker recognition method described above.

[0043] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the above-described speaker recognition methods.

[0044] The aforementioned speaker recognition method, model, computer equipment, and storage medium extract frame feature vectors corresponding to each speech frame in the target speech data, using speech frames as units. They then calculate the fusion weight coefficients corresponding to each speech frame using at least two different attention mechanisms. By assigning fusion weights to the frame feature vectors through at least two different attention mechanisms, they obtain embedded feature vectors of the target speech data and perform speaker recognition based on these embedded feature vectors. This method introduces at least two different attention mechanisms. Through the complementarity between these mechanisms, it can better allocate weights, better capture the intrinsic correlations of features, and utilize weight allocation to highlight the contribution of more effective features, thereby improving the accuracy of speaker recognition. Attached Figure Description

[0045] Figure 1 This is a flowchart illustrating a speaker recognition method in one embodiment;

[0046] Figure 2 This is a flowchart illustrating the steps for calculating the fusion weight coefficients corresponding to the feature vectors of each frame in one embodiment.

[0047] Figure 3 This is a flowchart illustrating the steps of calculating the first weight coefficient of the feature vector of each frame according to a multi-head attention mechanism in one embodiment.

[0048] Figure 4 This is a flowchart illustrating the steps of calculating the second weight coefficients of the feature vectors of each frame according to a self-attention mechanism in one embodiment.

[0049] Figure 5 This is a block diagram of a speaker recognition model in one embodiment;

[0050] Figure 6 This is a flowchart illustrating the speaker recognition method in another embodiment;

[0051] Figure 7 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0053] The speaker recognition method provided in this application can be applied to terminal devices. These terminal devices can be, but are not limited to, various personal computers, laptops, smartphones, tablets, portable wearable devices, and in-vehicle terminal devices.

[0054] In one embodiment, such as Figure 1 As shown, a speaker recognition method is provided. Taking the application of this method to a terminal as an example, the method includes the following steps:

[0055] Step S102: Obtain target speech data.

[0056] The target speech data refers to the speech data of the speaker to be identified. The voiceprint information corresponding to the target speech data can be pre-registered in a voiceprint database.

[0057] Specifically, the terminal can receive the user's voice data through a voice acquisition device and, in response to the speaker recognition command, use the received voice data as the target voice data to be recognized.

[0058] Step S104: Extract the frame feature vectors corresponding to each speech frame from the target speech data.

[0059] The target speech data includes at least one speech frame. Specifically, the terminal can perform frame segmentation processing on the target speech data to obtain at least one speech frame, and extract the corresponding frame feature vectors from each of the segmented speech frames using a feature extraction method.

[0060] In one embodiment, before extracting the frame feature vectors corresponding to each speech frame from the target speech data, the method may further include: preprocessing the target speech data. The preprocessing may include at least one of the following: speech enhancement processing, VAD (Voice Activity Detection) processing, audio quality detection processing, and effective audio extraction processing. By preprocessing the target speech data, effective speech segments can be extracted, improving the accuracy of the speech data and thus further improving the accuracy of speaker recognition.

[0061] For example, a feature extraction algorithm based on FBank (Filter Bank) can be used to extract the filter bank feature vector corresponding to each speech frame from the target speech data, and then the frame feature vector of each speech frame can be determined according to the filter bank feature vector corresponding to each speech frame.

[0062] In one embodiment, extracting frame feature vectors corresponding to each speech frame from the target speech data includes: preprocessing the target speech data to obtain a target audio segment; extracting filter bank feature vectors of each speech frame from the target audio segment; and performing neural network operations on the filter bank feature vectors of each speech frame using at least one residual block to obtain frame feature vectors corresponding to each speech frame output by at least one residual block.

[0063] In this embodiment, the terminal can first preprocess the target speech data. The preprocessing can include at least one of speech enhancement processing, VAD processing, audio quality detection processing, and effective audio extraction processing. After preprocessing, an effective target audio segment is obtained. The FBank algorithm is used to extract the filter bank feature vector of each speech frame from the target audio segment. Furthermore, the extracted filter bank feature vector of each speech frame can be input into at least one residual block of the residual network (ResNet Block) for neural network operation to obtain the frame feature vector corresponding to each speech frame after neural network operation.

[0064] For example, it may include four residual blocks, which can be connected in series. These four residual blocks are used to perform convolution, fully connected operations, nonlinear ReLU (Rectified Linear Unit) function operations, and BN (Batch Normalization) operations, respectively. The filter bank feature vectors of each extracted speech frame are iteratively processed by these four residual blocks, and the frame feature vector h corresponding to each speech frame is output. t, , where t is the frame index of the speech frame.

[0065] Step S106: Calculate the fusion weight coefficients corresponding to the feature vectors of each frame. The fusion weight coefficients are obtained by fusing the weight coefficients calculated by at least two different attention mechanisms.

[0066] The fusion weight coefficients are obtained by fusion calculation based on the weight coefficients calculated from two different attention mechanisms. The trainable parameter terms in the relevant calculation functions corresponding to each attention mechanism can be pre-trained by deep learning.

[0067] Specifically, the terminal can assign corresponding weight coefficients to the feature vectors of each frame according to at least two pre-built different attention mechanisms, and perform fusion algorithms such as averaging or weighted calculation on the weight coefficients assigned by each attention mechanism to obtain the fusion weight coefficients assigned to the feature vectors of each frame based on at least two different attention mechanisms.

[0068] In one embodiment, reference Figure 2As shown, the steps for calculating the fusion weight coefficients corresponding to the feature vectors of each frame may include:

[0069] S202: Calculate the first weight coefficient of the feature vector of each frame according to the multi-head attention mechanism.

[0070] S204: Calculate the second weight coefficient of the feature vector of each frame according to the self-attention mechanism.

[0071] S206: Obtain the fusion weight coefficients corresponding to the feature vectors of each frame based on the average of each first weight coefficient and each second weight coefficient.

[0072] In this embodiment, the weight coefficients calculated based on the multi-head attention mechanism and the weight coefficients calculated based on the self-attention mechanism can be averaged to obtain the fusion weight coefficients of the feature vectors of each frame. These two different attention mechanisms can allocate different proportions of attention weights at different speech frame positions. Through the cooperation of these two different attention weight allocation mechanisms, they can compensate for each other's shortcomings and improve the accuracy of feature extraction.

[0073] In one embodiment, reference Figure 3 As shown, the multi-head attention mechanism includes multiple first-head attention units. The step of calculating the first weight coefficients of the feature vectors of each frame according to the multi-head attention mechanism may include:

[0074] S302: Obtain the resolution information corresponding to each first-head attention unit.

[0075] Here, the first head attention unit refers to the head attention unit that calculates the first score based on the multi-head attention mechanism. Different first head attention units can be pre-configured with different resolution information. By configuring different resolution information for different first head attention units, different resolution first head attention units can have different weight allocation strengths. Each first head attention unit can score the feature vector of each speech frame based on different weight allocation strengths, and output diverse scoring results. That is, different first head attention units can output different first scores, thereby further improving the diversity of feature extraction.

[0076] S304: Calculate the first score of the feature vector of each frame based on the first score calculation function and the resolution information corresponding to each first head attention unit.

[0077] For example, the first score calculation function can be constructed based on a non-linear activation function, resolution information, trainable parameter terms, and a preset bias term. The specific formula for the first score calculation function can be found below:

[0078]

[0079] Where f is a nonlinear activation function, W i For trainable parameter terms (trainable matrix), v i b represents the trainable parameter term (trainable vector). i k i N represents the preset bias term. i The resolution information is represented by T, which indicates that the transpose operation is performed, and i represents the head index of the first head attention unit.

[0080] S306: Normalize each first score to obtain the first weight coefficient corresponding to the feature vector of each frame.

[0081] For example, each first attention unit outputs its own first score for the feature vector of each frame. After that, you can Normalization is performed to obtain the first weight coefficients of each first-head attention unit for the feature vector of each frame. Among them, the first score Normalization is performed to obtain the first weight coefficient. The calculation formula can be found as follows:

[0082]

[0083] in, h represents the first fraction. t τ represents the frame feature vector of each speech frame, starting from 1 and taking positive integer values, T represents the total number of speech frames, and exp represents the exponential function.

[0084] In one embodiment, reference Figure 4 As shown, the steps for calculating the second weight coefficients of the feature vectors of each frame based on the self-attention mechanism may include:

[0085] Step S402: Determine the target Q vector corresponding to the feature vector of each frame according to the statistical pooling method. The Q vector is pre-constructed based on each sentence in the target speech data. Different sentences correspond to different Q vectors. A sentence includes at least one speech frame.

[0086] In this process, different Q-vectors can be pre-constructed for different statements. A statement can include at least one speech frame. During computation, speech frames in the same statement can be mapped to the same Q-vector using a statistical pooling method. Speech frames in different statements can be mapped to the Q-vectors constructed for their respective statements.

[0087] For example, considering that the mean and standard deviation can capture global information and the dynamic changes in speech, a statistical pooling function can be used to map the Q vector, as shown in the following formula:

[0088]

[0089] Where g represents the statistical pooling method. q represents the trainable parameter term (trainable matrix). i Let represent the target Q-vector, and ...

[0090] Step S404: Determine the target K vector corresponding to each speech frame based on the linear mapping function.

[0091] For example, the target K vector corresponding to each speech frame can be determined according to a linear mapping function. The specific formula is as follows:

[0092] k t =W k h t

[0093] Among them, W k This represents the trainable parameter term (trainable matrix). k t Let h represent the K-vector. t represents the frame feature vector, and t represents the frame index.

[0094] Step S406: Calculate the second score of the feature vector of each frame based on the target Q vector, the target K vector, and the second score calculation function.

[0095] Specifically, the self-attention mechanism employed outputs the second score by mapping the Q and K vectors to the second score calculation function.

[0096] The self-attention mechanism can include at least one second head attention unit, which refers to a head attention unit based on the second score calculation function of the self-attention mechanism. Different second head attention units can be pre-configured with different resolution information. By configuring different resolution information for different second head attention units, second head attention units with different resolutions can have different weight allocation strengths. Each second head attention unit can score the feature vector of each speech frame based on different weight allocation strengths, outputting diverse scoring results. That is, the second scores output by different second head attention units can be different, thereby further improving the diversity of feature extraction.

[0097] In one embodiment, the step of calculating the second score of each frame feature vector based on the target Q vector, the target K vector, and the second score calculation function may include: obtaining the resolution information corresponding to each second head attention unit; and calculating the second score corresponding to each frame feature vector based on the target Q vector, the target K vector, and the resolution information corresponding to each second head attention unit.

[0098] For example, a second score calculation function can be constructed based on the scaled dot product attention algorithm, and the second score can be calculated based on a multi-resolution self-attention mechanism. The specific calculation formula is as follows:

[0099]

[0100] in, N represents the second fraction. i Represents resolution information, d k The scaling factor (which can be pre-configured) is k, where q represents the target Q vector, and k is k. t Let h represent the target K vector. t Let t represent the frame feature vector, i represent the frame index, and i represent the header index.

[0101] Step S406: Normalize each second score to obtain the second weight coefficients corresponding to the feature vectors of each frame.

[0102] For example, all input speech frames can be aggregated to obtain the second score. After performing softmax normalization, the second weight coefficients for each speech frame are obtained. The specific calculation formula can be found below:

[0103]

[0104] in, This represents the second weighting coefficient. Let represent the second fraction, exp represent the exponential function, T represent the total number of speech frames, τ start from 1 and take positive integer values, i represent the head index, and t represent the frame index.

[0105] In one embodiment, the fusion weight coefficients corresponding to the feature vectors of each frame can be obtained based on the average of each first weight coefficient and each second weight coefficient.

[0106] For example, the following calculation formula can be used as a reference:

[0107]

[0108] in, Indicates the fusion weighting coefficient. This represents the first weighting coefficient. represents the second weighting coefficient, t represents the frame index, and i represents the header index.

[0109] Step S108: Extract the embedded feature vector of the target speech data based on the feature vector of each frame and its corresponding fusion weight coefficient.

[0110] Specifically, the terminal can calculate the weighted mean vector and the weighted standard deviation vector based on the feature vectors of each frame and the fusion weight coefficients corresponding to each frame feature vector. Furthermore, the weighted mean vector and the weighted standard deviation vector can be concatenated to obtain the embedded feature vector corresponding to the target speech data.

[0111] For example, the formula for calculating the weighted mean vector can be found as follows:

[0112]

[0113] Where, μ i Represents the weighted mean vector. h represents the fusion weight coefficient. t Let i represent the frame feature vector corresponding to each speech frame, i represent the head index, t represent the frame index, and T represent the total number of speech frames.

[0114] For example, the formula for calculating the weighted standard deviation vector can be found as follows:

[0115]

[0116] Where, σ i μ represents the weighted standard deviation vector. i Represents the weighted mean vector. h represents the fusion weight coefficient. t Let i represent the frame feature vector, i represent the header index, and t represent the frame index.

[0117] Step S110: Determine the target speaker corresponding to the target speech data based on the embedded feature vector and the pre-trained voiceprint scoring unit.

[0118] The target speaker refers to the speaker identified based on the target speech data.

[0119] Specifically, the terminal can input the embedded feature vector corresponding to the extracted target speech data into the voiceprint scoring unit for scoring based on the pre-trained voiceprint scoring unit, for example, using PLDA (Probabilistic Linear Discriminant Analysis, channel compensation algorithm), and determine the corresponding target speaker based on the scoring result.

[0120] For example, at least one speaker's voiceprint features can be pre-registered in a voiceprint database. Based on the voiceprint scoring unit, the probability scores of the target speech data falling into each registered voiceprint feature are calculated according to the embedded feature vector of the target speech data. Based on the calculation results and a preset threshold, when the probability score of the target speech data falling into a certain registered voiceprint feature calculated by the voiceprint scoring unit is greater than the preset threshold, it is determined that the target speech data belongs to the speaker corresponding to that registered voiceprint feature.

[0121] The aforementioned speaker recognition method extracts frame feature vectors corresponding to each speech frame in the target speech data, and calculates the fusion weight coefficients for each speech frame using at least two different attention mechanisms. By assigning fusion weights to the feature vectors of each frame through at least two different attention mechanisms, an embedded feature vector of the target speech data is obtained, and speaker recognition is performed based on this embedded feature vector. This method introduces at least two different attention mechanisms, and through the complementarity between these mechanisms, better weight allocation can be performed, better capturing the intrinsic correlation of features. Weight allocation is used to highlight the contribution of more effective features, thereby improving the accuracy of speaker recognition.

[0122] In one embodiment, the method further includes: acquiring sample speech data and user tags corresponding to multiple users; training each attention mechanism using the multiple sample speech data to obtain sample embedded feature vectors corresponding to each sample speech data; registering each sample embedded feature vector as the voiceprint feature of the corresponding user to the voiceprint database; and training the voiceprint scoring unit based on each sample embedded feature vector and user tags.

[0123] In this embodiment, the voiceprint scoring unit can be pre-trained based on the embedded feature vectors of sample speech data corresponding to multiple users. The method for obtaining the embedded feature vectors of the sample speech data is similar to the method for obtaining the embedded feature vectors of the target speech data, and specifically includes the following steps: extracting frame feature vectors corresponding to each speech frame from the sample speech data; calculating the fusion weight coefficients corresponding to each frame feature vector, wherein the fusion weight coefficients are obtained by fusing weight coefficients calculated based on at least two different attention mechanisms; and calculating the embedded feature vectors of the sample speech data based on each frame feature vector and its corresponding fusion weight coefficient.

[0124] More specifically, the training method for the voiceprint scoring unit may include: using the sample embedded feature vector of the sample speech data and the user label (speaker label) as input, and using negative cross-entropy as the objective function to train the voiceprint scoring unit.

[0125] For example, the formula for the negative cross-entropy mentioned above can be referenced as follows:

[0126]

[0127] Where N is the number of samples in the training set, S is the number of outputs of the neural network, and p(y) i |x t ) is the posterior probability. This refers to the alignment target probability. When the output layer of the neural network is softmax, the model can be the output 'a' of a fully connected network, with S outputs, i.e., outputs a1, a2, ..., a2. S Therefore, for each sample of speech data, the probability that it belongs to category i is:

[0128]

[0129] The above formula can guarantee that That is, the sum of the probabilities of belonging to each category is 1.

[0130] In one embodiment, such as Figure 5 As shown, a speaker recognition model is provided for speaker recognition. The model includes: an input layer 502, a residual network layer 504, a hybrid pooling layer 506, an embedded extraction layer 508, and an output layer 510, wherein:

[0131] Input layer 502 is used to acquire target speech data;

[0132] Residual network layer 504 is used to extract the frame feature vectors corresponding to each speech frame from the target speech data:

[0133] Hybrid pooling layer 506 is used to calculate the fusion weight coefficients corresponding to the feature vectors of each frame. The fusion weight coefficients are obtained by fusing the weight coefficients calculated by at least two different attention mechanisms.

[0134] The embedded extraction layer 508 is used to calculate the embedded feature vector of the target speech data based on the feature vectors of each frame and their respective fusion weight coefficients.

[0135] Output layer 510 is used to determine the target speaker corresponding to the target speech data based on the embedded feature vector and the pre-trained speaker scoring unit.

[0136] In one embodiment, the hybrid pooling layer 506 may include a multi-head attention unit 5062, a self-attention unit 5064, and a fusion unit 5066. The multi-head attention unit 5062 is used to calculate a first weight coefficient of each frame feature vector according to the multi-head attention mechanism; the attention unit 5064 is used to calculate a second weight coefficient of each frame feature vector according to the self-attention mechanism; and the fusion unit 5066 obtains the fusion weight coefficient corresponding to each frame feature vector according to the average of each first weight coefficient and each second weight coefficient.

[0137] In one embodiment, the multi-head attention unit 5062 is used to obtain the resolution information corresponding to each first head attention unit; calculate the first score of each frame feature vector according to the first score calculation function and the resolution information corresponding to each first head attention unit; and normalize each first score to obtain the first weight coefficient corresponding to each frame feature vector.

[0138] In one embodiment, the self-attention unit 5064 determines the target Q vector corresponding to each frame feature vector according to the statistical pooling method. The Q vector is pre-constructed based on each sentence in the target speech data. Different sentences correspond to different Q vectors. A sentence includes at least one speech frame. The target K vector corresponding to each frame feature vector is determined according to the linear mapping function. The second score of each frame feature vector is calculated according to the target Q vector, the target K vector, and the second score calculation function. The second scores are normalized to obtain the second weight coefficient corresponding to each frame feature vector.

[0139] In one embodiment, the self-attention unit 5064 obtains the resolution information corresponding to each of the second head attention units; and calculates the second score corresponding to the feature vector of each frame based on the target Q vector, the target K vector, the second score calculation function, and the resolution information corresponding to each of the second head attention units.

[0140] In one embodiment, the residual network layer 504 may include at least one residual block 5042, which performs neural network operations on the filter bank feature vectors of each speech frame to obtain the frame feature vectors corresponding to each speech frame output by at least one residual block 5042.

[0141] In one embodiment, the input layer 502 is further used to obtain sample speech data and user tags corresponding to multiple users, the hybrid pooling layer 506 is further used to train each attention mechanism using multiple sample speech data, the embedded extraction layer 508 is further used to obtain the sample embedded feature vector corresponding to each sample speech data, the output layer 510 is further used to register each sample embedded feature vector as the voiceprint feature of the corresponding user to the voiceprint database, and the output layer 510 is further used to train the voiceprint scoring unit 5102 according to each sample embedded feature vector and user tags.

[0142] For specific limitations regarding the speaker recognition model, please refer to the limitations of the speaker recognition method above, which will not be repeated here. Each module in the aforementioned speaker recognition model can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0143] The speaker recognition method involved in this application will be further explained below with an application example. (Refer to...) Figure 6 As shown, Figure 6 The diagram illustrates a speaker recognition method in an application example. Specifically, it can include two stages: a voiceprint registration stage and a voiceprint recognition stage. More specifically, it can include the following steps:

[0144] Step S1: Preprocess the sample speech data and perform speech enhancement processing.

[0145] Step S2: Perform VAD silence detection processing on the audio signal after speech enhancement, and retain the non-silent areas.

[0146] Step S3: Perform audio quality detection processing on the retained non-silent areas, and extract valid audio segments based on the signal-to-noise ratio, clipping size, and volume.

[0147] Step S4: Extract FBank features from valid audio segments and input them into the speaker recognition model.

[0148] Step S5: Train the speaker recognition model with a sample batch size of 64 and an initial learning rate of 0.01.

[0149] Step S6: Based on the trained speaker recognition model, perform embedded feature vector extraction. During the voiceprint registration stage, the extracted embedded feature vector is saved as the corresponding user's voiceprint feature in the voiceprint database, which can be used directly during the voiceprint recognition stage.

[0150] Step S7: Train the voiceprint scoring unit using the embedded feature vectors of the extracted sample speech data (e.g., a PLDA scoring model can be used to construct it);

[0151] Step S8: Process the target speech data of the target speaker, for example, by performing LDA transformation, mean removal and whitening.

[0152] Step S9: Input the embedded feature vector of the processed target speech data into the voiceprint scoring unit, and then obtain the recognition result through threshold discrimination.

[0153] It should be understood that, although Figures 1 to 4 as well as Figure 6 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figures 1 to 4 as well as Figure 6 At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0154] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 7 As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements a speaker recognition method. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.

[0155] Those skilled in the art will understand that Figure 7 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0156] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it performs the following steps: acquiring target speech data; extracting frame feature vectors corresponding to each speech frame from the target speech data; calculating fusion weight coefficients corresponding to each frame feature vector, wherein the fusion weight coefficients are obtained by fusing weight coefficients calculated based on at least two different attention mechanisms; calculating embedded feature vectors of the target speech data based on each frame feature vector and its corresponding fusion weight coefficients; and determining the target speaker corresponding to the target speech data based on the embedded feature vectors and a pre-trained speaker scoring unit.

[0157] In one embodiment, when the processor executes a computer program to calculate the fusion weight coefficients corresponding to the feature vectors of each frame, it specifically implements the following steps: calculating the first weight coefficients of the feature vectors of each frame according to the multi-head attention mechanism; calculating the second weight coefficients of the feature vectors of each frame according to the self-attention mechanism; and obtaining the fusion weight coefficients corresponding to the feature vectors of each frame according to the average of the first weight coefficients and the second weight coefficients.

[0158] In one embodiment, when the processor executes a computer program to calculate the first weight coefficient of each frame feature vector according to the multi-head attention mechanism, it specifically implements the following steps: obtaining the resolution information corresponding to each first head attention unit; calculating the first score of each frame feature vector according to the first score calculation function and the resolution information corresponding to each first head attention unit; and normalizing each first score to obtain the first weight coefficient corresponding to each frame feature vector.

[0159] In one embodiment, when the processor executes a computer program to calculate the second weight coefficients of each frame feature vector according to the self-attention mechanism, it specifically implements the following steps: determining the target Q vector corresponding to each frame feature vector according to the statistical pooling method, wherein the Q vector is pre-constructed based on each sentence in the target speech data, and different sentences correspond to different Q vectors, and a sentence includes at least one speech frame; determining the target K vector corresponding to each frame feature vector according to the linear mapping function; calculating the second score of each frame feature vector according to the target Q vector, the target K vector, and the second score calculation function; and normalizing each second score to obtain the second weight coefficients corresponding to each frame feature vector.

[0160] In one embodiment, when the processor executes a computer program to calculate the second score of each frame feature vector based on the target Q vector, the target K vector, and the second score calculation function, it specifically implements the following steps: obtaining the resolution information corresponding to each second head attention unit; calculating the second score corresponding to each frame feature vector based on the target Q vector, the target K vector, the second score calculation function, and the resolution information corresponding to each second head attention unit.

[0161] In one embodiment, when the processor executes a computer program to extract the frame feature vectors corresponding to each speech frame from the target speech data, it specifically implements the following steps: preprocessing the target speech data to obtain a target audio segment; extracting the filter bank feature vectors of each speech frame from the target audio segment; and performing neural network operations on the filter bank feature vectors of each speech frame using at least one residual block to obtain the frame feature vectors corresponding to each speech frame output by at least one residual block.

[0162] In one embodiment, when the processor executes the computer program, it further performs the following steps: acquiring sample speech data and user tags corresponding to multiple users; training each attention mechanism using the multiple sample speech data to obtain sample embedded feature vectors corresponding to each sample speech data; registering each sample embedded feature vector as the voiceprint feature of the corresponding user to the voiceprint database; and training the voiceprint scoring unit based on each sample embedded feature vector and user tags.

[0163] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, performs the following steps: acquiring target speech data; extracting frame feature vectors corresponding to each speech frame from the target speech data; calculating fusion weight coefficients corresponding to each frame feature vector, the fusion weight coefficients being obtained by fusing weight coefficients calculated based on at least two different attention mechanisms; calculating embedded feature vectors of the target speech data based on each frame feature vector and its corresponding fusion weight coefficients; and determining the target speaker corresponding to the target speech data based on the embedded feature vectors and a pre-trained speaker scoring unit.

[0164] In one embodiment, when the computer program is executed by the processor to calculate the fusion weight coefficients corresponding to the feature vectors of each frame, the following steps are specifically implemented: calculating the first weight coefficients of the feature vectors of each frame according to the multi-head attention mechanism; calculating the second weight coefficients of the feature vectors of each frame according to the self-attention mechanism; and obtaining the fusion weight coefficients corresponding to the feature vectors of each frame according to the average of the first weight coefficients and the second weight coefficients.

[0165] In one embodiment, when the computer program is executed by the processor to calculate the first weight coefficient of each frame feature vector according to the multi-head attention mechanism, the specific steps are as follows: obtaining the resolution information corresponding to each first head attention unit; calculating the first score of each frame feature vector according to the first score calculation function and the resolution information corresponding to each first head attention unit; and normalizing each first score to obtain the first weight coefficient corresponding to each frame feature vector.

[0166] In one embodiment, when the computer program is executed by the processor to calculate the second weight coefficients of the feature vectors of each frame according to the self-attention mechanism, the specific steps are as follows: the target Q vector corresponding to the feature vector of each frame is determined according to the statistical pooling method. The Q vector is pre-constructed according to each sentence in the target speech data. Different sentences correspond to different Q vectors. Each sentence includes at least one speech frame. The target K vector corresponding to the feature vector of each frame is determined according to the linear mapping function. The second score of the feature vector of each frame is calculated according to the target Q vector, the target K vector, and the second score calculation function. The second scores are normalized to obtain the second weight coefficients corresponding to the feature vector of each frame.

[0167] In one embodiment, when the computer program is executed by the processor to calculate the second score of each frame feature vector based on the target Q vector, the target K vector, and the second score calculation function, the specific steps are as follows: obtaining the resolution information corresponding to each second head attention unit respectively; calculating the second score corresponding to each frame feature vector based on the target Q vector, the target K vector, the second score calculation function, and the resolution information corresponding to each second head attention unit.

[0168] In one embodiment, when a computer program is executed by a processor to extract frame feature vectors corresponding to each speech frame from target speech data, the specific steps are as follows: preprocessing the target speech data to obtain a target audio segment; extracting filter bank feature vectors of each speech frame from the target audio segment; and performing neural network operations on the filter bank feature vectors of each speech frame using at least one residual block to obtain frame feature vectors corresponding to each speech frame output by at least one residual block.

[0169] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: acquiring sample speech data and user tags corresponding to multiple users; training each attention mechanism using the multiple sample speech data to obtain sample embedded feature vectors corresponding to each sample speech data; registering each sample embedded feature vector as the voiceprint feature of the corresponding user to the voiceprint database; and training the voiceprint scoring unit based on each sample embedded feature vector and user tags.

[0170] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0171] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0172] Furthermore, the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the characters in this article generally indicate that the preceding and following related objects have an "or" relationship.

[0173] It should be noted that in this invention, terms such as "first" and "second" are used only for descriptive purposes and should not be construed as indicating or implying relative importance or order; terms such as "S102" and "S104" are used to distinguish steps and should not be construed as performing method steps in a specific order or sequence; when the following description relates to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The term "multiple" in this invention includes two or more, unless otherwise stated.

[0174] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A speaker recognition method, the method comprising: Acquire target speech data; Extract the frame feature vectors corresponding to each speech frame from the target speech data; Calculate the fusion weight coefficient corresponding to each frame feature vector. The fusion weight coefficient is obtained by fusing at least the first weight coefficient of each frame feature vector calculated based on the multi-head attention mechanism and the second weight coefficient of each frame feature vector calculated based on the self-attention mechanism. The Q vector in the self-attention mechanism is constructed according to the sentence to which each speech frame belongs. The embedded feature vector of the target speech data is calculated based on the feature vectors of each frame and their respective fusion weight coefficients. The target speaker corresponding to the target speech data is determined based on the embedded feature vector and the pre-trained voiceprint scoring unit.

2. The method according to claim 1, characterized in that, The calculation of the fusion weight coefficients corresponding to the feature vectors of each frame includes: The first weight coefficient of each frame feature vector is calculated based on the multi-head attention mechanism; The second weight coefficient of each frame feature vector is calculated based on the self-attention mechanism; The fusion weight coefficient corresponding to each frame feature vector is obtained by averaging the first weight coefficient and the second weight coefficient.

3. The method according to claim 2, characterized in that, The multi-head attention mechanism includes at least one first-head attention unit, and the calculation of the first weight coefficients of each frame feature vector according to the multi-head attention mechanism includes: Obtain the resolution information corresponding to each of the first head attention units; The first score of each frame feature vector is calculated based on the first score calculation function and the resolution information corresponding to each of the first head attention units. The first scores are normalized to obtain the first weight coefficients corresponding to the feature vectors of each frame.

4. The method according to claim 2, characterized in that, The calculation of the second weight coefficient of each frame feature vector based on the self-attention mechanism includes: The target Q vector corresponding to each frame feature vector is determined according to the statistical pooling method. The Q vector is pre-constructed based on each sentence in the target speech data. Different sentences correspond to different Q vectors. Each sentence includes at least one speech frame. The target K vector corresponding to each frame feature vector is determined based on the linear mapping function; Calculate the second score of each frame feature vector based on the target Q vector, the target K vector, and the second score calculation function; The second scores are normalized to obtain the second weight coefficients corresponding to the feature vectors of each frame.

5. The method according to claim 4, characterized in that, The self-attention mechanism includes at least one second-head attention unit, and the calculation of the second score of each frame feature vector based on the target Q vector, the target K vector, and the second score calculation function includes: Each of the second head attention units acquires its corresponding resolution information. The second score corresponding to each frame feature vector is calculated based on the target Q vector, the target K vector, the second score calculation function, and the resolution information corresponding to each second head attention unit.

6. The method according to claim 1, characterized in that, The step of extracting the frame feature vector corresponding to each speech frame from the target speech data includes: The target speech data is preprocessed to obtain the target audio segment; Extract the filter bank feature vector of each speech frame from the target audio segment; By using at least one residual block to perform neural network operations on the filter bank feature vectors of each of the speech frames, the frame feature vectors of each of the speech frames output by at least one of the residual blocks are obtained.

7. The method according to any one of claims 1 to 6, characterized in that, The method further includes: Obtain sample voice data and user tags for each of the multiple users; The attention mechanism is trained using multiple sample speech data to obtain the sample embedded feature vector corresponding to each sample speech data. The embedded feature vectors of each sample are registered as the voiceprint features of the corresponding user in the voiceprint database; The voiceprint scoring unit is trained based on the embedded feature vectors of each sample and the user tags.

8. A speaker recognition device for performing speaker recognition, comprising: The input module is used to acquire the target speech data; The residual network module is used to extract the frame feature vectors corresponding to each speech frame from the target speech data; A hybrid pooling module is used to calculate the fusion weight coefficients corresponding to each frame feature vector. The fusion weight coefficients are obtained by fusing at least the first weight coefficients of each frame feature vector calculated based on the multi-head attention mechanism and the second weight coefficients of each frame feature vector calculated based on the self-attention mechanism. The Q vector in the self-attention mechanism is constructed according to the sentence to which each speech frame belongs. An embedded extraction module is used to calculate the embedded feature vector of the target speech data based on the feature vectors of each frame and their respective fusion weight coefficients. The output module is used to determine the target speaker corresponding to the target speech data based on the embedded feature vector and the pre-trained voiceprint scoring unit.

9. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images