A method and system for speaker recognition based on time-frequency domain dynamic feature matrix

Abstract

The application provides a speaker recognition method and system based on a time-frequency domain dynamic feature matrix, which maps a time dynamic feature sequence of original speech into a two-dimensional image, calculates a similarity through a similarity matrix, and enhances time domain dynamic features in the time dynamic feature sequence through an adaptive weighting method; performs short-time Fourier transform on the original speech to obtain frequency spectrum values and calculate frequency domain dynamic features, and dynamically adjusts a similarity threshold value; trains in a convolutional neural network model (CNN), extracts acoustic features through a traditional method to obtain initial speaker features of the original speech; and a feature fusion classifier calculates a class probability distribution of the speaker according to a weighted and adaptively fused feature vector of a full connection layer, and takes a class with the maximum probability as a final result. The application improves the accuracy and robustness of speaker recognition in a complex scene, enhances the adaptability to different speech scenes, and better retains information for identifying the identity of a speaker in a speech signal.

Description

Technical Field

[0001] This invention relates to the field of speaker recognition technology, and specifically to a speaker recognition method and system based on a time-frequency domain dynamic feature matrix. Background Technology

[0002] Currently, speaker recognition is a technology that identifies or authenticates an individual by analyzing their voice characteristics. Compared with other biometric identification technologies such as fingerprints and irises, voiceprints have advantages such as uniqueness, ease of acquisition, and non-invasiveness.

[0003] Commonly used speaker recognition systems consist of three parts: acoustic feature extraction, model training, and backend speaker discrimination. Typically, speaker features are represented using spectral space vectors, such as spectrograms, FBank, and MFCC. The original speech file is input, and preprocessing operations such as pre-emphasis and windowing are performed on the speech signal. Based on human vocalization mechanisms or human ear perception mechanisms, different algorithms are used to extract features from the signal spectrum, ultimately obtaining vector-form acoustic features. To enhance the model's discriminative ability, the speaker features are processed in the backend using LDA, mean removal, length normalization, and score normalization. Finally, similarity discrimination is performed using discriminative models such as Cosine scoring and PLDA.

[0004] Meanwhile, with the development of deep learning, many works use raw waveforms as input to train speaker recognition models and have achieved performance comparable to feature extraction methods such as FBank. Some works use pre-trained models as front-end feature extraction modules, replacing traditional acoustic features. However, simply replacing traditional feature extraction methods with the general representation of pre-trained models may result in the underutilization of speaker-specific information contained in the raw speech.

[0005] Commonly used speech feature extraction methods focus on extracting features related to the spectral envelope of the speech signal, essentially ignoring the phase relationship between frequency components. Although Khadar Nawas et al. used a recurrence plot-based method to extract features and extracted the cyclic patterns of the vocal cord vibration system through phase space reconstruction, this approach only processes speech in the time domain and ignores the key spectral information that distinguishes speaker characteristics. Therefore, finding a feature extraction scheme that can comprehensively model the dynamic features of speech in both the time and frequency domains requires further research.

[0006] To address the aforementioned issues, this application proposes a speaker recognition method and system based on a time-frequency domain dynamic feature matrix. Summary of the Invention

[0007] This application addresses one or more technical deficiencies in the prior art by proposing the following technical solution.

[0008] Based on the first aspect of this application, a speaker recognition method based on a time-frequency domain dynamic feature matrix is ​​proposed, comprising:

[0009] S1: Map the temporal dynamic feature sequence of the original speech to a two-dimensional image, and calculate the similarity of each frame of the original speech signal through the similarity matrix. Use an adaptive weighting method to enhance the temporal dynamic features in the temporal dynamic feature sequence to obtain the temporal dynamic feature matrix.

[0010] The time-domain dynamic feature matrix is ​​represented as follows:

[0011] R ij = w(i, j)·θ(∈-||X||);

[0012]

[0013] Among them, R ij denoted by , where ∈ represents the value of the time-domain dynamic feature matrix, ∈ represents the preset time-domain dynamic feature threshold, ||X|| represents the distance between time points i and j, θ represents the step function, w(i,j) is the Gaussian weight of the time position, and σ represents the standard deviation used to adjust the weight decay rate;

[0014] S2: Perform a short-time Fourier transform on the original speech to obtain the spectral value of each frame of speech signal, calculate the frequency domain dynamic features of each frame of speech signal of the original speech, and dynamically adjust the similarity threshold to obtain the frequency domain dynamic feature matrix.

[0015] The frequency domain dynamic feature matrix is ​​represented as follows:

[0016] R f (i, j) = θ(ε) f -‖‖S(i)-S(j)||);

[0017] Among them, R f (i, j) represents the frequency domain dynamic characteristic matrix, ε f Let S(i) represent the similarity threshold of the frequency domain dynamic feature matrix, ||S(i)-S(j)|| represent the distance between the spectral energies of frame i and frame j, and θ represent the step function.

[0018] S3: Input the frequency domain dynamic features and the time domain dynamic features into the convolutional neural network (CNN) for training, extract the acoustic features of the original speech signal using traditional feature extraction methods, and input them into the Conformer model for processing to obtain the initial speaker features of the original speech.

[0019] S4: The trained time-domain dynamic features, frequency-domain dynamic features and speaker initial features are weighted and adaptively fused, and the fused feature vector is fed into the fully connected layer and mapped to a low-dimensional space;

[0020] S5: The feature fusion classifier calculates the speaker's category probability distribution based on the feature vector output by the fully connected layer, and takes the category with the highest probability as the final speaker recognition result.

[0021] By analyzing the hidden patterns, periodicity, and nonlinear dynamics of time series through time dynamic feature sequences, the time-domain dynamic feature matrix can extract the nonlinear dynamic changes of speech signals over time, identify diagonal lines and block structures in images, and better extract speech signal similarity and speaker features.

[0022] Furthermore, before S1, the process includes extracting a time-dynamic feature sequence from the original speech and dividing the original speech into 25ms frames.

[0023] Furthermore, the formula for calculating the similarity threshold of the frequency domain dynamic feature matrix is ​​as follows:

[0024] εf=α·mean(||S(i)-S(j)||);

[0025] Where α represents the adjustment factor, S(i) and S(j) represent the spectral energy, and mean represents the average value.

[0026] The frequency domain dynamic feature matrix can reflect the variation pattern of speech signals in the spectrum.

[0027] Furthermore, the formula for calculating the spectral energy is as follows:

[0028] S(t)=[|X(t, f1)|, |X(t, f2)|, ..., |X(t, f F )|];

[0029]

[0030] Where X(t, f) F ) represents the spectral values ​​of time t and frequency f, ω[nt] represents the short time window function, and N represents the number of points of the Fourier transform.

[0031] Furthermore, the calculation formula for the weighted adaptive fusion of the time-domain dynamic features, the frequency-domain dynamic features, and the speaker's initial features is as follows:

[0032] e fusion =β1·e time +β2·e freq +β3·e feature ;

[0033] Among them, e fusion e represents the fused feature vector. time e represents the time-domain dynamic characteristics. freq Represents the dynamic characteristics in the frequency domain, e feature β1, β2, and β3 represent the speaker's initial features, while β1, β2, and β3 represent the feature fusion weight parameters.

[0034] By learning the weight ratios of three sets of features and assigning different weight ratios, the contribution of different features is dynamically adjusted, enabling the model to fully extract the dynamic and static features in the speaker's speech. This integrates multi-dimensional information in the time and frequency domains while enhancing the expressive and discriminative capabilities of the features.

[0035] Based on a second aspect of this application, a speaker recognition system based on a time-frequency domain dynamic feature matrix is ​​also proposed, comprising:

[0036] Temporal module: The temporal dynamic feature sequence of the original speech is mapped to a two-dimensional image, and the similarity of each frame of the original speech signal is calculated through a similarity matrix. An adaptive weighting method is used to enhance the temporal dynamic features in the temporal dynamic feature sequence to obtain the temporal dynamic feature matrix.

[0037] The time-domain dynamic feature matrix is ​​represented as follows:

[0038] R ij = w(i, j)·θ(∈-||X||);

[0039]

[0040] Among them, R ij denoted by , where ∈ represents the value of the time-domain dynamic feature matrix, ∈ represents the preset time-domain dynamic feature threshold, ||X|| represents the distance between time points i and j, θ represents the step function, w(i,j) is the Gaussian weight of the time position, and σ represents the standard deviation used to adjust the weight decay rate;

[0041] Frequency domain module: Performs short-time Fourier transform on the original speech to obtain the spectrum value of each frame of speech signal, calculates the frequency domain dynamic features of each frame of speech signal of the original speech, and dynamically adjusts the similarity threshold to obtain the frequency domain dynamic feature matrix.

[0042] The frequency domain dynamic feature matrix is ​​represented as follows:

[0043] R f (i, j) = θ(ε) f -||S(i)-S(j)||);

[0044] Among them, R f(i, j) represents the frequency domain dynamic characteristic matrix, ε f Let S(i) represent the similarity threshold of the frequency domain dynamic feature matrix, ||S(i)-S(j)|| represent the distance between the spectral energies of frame i and frame j, and θ represent the step function.

[0045] Feature module: The frequency domain dynamic features and the time domain dynamic features are input into the convolutional neural network (CNN) for training. The acoustic features of the original speech signal are extracted by traditional feature extraction methods and input into the Conformer model for processing to obtain the initial speaker features of the original speech.

[0046] Adaptive fusion module: performs weighted adaptive fusion of the trained time-domain dynamic features, frequency-domain dynamic features and speaker initial features, and feeds the fused feature vector into a fully connected layer and maps it to a low-dimensional space;

[0047] Recognition Module: The feature fusion classifier calculates the speaker's category probability distribution based on the feature vector output by the fully connected layer, and takes the category with the highest probability as the final speaker recognition result.

[0048] Based on a third aspect of this application, a computer program product is also proposed, having one or more computer programs thereon that, when executed by a computer processor, perform the method as described in any of the above.

[0049] The technical advantages of this application are as follows: By adding and replacing the feature extraction methods and models, and combining the dynamic information and time-frequency domain features of the speech signal, this application solves the problem of the current single feature extraction method, more fully preserves the information in the speech signal that distinguishes the speaker's identity, and uses different models to model the extracted features, extracting the global and local information of the speaker more comprehensively. It can adapt to a variety of feature extraction schemes and enhance the accuracy and robustness of speaker recognition in complex scenarios. Attached Figure Description

[0050] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings.

[0051] Figure 1 This is a general flowchart of a speaker recognition method based on a time-frequency domain dynamic feature matrix provided according to an embodiment of this application.

[0052] Figure 2 This is a framework diagram of a speaker recognition system based on a time-frequency domain dynamic feature matrix, provided according to an embodiment of this application.

[0053] Figure 3This is a schematic diagram of the structure of a computer system for an electronic device applicable to embodiments of this application. Detailed Implementation

[0054] The present application will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings.

[0055] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0056] Figure 1 This application illustrates a speaker recognition method based on a time-frequency domain dynamic feature matrix, comprising:

[0057] S1: Map the temporal dynamic feature sequence of the original speech to a two-dimensional image, and calculate the similarity of each frame of the original speech signal through the similarity matrix. Use an adaptive weighting method to enhance the temporal dynamic features in the temporal dynamic feature sequence to obtain the temporal dynamic feature matrix.

[0058] The time-domain dynamic feature matrix is ​​represented as follows:

[0059] R ij = w(i, j)·θ(∈-||X||);

[0060]

[0061] Among them, R ij denoted by , where ∈ represents the value of the time-domain dynamic feature matrix, ∈ represents the preset time-domain dynamic feature threshold, ||X|| represents the distance between time points i and j, θ represents the step function, w(i,j) is the Gaussian weight of the time position, and σ represents the standard deviation used to adjust the weight decay rate;

[0062] S2: Perform a short-time Fourier transform on the original speech to obtain the spectral value of each frame of speech signal, calculate the frequency domain dynamic features of each frame of speech signal of the original speech, and dynamically adjust the similarity threshold to obtain the frequency domain dynamic feature matrix.

[0063] The frequency domain dynamic feature matrix is ​​represented as follows:

[0064] R f (i, j) = θ(ε) f -||S(i)-S(j)||);

[0065] Among them, R f (i, j) represents the frequency domain dynamic characteristic matrix, εf Let S(i) represent the similarity threshold of the frequency domain dynamic feature matrix, ||S(i)-S(j)|| represent the distance between the spectral energies of frame i and frame j, and θ represent the step function.

[0066] S3: Input the frequency domain dynamic features and the time domain dynamic features into the convolutional neural network (CNN) for training, extract the acoustic features of the original speech signal using traditional feature extraction methods, and input them into the Conformer model for processing to obtain the initial speaker features of the original speech.

[0067] S4: The trained time-domain dynamic features, frequency-domain dynamic features and speaker initial features are weighted and adaptively fused, and the fused feature vector is fed into the fully connected layer and mapped to a low-dimensional space;

[0068] S5: The feature fusion classifier calculates the speaker's category probability distribution based on the feature vector output by the fully connected layer, and takes the category with the highest probability as the final speaker recognition result.

[0069] It should be noted that the formula for calculating the similarity threshold of the frequency domain dynamic feature matrix is ​​as follows:

[0070] ε f =α·mean(||S(i)-S(j)||);

[0071] Where α represents the adjustment factor, S(i) and S(j) represent the spectral energy, and mean represents the average value.

[0072] It should be noted that the formula for calculating the spectral energy is as follows:

[0073] S(t)=[|X(t, f1)|, |X(t, f2)|, ..., |X(t, f F )|];

[0074]

[0075] Where X(t, f) F ) represents the spectral values ​​of time t and frequency f, ω[nt] represents the short time window function, and N represents the number of points of the Fourier transform.

[0076] It should be noted that the calculation formula for the weighted adaptive fusion of the time-domain dynamic features, the frequency-domain dynamic features, and the speaker's initial features is as follows:

[0077] e fusion =β1·e time +β2·e freq +β3·e feature ;

[0078] Among them, e fusion e represents the fused feature vector. time e represents the time-domain dynamic characteristics. freq Represents the dynamic characteristics in the frequency domain, e feature β1, β2, and β3 represent the speaker's initial features, while β1, β2, and β3 represent the feature fusion weight parameters.

[0079] It should be noted that the CNN model was trained using the PyTorch Lightning training framework for 50 iterations and optimized using the Adam W optimizer.

[0080] It should be noted that optimizing the convolutional neural network model (CNN) using the optimizer adam W is not only simple to implement but also computationally efficient. It can adaptively adjust the learning rate, ensuring a stable training process and enabling the model to converge better.

[0081] It should be noted that time-domain dynamic features and frequency-domain features together provide a comprehensive description of the original speech signal. Time-domain dynamic features describe the waveform of the speech signal as it changes over time, including the amplitude, period, and pulse width of the original speech. Frequency-domain dynamic features describe the frequency component distribution of the speech signal, including frequency distribution, frequency density, and frequency components.

[0082] It should be noted that, preferably, the batch size during training is set to 256, the initial learning rate is set to 5e-4, and a linear decay learning rate strategy is adopted during training.

[0083] It should be noted that by learning the weight ratios of time-domain dynamic features, frequency-domain dynamic features, and traditional acoustic features, different weight ratios are assigned to the three features, and the contributions of different features are dynamically adjusted, so that the model can fully extract the dynamic and static features of the speaker's speech, and at the same time integrate multi-dimensional information from the time and frequency domains to enhance the expressive and discriminative capabilities of the features.

[0084] It should be noted that by inputting the fused features into a fully connected layer for feature compression and nonlinear mapping, the fused features are adapted to the classification task and classified in a classifier (such as AAM-Softmax). By introducing angular boundaries in the feature space, the interval between different types of features is increased, which can effectively improve the accuracy and robustness of speaker classification and enhance the system's adaptability to different speech scenarios.

[0085] In specific embodiments, by using time dynamic feature sequences with values ​​of 0 or 1, it is possible to analyze the hidden patterns, periodicity, and nonlinear dynamics of a time series. The time-domain dynamic feature matrix can extract the nonlinear dynamic changes of speech signals over time, identify diagonals and block structures in images, and better extract speech signal similarity and speaker features.

[0086] In a specific embodiment, the value of β ranges from [0, 1], and the sum of β1, β2 and β3 is 1.

[0087] It should be noted that this application solves the problem of the current single feature extraction method by adding and replacing the feature extraction method and model, and combining the dynamic information and time-frequency domain features of the speech signal. It more fully preserves the information in the speech signal that distinguishes the speaker's identity, and uses different models to model the extracted features, so as to extract the global and local information of the speaker more comprehensively. It can adapt to a variety of feature extraction schemes and enhance the accuracy and robustness of speaker recognition in complex scenarios.

[0088] The following is for reference. Figure 2 , Figure 2 A speaker recognition system based on a time-frequency domain dynamic feature matrix is ​​shown, including a time-domain module a, a frequency-domain module b, a feature module c, an adaptive fusion module d, and a recognition module e.

[0089] In a specific embodiment, the time-domain module a is configured to: map the time-dynamic feature sequence of the original speech into a two-dimensional image, calculate the similarity of each frame of the original speech signal through a similarity matrix, and enhance the time-domain dynamic features in the time-dynamic feature sequence using an adaptive weighting method to obtain a time-domain dynamic feature matrix.

[0090] The time-domain dynamic feature matrix is ​​represented as follows:

[0091] R ij = w(i, j)·θ(∈-||X||);

[0092]

[0093] Among them, R ij denoted by , where ∈ represents the value of the time-domain dynamic feature matrix, ∈ represents the preset time-domain dynamic feature threshold, ||X|| represents the distance between time points i and j, θ represents the step function, w(i,j) is the Gaussian weight of the time position, and σ represents the standard deviation used to adjust the weight decay rate.

[0094] In a specific embodiment, the frequency domain module b is configured to: perform a short-time Fourier transform on the original speech to obtain the spectral value of each frame of speech signal, calculate the frequency domain dynamic features of each frame of speech signal of the original speech, and dynamically adjust the similarity threshold to obtain the frequency domain dynamic feature matrix.

[0095] The frequency domain dynamic feature matrix is ​​represented as follows:

[0096] R f (i, j) = θ(ε) f -||S(i)-S(j)||);

[0097] Among them, R f (i, j) represents the frequency domain dynamic characteristic matrix, ε f Let S(i) represent the similarity threshold of the frequency domain dynamic feature matrix, ||S(i)-S(j)|| represent the distance between the spectral energies of frame i and frame j, and θ represent the step function.

[0098] In a specific embodiment, the feature module c is configured to: input the frequency domain dynamic features and the time domain dynamic features into a convolutional neural network (CNN) for training, extract the acoustic features of the original speech signal using traditional feature extraction methods, and input them into a conformer model for processing to obtain the initial speaker features of the original speech.

[0099] In a specific embodiment, the adaptive fusion module d is configured to: perform weighted adaptive fusion of the trained time-domain dynamic features, the frequency-domain dynamic features, and the speaker's initial features, and then send the fused feature vector to a fully connected layer and map it to a low-dimensional space.

[0100] In a specific embodiment, the recognition module e is configured such that: the feature fusion classifier calculates the speaker's category probability distribution based on the feature vector output by the fully connected layer, and takes the category with the highest probability as the final speaker recognition result.

[0101] It should be noted that the CNN model was trained using the PyTorch Lightning training framework for 50 iterations and optimized using the AdamW optimizer.

[0102] It should be noted that, preferably, the batch size during training is set to 256, the initial learning rate is set to 5e-4, and a linear decay learning rate strategy is adopted during training.

[0103] It should be noted that this application solves the problem of the current single feature extraction method by adding and replacing the feature extraction method and model, and combining the dynamic information and time-frequency domain features of the speech signal. It more fully preserves the information in the speech signal that distinguishes the speaker's identity, and uses different models to model the extracted features, so as to extract the global and local information of the speaker more comprehensively. It can adapt to a variety of feature extraction schemes and enhance the accuracy and robustness of speaker recognition in complex scenarios.

[0104] The following is for reference. Figure 3 It shows a schematic diagram of the structure of a computer system suitable for implementing the electronic device of the present application. Figure 3 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0105] like Figure 3 As shown, the computer system includes a central processing unit (CPU) 301, which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) 302 or programs loaded from storage section 308 into random access memory (RAM) 303. The RAM 303 also stores various programs and data required for system operation. The CPU 301, ROM 302, and RAM 303 are interconnected via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0106] The following components are connected to I / O interface 305: an input section 306 including a keyboard, mouse, etc.; an output section 307 including a liquid crystal display (LCD) and speakers, etc.; a storage section 308 including a hard disk, etc.; and a communication section 309 including a network interface card such as a LAN card and a modem, etc. The communication section 309 performs communication processing via a network such as the Internet. A drive 310 is also connected to I / O interface 305 as needed. A removable medium 311, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 310 as needed so that computer programs read from it can be installed into storage section 308 as needed.

[0107] Specifically, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable storage medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 309, and / or installed from removable medium 311. When the computer program is executed by central processing unit (CPU) 301, it performs the functions defined in the methods of this application. It should be noted that the computer-readable storage medium of this application can be a computer-readable signal medium or a computer-readable storage medium or any combination thereof. The computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can also be any computer-readable storage medium other than a computer-readable storage medium that can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. Program code contained on a computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0108] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages—such as Java, Smalltalk, and C++—as well as conventional procedural programming languages—such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0109] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0110] The modules described in the embodiments of this application can be implemented in software or in hardware.

[0111] In another aspect, this application also provides a computer-readable storage medium, which may be included in the electronic device described in the above embodiments; or it may exist independently and not assembled into the electronic device. The computer-readable storage medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to: map the temporal dynamic feature sequence of the original speech into a two-dimensional image, calculate the similarity of each frame of the original speech signal using a similarity matrix, enhance the temporal dynamic features in the temporal dynamic feature sequence using an adaptive weighting method, and obtain a temporal dynamic feature matrix; perform a short-time Fourier transform on the original speech to obtain the spectral value of each frame of the speech signal, calculate the frequency domain dynamic features of each frame of the original speech signal, and dynamically adjust the similarity threshold to obtain a frequency domain dynamic feature matrix; and... The frequency domain dynamic features and the time domain dynamic features are input into a convolutional neural network (CNN) for training. Acoustic features of the original speech signal are extracted using traditional feature extraction methods and input into a conformer model for processing to obtain the initial speaker features of the original speech. The trained time domain dynamic features, frequency domain dynamic features, and initial speaker features are then weighted and adaptively fused. The fused feature vector is fed into a fully connected layer and mapped to a low-dimensional space. The feature fusion classifier calculates the speaker's category probability distribution based on the feature vector output by the fully connected layer and uses the category with the highest probability as the final speaker recognition result.

[0112] Finally, it should be noted that the above description is merely a preferred embodiment of this application and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in this application is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features with similar functions disclosed in this application.

Claims

1. A speaker recognition method based on a time-frequency domain dynamic feature matrix, characterized in that, include: S1: Map the temporal dynamic feature sequence of the original speech to a two-dimensional image, and calculate the similarity of each frame of the original speech signal through the similarity matrix. Use an adaptive weighting method to enhance the temporal dynamic features in the temporal dynamic feature sequence to obtain the temporal dynamic feature matrix. The time-domain dynamic feature matrix is ​​represented as follows: R ij =w(i,j)·θ(∈-||X||); Among them, R ij denoted as the value of the time-domain dynamic feature matrix, ∈ represents the preset time-domain dynamic feature threshold; ||X|| represents the distance between time points i and j, θ represents the step function, w(i,j) is the Gaussian weight of the time position, and σ represents the standard deviation used to adjust the weight decay rate; S2: Perform a short-time Fourier transform on the original speech to obtain the spectral value of each frame of speech signal, calculate the frequency domain dynamic features of each frame of speech signal of the original speech, and dynamically adjust the similarity threshold to obtain the frequency domain dynamic feature matrix. The frequency domain dynamic feature matrix is ​​represented as follows: R f (i,j)=θ(ε f -||S(i)-S(j)||); Among them, R f (i, j) represents the frequency domain dynamic characteristic matrix, ε f Let S(i) represent the similarity threshold of the frequency domain dynamic feature matrix, ||S(i)-S(j)|| represent the distance between the spectral energies of frame i and frame j, and θ represent the step function. S3: Input the frequency domain dynamic features and the time domain dynamic features into the convolutional neural network (CNN) for training, extract the acoustic features of the original speech signal using traditional feature extraction methods, and input them into the Conformer model for processing to obtain the initial speaker features of the original speech. S4: The trained time-domain dynamic features, frequency-domain dynamic features, and speaker initial features are weighted and adaptively fused, and the fused feature vector is fed into a fully connected layer and mapped to a low-dimensional space. S5: The feature fusion classifier calculates the speaker's category probability distribution based on the feature vector output by the fully connected layer, and takes the category with the highest probability as the final speaker recognition result.

2. The method according to claim 1, characterized in that, Before S1, the process also includes extracting a time dynamic feature sequence from the original speech and dividing the original speech into 25ms frames.

3. The method according to claim 1, characterized in that, The formula for calculating the similarity threshold of the frequency domain dynamic feature matrix is ​​as follows: ε f =α·mean(||S(i)-S(j)||); Where α represents the adjustment factor, S(i) and S(j) represent the spectral energy, and mean represents the average value.

4. The method according to claim 3, characterized in that, The formula for calculating the spectral energy is: S(t)=[|X(t,f1)|,|X(t,f2)|,…,|X(t,f F )|]; Where X(t,f) F ) represents the spectral values ​​of time t and frequency f, ω[nt] represents the short time window function, and N represents the number of points of the Fourier transform.

5. The method according to claim 1, characterized in that, The calculation formula for the weighted adaptive fusion of the time-domain dynamic features, the frequency-domain dynamic features, and the speaker's initial features is as follows: and fusion =β1·e time +β2·e freq +β3·e featue ; Among them, e fusion e represents the fused feature vector. time e represents the time-domain dynamic characteristics. freq Represents the dynamic characteristics in the frequency domain, e feature β1, β2, and β3 represent the speaker's initial features, while β1, β2, and β3 represent the feature fusion weight parameters.

6. A speaker recognition system based on a time-frequency domain dynamic feature matrix, characterized in that, include: Temporal module: The temporal dynamic feature sequence of the original speech is mapped into a two-dimensional image, and the similarity of each frame of the original speech signal is calculated through a similarity matrix. An adaptive weighting method is used to enhance the temporal dynamic features in the temporal dynamic feature sequence to obtain the temporal dynamic feature matrix. The time-domain dynamic feature matrix is ​​represented as follows: R ij =w(i,j)·θ(∈-||X||); Among them, R ij denoted as the value of the time-domain dynamic feature matrix, ∈ represents the preset time-domain dynamic feature threshold; ||X|| represents the distance between time points i and j, θ represents the step function, w(i,j) is the Gaussian weight of the time position, and σ represents the standard deviation used to adjust the weight decay rate; Frequency domain module: Performs short-time Fourier transform on the original speech to obtain the spectral value of each frame of speech signal, calculates the frequency domain dynamic features of each frame of speech signal of the original speech, and dynamically adjusts the similarity threshold to obtain the frequency domain dynamic feature matrix. The frequency domain dynamic feature matrix is ​​represented as follows: R f (i,j)=θ(ε f -||S(i)-S(j)||); Among them, R f (i, j) represents the frequency domain dynamic characteristic matrix, ε f Let S(i) represent the similarity threshold of the frequency domain dynamic feature matrix, ||S(i)-S(j)|| represent the distance between the spectral energies of frame i and frame j, and θ represent the step function. Feature module: The frequency domain dynamic features and the time domain dynamic features are input into the convolutional neural network (CNN) for training. The acoustic features of the original speech signal are extracted by traditional feature extraction methods and input into the Conformer model for processing to obtain the initial speaker features of the original speech. Adaptive fusion module: performs weighted adaptive fusion of the trained time-domain dynamic features, frequency-domain dynamic features, and speaker initial features, and feeds the fused feature vector into a fully connected layer and maps it to a low-dimensional space; Recognition Module: The feature fusion classifier calculates the speaker's category probability distribution based on the feature vector output by the fully connected layer, and takes the category with the highest probability as the final speaker recognition result.

7. A computer program product having one or more computer programs thereon, characterized in that, When the computer program is executed by a computer processor, the method described in any one of claims 1-5 is performed.

Images