Method and apparatus for training a voiceprint recognition model

Abstract

The application discloses a voiceprint recognition model training method and device, and belongs to the technical field of artificial intelligence. The method comprises the following steps: obtaining audio information of first audio data, wherein the audio information comprises an audio signal and a target voiceprint feature; based on the audio signal, first STFT feature information is obtained, wherein the first STFT feature information comprises time domain features and frequency domain features of the first audio data; the first STFT feature information is subjected to feature processing to obtain a first feature vector, wherein the first feature vector represents the voiceprint features of the first audio data; and based on the target voiceprint feature and the first feature vector, a first voiceprint recognition model is trained to obtain a second voiceprint recognition model.

Description

Technical Field

[0001] This application belongs to the field of artificial intelligence technology, specifically relating to a training method and apparatus for a voiceprint recognition model. Background Technology

[0002] With the continuous development of artificial intelligence technology, its applications are becoming increasingly widespread in various fields, and voiceprint recognition is one of the important application scenarios. Voiceprint recognition technology plays an increasingly important role in ensuring security and improving user experience. People's demand for voiceprint recognition continues to grow, and the requirements for the accuracy of voiceprint recognition results are becoming increasingly stringent.

[0003] Existing voiceprint recognition models primarily utilize Mel filter bank (FBank) and Mel-Frequency Cepstral Coefficients (MFCC) features in their audio signal features. FBank features are spectral energy characteristics directly extracted from the Mel filter bank; they simulate the auditory characteristics of the human ear, decomposing the audio signal into energy distributions across multiple Mel frequency bands. MFCC features are derived from FBank features through discrete cosine transform, further compressing the data and removing correlations between different signal dimensions.

[0004] However, since the aforementioned FBank and MFCC features are obtained through operations such as Mel filtering, which lead to feature loss in the audio (e.g., discarding high-frequency details in the original audio), the original audio signal suffers from severe distortion. Therefore, current voiceprint recognition models perform poorly in voiceprint identification. Summary of the Invention

[0005] The purpose of this application is to provide a training method and apparatus for a voiceprint recognition model, which can improve the training efficiency of electronic devices for voiceprint recognition models.

[0006] In a first aspect, embodiments of this application provide a training method for a voiceprint recognition model. The method includes: acquiring audio information of first audio data, the audio information including an audio signal and target voiceprint features; acquiring first short-time Fourier transform (STFT) feature information based on the audio signal, the first STFT feature information including time-domain features and frequency-domain features of the first audio data; performing feature processing on the first STFT feature information to obtain a first feature vector, the first feature vector representing the voiceprint features of the first audio data; and training a first voiceprint recognition model based on the target voiceprint features and the first feature vector to obtain a second voiceprint recognition model.

[0007] Secondly, embodiments of this application provide a training apparatus for a voiceprint recognition model, the apparatus comprising: an acquisition module and a processing module. The acquisition module is configured to acquire audio information of first audio data, the audio information including an audio signal and target voiceprint features; and, based on the audio signal, acquire first STFT feature information, the first STFT feature information including time-domain features and frequency-domain features of the first audio data. The processing module is configured to perform feature processing on the first STFT feature information acquired by the acquisition module to obtain a first feature vector, the first feature vector representing the voiceprint features of the first audio data; and, based on the target voiceprint features acquired by the acquisition module and the first feature vector, train a first voiceprint recognition model to obtain a second voiceprint recognition model.

[0008] Thirdly, embodiments of this application provide an electronic device including a processor and a memory, wherein the memory stores programs or instructions executable on the processor, and the programs or instructions, when executed by the processor, implement the steps of the method described in the first aspect.

[0009] Fourthly, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method described in the first aspect.

[0010] Fifthly, embodiments of this application provide a chip, the chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run programs or instructions to implement the method as described in the first aspect.

[0011] In a sixth aspect, embodiments of this application provide a computer program product stored in a storage medium, which is executed by at least one processor to implement the method described in the first aspect.

[0012] In this embodiment of the application, an audio signal and target voiceprint features corresponding to the first audio data are obtained. Based on the audio signal, a first STFT feature information including the time domain features and frequency domain features of the first audio data is obtained. The first STFT feature information is processed to obtain a first feature vector characterizing the voiceprint features of the first audio data. Based on the target voiceprint features and the first feature vector, a first voiceprint recognition model is trained to obtain a second voiceprint recognition model. In this scheme, after acquiring the first audio data, the first STFT feature information, which includes the time-domain and frequency-domain features of the first audio data, can be obtained based on the audio signal corresponding to the first audio data. Since the first STFT feature information is directly extracted from the audio signal, it retains the detailed distribution information of the audio signal in time and frequency, and is closer to the audio signal corresponding to the first audio data. It can provide more information about the first audio data itself. In the process of training the first voiceprint recognition model, this first STFT feature information, which is closer to the audio signal and includes more comprehensive features of the first audio data, is used as input. This allows the first voiceprint recognition model to capture the unique voiceprint features of the first audio data more quickly and accurately for model training, thus obtaining the second voiceprint recognition model. Therefore, the trained second voiceprint recognition model can accurately perform voiceprint recognition, improving the voiceprint recognition effect of the voiceprint recognition model. Attached Figure Description

[0013] Figure 1 This is one of the flowcharts illustrating the training method of the voiceprint recognition model provided in the embodiments of this application;

[0014] Figure 2 This is a second schematic flowchart of the training method for the voiceprint recognition model provided in the embodiments of this application;

[0015] Figure 3 This is a schematic diagram of an example of the STFT time-frequency spectrum provided in an embodiment of this application;

[0016] Figure 4 This is the third flowchart illustrating the training method of the voiceprint recognition model provided in the embodiments of this application;

[0017] Figure 5 This is a flowchart illustrating the SENet module provided in an embodiment of this application;

[0018] Figure 6 This is a flowchart illustrating the Res2Net module provided in an embodiment of this application;

[0019] Figure 7 This is a flowchart illustrating the Attention module provided in an embodiment of this application;

[0020] Figure 8This is one of the schematic diagrams of the replicated audio upload interface provided in the embodiments of this application;

[0021] Figure 9 This is a second schematic diagram of the replicated audio upload interface provided in the embodiments of this application;

[0022] Figure 10 This is the fourth flowchart illustrating the training method of the voiceprint recognition model provided in the embodiments of this application;

[0023] Figure 11 This is a data flow diagram of the training method for the voiceprint recognition model provided in the embodiments of this application;

[0024] Figure 12 This is a schematic diagram of the structure of the training device for the voiceprint recognition model provided in the embodiments of this application;

[0025] Figure 13 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application;

[0026] Figure 14 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0027] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0028] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0029] The terms "at least one," "at least one," etc., in this application refer to any one, any two, or a combination of two or more of the included objects. For example, at least one of a, b, and c can mean: "a," "b," "c," "a and b," "a and c," "b and c," and "a, b, and c," where a, b, and c can be single or multiple. Similarly, "at least two" refers to two or more, and its meaning is similar to that of "at least one."

[0030] The training method of the voiceprint recognition model provided in this application will be described in detail below with reference to the accompanying drawings, through specific embodiments and application scenarios.

[0031] The training method for the voiceprint recognition model provided in this application can be applied to scenarios where a voiceprint recognition model is required to process voiceprint recognition tasks.

[0032] For example, a trained second voiceprint recognition model can be used to detect whether user-submitted audio data belongs to a sensitive individual. When training the first voiceprint recognition model to obtain the trained second voiceprint recognition model, firstly, audio information corresponding to multiple audio data points is obtained from an open-source audio dataset. This audio information includes audio signals, audio sampling rates, and target voiceprint features. With the audio sampling rate of each audio signal equal to the target sampling rate, the audio signals are processed to ensure that all audio signals have equal lengths. Then, a Fourier transform is performed on the audio signals to obtain first STFT feature information. This first STFT feature information is input into the first voiceprint recognition model for model training. Weights are applied to each frequency feature in the first STFT feature information. The second STFT feature information is obtained by readjustment. The frequency features in the second STFT feature information are convolved to obtain the third STFT feature information. Based on the third STFT feature information, the audio data is processed to obtain the first feature vector. Based on the target voiceprint feature and the first feature vector, the loss value is calculated. Based on the loss value, the parameters in the first voiceprint recognition model are updated to obtain the second voiceprint recognition model. The second voiceprint recognition model is used to identify the voiceprint features of the audio data submitted by the user and compare them with the voiceprint features of the audio data of sensitive persons to determine whether the audio data submitted by the user belongs to a sensitive person.

[0033] The voiceprint recognition model training method provided in this application embodiment obtains the audio signal and target voiceprint features corresponding to the first audio data, obtains the first STFT feature information including the time domain features and frequency domain features of the first audio data based on the audio signal, performs feature processing on the first STFT feature information to obtain the first feature vector characterizing the voiceprint features of the first audio data, and trains the first voiceprint recognition model based on the target voiceprint features and the first feature vector to obtain the second voiceprint recognition model. In this scheme, after acquiring the first audio data, the first STFT feature information, which includes the time-domain and frequency-domain features of the first audio data, can be obtained based on the audio signal corresponding to the first audio data. Since the first STFT feature information is directly extracted from the audio signal, it retains the detailed distribution information of the audio signal in time and frequency, and is closer to the audio signal corresponding to the first audio data. It can provide more information about the first audio data itself. In the process of training the first voiceprint recognition model, this first STFT feature information, which is closer to the audio signal and includes more comprehensive features of the first audio data, is used as input. This allows the first voiceprint recognition model to capture the unique voiceprint features of the first audio data more quickly and accurately for model training, thus obtaining the second voiceprint recognition model. Therefore, the trained second voiceprint recognition model can accurately perform voiceprint recognition, improving the voiceprint recognition effect of the voiceprint recognition model.

[0034] The voiceprint recognition model training method provided in this application can be executed by a voiceprint recognition model training device. Exemplarily, the voiceprint recognition model training device can be an electronic device or a component within that electronic device, such as an integrated circuit or a chip. The specific device can be determined according to actual usage requirements, and this application does not impose any limitations. The following description uses an electronic device as the voiceprint recognition model training device and an electronic device executing the voiceprint recognition model training method as an example to illustrate the voiceprint recognition model training method provided in this application.

[0035] Figure 1 A flowchart illustrating a training method for a voiceprint recognition model provided in an embodiment of this application is shown. Figure 1 As shown, the training method for the voiceprint recognition model provided in this application embodiment may include the following steps 201 to 204.

[0036] Step 201: The electronic device acquires the audio information of the first audio data.

[0037] In this embodiment of the application, the audio information includes audio signals and target voiceprint features.

[0038] It should be noted that the audio signal mentioned above refers to the signal waveform of the audio signal.

[0039] In this embodiment of the application, the first audio data includes multiple audio data, which can also be called an open-source audio dataset, as a dataset for training the voiceprint recognition model.

[0040] Optionally, in this embodiment of the application, the electronic device can obtain the first audio data by traversing the data directory structure where the first audio data is located, thereby obtaining the audio information of the first audio data.

[0041] For example, taking the open-source audio dataset CN-Celeb2 as an example, SPK_ID represents the speaker's identity, which represents the voiceprint feature vector, i.e., the target voiceprint feature mentioned above. audio_file represents the first audio data of the speaker, and its data directory structure is CN-Celeb2_flac / data / ${SPK_ID} / ${audio_file}. The specific method to obtain the audio information of the first audio data in this open-source audio dataset is as follows: traverse each audio data audio_file in the open-source audio dataset, and use torchaudio to read the audio signal SIGNAL and audio sampling rate SR corresponding to each audio data; according to the absolute file path corresponding to the audio data audio_file, use Python code to parse the speaker identity SPK_ID corresponding to each audio data; combine the obtained audio information into a triplet data [SIGNAL, SR, SPK_ID].

[0042] It is understandable that the aforementioned audio signal represents a mechanical wave, serving as a carrier of information regarding the wavelength and intensity changes of that wave. Audio signals possess several key characteristic parameters, such as frequency, amplitude, phase, and bandwidth. These parameters determine the characteristics of the audio signal, and by analyzing these characteristics, the speaker's identity can be identified—a process known as voiceprint recognition.

[0043] It should be noted that voiceprint refers to the unique acoustic characteristics of each person when speaking, similar to the uniqueness of a fingerprint. Voiceprint recognition is a biometric-based identity verification technology.

[0044] In this embodiment of the application, the aforementioned target voiceprint feature refers to the actual voiceprint feature of the first audio data, that is, the voiceprint feature expected to be output after the first audio data is input into the voiceprint recognition model, which is used to compare with the voiceprint feature predicted by the voiceprint recognition model, thereby judging the accuracy of the recognition result of the voiceprint recognition model.

[0045] Step 202: The electronic device acquires the first STFT feature information based on the audio signal.

[0046] In this embodiment of the application, the first STFT feature information includes the time-domain features and frequency-domain features of the first audio data.

[0047] It is understood that the aforementioned first STFT feature information includes the frequency characteristics of the aforementioned first audio data at each time. STFT features are crucial time-frequency analysis features in signal processing and audio processing. STFTs can analyze audio signals in both the time and frequency domains, exhibiting excellent time-frequency localization characteristics and effectively capturing the instantaneous features of audio signals. STFT features can be displayed through time-frequency graphs, where the horizontal axis represents time, the vertical axis represents frequency, and color or brightness represents the intensity of the audio signal at different times and frequencies.

[0048] Optionally, in this embodiment of the application, the first audio data includes multiple audio data items, and the audio information further includes the audio sampling rate, combined with... Figure 1 ,like Figure 2 As shown, step 202 can be implemented through steps 202a and 202b below.

[0049] Step 202a: If the lengths of the audio signals of multiple audio data are not equal when the audio sampling rate is equal to the target sampling rate, the electronic device performs signal processing on the audio signals of the multiple audio data.

[0050] In the embodiments of this application, the above signal processing includes at least one of signal trimming and signal padding.

[0051] In this embodiment of the application, the audio sampling rate refers to the number of times per second that a continuously changing analog audio signal is sampled when it is converted into a discrete digital signal.

[0052] It should be noted that the higher the audio sampling rate, the more points are sampled per unit time, the richer the original audio signal information can be retained, and the better the sound quality is reproduced. In addition, the higher the audio sampling rate, the more sound details are captured, which can more realistically reproduce the sound quality and timbre of the original audio signal, making the listening experience closer to the original analog signal.

[0053] In this embodiment of the application, the target sampling rate can be a pre-set audio sampling rate that meets the requirements of the voiceprint recognition model, such as 16KHz. The target sampling rate is used to determine whether the audio sampling rate of the acquired audio information meets the requirements.

[0054] It should be noted that after acquiring the audio signals and target voiceprint features of multiple audio data points, the electronic device can also acquire the audio sampling rates corresponding to these multiple audio data points to determine whether the audio signals corresponding to these multiple audio data points meet the requirements of the voiceprint recognition model for the input signal. That is, if the audio sampling rate is equal to the target sampling rate, the audio signals corresponding to these multiple audio data points are considered to be usable as input to the voiceprint recognition model. Then, it further determines whether the lengths of the audio signals corresponding to the multiple audio data points are consistent. If they are consistent, the audio signals corresponding to the multiple audio data points are directly converted into the corresponding STFT features. If they are inconsistent, signal processing is performed on the audio signals corresponding to the multiple audio data points to avoid errors in the voiceprint recognition model training process caused by inconsistent lengths of multiple audio signals.

[0055] Optionally, in this embodiment of the application, the electronic device may perform signal processing on the audio signals of multiple audio data in a manner that includes at least one of the following: (1) padding audio signals shorter than the required length by adding blank audio at the end of the audio signal; (2) trimming audio signals longer than the required length by randomly selecting a segment of audio signal that meets the required length. This ensures that the lengths of the processed audio signals of the multiple audio data are all equal.

[0056] For example, the electronic device defines a hyperparameter target sampling rate (target_sample_rate) and a training audio length (train_audio_len); it determines whether target_sample_rate is equal to the audio sampling rate SR obtained in step 201 above. If they are not equal, the resample method of torchaudio is used to resample to ensure that the audio sampling rate SR conforms to the hyperparameter specification; if they are equal, it further determines whether train_audio_len is equal to the length of the audio signal SIGNAL obtained by resampling. If the audio signal SIGNAL is short, zeros are added to the end of the audio signal SIGNAL, i.e., blank audio is added. If the audio signal SIGNAL is long, the audio signal SIGNAL is randomly truncated. Finally, the electronic device will trim / padded all the sampled audio signals SIGNAL into audio signals of the same length.

[0057] Step 202b: The electronic device performs Fourier transform on the audio signals of the multiple audio data after signal processing to obtain the first STFT feature information.

[0058] In this embodiment, the Fourier transform described above is a mathematical method for converting a signal from the time domain (or spatial domain) to the frequency domain. In audio signal processing, the Fourier transform can decompose an audio signal into sinusoidal components of different frequencies, thereby revealing the frequency components of the audio signal. STFT is an extension of the Fourier transform, suitable for processing non-stationary signals. STFT obtains the frequency components of the signal as it changes over time by dividing the signal into multiple short time intervals and performing a Fourier transform on each interval. This allows STFT to capture transient changes in the signal, such as sudden changes in pitch or bursts of noise, thus obtaining the characteristics of the signal at different times and frequencies, providing a basis for subsequent analysis and processing.

[0059] Typically, audio data is composed of the superposition of periodic signals at multiple frequencies. The influence factors of signals at different frequencies on the voiceprint recognition task vary. In this embodiment, the electronic device obtains STFT features by performing Fourier transform on the audio signal, thereby converting the audio signal into component signals at multiple frequencies and reducing the learning difficulty of the voiceprint recognition model.

[0060] For example, the electronic device performs a Fourier transform on the audio signal SIGNAL obtained in step 202a according to the hyperparameter n_fft, converting the time-domain feature SIGNAL into a time-frequency domain feature STFT, i.e., the first STFT feature information. In this way, the electronic device transforms the one-dimensional unaligned audio signal SIGNAL into a two-dimensional aligned STFT feature, i.e., the first STFT feature information, through a short-time Fourier transform. The length of the audio signal becomes train_audio_len, and the audio signal SIGNAL is separated into a superposition of multiple periodic waveforms of different frequencies, which can be divided into two dimensions: time and frequency. Figure 3 As shown, the irregular audio signal SIGNAL30 is separated into three periodic signal waveforms with different frequencies, as follows: Figure 3 Signal waveforms 31, 32 and 33 are shown in the figure.

[0061] It's important to note that `n_fft` is a key parameter in STFT, representing the window length used when performing the Fast Fourier Transform (FFT), which is the number of sample points considered in each analysis. In STFT, the audio signal is divided into multiple overlapping windows (or frames), and the signal within each window is independently FFTed. `n_fft` determines the number of samples within each window, thus affecting both frequency and temporal resolution.

[0062] In this embodiment, when the audio sampling rate is equal to the target sampling rate, the electronic device performs signal trimming and signal padding on the audio signals of multiple audio data, so that the audio signal lengths of the multiple audio data are consistent, thereby reducing the impact of audio data defects on the training results of the voiceprint recognition model.

[0063] Step 203: The electronic device performs feature processing on the first STFT feature information to obtain the first feature vector.

[0064] In this embodiment of the application, the first feature vector represents the voiceprint feature of the first audio data.

[0065] In this embodiment of the application, the electronic device inputs the first STFT feature information corresponding to the audio information into the first voiceprint recognition model (i.e., the initial voiceprint recognition model). The first voiceprint recognition model performs feature processing on the first STFT feature information to obtain the voiceprint features of the predicted first audio data, that is, the first audio data is mapped to the corresponding first feature vector.

[0066] Optionally, in the embodiments of this application, the above-mentioned frequency domain features include at least one frequency feature, combined with Figure 1 ,like Figure 4 As shown, step 203 can be implemented through steps 203a to 203c as described below.

[0067] Step 203a: The electronic device performs weight adjustment on each frequency feature in the first STFT feature information to obtain the second STFT feature information.

[0068] In this embodiment, the audio signal is converted into a periodic signal within A frequencies by using the first STFT feature information obtained in step 202 above. The first STFT feature information and each frequency feature C are represented by the following formula (1). x The relationship, where x∈[1,A]:

[0069] F STFT =[C1,C2,……,C A (1)

[0070] Among them, F STFT This represents the first STFT feature information mentioned above, C A This represents the A-th frequency feature.

[0071] It should be noted that different frequency characteristics C x With C y (x≠y) differs in human hearing, thus having different impacts on voiceprint recognition tasks.

[0072] Optionally, in this embodiment of the application, the electronic device may use a Squeeze-and-Excitation Network (SENet) module to adjust the weight of each frequency feature in the first STFT feature information to obtain the second STFT feature information, so as to reduce the influence of minor frequency features in the first STFT feature information and improve the recognition of important frequency features in the first STFT feature information.

[0073] Understandably, the SENet module is an innovative neural network architecture designed to improve network performance by explicitly modeling the dependencies between feature channels. The core idea of ​​the SENet module is to enhance the representational capabilities of convolutional neural networks by introducing a new structural unit—the "Squeeze-and-Excitation" (SE) block—which measures the correlation between channels in a multi-channel input and dynamically adjusts the corresponding weights. The SE block consists of two main operations: squeezing and excitation.

[0074] Optionally, in this embodiment of the application, the electronic device can adjust the weight of each frequency feature in the first STFT feature information by means of compression and excitation operations. For example, step 203a can be implemented by steps 203a1 to 203a3 as described below.

[0075] Step 203a1: The electronic device performs global average pooling on each frequency feature in the first STFT feature information to obtain the average STFT feature information.

[0076] In this embodiment of the application, the electronic device uses a compression operation to perform global average pooling on each frequency feature in the first STFT feature information to obtain average STFT feature information.

[0077] Understandingly, compression refers to aggregating the spatial dimensions of the input feature map through global average pooling, generating a channel descriptor for each channel. Specifically, by performing global average pooling on the input feature map, the feature values ​​of all spatial locations for each channel are summed, and then divided by the number of spatial locations, resulting in a vector equal to the number of channels. This vector contains the global information of each channel. In this way, the global spatial information is compressed into a single channel vector, capturing the global distribution of the channel feature responses.

[0078] For example, such as Figure 5As shown, the electronic device uses the SENet module to perform global average pooling on the input first STFT feature information in the frequency dimension, reducing the feature value of each frequency in the first STFT feature information to a scalar. In this way, the spatial information of each frequency feature is compressed into a global description, resulting in the average STFT feature information STFT_AVG.

[0079] Step 203a2: The electronic device performs frequency adjustment processing on the average STFT feature information to obtain the weight value corresponding to each frequency feature.

[0080] In this embodiment, the electronic device uses an excitation operation to perform frequency adjustment processing on the average STFT feature information to obtain the weight value corresponding to each frequency feature.

[0081] Understandably, the activation operation refers to learning the weights of each channel through a two-layer fully connected network (FC network) after the channel descriptors are obtained through compression. The role of these two fully connected layers is to capture the dependencies between channels and generate a weight value (or importance coefficient) for each channel based on these dependencies. The first FC layer is typically used to reduce the dimensionality of the channel descriptors to reduce computation and applies the Rectified Linear Unit (ReLU) function, i.e., a non-linear activation function. The second FC layer then projects it back to the original channel dimension and outputs the importance coefficient for each channel through a sigmoid activation function.

[0082] For example, such as Figure 5 As shown, the electronic device utilizes the SENet module with a network structure comprising fully connected layer 1 and fully connected layer 2. It combines activation function 1 (e.g., ReLU) and activation function 2 (e.g., Sigmoid activation function) respectively to readjust the compressed average STFT feature information STFT_AVG, thereby generating the weight value STFT_WEIGHT for each channel. Fully connected layer 1 is typically used for frequency dimensionality reduction, while fully connected layer 2 is used to restore the original frequency count.

[0083] Step 203a3: The electronic device performs weighted processing on the frequency features in the first STFT feature information based on the weight value corresponding to each frequency feature to obtain the second STFT feature information.

[0084] For example, after obtaining the weight value corresponding to each frequency feature, the electronic device can use the generated weight value STFT_WEIGHT to weight the original input first STFT feature information by multiplying by frequency, thereby dynamically adjusting the feature response of each frequency and generating a new second STFT feature information W_STFT with weight information.

[0085] In this embodiment, the electronic device uses the SENet module to compress, excite, and weight the input first STFT feature information to obtain a new second STFT feature information W_STFT with weight information. This second STFT feature information can effectively enhance the feature representation capability of the voiceprint recognition model, thereby improving the performance of the voiceprint recognition model in the voiceprint recognition task.

[0086] Step 203b: The electronic device performs convolution processing on the frequency features in the second STFT feature information to obtain the third STFT feature information.

[0087] It should be noted that the second STFT feature information obtained through the above step 203a takes into account the weight of each frequency feature in the first STFT feature information. Based on this, and by combining the results of various combinations of different audio frequency features, the speaker's identity, i.e., voiceprint features, can be accurately identified.

[0088] Optionally, in this embodiment of the application, the electronic device can utilize a Residual Networks with Multi-Scale Feature Aggregation (Res2Net) module to group and convolve different frequency features to achieve the effect of integrating multiple frequency features, thereby extracting features that are more conducive to voiceprint recognition.

[0089] Understandably, the Res2Net module is a core component of a novel convolutional neural network architecture. It incorporates a series of innovations on the basis of traditional residual networks, aiming to enhance the feature representation capabilities of the model. Specifically, it refers to the mechanism of adding multi-scale features through combination, thereby strengthening the ability to represent multi-channel input features.

[0090] Optionally, in the embodiments of this application, step 203b can be implemented by steps 203b1 to 203b4 as described below.

[0091] Step 203b1: The electronic device groups the frequency features in the second STFT feature information to obtain N groups of frequency features.

[0092] Where N is an integer greater than 1.

[0093] For example, the electronic device divides the frequency features in the previously generated second STFT feature information W_STFT with weighted information into multiple groups, which represent multiple different scales (e.g., frequency bands), that is, each group represents a scale (e.g., low frequency band, mid frequency band, or high frequency band). Assuming that there are nine frequency features in the second STFT feature information W_STFT, these frequency features are divided into three groups, which can be represented by the following formula (2):

[0094] W_STFT=[[C1,C2,C3],[C4,C5,C6],[C7,C8,C9]] (2)

[0095] Among them, frequency features C1, C2, and C3 form one group, frequency features C4, C5, and C6 form another group, and frequency features C7, C8, and C9 form yet another group.

[0096] Step 203b2: The electronic device performs convolution processing on the first group of frequency features in the N groups of frequency features to obtain the first output feature.

[0097] Step 203b3: The electronic device performs convolution processing on the i-th frequency feature and the (i-1)-th output feature in the N-group frequency features to obtain the i-th output feature, i∈[2,N], and i is an integer.

[0098] Step 203b4: The electronic device performs fusion processing on the N output features corresponding to the N groups of frequency features to obtain the third STFT feature information.

[0099] For example, such as Figure 6 As shown, the electronic device performs convolution operations on each frequency feature group from front to back. In each step, the output feature of the previous group is fused with the input feature of the current group by addition. For example, in the first step, the frequency feature group [C1,C2,C3] is convolved to obtain Conv1. In the second step, the frequency feature group [C4,C5,C6] is convolved, and the result of this convolution operation is fused with Conv1 by addition to obtain Conv2. In the third step, the frequency feature group [C7,C8,C9] is convolved, and the result of this convolution operation is fused with Conv2 by addition to obtain Conv3. Finally, Conv1, Conv2, and Conv3 are fused to generate the final output feature, namely the third STFT feature information N_STFT.

[0100] In this embodiment, the electronic device divides the frequency features in the second STFT feature information into multiple groups, and then performs convolution operation on each frequency feature group from front to back. In each step, the electronic device fuses the output features of the previous group with the input features of the current group by addition. This stepwise convolution and fusion method enables each group to combine the information of the previous group, thereby gradually capturing richer multi-scale features. In this way, the newly obtained third STFT feature information can effectively capture frequency features of different scales, thereby improving the multi-scale feature representation capability of the voiceprint recognition model.

[0101] Step 203c: The electronic device processes the first audio data based on the third STFT feature information to obtain the first feature vector.

[0102] In this embodiment, the third STFT feature information adjusts the weight values ​​of each frequency feature in the first STFT feature information in the frequency domain and obtains multi-scale features. For the temporal portion of the first audio data, such as blank audio or background noise, the electronic device can further process the first audio data using the third STFT feature information to focus on audio data with specific temporal meaning, thereby reducing the impact of blank audio and background noise on the recognition effect of the voiceprint recognition model and improving the performance of the voiceprint recognition model.

[0103] It should be noted that the audio data with specific meaning in the above time sequence refers to: valid audio data within a certain time period. Valid means audio data with actual meaning, that is, it does not include blank audio or background noise audio.

[0104] Optionally, in this embodiment of the application, the electronic device may utilize the Attention module to process the first audio data based on the aforementioned third STFT feature information, thereby reducing the impact of blank audio and noise on the audio signal in the time domain.

[0105] Understandably, the Attention module is a crucial component of deep learning. It mimics the human attention mechanism, enabling the model to dynamically focus on important information based on task requirements when processing input data, thereby improving model performance and accuracy. The core idea of ​​the Attention module is to introduce an attention mechanism that helps the model assign different weights to each part of the input, thus extracting more critical and important information and allowing the model to make more accurate judgments. This mechanism is similar to how humans observe the external world; people typically focus on important local information first, and then combine it to form an overall impression.

[0106] Optionally, in the embodiments of this application, step 203c can be implemented by steps 203c1 to 203c4 as described below.

[0107] Step 203c1: The electronic device converts the third STFT feature information into a feature matrix.

[0108] In this embodiment of the application, the feature matrix includes frequency features corresponding to different audio durations.

[0109] For example, the third STFT feature information N_STFT is converted into an F_audio feature matrix of shape n*d, which can be represented by the following formula (3):

[0110] F_audio=[N_STFT1,N_STFT2,…,N_STFT n F_audio∈R n*d (3)

[0111] Where n represents the audio duration and d represents the number of frequency domain feature dimensions.

[0112] Step 203c2: The electronic device maps the feature matrix into three vectors.

[0113] In this embodiment of the application, the three vectors mentioned above include a query vector, a key vector, and a value vector.

[0114] In this embodiment of the application, the electronic device can use the processing logic in the Attention module to map the feature matrix obtained in step 203c1 above into three vectors: Query (query vector), Key (key vector), and Value (value vector), so as to reduce the impact of blank parts and noise in the first audio data on the prediction of the voiceprint recognition model and ensure that the voiceprint recognition task focuses on the effective audio part.

[0115] Step 203c3: The electronic device performs dot product and normalization on the query vector and key vector to obtain the first vector, and performs a weighted summation on the first vector and the value vector to obtain the intermediate vector.

[0116] Understandably, the core architecture of an Attention module typically includes three components: Query (query vector), Key (key vector), and Value (value vector), as well as steps such as Softmax normalization of attention scores and weighted summation. Specifically, it includes the following steps:

[0117] Query, Key, and Value Calculation: For the input sequence, the model first calculates the Query, Key, and Value vectors through three different linear transformations (usually implemented by learnable weight matrices).

[0118] Attention score calculation: Then, the matching degree between each query vector and all key vectors is calculated to form an attention score matrix. This matching degree is usually calculated using dot product or cosine similarity.

[0119] Attention weights are calculated by applying the Softmax function to normalize each row of the attention score matrix (corresponding to a query vector) to obtain the attention weight vector.

[0120] Context vector calculation: Finally, the attention weight vector is weighted and summed with the corresponding value vector set to obtain the context vector for each position. This context vector contains the most relevant information about the current task in the input sequence, which helps the model generate better output.

[0121] For example, such as Figure 7 As shown, the electronic device uses the processing logic in the Attention module to map the input feature matrix into three vectors: query vector, key vector, and value vector. By performing dot product and normalization (e.g., using the softmax function) on the query vector and key vector, a 1*n vector is obtained, which is used to represent the influence factors of n inputs in the time domain on the final output. The vector is then weighted and summed with the corresponding value vector to generate the state representation token of the entire first audio data, i.e., the intermediate vector. The specific expression is shown in the following formula (4).

[0122]

[0123] Understandably, the aforementioned state feature token contains key information about the initial audio input data and can be used for subsequent task processing.

[0124] Step 203c4: The electronic device transforms the intermediate vector into the first feature vector.

[0125] In this embodiment of the application, the electronic device transforms the above intermediate vector into a first feature vector through an embedding layer. The first feature vector is used to characterize the voiceprint features of the audio data predicted by the first voiceprint recognition model.

[0126] Understandably, embedding layers can map raw data from a high-dimensional space to a low-dimensional space, which helps reduce data complexity and computational resource requirements, while improving the training and inference efficiency of the model.

[0127] For example, the electronic device processes the aforementioned state representation token through the embedding layer to obtain the embedding vector emb_pred corresponding to the first audio data.

[0128] In this embodiment, the electronic device introduces an Attention module, which enables the training of the first voiceprint recognition model to obtain the trained second voiceprint recognition model, and can focus more on the first audio data with specific temporal meaning, thereby reducing the impact of blank parts and noise in the first audio data on the prediction of the voiceprint recognition model and improving the recognition effect of the voiceprint recognition model.

[0129] Step 204: The electronic device trains the first voiceprint recognition model based on the target voiceprint features and the first feature vector to obtain the second voiceprint recognition model.

[0130] Optionally, in the embodiments of this application, step 204 can be implemented by steps 204a and 204b as described below.

[0131] Step 204a: The electronic device calculates the loss value based on the target voiceprint features and the first feature vector.

[0132] For example, such as Figure 7 As shown, the electronic device uses one-hot encoding to encode the audio speaker identity SPK_ID obtained in step 201 to obtain emb_real, which is the target voiceprint feature, and uses a loss function, such as the additive angular margin loss (AAMLoss) function, to calculate the loss value between emb_real and emb_pred obtained in step 203c4.

[0133] Understandably, AAMLoss is a loss function commonly used in fields such as face recognition and voiceprint recognition. It aims to improve the model's discriminative ability by optimizing the angular relationship between embedding vectors. AAMLoss enhances intra-class compactness and inter-class separability by introducing an additive angular margin.

[0134] Step 204b: The electronic device updates the parameters in the first voiceprint recognition model based on the loss value to obtain the second voiceprint recognition model.

[0135] Optionally, in this embodiment of the application, the electronic device can calculate the gradient of each parameter in the first voiceprint recognition model based on the loss value and the gradient of the loss function through the backpropagation algorithm, and use the optimization algorithm to update the weights and bias parameters of the model according to the calculated gradient, thereby obtaining the trained second voiceprint recognition model.

[0136] This application provides a method for training a voiceprint recognition model. After acquiring first audio data, a first STFT feature information, including the time-domain and frequency-domain features of the first audio data, can be obtained based on the audio signal corresponding to the first audio data. Since the first STFT feature information is directly extracted from the audio signal, it retains detailed distribution information of the audio signal in time and frequency, and is closer to the audio signal corresponding to the first audio data, providing more information about the first audio data itself. During the training of the voiceprint recognition model, this first STFT feature information, which is closer to the audio signal and includes more comprehensive features of the first audio data, is used as input, enabling the first voiceprint recognition model to capture the unique voiceprint features of the first audio data more quickly and accurately for model training, resulting in a second voiceprint recognition model. Therefore, the trained second voiceprint recognition model can accurately perform voiceprint recognition, improving the voiceprint recognition effect of the voiceprint recognition model.

[0137] Optionally, in this embodiment of the application, after step 204 above, the training method of the voiceprint recognition model provided in this embodiment of the application further includes steps 205 to 207 as described below.

[0138] Step 205: The electronic device inputs the second audio data into the second voiceprint recognition model to obtain the audio signal of the second audio data.

[0139] For example, taking a mobile phone as an electronic device, the phone uses a second voiceprint recognition model to detect whether the voiceprint features of the audio uploaded by the user to be replicated are those of a sensitive person. Figure 8 As shown, the mobile phone displays the audio upload interface 60, where the user uploads the audio to be copied to the mobile phone, and the mobile phone inputs the audio to be copied into the second voiceprint recognition model to obtain the audio signal of the audio to be copied.

[0140] Step 206: The electronic device acquires the fourth STFT feature information based on the audio signal of the second audio data.

[0141] In this embodiment of the application, the fourth STFT feature information includes the time-domain features and frequency-domain features of the second audio data.

[0142] For example, the mobile phone obtains the corresponding fourth STFT feature information based on the audio signal of the audio to be replicated.

[0143] It should be noted that the specific scheme for the electronic device to obtain the fourth STFT feature information based on the audio signal of the second audio data can be found in the description of the specific scheme for obtaining the first STFT feature information above, and will not be repeated here in the embodiments of this application.

[0144] Step 207: The electronic device performs feature processing on the fourth STFT feature information and outputs the second feature vector.

[0145] In this embodiment of the application, the second feature vector represents the voiceprint features of the second audio data.

[0146] For example, the mobile phone performs feature processing on the fourth STFT feature information and outputs a second feature vector representing the voiceprint features of the audio to be replicated. The mobile phone compares this second feature vector with the voiceprint features of pre-set sensitive individuals and finds that the second feature vector is similar to the voiceprint features of the pre-set sensitive individuals. Figure 9 As shown, a prompt message 61 is displayed on the audio upload interface 60, such as "The audio you uploaded is similar to the pronunciation of a sensitive person. Please select a new audio to copy."

[0147] It should be noted that the specific scheme for the electronic device to perform feature processing on the fourth STFT feature information and output the second feature vector can be found in the description of the specific scheme for the first feature vector above, and will not be repeated here in the embodiments of this application.

[0148] In this embodiment, the electronic device can use the trained second voiceprint recognition model to perform voiceprint recognition on the second audio data, and can accurately output the second feature vector representing the voiceprint features of the second audio data, thereby improving the voiceprint recognition effect of the voiceprint recognition model.

[0149] The implementation process of the training method for the voiceprint recognition model provided in this application embodiment will be illustrated below through specific implementation methods. Figure 10 As shown, the implementation process of the training method for the voiceprint recognition model provided in this application embodiment may include the following steps 81 to 90.

[0150] Step 81: The mobile phone reads the original audio information to obtain the audio signal, audio sampling rate, and audio speaker number.

[0151] Step 82: The mobile phone obtains multiple audio signals of equal length through resampling, signal cropping, signal filling, etc., and performs Fourier transform on the audio signals to obtain the first STFT feature information.

[0152] Step 83: The mobile phone uses the SENet module to adjust the weights of each frequency feature in the first STFT feature information to obtain the second STFT feature information.

[0153] Step 84: The mobile phone uses the Res2Net module to group, convolutionally fuse, and combine the frequency features in the second STFT feature information to obtain the third STFT feature information.

[0154] Step 85: The mobile phone uses the Attention module to remove the influence of blank audio and background noise on the original audio data, and obtains the predicted voiceprint features of the original audio data, i.e., the first feature vector.

[0155] Step 86: The mobile phone obtains the target voiceprint features using the audio speaker number. Based on the target voiceprint features and the first feature vector mentioned above, the loss value is calculated, and the model parameters of the first voiceprint recognition model are adjusted according to the loss value to obtain the second voiceprint recognition model.

[0156] Step 87: The mobile phone uses the trained second voiceprint recognition model to detect whether the voiceprint features of the audio to be copied uploaded by the user are the voiceprint features of a sensitive person. The user then uploads the audio to be copied to the mobile phone.

[0157] Step 88: The mobile phone inputs the audio to be copied into the second voiceprint recognition model to obtain the audio signal of the audio to be copied.

[0158] Step 89: The mobile phone obtains the corresponding fourth STFT feature information based on the audio signal of the audio to be replicated.

[0159] Step 90: The mobile phone performs feature processing on the above-mentioned fourth STFT feature information and outputs a second feature vector representing the voiceprint features of the audio to be replicated.

[0160] In this embodiment, the electronic device uses the STFT features obtained after Fourier transform as the model input. Compared with the existing model that uses FBank features, the STFT features are closer to the original appearance of the audio waveform and can learn more audio features. At the same time, the electronic device reduces the impact of blank audio and background noise on voiceprint recognition through the Attention module, so that as many audio samples as possible can be used during model training, thereby improving the model performance.

[0161] It should be noted that the implementation process of steps 81 to 90 above is the same as the implementation process in the embodiment of the training method of the above voiceprint recognition model. For the parts not described in detail in the embodiments of this application, please refer to the relevant descriptions in the above embodiments. The embodiments of this application will not be repeated here.

[0162] The following detailed implementation method illustrates the data stream transmission in the training method of the voiceprint recognition model provided in this application. For example... Figure 11 As shown, Figure 11The training data is the first audio data, SIGNAL is the audio signal of the first audio data, B (Batch) is the batch size, representing the number of audio signals processed simultaneously, and T (Time Frames) is the number of time frames, representing the number of frames the audio signal is divided into. SIGNAL[B,T] represents a batch containing B audio signals, each signal being divided into T time frames. STFT is the aforementioned first STFT feature information, C is the number of channels or frequency bins, representing the number of frequency features. STFT[B,C,T] represents a batch containing B audio signals. For each audio signal, it is converted into a spectrogram or feature map through STFT. This spectrogram or feature map has C channels, each channel corresponding to a different frequency component. At the same time, this spectrogram or feature map is also divided into T time frames to represent the frequency characteristics of the audio signal in different time periods. Similarly, W_STFT represents the second STFT feature information mentioned above, N_STFT represents the third STFT feature information mentioned above, and token is an intermediate vector containing key information of the input first audio data. The meanings of W_STFT[B,C,T], N_STFT[B,C,T], and token[B,C] can be referred to STFT[B,C,T], and will not be elaborated here. SPK_ID represents the identity of the audio speaker. One-hot encoding of SPK_ID can yield emb_real, which is used to characterize the target voiceprint features. Processing the intermediate vector token through a linear layer can yield emb_pred, which is used to characterize the voiceprint features of the first audio data predicted by the first voiceprint recognition model. dim represents the dimension of the embedding vector. emb_pred[B,dim] represents a batch containing B audio signals, each of which is predicted by the first voiceprint recognition model as a dim-dimensional voiceprint feature vector. emb_real[B,dim] represents a batch containing B audio signals, each of which corresponds to a dim-dimensional voiceprint feature vector.

[0163] This application provides a voiceprint recognition method, which may include the following steps 301 to 303.

[0164] Step 301: The electronic device inputs the second audio data into the second voiceprint recognition model to obtain the audio signal of the second audio data.

[0165] In this embodiment of the application, the second voiceprint recognition model is trained based on STFT feature information, which includes the time domain features and frequency domain features of the audio data.

[0166] Step 302: The electronic device acquires the fourth STFT feature information based on the audio signal of the second audio data.

[0167] In this embodiment of the application, the fourth STFT feature information includes the time-domain features and frequency-domain features of the second audio data.

[0168] Step 303: The electronic device performs feature processing on the fourth STFT feature information and outputs the second feature vector.

[0169] In this embodiment of the application, the second feature vector represents the voiceprint features of the second audio data.

[0170] It should be noted that the specific implementation methods of steps 301 to 303 above can be found in the specific descriptions in the above embodiments, and will not be repeated here in the embodiments of this application.

[0171] Optionally, in the embodiments of this application, before step 301 above, the training method of the above voiceprint recognition model can also be executed, namely steps 201 to 204 above, and related schemes. Specific embodiments of this application will not be elaborated here.

[0172] This application provides a voiceprint recognition method. Second audio data is input into a second voiceprint recognition model to obtain the audio signal of the second audio data. Based on the audio signal, fourth STFT feature information, including the time-domain and frequency-domain features of the second audio data, is obtained. Feature processing is performed on the fourth STFT feature information to output a second feature vector representing the voiceprint features of the second audio data. In this scheme, since the second voiceprint recognition model is trained based on STFT feature information, which is directly extracted from the audio signal, it retains detailed distribution information of the audio signal in time and frequency, and is closer to the audio signal corresponding to the audio data, providing more information about the audio data itself. Therefore, the second voiceprint recognition model trained based on STFT feature information can output more accurate voiceprint features. In this application embodiment, inputting the second audio data into the second voiceprint recognition model can obtain accurate voiceprint recognition results for the second audio data, improving the voiceprint recognition effect.

[0173] It should be noted that each of the above method embodiments, or various possible implementations of each method embodiment, can be executed individually or in combination of any two or more. The specific implementation can be determined according to actual usage requirements, and some embodiments of this application do not impose such restrictions.

[0174] The voiceprint recognition model training method provided in this application can be executed by a voiceprint recognition model training device. This application uses the example of a voiceprint recognition model training device executing the voiceprint recognition model training method to illustrate the voiceprint recognition model training device provided in this application.

[0175] Figure 12 A schematic diagram of a possible structure of a training device for a voiceprint recognition model involved in some embodiments of this application is shown. For example... Figure 12 As shown, the training device 20 for the voiceprint recognition model may include an acquisition module 21 and a processing module 22. The acquisition module 21 is used to acquire audio information of first audio data, the audio information including an audio signal and target voiceprint features; and, based on the audio signal, acquire first STFT feature information, the first STFT feature information including time-domain features and frequency-domain features of the first audio data. The processing module 22 is used to perform feature processing on the first STFT feature information acquired by the acquisition module 21 to obtain a first feature vector, the first feature vector representing the voiceprint features of the first audio data; and, based on the target voiceprint features acquired by the acquisition module 21 and the first feature vector, train a first voiceprint recognition model to obtain a second voiceprint recognition model.

[0176] In one possible implementation, the first audio data includes multiple audio data segments, and the audio information also includes an audio sampling rate. Specifically, the acquisition module 21 is configured to, when the audio sampling rate is equal to the target sampling rate, if the lengths of the audio signals of the multiple audio data segments are unequal, perform signal processing on the audio signals of the multiple audio data segments, the signal processing including at least one of signal trimming and signal padding; and perform a Fourier transform on the processed audio signals of the multiple audio data segments to obtain the first STFT feature information.

[0177] In one possible implementation, the frequency domain features include at least one frequency feature. The processing module 22 is specifically configured to perform weight adjustment on each frequency feature in the first STFT feature information to obtain second STFT feature information; and to perform convolution processing on the frequency features in the second STFT feature information to obtain third STFT feature information; and to process the first audio data based on the third STFT feature information to obtain the first feature vector.

[0178] In one possible implementation, the processing module 22 is specifically used to perform global average pooling on each frequency feature in the first STFT feature information to obtain average STFT feature information; and to perform frequency adjustment processing on the average STFT feature information to obtain the weight value corresponding to each frequency feature; and to perform weighted processing on the frequency features in the first STFT feature information based on the weight value corresponding to each frequency feature to obtain the second STFT feature information.

[0179] In one possible implementation, the processing module 22 is specifically used to: group the frequency features in the second STFT feature information to obtain N groups of frequency features, where N is an integer greater than 1; and perform convolution processing on the first group of frequency features in the N groups of frequency features to obtain the first output feature; and perform convolution processing on the i-th group of frequency features and the (i-1)-th output feature in the N groups of frequency features to obtain the i-th output feature, where i∈[2,N] and i is an integer; and perform fusion processing on the N output features corresponding to the N groups of frequency features to obtain the third STFT feature information.

[0180] This application provides a training device for a voiceprint recognition model. After the training device acquires first audio data, it can obtain first STFT feature information, including the time-domain and frequency-domain features of the first audio data, based on the audio signal corresponding to the first audio data. Since the first STFT feature information is directly extracted from the audio signal, it retains detailed distribution information of the audio signal in time and frequency, and is closer to the audio signal corresponding to the first audio data, providing more information about the first audio data itself. During the training of the first voiceprint recognition model, the training device uses this first STFT feature information, which is closer to the audio signal and includes more comprehensive features of the first audio data, as input, enabling the first voiceprint recognition model to capture the unique voiceprint features of the first audio data more quickly and accurately for model training, thus obtaining a second voiceprint recognition model. Therefore, the trained second voiceprint recognition model can accurately perform voiceprint recognition, improving the voiceprint recognition effect of the voiceprint recognition model.

[0181] The training device for the voiceprint recognition model in this application embodiment can be an electronic device or a component within an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television (TV), ATM, or self-service machine, etc. This application embodiment does not specifically limit the specific device.

[0182] The training device for the voiceprint recognition model in this application embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit the specific operating system used.

[0183] The training device for the voiceprint recognition model provided in this application embodiment can realize the various processes implemented in the various embodiments of the above-described voiceprint recognition model training method. To avoid repetition, it will not be described again here.

[0184] Optionally, such as Figure 13 As shown, this application embodiment also provides an electronic device 1000, including a processor 1001 and a memory 1002. The memory 1002 stores a program or instructions that can run on the processor 1001. When the program or instructions are executed by the processor 1001, they implement the various steps of the various embodiments of the above-described voiceprint recognition model training method and can achieve the same technical effect. To avoid repetition, they will not be described again here.

[0185] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0186] Figure 14 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of this application.

[0187] The electronic device 100 includes, but is not limited to, components such as: radio frequency unit 101, network module 102, audio output unit 103, input unit 104, sensor 105, display unit 106, user input unit 107, interface unit 108, memory 109, and processor 110.

[0188] Those skilled in the art will understand that the electronic device 100 may also include a power supply (such as a battery) for supplying power to various components. The power supply may be logically connected to the processor 110 through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system. Figure 14 The electronic device structure shown does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0189] The processor 110 is configured to: acquire audio information of first audio data, the audio information including an audio signal and target voiceprint features; acquire first STFT feature information based on the audio signal, the first STFT feature information including time-domain features and frequency-domain features of the first audio data; perform feature processing on the first STFT feature information to obtain a first feature vector, the first feature vector representing the voiceprint features of the first audio data; and train a first voiceprint recognition model based on the target voiceprint features and the first feature vector to obtain a second voiceprint recognition model.

[0190] Optionally, the first audio data includes multiple audio data segments, and the audio information also includes an audio sampling rate. The processor 110 is further configured to, when the audio sampling rate is equal to the target sampling rate, if the lengths of the audio signals of the multiple audio data segments are unequal, perform signal processing on the audio signals of the multiple audio data segments, the signal processing including at least one of signal trimming and signal padding; and perform a Fourier transform on the signal-processed audio signals of the multiple audio data segments to obtain the first STFT feature information.

[0191] Optionally, the frequency domain features include at least one frequency feature. The processor 110 is specifically configured to perform weight adjustment on each frequency feature in the first STFT feature information to obtain second STFT feature information; and to perform convolution processing on the frequency features in the second STFT feature information to obtain third STFT feature information; and to process the first audio data based on the third STFT feature information to obtain the first feature vector.

[0192] Optionally, the processor 110 is specifically configured to perform global average pooling on each frequency feature in the first STFT feature information to obtain average STFT feature information; and to perform frequency adjustment processing on the average STFT feature information to obtain a weight value corresponding to each frequency feature; and to perform weighted processing on the frequency features in the first STFT feature information based on the weight value corresponding to each frequency feature to obtain the second STFT feature information.

[0193] Optionally, the processor 110 is specifically configured to group the frequency features in the second STFT feature information to obtain N groups of frequency features, where N is an integer greater than 1; and to perform convolution processing on the first group of frequency features in the N groups of frequency features to obtain the first output feature; and to perform convolution processing on the i-th group of frequency features and the (i-1)-th output feature in the N groups of frequency features to obtain the i-th output feature, where i∈[2,N] and i is an integer; and to perform fusion processing on the N output features corresponding to the N groups of frequency features to obtain the third STFT feature information.

[0194] This application provides an electronic device that, after acquiring first audio data, can obtain first STFT feature information, including the time-domain and frequency-domain features of the first audio data, based on the audio signal corresponding to the first audio data. Since the first STFT feature information is directly extracted from the audio signal, it retains detailed distribution information of the audio signal in time and frequency, and is closer to the audio signal corresponding to the first audio data, providing more information about the first audio data itself. During the training of the first voiceprint recognition model, the electronic device uses this first STFT feature information, which is closer to the audio signal and includes more comprehensive features of the first audio data, as input, enabling the first voiceprint recognition model to capture the unique voiceprint features of the first audio data more quickly and accurately for model training, thus obtaining a second voiceprint recognition model. Therefore, the trained second voiceprint recognition model can accurately perform voiceprint recognition, improving the voiceprint recognition effect of the voiceprint recognition model.

[0195] The electronic device provided in this application embodiment can implement all the processes implemented in the above method embodiments and achieve the same technical effects. To avoid repetition, it will not be described again here. The beneficial effects of the various implementation methods in this embodiment can be found in the beneficial effects of the corresponding implementation methods in the above method embodiments. To avoid repetition, it will not be described again here.

[0196] It should be understood that, in this embodiment, the input unit 104 may include a graphics processing unit (GPU) 1041 and a microphone 1042. The GPU 1041 processes image data of still images or videos obtained by an image capture device (such as a camera) in video capture mode or image capture mode. The display unit 106 may include a display panel 1061, which may be configured in the form of a liquid crystal display, an organic light-emitting diode, or the like. The user input unit 107 includes at least one of a touch panel 1071 and other input devices 1072. The touch panel 1071 is also called a touch screen. The touch panel 1071 may include a touch detection device and a touch controller. Other input devices 1072 may include, but are not limited to, a physical keyboard, function keys (such as volume control buttons, power buttons, etc.), a trackball, a mouse, and a joystick, which will not be described in detail here.

[0197] The memory 109 can be used to store software programs and various data. The memory 109 may primarily include a first storage area for storing programs or instructions and a second storage area for storing data. The first storage area may store the operating system, application programs or instructions required for at least one function (such as sound playback, image playback, etc.). Furthermore, the memory 109 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DRRAM). The memory 109 in the embodiments of this application includes, but is not limited to, these and any other suitable types of memory.

[0198] Processor 110 may include one or more processing units; optionally, processor 110 integrates an application processor and a modem processor, wherein the application processor mainly handles operations involving the operating system, user interface, and applications, and the modem processor mainly handles wireless communication signals, such as a baseband processor. It is understood that the aforementioned modem processor may also not be integrated into processor 110.

[0199] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described voiceprint recognition model training method embodiment and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0200] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0201] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above-described voiceprint recognition model training method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0202] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0203] This application provides a computer program product, which is stored in a storage medium and executed by at least one processor to implement the various processes of the training method embodiment of the voiceprint recognition model described above, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0204] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0205] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0206] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A training method for a voiceprint recognition model, characterized in that, include: Acquire audio information from the first audio data, the audio information including audio signal and target voiceprint features; Based on the audio signal, first short-time Fourier transform (STFT) feature information is obtained, the first STFT feature information including the time domain features and frequency domain features of the first audio data; The first STFT feature information is processed to obtain a first feature vector, which represents the voiceprint feature of the first audio data. Based on the target voiceprint features and the first feature vector, the first voiceprint recognition model is trained to obtain the second voiceprint recognition model. The frequency domain features include at least one frequency feature, and the feature processing of the first STFT feature information to obtain the first feature vector includes: The weights of each frequency feature in the first STFT feature information are adjusted to obtain the second STFT feature information. The frequency features in the second STFT feature information are convolved to obtain the third STFT feature information. Based on the third STFT feature information, the first audio data is processed to obtain the first feature vector; The step of adjusting the weights of each frequency feature in the first STFT feature information to obtain the second STFT feature information includes: Global average pooling is performed on each frequency feature in the first STFT feature information to obtain the average STFT feature information. The average STFT feature information is subjected to frequency adjustment processing to obtain the weight value corresponding to each frequency feature; Based on the weight value corresponding to each frequency feature, the frequency features in the first STFT feature information are weighted to obtain the second STFT feature information.

2. The method according to claim 1, characterized in that, The first audio data includes multiple audio data points, and the audio information also includes the audio sampling rate; The step of obtaining the first STFT feature information based on the audio signal includes: If the lengths of the audio signals of the multiple audio data are not equal when the audio sampling rate is equal to the target sampling rate, then signal processing is performed on the audio signals of the multiple audio data, and the signal processing includes at least one of signal trimming and signal padding. The audio signals of the multiple audio data after signal processing are subjected to Fourier transform to obtain the first STFT feature information.

3. The method according to claim 1, characterized in that, The process of convolving the frequency features in the second STFT feature information to obtain the third STFT feature information includes: The frequency features in the second STFT feature information are grouped to obtain N groups of frequency features, where N is an integer greater than 1; The first set of frequency features in the N sets of frequency features is convolved to obtain the first output feature; Convolution is performed on the i-th frequency feature and the (i-1)-th output feature in the N groups of frequency features to obtain the i-th output feature, where i ∈ [2, N] and i is an integer; The N output features corresponding to the N sets of frequency features are fused to obtain the third STFT feature information.

4. A training device for a voiceprint recognition model, characterized in that, include: Acquisition module and processing module; The acquisition module is used to acquire audio information of the first audio data, the audio information including audio signal and target voiceprint features; Furthermore, based on the audio signal, first STFT feature information is obtained, the first STFT feature information including the time domain features and frequency domain features of the first audio data; The processing module is used to perform feature processing on the first STFT feature information acquired by the acquisition module to obtain a first feature vector, wherein the first feature vector represents the voiceprint feature of the first audio data; and to train a first voiceprint recognition model based on the target voiceprint feature acquired by the acquisition module and the first feature vector to obtain a second voiceprint recognition model. The frequency domain features include at least one frequency feature. The processing module is specifically used to adjust the weight of each frequency feature in the first STFT feature information to obtain the second STFT feature information; and to perform convolution processing on the frequency features in the second STFT feature information to obtain the third STFT feature information; and to process the first audio data based on the third STFT feature information to obtain the first feature vector. The processing module is specifically configured to perform global average pooling on each frequency feature in the first STFT feature information to obtain average STFT feature information; and to perform frequency adjustment processing on the average STFT feature information to obtain a weight value corresponding to each frequency feature; and to perform weighted processing on the frequency features in the first STFT feature information based on the weight value corresponding to each frequency feature to obtain the second STFT feature information.

5. The apparatus according to claim 4, characterized in that, The first audio data includes multiple audio data points, and the audio information also includes the audio sampling rate; The acquisition module is specifically configured to, when the audio sampling rate is equal to the target sampling rate, if the lengths of the audio signals of the multiple audio data are not equal, perform signal processing on the audio signals of the multiple audio data, the signal processing including at least one of signal trimming and signal padding; and perform Fourier transform on the signal-processed audio signals of the multiple audio data to obtain the first STFT feature information.

6. The apparatus according to claim 4, characterized in that, The processing module is specifically used to group the frequency features in the second STFT feature information to obtain N groups of frequency features, where N is an integer greater than 1; and to perform convolution processing on the first group of frequency features in the N groups of frequency features to obtain the first output feature; and to perform convolution processing on the i-th group of frequency features and the (i-1)-th output feature in the N groups of frequency features to obtain the i-th output feature, where i∈[2,N] and i is an integer; and to perform fusion processing on the N output features corresponding to the N groups of frequency features to obtain the third STFT feature information.

Images