Method, device, server and storage medium for identifying sound-emitting objects

Abstract

This disclosure relates to a method, apparatus, server, and storage medium for identifying a voice-generating object. The method includes: extracting a first speech vector from first speech data of a voice-generating object to be identified, and extracting a second speech vector from second speech data of a target voice-generating object; inputting the first and second speech vectors into a voiceprint feature recognition model, and using the activation function of the hidden layer in the voiceprint feature recognition model to extract voiceprint features from the first and second speech vectors respectively, obtaining the first voiceprint feature of the voice-generating object to be identified and the second voiceprint feature of the target voice-generating object; calculating the similarity between the first and second voiceprint features; and if the similarity is greater than or equal to a similarity threshold, determining that the voice-generating object to be identified matches the target voice-generating object. This method can accurately determine whether the voice-generating object to be identified matches the target voice-generating object.

Description

Technical Field

[0001] This disclosure relates to the field of communication technology, and in particular to a method, apparatus, server and storage medium for identifying a sound-emitting object. Background Technology

[0002] With the development of mobile internet, various online platforms have developed rapidly, and a large number of users upload their audio and video content to these platforms. In some cases, online platforms need to determine the source of the audio or video.

[0003] Voice object recognition can determine whether a voice object is a registered target object based on the speech spoken by the voice object being tested. However, currently, due to the limited number of speech samples used for training, the accuracy and speed of voice object recognition are not high. Summary of the Invention

[0004] This disclosure provides a method, apparatus, server, and storage medium for identifying sound-emitting objects, thereby at least solving the problems of low accuracy and speed in identifying sound-emitting objects in related technologies. The technical solution of this disclosure is as follows:

[0005] According to a first aspect of the present disclosure, a method for identifying a voice-producing object is provided, comprising: extracting a first voice vector from first voice data of a voice-producing object to be identified, and extracting a second voice vector from second voice data of a target voice-producing object; inputting the first and second voice vectors into a voiceprint feature recognition model, and using activation functions of hidden layers in the voiceprint feature recognition model to extract voiceprint features from the first and second voice vectors respectively, thereby obtaining a first voiceprint feature of the voice-producing object to be identified and a second voiceprint feature of the target voice-producing object, wherein the voiceprint feature recognition model includes multiple cascaded hidden layers, the number of hidden layers and the number of hidden neurons in each hidden layer are determined according to the number of training samples; calculating the similarity between the first and second voiceprint features; and if the similarity is greater than or equal to a similarity threshold, determining that the voice-producing object to be identified matches the target voice-producing object.

[0006] According to a second aspect of the present disclosure, a device for recognizing a voice-producing object is provided, comprising: an extraction module configured to extract a first voice vector from first voice data of a voice-producing object to be recognized, and extract a second voice vector from second voice data of a target voice-producing object; an input module configured to input the first and second voice vectors into a voiceprint feature recognition model, and extract voiceprint features from the first and second voice vectors respectively using activation functions of hidden layers in the voiceprint feature recognition model to obtain a first voiceprint feature of the voice-producing object to be recognized and a second voiceprint feature of the target voice-producing object, wherein the voiceprint feature recognition model includes multiple cascaded hidden layers, the number of hidden layers and the number of hidden neurons in each hidden layer are determined according to the number of training samples; a calculation module configured to calculate the similarity between the first and second voiceprint features; and a matching module configured to determine that the voice-producing object to be recognized matches the target voice-producing object if the similarity is greater than or equal to a similarity threshold.

[0007] According to a third aspect of the present disclosure, a server is provided, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to execute instructions to implement the sound-emitting object recognition method as described in the first or second aspect.

[0008] According to a fourth aspect of the present disclosure, a storage medium is provided that, when instructions in the storage medium are executed by a server's processor, enables the server to perform the sound-emitting object recognition method as described in the first or second aspect.

[0009] According to a fifth aspect of the present disclosure, a computer program product is provided, which, when executed by a processor of a server, enables the server to perform the sound-emitting object recognition method as described in the first or second aspect.

[0010] The technical solutions provided by the embodiments of this disclosure bring at least the following beneficial effects:

[0011] In this embodiment, a first speech vector and a second speech vector extracted from the first speech data of the voice-producing object to be identified and the second speech data of the target voice-producing object are input into a voiceprint feature recognition model. The activation functions of the hidden layers in the voiceprint feature recognition model are used to extract voiceprint features from the first and second speech vectors, respectively, to obtain the first voiceprint feature of the voice-producing object to be identified and the second voiceprint feature of the target voice-producing object. Since the voiceprint feature recognition model includes multiple cascaded hidden layers, and the number of hidden layers and the number of hidden neurons in each hidden layer are determined based on the number of training samples, the voiceprint feature recognition model extracts voiceprint features quickly and accurately, accurately representing the voiceprint characteristics of the voice-producing object to be identified. Therefore, the first voiceprint feature accurately represents the characteristics of the voice-producing object to be identified, and the second voiceprint feature accurately represents the characteristics of the target voice-producing object. Thus, by determining the similarity between the first and second voiceprint features, it is possible to quickly and accurately determine whether the voice-producing object to be identified matches the target voice-producing object.

[0012] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0013] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure, and are not intended to unduly limit this disclosure.

[0014] Figure 1 This is a diagram illustrating a time-delay neural network structure according to an exemplary embodiment.

[0015] Figure 2 This is a schematic diagram illustrating the principle architecture of a sound-emitting object recognition according to an exemplary embodiment.

[0016] Figure 3 This is a schematic diagram illustrating an application environment of a sound-emitting object recognition method, apparatus, electronic device, and storage medium according to an exemplary embodiment.

[0017] Figure 4 This is a flowchart illustrating a method for recognizing a sound-producing object according to an exemplary embodiment.

[0018] Figure 5 This is a flowchart illustrating the extraction of voiceprint features according to an exemplary embodiment.

[0019] Figure 6 This is a schematic diagram illustrating a framing method according to an exemplary embodiment.

[0020] Figure 7This is a flowchart illustrating another training model for voiceprint feature recognition according to an exemplary embodiment.

[0021] Figure 8 This is a schematic diagram of a time-delay neural network according to an exemplary embodiment.

[0022] Figure 9 This is a flowchart illustrating a process for calculating voiceprint feature similarity according to an exemplary embodiment.

[0023] Figure 10 This is a schematic diagram illustrating a cosine similarity algorithm according to an exemplary embodiment.

[0024] Figure 11 This is a schematic diagram illustrating a voiceprint feature recognition scenario according to an exemplary embodiment.

[0025] Figure 12 This is a block diagram illustrating a sound-emitting object recognition device according to an exemplary embodiment.

[0026] Figure 13 This is a block diagram illustrating a server according to an exemplary embodiment.

[0027] Figure 14 This is a block diagram of a device for data processing according to an exemplary embodiment. Detailed Implementation

[0028] To enable those skilled in the art to better understand the technical solutions of this disclosure, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings.

[0029] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0030] As the scale of business on the network platform continues to expand, the amount of data that the network platform's servers need to process also increases. A single instance node on the server may no longer be able to complete the data processing of all business operations in a timely manner. Therefore, it is necessary to deploy multiple instance nodes on the server to achieve parallel processing of massive and continuously increasing data.

[0031] Before introducing the specific embodiments of this disclosure, the technical terms used in describing the embodiments of this disclosure will be introduced first.

[0032] First, let's introduce voiceprint recognition. Voiceprint recognition is a type of biometric technology, also known as speaker recognition, which includes speaker identification and speaker verification. Speaker verification is used to confirm whether a certain piece of speech was spoken by a specific person; it's a "one-to-one" identification problem. Voiceprint recognition converts sound signals into electrical signals, which are then used by a computer for identification. The voice target identification disclosed in this paper can be understood as speaker verification, that is, confirming whether the voice target is the target voice target.

[0033] Voiceprints, in particular, are the sound wave spectra carrying speech information displayed using electroacoustic instruments. Voiceprints are not only specific but also relatively stable. This is because the vocal organs used by individuals when speaking vary greatly in size and shape from person to person, resulting in differences in the voiceprint spectra of any two people.

[0034] The characteristics of vocalization are mainly reflected in the following aspects: (1) Resonance mode characteristics: pharyngeal resonance, nasal resonance and oral resonance; (2) Voice purity characteristics: the purity of different people's voices is generally different, and can be roughly divided into three levels: high purity (bright), low purity (hoarse) and medium purity; (3) Average pitch characteristics: the high or low of the average pitch is what is generally referred to as whether the voice is high-pitched or low-pitched; (4) Vocal range characteristics: the high or low of the vocal range is what is generally referred to as whether the voice is full or dry.

[0035] Due to the aforementioned vocal characteristics, the distribution of formants in the spectrograms of different people's voices is different. Voiceprint recognition determines whether the speakers of two audio segments are the same person by comparing their pronunciation of the same phonemes, thus achieving the function of "identifying people by their voices".

[0036] Next, we introduce the Time-Delay Neural Network (TDNN). The output of each hidden layer is extended in the time domain, meaning that the input received by each hidden layer is not only the output of the previous layer at the current moment, but also the output of the previous layer at some moments before and after. TDNN also extends the output of each hidden layer during network propagation.

[0037] A time-delay neural network is a neural network that takes input information from an input layer and contains multiple frames, such as... Figure 1 As shown, Figure 1This paper illustrates a time-delay neural network (TDNN) structure. Assuming a delay of 2, three consecutive frames are considered. The hidden layers are used for feature extraction. Each rectangle in the input layer contains 13 small circles, representing the 13-dimensional Mel-scale frequency cepstral coefficients (MFCC) features of that frame. The first hidden layer has 256 hidden neurons, resulting in a total of 3 * 13 * 256 = 9984 weights. The small amount of weight data required for determination leads to fast feature extraction. Due to this characteristic, TDNNs are widely used in feature extraction applications.

[0038] However, the inventors discovered through research that the existing TDNN network structure is not effective in extracting voiceprint features when there are few voice data samples. The extracted voiceprint features are prone to overfitting, which leads to low accuracy in subsequent identification of the voice source.

[0039] Based on this, the present disclosure provides a method for recognizing a voice-producing object. A first speech vector and a second speech vector, extracted from the first speech data of the voice-producing object to be identified and the second speech data of the target voice-producing object, are input into a voiceprint feature recognition model. The activation function of the hidden layer in the voiceprint feature recognition model is used to extract voiceprint features from the first and second speech vectors, respectively, to obtain the first voiceprint feature of the voice-producing object to be identified and the second voiceprint feature of the target voice-producing object. Since the voiceprint feature recognition model extracts voiceprint features quickly and accurately, and can accurately represent the voiceprint characteristics of the voice-producing object to be identified, by determining the similarity between the first and second voiceprint features, it is possible to quickly and accurately determine whether the voice-producing object to be identified matches the target voice-producing object.

[0040] It should be noted that the embodiments of this disclosure provide a principle architecture 200 for recognizing a sound-emitting object, such as... Figure 2 As shown:

[0041] First, regarding the training process, a voiceprint feature recognition model is trained based on multiple first training samples. Each first training sample includes a third speech vector 212 of a first voice-speaking object. The voiceprint feature recognition model 214 extracts a third voiceprint feature 216 from the third speech vector 212 and trains the voiceprint feature recognition model 214 based on the third voiceprint feature 216 and its corresponding target identification information 218.

[0042] Then, in the part involving the voice object recognition component, the first speech vector 2221 of the voice object to be recognized and the second speech vector 2222 of the target voice object are obtained; the first speech vector and the second speech vector are input into the trained voiceprint feature recognition model 224 to determine the first voiceprint feature 2261 of the voice object to be recognized and the second voiceprint feature 2262 of the target voice object, and the first similarity 228 between the first voiceprint feature and the second voiceprint feature is calculated; if the first similarity is greater than or equal to the first similarity threshold, it is determined that the voice object to be recognized matches the target voice object, that is, the recognition result is obtained.

[0043] Depend on Figure 2 It is known that a pre-trained voiceprint feature recognition model is needed to extract the first voiceprint feature of the voice-producing object to be identified and the second voiceprint feature of the target voice-producing object. Therefore, the voiceprint feature recognition model needs to be trained before using it for feature extraction.

[0044] like Figure 3 The diagram shown is an application environment illustration of the sound-emitting object recognition device, electronic device, and storage medium provided in one or more embodiments of this disclosure. Figure 3 As shown, server 100 communicates with one or more user terminals 200 via network 300 for data communication or interaction. Server 100 may be a web server, database server, etc. User terminals 200 may be, but are not limited to, personal computers (PCs), smartphones, tablets, personal digital assistants (PDAs), etc. Network 300 may be wired or wireless.

[0045] The following will provide a detailed description of the sound-emitting object recognition provided in the embodiments of this disclosure.

[0046] The voice-generating object recognition provided in this disclosure can be applied to the user terminal 200. For ease of description, unless otherwise specified, this disclosure describes the user terminal 200 as the execution subject. It is understood that the execution subject does not constitute a limitation of this disclosure.

[0047] The following describes, with reference to the accompanying drawings, a specific implementation of the speech object recognition method provided in this disclosure.

[0048] like Figure 4 The diagram shown is a flowchart of a method for recognizing a sound-producing object provided in an embodiment of this disclosure. The method includes:

[0049] S410, extract a first speech vector from the first speech data of the target speech object, and extract a second speech vector from the second speech data of the target speech object.

[0050] S420, the first speech vector and the second speech vector are input into the voiceprint feature recognition model. The activation function of the hidden layer in the voiceprint feature recognition model is used to extract voiceprint features from the first speech vector and the second speech vector respectively, so as to obtain the first voiceprint feature of the voice-producing object to be identified and the second voiceprint feature of the target voice-producing object.

[0051] S430, calculate the similarity between the first voiceprint feature and the second voiceprint feature.

[0052] S440, if the similarity is greater than or equal to the similarity threshold, then it is determined that the voice object to be identified matches the target voice object.

[0053] In this embodiment, a first speech vector and a second speech vector extracted from the first speech data of the voice-producing object to be identified and the second speech data of the target voice-producing object are input into a voiceprint feature recognition model. The activation function of the hidden layer in the voiceprint feature recognition model is used to extract voiceprint features from the first speech vector and the second speech vector respectively, thereby obtaining the first voiceprint feature of the voice-producing object to be identified and the second voiceprint feature of the target voice-producing object. Since the voiceprint feature recognition model extracts voiceprint features quickly and accurately, it can accurately represent the voiceprint characteristics of the voice-producing object to be identified. Therefore, by determining the similarity between the first voiceprint feature and the second voiceprint feature, it is possible to quickly and accurately determine whether the voice-producing object to be identified matches the target voice-producing object.

[0054] The specific implementation methods for each of the above steps are described below.

[0055] First, let's introduce the S410.

[0056] Extract a first speech vector from the first speech data of the target speech object, and extract a second speech vector from the second speech data of the target speech object.

[0057] As one implementation of this disclosure, in order to improve the stability of the speech vector, the above-mentioned step of extracting the first speech vector from the first speech data of the speech object to be identified may specifically include the following steps: determining the first speech signal based on the audio frame corresponding to the first speech data and a preset window function; performing a fast Fourier transform on the first speech signal to obtain the spectrum signal of the audio frame; filtering the spectrum signal to obtain the filtered spectrum signal; and performing a discrete cosine transform on the filtered spectrum signal to obtain the first speech vector.

[0058] First, in the aforementioned step of determining the first speech signal based on the audio frames corresponding to the first speech data and a preset window function, the audio signal corresponding to each audio frame can be multiplied by a smooth window function to form windowed speech, i.e., the first speech signal. This allows the audio frames to smoothly decay to zero at both ends, making the audio signal more continuous and enabling better short-time Fourier transforms to obtain higher-quality spectra. Specifically, a window function can be selected for each frame; the width of the window function is the frame length. Commonly used window functions include rectangular windows, Hamming windows, and Gaussian windows.

[0059] Secondly, in the aforementioned step of performing a Fast Fourier Transform (FFT) on the first speech signal to obtain the spectral signal of the audio frame, a FFT can be performed on the first speech signal to obtain the spectral signal of the audio / video frame, that is, to extract the discrete frequency band spectral signal from the first speech signal. Then, the spectral signal is filtered to obtain the filtered spectral signal.

[0060] Finally, in the aforementioned step of performing a discrete cosine transform on the filtered spectral signal to obtain the fifth speech vector, the resulting first speech vector (cephalic coefficients) can be obtained through discrete cosine transform. The cepstrum can be viewed as the logarithmic spectral signal of the spectral signal; that is, the spectrum transforms the time-domain signal into a frequency-domain signal, while the cepstrum transforms the frequency-domain signal back into a time-domain signal. In terms of waveform, the cepstrum and the spectrum have similar waveforms; if the spectrum has a peak at low frequencies, the cepstrum also has a peak at low cepstrum coefficients, and vice versa. The advantage of cepstrum coefficients is that the changes in their different coefficients are uncorrelated, thus significantly reducing the number of parameters required for subsequent model training.

[0061] In this way, the above processing can reduce the impact of the equipment used to collect the speech signal and environmental variables during the acquisition process on the signal, making the signal more uniform and smooth, and providing a more stable signal source (i.e., the fifth speech vector) for subsequent extraction of voiceprint features.

[0062] Therefore, the following will first be combined with the appendix Figure 5 This document describes a specific implementation method for extracting MFCCs provided in the embodiments of this disclosure.

[0063] S321, Input audio signal (i.e., first voice data).

[0064] S322 performs pre-emphasis processing on the audio signal to obtain the pre-emphasized audio signal.

[0065] Because speech signals have relatively high energy in the low-frequency range, and this energy is mainly distributed in the low-frequency band, the power spectral density of speech decreases as the frequency increases, leading to weakened high-frequency transmission and affecting signal quality. Therefore, it is necessary to pre-emphasize the audio signal, that is, to boost the high-frequency components. By emphasizing the high-frequency parts of the speech information, the influence of lip radiation can be removed, and the high-frequency resolution of the speech can be increased, making the signal spectrum flatter.

[0066] S323, determine the first speech signal based on the pre-emphasized audio signal and the preset window function.

[0067] This step can include two steps: framing and windowing. This is because framing and windowing are actually continuous operations. In real life, audio signals are generally non-stationary signals, but they can be considered stationary over a certain period of time; that is, audio signals have short-term stationarity (for example, the speech signal can be considered approximately constant within 10-30ms). Therefore, the pre-emphasized audio signal can be divided into short segments for processing, which is called framing.

[0068] like Figure 6 As shown, there is a repetitive part between frame t and frame t+1. This repetitive part is called frame shift. By creating an overlapping area (frame shift) between two adjacent frames, we can avoid missing signal points between adjacent frames.

[0069] Optionally, a Hamming window can be used as the preset window function. We multiply each audio frame by the Hamming window, which increases the continuity of the left and right sides of the audio frame, which is called windowing.

[0070] Specifically, the framed audio signal is S(n), where n = N, and N is a natural number. The Hamming window is W(n). Based on the framed audio signal S(n) and the Hamming window W(n), the first speech signal S'(n) can be calculated using the following formula: S'(n) = S(n) * W(n). Windowing enhances the continuity between the left and right ends of the frame, thus making the overall audio signal more continuous.

[0071] S324 performs a short-time Fourier transform on the first speech signal to obtain the spectrum signal.

[0072] The short-time Fourier transform (STFT) is a mathematical transform related to the Fourier transform, used to determine the frequency and phase of a sinusoidal wave in a local region of a time-varying signal. After the above windowing process, the first audio signal of each frame needs to undergo another short-time Fourier transform to obtain the energy distribution in the spectrum. In this way, the characteristics of the signal can be observed simply and intuitively through the energy distribution.

[0073] S325 performs modulo operation on the spectrum signal to obtain the modulo-operated spectrum signal.

[0074] Modulus calculation refers to the positive square root of the sum of the squares of the real and imaginary parts of a complex number, which is called the modulus of the complex number.

[0075] S326 performs filtering on the modulo-calculated spectrum signal to obtain the filtered spectrum signal.

[0076] After undergoing a short-time Fourier transform, the signal is passed through a set of Mel filters to obtain the Mel spectrum. The Mel spectrum accurately reflects the auditory characteristics of the human ear, which is to convert ordinary frequencies into Mel frequencies.

[0077] S327 performs logarithmic processing on the filtered spectrum signal to obtain the logarithm-processed spectrum signal.

[0078] Since the signals emitted by the human vocal system are formed by the convolution of pitch information and vocal tract information, denoted as "s convolution v", after the above Fourier transform, convolution becomes multiplication, i.e., "STFT(s)*STFT(v)". Taking the logarithm transforms multiplication into addition, i.e., "Log(FFT(s))+Log(FFT(v))". Converting the convolution signal into an additive signal is the reason for taking STFT and the logarithm.

[0079] S328 performs a discrete cosine transform on the logarithmically processed spectral signal to obtain the first speech vector.

[0080] The Discrete Cosine Transform (DCT) is a transform related to the Fourier Transform. It is similar to the Discrete Fourier Transform but uses only real numbers. This completes the preprocessing of the speech information, generating the fifth speech vector.

[0081] S329, output the first speech vector.

[0082] This completes the process of extracting the fifth speech vector (MFCC) from the third speech data, and removing the silence interval from the fifth speech vector to obtain the third speech vector (MFCC after VAD processing).

[0083] The MFCC determined through the above process can accurately represent the spectral features in the speech data. Next, the third voiceprint feature can be extracted from the third speech vector, and the third voiceprint feature and its corresponding target identification information can be used to train the model to obtain the voiceprint feature recognition model involved in this disclosure, namely mini-TDNN.

[0084] Because mini-TDNN has fewer layers and fewer hidden neurons, it can extract features quickly. Even when the number of audio frames corresponding to the third speech vector is small, it can effectively extract the third voiceprint features from the third speech vector.

[0085] The method for extracting the second speech vector from the second speech data of the target speaker can be derived in the same way, and will not be elaborated here.

[0086] Next, let's introduce the S420.

[0087] In some embodiments of this disclosure, the following steps may be included prior to S420:

[0088] Obtain the third speech vector of the first speaker and its corresponding target identification information; determine the fourth speech vector based on the preset time delay parameter and the third speech vector, where the preset time delay parameter is the time delay parameter of the voiceprint feature recognition model, and each third speech vector corresponds to one frame of audio; determine the target speech vector based on the third and fourth speech vectors; input the target speech vector into the voiceprint feature recognition model so that the first hidden layer of the voiceprint feature recognition model uses an activation function to extract voiceprint features from the target speech vector, thereby obtaining the third voiceprint feature; train the voiceprint feature recognition model based on the third voiceprint feature and its corresponding target identification information.

[0089] The aforementioned third voice data can be voice data obtained by the server from video or audio files. When obtaining voice data, it is necessary to also obtain the identification information of the voice object corresponding to the voice data. For example, if second voice data A records voice data including the voice of voice object A, and second voice data B records voice data including the voice of voice object B, then the second voice data of voice object A and its corresponding target identification information "A" are obtained together, as well as the second voice data of voice object B and its corresponding target identification information "B".

[0090] Thirdly, speech data is a sound signal. The time-domain waveform of sound only represents the relationship between sound pressure and time, and cannot well represent the characteristics of the sound. Therefore, it is necessary to convert the sound signal into an acoustic feature vector. Acoustic feature vectors can include, for example, Mel-frequency cepstral coefficients and linear predictive cepstral coefficients.

[0091] Optionally, MFCC is used as the third speech vector. To facilitate explanation of the meaning of MFCC, before introducing MFCC, we first introduce Mel frequencies. Mel frequencies are proposed based on the characteristics of human hearing and have a non-linear relationship with frequency. MFCC utilizes this relationship to calculate the spectral characteristics. The human auditory system is a special non-linear system, and its sensitivity to signals of different frequencies varies. The human auditory system can extract the speaker's personal characteristics. MFCC takes human hearing into account, first mapping the linear spectrum to the Mel non-linear spectrum based on auditory perception, and then converting it to the cepstrum. In this way, by simulating the characteristics of human auditory perception processing, the speech recognition rate can be improved.

[0092] As one implementation of this disclosure, in order to eliminate the influence of silent intervals and improve the accuracy of voiceprint feature extraction, the following steps may be included before the steps involving obtaining the third speech vector of the first speaker and its corresponding target identification information:

[0093] The fifth speech vector is determined based on the third speech data; speech endpoint detection is performed on the fifth speech vector to obtain the target endpoint of the fifth speech vector; the silence interval vector in the fifth speech vector is determined based on the target endpoint of the fifth speech vector; the silence interval vector is removed from the fifth speech vector to obtain the third speech vector.

[0094] Voice Activity Detection (VAD) is used to accurately locate the start and end points of speech in noisy speech, as speech often contains long silences; in other words, it separates silences from the actual speech. A fifth speech vector is determined based on the third speech data. Voice activity detection is then performed on this fifth speech vector to obtain its target endpoints. For example, if the third speech data includes multiple silence points (i.e., target endpoints), these silence points form silence intervals. These silence intervals need to be removed from the original speech data; that is, the silence interval vector in the fifth speech vector is determined based on the target endpoints of the fifth speech vector. The silence interval vector is then removed from the fifth speech vector to obtain the third speech vector.

[0095] In this way, by accurately locating the start and end points (target endpoints) of speech from speech with silence or noise, the speech with silence or noise can be separated from the actual speech, the silent or noisy parts can be removed, and the third speech vector without silence or noise can be retained.

[0096] The following is combined with Figure 7This document describes a specific implementation method provided in this disclosure, which utilizes the activation function of the hidden layer in a voiceprint feature recognition model to extract voiceprint features from a first speech vector and a second speech vector, respectively, to obtain the first voiceprint feature of the voice-producing object to be identified and the second voiceprint feature of the target voice-producing object.

[0097] like Figure 7 As shown, the specific steps may include the following:

[0098] S332, determine the fourth speech vector based on the preset delay parameter and the third speech vector. The preset delay parameter is the delay parameter of the voiceprint feature recognition model. Each third speech vector corresponds to one frame of audio.

[0099] Specifically, the step of determining the fourth speech vector based on the preset time delay parameter and the third speech vector may include the following steps:

[0100] Based on the preset delay parameters, at least one audio frame is determined that is separated from the audio frame corresponding to the third speech vector by a preset delay; the speech vector corresponding to the at least one audio frame is determined as the fourth speech vector.

[0101] Specifically, at least one audio frame with a preset time delay is determined based on a preset time delay parameter, and the audio frame corresponding to the third speech vector is determined as the fourth speech vector.

[0102] For example, if the third speech vector corresponds to the audio of frame t and the preset delay parameters are +1, +2, -1, and -2, then the fourth speech vector may include the speech vectors corresponding to the audio of frame t-2, frame t-1, frame t+1, and frame t+2, respectively.

[0103] S334, determine the target speech vector based on the third speech vector and the fourth speech vector.

[0104] Accordingly, the target speech vector can be a speech vector that is concatenated with the third and fourth speech vectors, that is, the speech vectors corresponding to the audio of frame t-2, frame t-1, frame t, frame t+1, and frame t+2, respectively.

[0105] S336, The target speech vector is input into the voiceprint feature recognition model, so that the first hidden layer of the voiceprint feature recognition model uses the activation function to extract voiceprint features from the target speech vector and obtain the first extraction result.

[0106] The input to the voiceprint feature recognition model is the target speech vector (such as MFCC) extracted from the audio of a preset number of frames. The voiceprint feature recognition model takes a fixed number of speech vectors from the audio, with one frame of speech vector corresponding to one frame of audio.

[0107] The activation function can be a rectified linear unit (ReLU), also known as a modified linear unit. ReLU sets the output of some hidden neurons to 0, thus making the neural network sparser and reducing the interdependence of parameters. In other words, ReLU increases the nonlinear relationship between the hidden layers of the neural network, alleviating the overfitting problem. The sparse model achieved through ReLU can better extract relevant features and fit the training data.

[0108] Optionally, in order to make the input of each hidden layer have the same distribution as much as possible, BatchNorm can be used for further processing during the processing of the target speech vector using the ReLU activation function.

[0109] For each hidden layer neuron, BatchNorm can guide the input value of the nonlinear transformation function into a region that is more sensitive to the input, thus avoiding the vanishing gradient problem. The vanishing gradient problem refers to the situation where, when the gradient is less than 1, the error between the predicted and true values ​​decays with each propagation layer, causing the model to stall on convergence. Because BatchNorm consistently maintains relatively large gradients in the neural network, it is highly efficient at adjusting the network parameters, improving training speed and accelerating convergence.

[0110] This not only speeds up the convergence of the model but also alleviates the gradient vanishing problem in deep networks, making the trained deep network model more stable.

[0111] S338, the first extraction result is used as the input of the next hidden layer of the first hidden layer, and the activation function of the next hidden layer is used to extract the voiceprint features of the first extraction result to obtain the second extraction result, and so on, until the preset hidden layer of the voiceprint feature recognition model is reached, and the preset hidden layer outputs the third voiceprint feature corresponding to the third speech vector.

[0112] The first hidden layer includes a first number of hidden neurons, and the preset hidden layer includes a second number of hidden neurons.

[0113] The first extraction result output from the first hidden layer is used as the input to the next hidden layer (the second hidden layer). The activation function of the second hidden layer is used to extract the voiceprint features from the first extraction result to obtain the second extraction result. The second extraction result output from the second hidden layer is then used as the input to the third hidden layer, and so on, until the preset hidden layer of the mini-TDNN is reached. The preset hidden layer outputs the second voiceprint feature corresponding to the second speech vector. Each hidden layer has a corresponding preset number of hidden neurons.

[0114] Among them, the following is combined with the appendix Figure 8 This disclosure describes a specific implementation method for extracting voiceprint features from speech vectors using a voiceprint feature recognition model.

[0115] First, the MFCC corresponding to an audio frame of a preset duration is used as the input to the voiceprint feature recognition model. Second, the first hidden layer extracts a first extraction result from the MFCC corresponding to (t-2, t-1, t, t+1, t+2) and inputs this first extraction result into the second hidden layer. Next, the second hidden layer extracts the output of the first extraction result at (t-2, t, t+2) as the second extraction result and inputs this second extraction result into the third hidden layer.

[0116] Then, the third hidden layer extracts its output at (t-3, t, t+3) from the second extraction result, and uses this as the third extraction result, which is then input into the fourth hidden layer. Next, the fourth hidden layer extracts its output at t from the third extraction result, and uses this as the fourth extraction result, which is then input into the fifth hidden layer. Finally, the fifth hidden layer extracts its output at t from the fourth extraction result, and uses this as the fifth extraction result.

[0117] Next, the sixth hidden layer extracts its output at point t from the fifth extraction result, and uses this as the sixth extraction result. Finally, the seventh hidden layer extracts its output at point t from the sixth extraction result, and uses this as the seventh extraction result. The first, second, third, and fourth hidden layers each have 256 hidden neurons, the fifth hidden layer has 750 hidden neurons, and the sixth and seventh hidden layers each have 512 hidden neurons.

[0118] Among them, because the voiceprint feature recognition model has fewer layers and fewer hidden neurons, the feature extraction speed is relatively fast. When the number of audio frames corresponding to the third speech vector is small, it can effectively extract the third voiceprint feature from the third speech vector.

[0119] S340, Based on the third voiceprint feature and its corresponding target identification information, train the voiceprint feature recognition model to obtain the voiceprint feature recognition model.

[0120] Optionally, the output of the sixth hidden layer can be selected as the third voiceprint feature.

[0121] Optionally, a deep neural network can be used, specifically a DNN. A DNN model is a deep learning framework model whose structure mainly includes: one input layer, multiple hidden layers, and one output layer. Generally, the first layer of a DNN model is the input layer, the last layer is the output layer, and the middle layers are multiple hidden layers. The voiceprint feature recognition model inputs the extracted second voiceprint features into the input layer of the DNN. The DNN adjusts its training parameters based on the first identifier information and target identifier information determined by the third voiceprint features until the training stopping condition is met, thus obtaining the voiceprint feature recognition model. The parameters of the DNN model adjusted in this process mainly include the weights of the linear transformations connecting the layers of the DNN model.

[0122] Understandable Figure 8 The process shown can be used to represent two stages.

[0123] The first stage is the training of the neural network. Since the neural network is a combination of a feature extractor and a classifier, each layer has a very strong feature extraction capability. Therefore, the output of the sixth hidden layer is used as the embeddings (i.e., feature vectors) of the input audio frames.

[0124] The second stage is feature extraction. Here, we can remove the pre-trained neural network (i.e., DNN) and use the remaining structure to derive the embeddings for each segment of speech. Embedding uses a low-dimensional vector to represent information. The property of embedding vectors is that vectors that are close in distance have similar meanings.

[0125] Optionally, x-vector is used to describe the second voiceprint features (i.e., embeddings) extracted from the second speech vector using mini-TDNN.

[0126] like Figure 8 As shown, the output of the sixth hidden layer of the voiceprint feature recognition model can be used as the xvector feature of the embedding.

[0127] Before obtaining the voiceprint feature recognition model involved in this disclosure, namely mini-TDNN, in order to improve the extraction accuracy of mini-TDNN, such as Figure 8 As shown, a neural network (such as a DNN) can be added to the last hidden layer of the mini-TDNN so that the output of the mini-TDNN serves as the input to the DNN. A network including both the mini-TDNN and the DNN can be used as the aforementioned voiceprint feature recognition model. After the voiceprint feature recognition model is trained, the trained DNN can be removed, leaving the mini-TDNN with high-precision feature extraction capabilities, capable of extracting voiceprint features from speech vectors.

[0128] Next, let's introduce the S430.

[0129] In some embodiments of this disclosure, the average value of multiple third voiceprint features of the first voice-emitting object is calculated to determine the fourth voiceprint feature; the fourth voiceprint feature is preprocessed to determine the calculation parameters of the PLDA algorithm; the PLDA algorithm is adjusted using the calculation parameters to determine the first PLDA algorithm; and the similarity between the first voiceprint feature and the second voiceprint feature is calculated using the first PLDA algorithm.

[0130] Before introducing the computational parameters of the Probabilistic Linear Discriminant Analysis (PLDA) algorithm, let's first explain its principle. PLDA is a channel compensation algorithm. The presence of channel information interferes with speaker recognition, even severely impacting the system's accuracy. Therefore, it's necessary to minimize this impact, i.e., to perform channel compensation on voiceprint features.

[0131] Channel compensation is specifically manifested in the following process: In the field of voiceprint recognition, suppose the training data speech consists of the speech of I speakers, where each speaker has J distinct speech segments. Then, we can define the j-th speech segment of the i-th speaker as Xij. Then, according to factor analysis, the generation model of Xij is defined as: xij = u + Fhi + Gwij + εij. This algorithm can be viewed as two parts. The first two terms on the right-hand side are only related to the speaker and not to any specific speech segment of the speaker; this is called the signal part, which describes the differences between speakers. The last two terms on the right-hand side describe the differences between different speech segments of the same speaker; this is called the noise part. Thus, the data structure of a speech segment can be described using two hypothetical variables. Assume the two hypothetical variables are matrices F and G. These matrices F and G contain the basic factors in their respective hypothetical variable spaces, which can be considered as eigenvectors of their respective spaces. For example, each column of F is equivalent to the eigenvector of the inter-class space, and each column of G is equivalent to the eigenvector of the intra-class space. The two vectors can be seen as feature representations in their respective spaces. For example, hi can be seen as the feature representation of Xij in the speaker space.

[0132] Thus, in the process of calculating the first similarity between the first voiceprint feature and the second voiceprint feature using the PLDA algorithm, if the likelihood of identical features is greater, then it can be determined that the other voiceprint object matches the target voiceprint object.

[0133] In this embodiment, the first similarity between the first voiceprint feature and the second voiceprint feature can be calculated using the PLDA algorithm. If the first similarity is greater than or equal to a first similarity threshold, it is determined that the voiceprint object to be identified matches the target voiceprint object, thus confirming that the first voiceprint feature belongs to the target voiceprint object. If the first similarity is less than the first similarity threshold, it is determined that the voiceprint object to be identified does not match the target voiceprint object, thus confirming that the first voiceprint feature does not belong to the target voiceprint object. Here, PLDA is used as the channel compensation algorithm, which has better channel compensation capabilities, thereby improving the reliability of the voiceprint recognition results.

[0134] The steps mentioned above, which involve preprocessing the fourth voiceprint feature to determine the calculation parameters of the PLDA algorithm, may specifically include the following steps:

[0135] Linear discriminant analysis is performed on the fourth voiceprint feature to determine the fifth voiceprint feature; mean normalization is performed on the fifth voiceprint feature to determine the sixth voiceprint feature; length normalization is performed on the sixth voiceprint feature to obtain the seventh voiceprint feature; and the calculation parameters of the PLDA algorithm are determined based on the seventh voiceprint feature.

[0136] First, the aforementioned step involving linear discriminant analysis (LDA) processing of the fourth voiceprint feature to determine the fifth voiceprint feature can specifically be: performing LDA processing on the first voiceprint feature. LDA can be understood as a dimensionality reduction method that tries to remove unnecessary classification directions, i.e., maximizing inter-class distance and minimizing intra-class distance. In speaker recognition, most cases are binary classification problems, so the role of LDA here is to reduce the original high-dimensional feature data to one dimension. When a speaker has many voices, if these voices are affected by the channel, then the variance of this speaker's voices will be very large. Then, LDA attempts to find a new direction to project all the original data into this direction, so that the data of the same speaker in this direction has the minimum intra-class variance, while maximizing the distance between different speakers. In this way, the effect of reducing channel differences is achieved.

[0137] When LDA is used to process the test data (first voiceprint feature) and the x-vector (target voiceprint feature) of the voiceprint feature recognition model, channel compensation can be performed on the first voiceprint feature and the target voiceprint feature, thereby improving the accuracy of subsequent voice object recognition results.

[0138] Secondly, the steps mentioned above involving mean normalization of the fifth voiceprint feature to determine the sixth voiceprint feature can specifically be: mean normalization of the fifth voiceprint feature to determine the sixth voiceprint feature.

[0139] Since the magnitude of the fifth voiceprint feature may differ, and the magnitude of the first voiceprint feature differs greatly, it is necessary to perform mean normalization on the original index data.

[0140] Specifically, the standardization process can be performed using the following formula: x = x - mean(x) / max(x) - min(x). This mean normalization process converts the original data into dimensionless index evaluation values ​​(i.e., the sixth voiceprint feature), ensuring they are as close to the same order of magnitude as possible. This further guarantees the accuracy of calculating the similarity between voiceprint features and the reliability of the object recognition results.

[0141] Then, in the aforementioned step of normalizing the length of the sixth voiceprint feature to obtain the seventh voiceprint feature, the specific steps can be as follows: If the length of the sixth voiceprint feature is greater than the first length, then determine the difference between the length of the sixth voiceprint feature and the first length, remove the difference portion from the length of the sixth voiceprint feature, so that the length of the sixth voiceprint feature after length normalization is equal to the first length. If the length of the sixth voiceprint feature is less than the first length, then determine the difference between the length of the first length and the length of the sixth voiceprint feature, and fill in the missing difference portion.

[0142] For example, if the length of the sixth voiceprint feature is 20s, it can be reduced to 10s. If the length of the sixth voiceprint feature is 5s, a blank voiceprint feature can be added to make its length equal to 10s, or the original voiceprint can be copied and added to the sixth voiceprint feature to make its length equal to 10s.

[0143] Finally, the steps mentioned above regarding determining the calculation parameters of the PLDA algorithm based on the seventh voiceprint feature can be specifically as follows: the seventh voiceprint feature obtained through linear discriminant analysis, mean normalization, and length normalization has features suitable for similarity calculation. Based on the length and variance parameters of the seventh voiceprint feature, the voiceprint features subsequently input into the PLDA algorithm can be normalized to facilitate uniformity of magnitude and calculation of similarity between voiceprint features.

[0144] Specifically, the step involving calculating the first similarity between the first and second voiceprint features using the Probabilistic Linear Discriminant Analysis (PLDA) algorithm can include the following steps: adjusting the PLDA algorithm using calculation parameters to determine the first PLDA algorithm; and calculating the first similarity between the first and second voiceprint features using the first PLDA algorithm. The overall process described above can be found in [reference needed]. Figure 9 .

[0145] Finally, let's introduce the S440.

[0146] If the similarity is greater than or equal to the similarity threshold, then the voice to be identified is determined to match the target voice. Matching the voice to be identified with the target voice can be used to indicate that the voice to be identified and the target voice are the same voice.

[0147] Therefore, by inputting the first speech vector (first MFCC) and the second speech vector (second MFCC) extracted from the first speech data of the voice-producing object to be identified and the second speech data of the target voice-producing object into the voiceprint feature recognition model, the first voiceprint feature of the voice-producing object to be identified and the second voiceprint feature of the target voice-producing object are determined. By using PLDA to score the similarity between the first voiceprint feature of the voice-producing object to be identified and the second voiceprint feature of the target object, a faster processing speed and a lower equal error rate (EER) can be achieved.

[0148] It should be noted that EER is the value when the False Rejection Rate (FRR) is equal to the False Acceptance Rate (FAR). At this point, the values ​​of FAR and FRR are called equal error rates.

[0149] In classification problems, if two samples are of the same class (the same person) but are mistakenly identified as different classes (not the same person) by the system, this is a false rejection. The FRR (False Rejection Rate) is the proportion of false rejections among all matching samples of the same class. Conversely, if two samples are of different classes (not the same person) but are mistakenly identified as the same class (the same person) by the system, this is a false acceptance. The FAR (False Acceptance Rate) is the proportion of false acceptances among all matching samples of different classes.

[0150] The higher the accuracy required by a voiceprint recognition model, the lower the false acceptance rate, but the higher the false rejection rate may be. Conversely, if the voiceprint recognition model prioritizes usability, i.e., a high pass rate, the false acceptance rate will be higher, and the false rejection rate will be lower. Here, the lower the error rate, the better the model's performance.

[0151] In addition, this disclosure also provides a specific implementation method for determining the similarity between a first voiceprint feature and a second voiceprint feature. Specifically, it includes the following steps:

[0152] The similarity between the first and second voiceprint features is calculated using the cosine similarity algorithm.

[0153] For ease of explanation, the following will be combined with... Figure 10The cosine similarity algorithm is introduced below. A schematic diagram of a cosine similarity algorithm provided in this embodiment of the invention is shown in the figure. First, the angle θ between vectors a and b is calculated, and then the cosine value cosθ corresponding to the angle θ is obtained. This cosine value can be used to characterize the similarity between the two vectors. The smaller the angle, the closer the cosine value is to 1, and the more similar vectors a and b are.

[0154] If the cosine similarity (e.g., 0.8) reaches the second similarity threshold (e.g., 0.7), then the voice to be identified matches the target voice. If the cosine similarity (e.g., 0.6) does not reach the first similarity threshold (e.g., 0.7), then the voice to be identified does not match the target voice.

[0155] Since the cosine similarity algorithm distinguishes the differences between different voiceprint features in terms of direction, it can correct the problem of inconsistent measurement standards between different voiceprint features and improve the reliability of voiceprint recognition results.

[0156] In addition to using the PLDA algorithm and cosine similarity algorithm described above to determine the voiceprint recognition result, Euclidean distance, Manhattan distance, or Pearson correlation coefficient can also be used for similarity detection.

[0157] Therefore, the first and second speech vectors extracted from the first speech data of the voice-producing object to be identified and the second speech data of the target voice-producing object are input into the voiceprint feature recognition model to determine the first voiceprint feature of the voice-producing object to be identified and the second voiceprint feature of the target voice-producing object. Since the voiceprint feature recognition model is obtained by training a mini-TDNN based on multiple first training samples, the mini-TDNN extracts voiceprint features quickly and accurately, and can accurately represent the voiceprint characteristics of the voice-producing object to be identified. Therefore, the first voiceprint feature can accurately represent the characteristics of the voice-producing object to be identified, and the second voiceprint feature can accurately represent the characteristics of the target voice-producing object. Therefore, by determining the similarity between the first and second voiceprint features, it is possible to quickly and accurately determine whether the voice-producing object to be identified matches the target voice-producing object.

[0158] The following section will illustrate the speech object recognition method provided in this publication using several specific scenarios.

[0159] First, the voice-speaker identification method disclosed herein can be applied to the field of online platforms. With the development of mobile internet, various online platforms are developing rapidly, and a large number of users upload their audio and video content to these platforms. In some cases, online platforms need to determine the voice-speaker corresponding to the audio and video content.

[0160] For example, if the title or description of audio or video content on an online platform lacks keywords related to the target speaker, it becomes extremely difficult for both users and administrators to find the relevant audio or video content among the vast amount of available resources. Figure 11 As shown, network platform users or administrators hope to find audio and video data including the voice data of singer "Jackson Wang" from the massive amount of audio and video data on the network platform.

[0161] First, we acquire the first voice data of multiple objects (including "Jackson Wang," Xiao M, Xiao P, and Xiao Q) and the second voice data of "Jackson Wang" pre-stored on the server. Then, we extract the first voice vector from the multiple first voice data and the second voice vector of "Jackson Wang" from the second voice data. We input these first and second voice vectors into the voiceprint feature recognition model defined above to determine multiple first voiceprint features and the second voiceprint feature of "Jackson Wang." Finally, we calculate the first similarity between each first voiceprint feature to be identified and the second voiceprint feature of "Jackson Wang" using the PLDA algorithm, and identify the voicer corresponding to the first voice vector with the highest similarity value as "Jackson Wang."

[0162] Therefore, by using the voice object recognition method, it is possible to find the audio and video corresponding to the target voice object from the massive amount of audio and video on the network platform. This not only provides a better user experience for network platform users, but also makes it easier for network platform administrators to manage the massive amount of data on the network platform.

[0163] Secondly, the voice-based identification method disclosed herein can be applied to the security field. In recent years, due to the development of the Internet, voice-related crimes have increased significantly. In these cases, the voice-based identification method can effectively serve as a technical means of investigation. For example, fraudster A frequently calls elderly people to defraud them of their property.

[0164] At this point, the first step is to acquire the suspect's first voice data and the fraudster A's second voice data. Then, the suspect's first voice vector is extracted from the first voice data, and the fraudster A's second voice vector is extracted from the second voice data. These first and second voice vectors are input into the voiceprint feature recognition model defined above to determine the suspect's first voiceprint feature and the fraudster A's second voiceprint feature. Finally, the first similarity between the first and second voiceprint features is calculated using the PLDA algorithm. If the first similarity is greater than or equal to a first similarity threshold, it can be basically determined that the suspect is indeed fraudster A.

[0165] Therefore, by using methods to identify the voice source, it is possible to quickly and accurately prevent crimes such as telecommunications fraud and build a safe social environment.

[0166] Finally, the voice-based object recognition method disclosed herein can be applied to smart hardware interaction. In recent years, due to the rapid development of smart homes, a large number of smart hardware products have emerged. However, many smart products can only recognize the content spoken by the user, but cannot distinguish the speaker's identity, thus failing to meet the user's personalized needs. For example, smart product B can extract valid instructions from the speech of the voice-based object. However, in multi-person scenarios, the instruction-receiving ability of smart products will decrease. In this case, the voice-based object recognition method can be used to solve this problem.

[0167] For example, in a scenario where homeowner C has other guests at home, and the desired outcome is for smart product B to only respond to homeowner C's voice messages and perform corresponding operations, the following steps can be taken: First, multiple sets of first voice data from various individuals (including homeowner C and guests D, E, and F) and second voice data pre-stored by homeowner C can be acquired. Then, first voice vectors are extracted from the multiple sets of first voice data, and second voice vectors from the second set of second voice data are extracted from homeowner C. These first and second voice vectors are then input into the voiceprint feature recognition model defined above to determine multiple first voiceprint features and homeowner C's second voiceprint features. Finally, the first similarity between each first voiceprint feature and homeowner C's second voiceprint feature is calculated using the PLDA algorithm. The voicer corresponding to the first voice vector with the highest similarity value is identified as homeowner C.

[0168] Therefore, by using voice recognition methods, smart products can distinguish different roles and achieve "recognizing people by their voices." This allows smart products to provide different content and services tailored to each individual, making human-computer interaction simpler and providing users with a more relaxed and personalized experience.

[0169] Based on the above-mentioned method for identifying the sound source, this disclosure also provides a device for identifying the sound source, specifically combined with... Figure 12 Please provide an explanation.

[0170] Figure 12 This is a block diagram illustrating a sound-emitting object recognition device according to an exemplary embodiment. (Refer to...) Figure 12 The sound-emitting object recognition device 1200 may include an extraction module 1210, an input module 1220, a calculation module 1230, and a matching module 1240.

[0171] Extraction module 1210 is configured to extract a first speech vector from the first speech data of the target speech object and extract a second speech vector from the second speech data of the target speech object.

[0172] The input module 1220 is configured to input the first speech vector and the second speech vector into the voiceprint feature recognition model, and use the activation function of the hidden layer in the voiceprint feature recognition model to extract voiceprint features from the first speech vector and the second speech vector respectively, so as to obtain the first voiceprint feature of the voice-producing object to be identified and the second voiceprint feature of the target voice-producing object. The voiceprint feature recognition model includes multiple cascaded hidden layers, and the number of hidden layers and the number of hidden neurons in each hidden layer are determined according to the number of training samples.

[0173] The calculation module 1230 is configured to perform calculations on the similarity between the first voiceprint feature and the second voiceprint feature.

[0174] The matching module 1240 is configured to determine that the voice object to be identified matches the target voice object if the similarity is greater than or equal to the similarity threshold.

[0175] In some embodiments of this disclosure, the sound-emitting object recognition device 1200 further includes:

[0176] The acquisition module is configured to acquire the third speech vector of the first vocal object and its corresponding target identification information.

[0177] The determination module is configured to determine the fourth speech vector based on a preset delay parameter and the third speech vector. The preset delay parameter is the delay parameter of the voiceprint feature recognition model, and each third speech vector corresponds to one frame of audio.

[0178] The determination module is also configured to perform the determination of the target speech vector based on the third speech vector and the fourth speech vector.

[0179] The input module is configured to input the target speech vector into the voiceprint feature recognition model, so that the first hidden layer of the voiceprint feature recognition model uses an activation function to extract voiceprint features from the target speech vector to obtain the third voiceprint feature.

[0180] The training module is configured to train a voiceprint feature recognition model based on the third voiceprint feature and its corresponding target identifier information.

[0181] In some embodiments of this disclosure, the determining module is further configured to perform the determination of at least one audio frame that is separated from the audio frame corresponding to the third speech vector by a preset time delay, based on a preset time delay parameter.

[0182] The determination module is further configured to determine the speech vector corresponding to at least one audio frame as the fourth speech vector.

[0183] In some embodiments of this disclosure, the extraction module 1210 includes:

[0184] The determination module is configured to execute a first speech signal based on the audio frame corresponding to the first speech data and a preset window function.

[0185] The first transformation module is configured to perform a fast Fourier transform on the first speech signal to obtain the spectral signal of the audio frame.

[0186] The filtering module is configured to perform filtering processing on the spectral signal to obtain the filtered spectral signal.

[0187] The second transformation module is configured to perform a discrete cosine transform on the filtered spectral signal to obtain the first speech vector.

[0188] In some embodiments of this disclosure, the calculation module 1230 is further configured to perform an average calculation on multiple third voiceprint features of the first voice-emitting object to determine a fourth voiceprint feature.

[0189] The computing module 1230 also includes:

[0190] The preprocessing module is configured to perform preprocessing on the fourth voiceprint feature to determine the computational parameters of the PLDA algorithm.

[0191] The adjustment module is configured to perform PLDA algorithm adjustment using computational parameters to determine the first PLDA algorithm.

[0192] The calculation module 1230 is further configured to perform the calculation of the similarity between the first voiceprint feature and the second voiceprint feature using the first PLDA algorithm.

[0193] In some embodiments of this disclosure, the preprocessing module mentioned above includes:

[0194] The determination module is configured to perform linear discriminant analysis on the fourth voiceprint feature to determine the fifth voiceprint feature.

[0195] The normalization module is configured to perform mean normalization on the fifth voiceprint feature to determine the sixth voiceprint feature.

[0196] The normalization module is also configured to perform length normalization on the sixth voiceprint feature to obtain the seventh voiceprint feature.

[0197] The determination module is also configured to perform calculations based on the seventh voiceprint feature to determine the computational parameters of the PLDA algorithm.

[0198] In some embodiments of this disclosure, the calculation module 1230 is further configured to perform calculation of the similarity between the first voiceprint feature and the second voiceprint feature using a cosine similarity algorithm.

[0199] In this embodiment, the voice-generating object recognition device 1300 can input the first and second speech vectors extracted from the first speech data of the voice-generating object to be identified and the second speech data of the target voice-generating object into a voiceprint feature recognition model. The activation functions of the hidden layers in the voiceprint feature recognition model are used to extract voiceprint features from the first and second speech vectors respectively, resulting in the first voiceprint feature of the voice-generating object to be identified and the second voiceprint feature of the target voice-generating object. Since the voiceprint feature recognition model includes multiple cascaded hidden layers, and the number of hidden layers and the number of hidden neurons in each hidden layer are determined based on the number of training samples, the voiceprint feature recognition model extracts voiceprint features quickly and accurately, accurately representing the voiceprint characteristics of the voice-generating object to be identified. Therefore, the first voiceprint feature accurately represents the characteristics of the voice-generating object to be identified, and the second voiceprint feature accurately represents the characteristics of the target voice-generating object. Thus, by determining the similarity between the first and second voiceprint features, it is possible to quickly and accurately determine whether the voice-generating object to be identified matches the target voice-generating object.

[0200] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0201] Figure 13 This is a block diagram illustrating a server according to an exemplary embodiment. (Refer to...) Figure 13 This disclosure also provides a server, including a processor 1310, a communication interface 1320, a memory 1330, and a communication bus 1340, wherein the processor 1310, the communication interface 1320, and the memory 1330 communicate with each other through the communication bus 1340.

[0202] The memory 1330 is used to store instructions that can be executed by the processor 1310.

[0203] When the processor 1310 executes the instructions stored in the memory 1330, it performs the following steps:

[0204] A first speech vector is extracted from the first speech data of the voice-producing object to be identified, and a second speech vector is extracted from the second speech data of the target voice-producing object. The first and second speech vectors are input into a voiceprint feature recognition model. The activation functions of the hidden layers in the voiceprint feature recognition model are used to extract voiceprint features from the first and second speech vectors respectively, so as to obtain the first voiceprint feature of the voice-producing object to be identified and the second voiceprint feature of the target voice-producing object. The voiceprint feature recognition model includes multiple cascaded hidden layers. The number of hidden layers and the number of hidden neurons in each hidden layer are determined according to the number of training samples. The similarity between the first and second voiceprint features is calculated. If the similarity is greater than or equal to the similarity threshold, the voice-producing object to be identified is determined to match the target voice-producing object.

[0205] As can be seen, by applying the embodiments of this disclosure, the first and second speech vectors extracted from the first speech data of the voice-producing object to be identified and the second speech data of the target voice-producing object can be input into the voiceprint feature recognition model. The activation functions of the hidden layers in the voiceprint feature recognition model are used to extract voiceprint features from the first and second speech vectors respectively, resulting in the first voiceprint feature of the voice-producing object to be identified and the second voiceprint feature of the target voice-producing object. Since the voiceprint feature recognition model includes multiple cascaded hidden layers, and the number of hidden layers and the number of hidden neurons in each hidden layer are determined based on the number of training samples, the voiceprint feature recognition model extracts voiceprint features quickly and accurately, accurately representing the voiceprint characteristics of the voice-producing object to be identified. Therefore, the first voiceprint feature accurately represents the characteristics of the voice-producing object to be identified, and the second voiceprint feature accurately represents the characteristics of the target voice-producing object. Thus, by determining the similarity between the first and second voiceprint features, it is possible to quickly and accurately determine whether the voice-producing object to be identified matches the target voice-producing object.

[0206] Figure 14 This is a block diagram illustrating a device for data processing according to an exemplary embodiment. For example, the device 1400 may be provided as a server. (Refer to...) Figure 14 Server 1400 includes processing component 1422, which further includes one or more processors, and memory resources represented by memory 1432 for storing instructions, such as applications, that can be executed by processing component 1422. The applications stored in memory 1432 may include one or more modules, each corresponding to a set of instructions. Furthermore, processing component 1422 is configured to execute instructions to perform the speech object recognition method described in any of the above embodiments.

[0207] The device 1400 may also include a power supply component 1426 configured to perform power management of the device 1400, a wired or wireless network interface 1450 configured to connect the device 1400 to a network, and an input / output (I / O) interface 1458. The device 1400 can operate on an operating system stored in memory 1432, such as Windows Server™, MacOS X™, Unix™, Linux™, FreeBSD™, or similar.

[0208] In some embodiments of this disclosure, a storage medium is also provided, which, when the instructions in the storage medium are executed by the processor of a server, enables the server to perform the sound-emitting object recognition method described in any of the above embodiments.

[0209] Optionally, the storage medium may be a non-transitory computer-readable storage medium, for example, a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device.

[0210] In some embodiments of this disclosure, a computer program product is also provided, which, when executed by the processor of a server, enables the server to perform the sound-emitting object recognition method described in any of the above embodiments.

[0211] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0212] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A method for recognizing a sound-producing object, characterized in that, include: Extract a first speech vector from the first speech data of the target speech object, and extract a second speech vector from the second speech data of the target speech object; The first speech vector and the second speech vector are input into the voiceprint feature recognition model. The activation function of the hidden layer in the voiceprint feature recognition model is used to extract voiceprint features from the first speech vector and the second speech vector respectively, so as to obtain the first voiceprint feature of the voice-producing object to be identified and the second voiceprint feature of the target voice-producing object. The voiceprint feature recognition model includes multiple cascaded hidden layers. The number of hidden layers and the number of hidden neurons in each hidden layer are determined according to the number of training samples. Calculate the similarity between the first voiceprint feature and the second voiceprint feature; If the similarity is greater than or equal to the similarity threshold, then it is determined that the voice-speaking object to be identified matches the target voice-speaking object; The voiceprint feature recognition model is a mini-TDNN; Before inputting the first speech vector and the second speech vector into the voiceprint feature recognition model, the method further includes: obtaining a third speech vector of a first voice-producing object and its corresponding target identification information; determining a fourth speech vector based on a preset delay parameter and the third speech vector; determining a target speech vector based on the third speech vector and the fourth speech vector, wherein the preset delay parameter is the delay parameter of the mini-TDNN, and each third speech vector corresponds to one frame of audio; inputting the target speech vector into the voiceprint feature recognition model, so that the first hidden layer of the voiceprint feature recognition model uses an activation function to extract voiceprint features from the target speech vector to obtain a first extraction result; using the first extraction result as the input of the next hidden layer of the first hidden layer, and using the activation function of the next hidden layer to extract voiceprint features from the first extraction result to obtain a second extraction result, and so on, until the preset hidden layer of the voiceprint feature recognition model is reached, and the preset hidden layer outputs a third voiceprint feature corresponding to the third speech vector; training the voiceprint feature recognition model based on the third voiceprint feature and its corresponding target identification information.

2. The method according to claim 1, characterized in that, The step of determining the fourth speech vector based on the preset delay parameter and the third speech vector includes: Based on the preset delay parameter, at least one audio frame is determined that is separated from the audio frame corresponding to the third speech vector by a preset delay. The speech vector corresponding to the at least one audio frame is determined as the fourth speech vector.

3. The method according to claim 1, characterized in that, The step of extracting the first speech vector from the first speech data of the speech object to be identified includes: The first speech signal is determined based on the audio frame corresponding to the first speech data and the preset window function; Perform a Fast Fourier Transform on the first speech signal to obtain the spectral signal of the audio frame; The spectrum signal is filtered to obtain the filtered spectrum signal; The first speech vector is obtained by performing a discrete cosine transform on the filtered spectral signal.

4. The method according to claim 1, characterized in that, The calculation of the similarity between the first voiceprint feature and the second voiceprint feature includes: The fourth voiceprint feature is determined by averaging the multiple third voiceprint features of the first voice-emitting object. The fourth voiceprint feature is preprocessed to determine the calculation parameters of the Probabilistic Linear Discriminant Analysis (PLDA) algorithm; The PLDA algorithm is adjusted using the calculation parameters to determine the first PLDA algorithm; The similarity between the first voiceprint feature and the second voiceprint feature is calculated using the first PLDA algorithm.

5. The method according to claim 4, characterized in that, The preprocessing of the fourth voiceprint feature to determine the calculation parameters of the Probabilistic Linear Discriminant Analysis (PLDA) algorithm includes: Linear discriminant analysis is performed on the fourth voiceprint feature to determine the fifth voiceprint feature; The fifth voiceprint feature is normalized to determine the sixth voiceprint feature; The sixth voiceprint feature is length-normalized to obtain the seventh voiceprint feature; The calculation parameters of the PLDA algorithm are determined based on the seventh voiceprint feature.

6. The method according to claim 1, characterized in that, The calculation of the similarity between the first voiceprint feature and the second voiceprint feature includes: The similarity between the first and second voiceprint features is calculated using the cosine similarity algorithm.

7. A device for identifying a sound-emitting object, characterized in that, include: The extraction module is configured to extract a first speech vector from the first speech data of the target speech object and extract a second speech vector from the second speech data of the target speech object. The input module is configured to input the first speech vector and the second speech vector into the voiceprint feature recognition model, and extract voiceprint features from the first speech vector and the second speech vector using the activation function of the hidden layer in the voiceprint feature recognition model, respectively, to obtain the first voiceprint feature of the voice-producing object to be identified and the second voiceprint feature of the target voice-producing object. The voiceprint feature recognition model includes multiple cascaded hidden layers, and the number of hidden layers and the number of hidden neurons in each hidden layer are determined according to the number of training samples. The calculation module is configured to perform a calculation of the similarity between the first voiceprint feature and the second voiceprint feature; The matching module is configured to determine that the voice object to be identified matches the target voice object if the similarity is greater than or equal to a similarity threshold. The voiceprint feature recognition model is a mini-TDNN; The sound-emitting object recognition device further includes: The acquisition module is configured to acquire the third voice vector of the first voice object and its corresponding target identification information before inputting the first voice vector and the second voice vector into the voiceprint feature recognition model; The determination module is configured to determine a fourth speech vector based on a preset delay parameter and the third speech vector; The determining module is further configured to determine a target speech vector based on the third speech vector and the fourth speech vector, wherein the preset delay parameter is the delay parameter of the mini-TDNN, and each third speech vector corresponds to one frame of audio. The input module is configured to input the target speech vector into the voiceprint feature recognition model, so that the first hidden layer of the voiceprint feature recognition model uses an activation function to extract voiceprint features from the target speech vector to obtain a first extraction result; the first extraction result is used as the input of the next hidden layer of the first hidden layer, and the activation function of the next hidden layer is used to extract voiceprint features from the first extraction result to obtain a second extraction result, and so on, until a preset hidden layer of the voiceprint feature recognition model is reached, and the preset hidden layer outputs a third voiceprint feature corresponding to the third speech vector; The training module is configured to train the voiceprint feature recognition model based on the third voiceprint feature and its corresponding target identification information.

8. The apparatus according to claim 7, characterized in that, The determining module is further configured to perform the function of determining at least one audio frame that is separated from the audio frame corresponding to the third speech vector by a preset time delay according to the preset time delay parameter. The determining module is further configured to determine the speech vector corresponding to the at least one audio frame as the fourth speech vector.

9. The apparatus according to claim 7, characterized in that, The extraction module includes: The determination module is configured to execute a first speech signal based on the audio frame corresponding to the first speech data and a preset window function; The first transformation module is configured to perform a fast Fourier transform on the first speech signal to obtain the spectral signal of the audio frame; The filtering module is configured to perform filtering processing on the spectrum signal to obtain a filtered spectrum signal; The second transformation module is configured to perform a discrete cosine transform on the filtered spectral signal to obtain the first speech vector.

10. The apparatus according to claim 7, characterized in that, The calculation module is further configured to perform an average calculation on multiple third voiceprint features of the first voice-emitting object to determine the fourth voiceprint feature; The computing module also includes: The preprocessing module is configured to perform preprocessing on the fourth voiceprint feature to determine the calculation parameters of the PLDA algorithm; The adjustment module is configured to adjust the PLDA algorithm using the calculation parameters to determine a first PLDA algorithm; The calculation module is further configured to perform calculation of the similarity between the first voiceprint feature and the second voiceprint feature using the first PLDA algorithm.

11. The apparatus according to claim 10, characterized in that, The preprocessing module includes: The determination module is configured to perform linear discriminant analysis on the fourth voiceprint feature to determine the fifth voiceprint feature; The normalization module is configured to perform mean normalization on the fifth voiceprint feature to determine the sixth voiceprint feature; The normalization module is also configured to perform length normalization processing on the sixth voiceprint feature to obtain the seventh voiceprint feature; The determining module is further configured to perform calculations based on the seventh voiceprint feature to determine the computational parameters of the PLDA algorithm.

12. The apparatus according to claim 7, characterized in that, The calculation module is further configured to perform a cosine similarity algorithm to calculate the similarity between the first voiceprint feature and the second voiceprint feature.

13. A server, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the instructions to implement the sound-emitting object recognition method as described in any one of claims 1 to 6.

14. A storage medium, characterized in that, When the instructions in the storage medium are executed by the server's processor, the server is able to perform the sound-emitting object recognition method as described in any one of claims 1 to 6.

Images