Voice traceability determination method and device, electronic equipment and storage medium

Abstract

The invention provides a voice traceability determination method and apparatus, an electronic device and a storage medium. The method comprises the steps of obtaining a to-be-detected voice signal; extracting voice features of the voice signal; generating model fingerprints according to the voice features, wherein the model fingerprints are feature sets used for distinguishing voice sources of different objects; performing clustering analysis on the model fingerprint to obtain a clustering result; and determining the voice traceability information of the voice signal according to the clustering result. In the embodiment of the invention, the model fingerprints are generated based on the extracted voice features, so that the system can capture fine voice features from massive voice features and perform clustering analysis on the model fingerprints to realize accurate voice classification and traceability, thereby improving the accuracy of voice traceability determination.

Description

Technical Field

[0001] The present invention relates to the field of speech processing technology, and in particular to a method, device, electronic device and computer-readable storage medium for determining speech source. Background Art

[0002] With the development of speech synthesis technology, generated speech has been widely used in various application scenarios, such as intelligent customer service, voice assistants, virtual character dialogues, etc. These technologies have greatly improved the naturalness of human-computer interaction and user experience.

[0003] In the related technologies, existing generated speech tracing technologies often have problems of low accuracy and low efficiency, and cannot meet the needs of actual applications. In particular, in multi-speaker scenarios, related technologies find it difficult to effectively distinguish different individual voices, thereby reducing the accuracy of determining the generated speech source. Summary of the invention

[0004] The present invention provides a method, device, electronic device and computer-readable storage medium for determining speech source, so as to at least solve the problem in the related art that the speech source determination accuracy is low due to the inability to effectively distinguish different individual voices in a multi-person speaking scene. The technical solution of the present invention is as follows:

[0005] According to a first aspect of an embodiment of the present invention, a method for determining a speech source is provided, comprising:

[0006] Acquire a speech signal to be detected;

[0007] Extracting speech features of the speech signal;

[0008] Generate a model fingerprint according to the speech feature, wherein the model fingerprint is a feature set used to distinguish speech sources of different objects;

[0009] Performing cluster analysis on the model fingerprint to obtain a clustering result;

[0010] The speech source tracing information of the speech signal is determined according to the clustering result.

[0011] Optionally, the method further includes:

[0012] Preprocessing the acquired speech signal to obtain a preprocessed speech signal;

[0013] The extracting the speech feature of the speech signal includes: extracting the speech feature of the speech signal after preprocessing.

[0014] Optionally, extracting speech features of the speech signal after preprocessing includes:

[0015] The speech features of the pre-processed speech signal are extracted using Mel-frequency cepstral coefficients and linear predictive coding.

[0016] Optionally, the method further includes:

[0017] The extracted speech features are subjected to feature vector normalization processing to obtain processed features.

[0018] Optionally, generating a model fingerprint according to the speech feature includes:

[0019] The speech features are extracted using a convolutional neural network to generate a model fingerprint, wherein the structure of the convolutional neural network includes, in sequence: an input layer, a first convolutional layer, a first maximum pooling layer, a second convolutional layer, a second maximum pooling layer, a first fully connected layer, a dropout layer, a second fully connected layer, and an output layer.

[0020] Optionally, the extracting the speech features by using a convolutional neural network to generate a model fingerprint includes:

[0021] The convolutional neural network is used to perform two convolutions and maximum pooling on the input speech features, and the obtained feature vectors are fully connected, discarded, and fully connected to output the model fingerprint.

[0022] Optionally, performing cluster analysis on the model fingerprint to obtain a clustering result includes:

[0023] The K-means algorithm is used to perform cluster analysis on the model fingerprint to obtain a clustering result.

[0024] Optionally, determining the speech source tracing information of the speech signal according to the clustering result includes:

[0025] Performing feature matching between the clustering result and the model fingerprint in the model fingerprint library;

[0026] The speech source information of the speech signal is determined according to the feature matching result.

[0027] Optionally, performing feature matching on the clustering result and the model fingerprint in the model fingerprint library includes:

[0028] Calculating the cosine similarity between the clustering results and the model fingerprints in the model fingerprint library;

[0029] When the cosine similarity is greater than or equal to a set threshold, determining that the clustering result matches the characteristics of the model fingerprint in the model fingerprint library;

[0030] The vector with the highest cosine similarity that matches the feature is selected as the matching result.

[0031] Optionally, the method further includes:

[0032] The determined voice source tracing information of the voice signal is displayed.

[0033] Optionally, the method further includes:

[0034] The model fingerprints in the model fingerprint library are updated regularly or in real time.

[0035] According to a second aspect of an embodiment of the present invention, a device for determining a speech source is provided, comprising:

[0036] An acquisition module, used for acquiring a voice signal to be detected;

[0037] An extraction module, used for extracting speech features of the speech signal;

[0038] A generating module, used to generate a model fingerprint according to the speech feature, wherein the model fingerprint is a feature set used to distinguish speech sources of different objects;

[0039] A clustering module, used to perform cluster analysis on the model fingerprint to obtain a clustering result;

[0040] A determination module is used to determine the speech source information of the speech signal according to the clustering result.

[0041] Optionally, the device further comprises:

[0042] A preprocessing module, used to preprocess the acquired speech signal to obtain a preprocessed speech signal;

[0043] The extraction module is also used to extract the speech features of the pre-processed speech signal.

[0044] Optionally, the extraction module is specifically used to extract speech features of the preprocessed speech signal using Mel-frequency cepstral coefficients and linear prediction coding.

[0045] Optionally, the device further comprises:

[0046] The standardization processing module is used to perform feature vector standardization processing on the extracted speech features to obtain processed features.

[0047] Optionally, the generation module is specifically used to use a convolutional neural network to extract features of the speech and generate a model fingerprint, wherein the structure of the convolutional neural network includes, in sequence: an input layer, a first convolutional layer, a first maximum pooling layer, a second convolutional layer, a second maximum pooling layer, a first fully connected layer, a Dropout layer, a second fully connected layer and an output layer.

[0048] Optionally, the generation model is specifically used to use a convolutional neural network to perform two convolutions and maximum pooling on the input speech features, and to fully connect, discard and fully connect the obtained feature vectors to output a model fingerprint.

[0049] Optionally, the clustering classification module is specifically used to perform clustering analysis on the model fingerprint using a K-means algorithm to obtain a clustering result.

[0050] Optionally, the determining module includes:

[0051] A matching module, used for performing feature matching between the clustering result and the model fingerprint in the model fingerprint library;

[0052] The source tracing determination module is used to determine the speech source tracing information of the speech signal according to the feature matching result.

[0053] Optionally, the matching module includes:

[0054] A calculation module, used for calculating the cosine similarity between the clustering results and the model fingerprints in the model fingerprint library;

[0055] A feature matching module, used for determining that the clustering result matches the feature of the model fingerprint in the model fingerprint library when the cosine similarity is greater than or equal to a set threshold;

[0056] The selection module is used to select the vector with the highest cosine similarity of the matching features as the matching result.

[0057] Optionally, the device further comprises:

[0058] The display module is used to display the determined voice source tracing information of the voice signal.

[0059] Optionally, the device further comprises:

[0060] The updating module is used to update the model fingerprint in the model fingerprint library regularly or in real time.

[0061] According to a third aspect of an embodiment of the present invention, there is provided an electronic device, including:

[0062] processor;

[0063] a memory for storing instructions executable by the processor;

[0064] Wherein, the processor is configured to execute the instructions to implement the method for determining the voice source as described above.

[0065] According to a fourth aspect of an embodiment of the present invention, a computer-readable storage medium is provided. When instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the method for determining the voice source as described above.

[0066] According to a fifth aspect of an embodiment of the present invention, there is provided a computer program product, comprising a computer program or instructions, which, when executed by a processor of an electronic device, implements the method for determining the voice source as described above.

[0067] The technical solution provided by the embodiments of the present invention brings at least the following beneficial effects:

[0068] In an embodiment of the present invention, a speech signal to be detected is obtained; speech features of the speech signal are extracted; a model fingerprint is generated based on the speech features, and the model fingerprint is a feature set used to distinguish speech sources of different objects; cluster analysis is performed on the model fingerprint to obtain a clustering result; and speech source tracing information of the speech signal is determined based on the clustering result. That is to say, in an embodiment of the present invention, a model fingerprint is generated based on the extracted speech features, so that the system can capture subtle sound characteristics from a large number of speech features, and cluster analysis is performed on the model fingerprint to achieve accurate speech classification and tracing, thereby improving the accuracy of speech source determination.

[0069] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] The drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present invention, and are used together with the specification to explain the principles of the present invention, and do not constitute an improper limitation of the present invention. In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings required for use in the embodiments or the prior art description will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0071] Figure 1 It is a flow chart of a method for determining speech source provided by an embodiment of the present invention.

[0072] Figure 2 It is a schematic diagram of extracting features from a speech signal provided by an embodiment of the present invention.

[0073] Figure 3 It is a schematic diagram of generating a model fingerprint using a convolutional neural network provided by an embodiment of the present invention.

[0074] Figure 4 It is a schematic diagram of clustering analysis of model fingerprints provided by an embodiment of the present invention.

[0075] Figure 5 It is a schematic diagram of normalizing feature vectors of speech features provided by an embodiment of the present invention.

[0076] Figure 6 This is another flow chart of a method for determining a speech source provided by an embodiment of the present invention.

[0077] Figure 7 It is a block diagram of a device for determining speech source tracing provided by an embodiment of the present invention.

[0078] Figure 8 It is a block diagram of an electronic device provided by an embodiment of the present invention.

[0079] Fig. 9 It is a block diagram of a device for determining speech source tracing provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0080] In order to enable ordinary persons in the art to better understand the technical solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings.

[0081] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Instead, they are merely examples of devices and methods consistent with some aspects of the present invention as detailed in the appended claims.

[0082] Technical terms:

[0083] Model Fingerprint: Model fingerprint refers to a unique and representative set of features extracted and generated from speech signals. These features can identify and distinguish different speech sources or speakers. In the field of speech processing and recognition, model fingerprints are similar to "digital signatures" used to identify and distinguish different individuals (or objects).

[0084] Voice Source Attribution: Voice source attribution refers to the process of determining the source of a voice signal, that is, identifying which speaker the voice is from or which device it is generated by. This has important application value in fields such as law, security, and media monitoring.

[0085] On the basis of understanding the above technical terms, please refer to the following embodiments. Figure 1 is a flow chart of a method for determining a speech source provided by an embodiment of the present invention, such as Figure 1 As shown, the method comprises the following steps:

[0086] Step 101: Acquire a speech signal to be detected.

[0087] Step 102: extracting speech features of the speech signal.

[0088] Step 103: Generate a model fingerprint according to the speech features, where the model fingerprint is a feature set used to distinguish speech sources of different objects.

[0089] Step 104: Perform cluster analysis on the model fingerprint to obtain a clustering result.

[0090] Step 105: Determine the speech source information of the speech signal according to the clustering result.

[0091] The method for determining the voice source tracing described in the present invention can be applied to terminals, servers, etc., without limitation herein. The terminal implementation device can be electronic devices such as smart phones, laptops, tablet computers, desktop computers, personal digital assistants (PDAs) and wearable devices. The server can be an independent server or a server cluster, or a server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, intermediate services, domain name services, security services, content distribution networks, or big data and artificial intelligence platforms, etc., without limitation herein.

[0092] Combine the following Figure 1 , the specific implementation steps of a method for determining the source of speech provided by an embodiment of the present invention are described in detail.

[0093] In step 101, a speech signal to be detected is obtained.

[0094] In this step, the voice signal to be detected can be collected by a high-precision microphone array, wherein the high-precision microphone array is a high-sensitivity microphone array composed of 128 miniature microphones, each microphone has a sampling rate of 48kHz, a bit depth of 24bit, and the microphone array adopts a ring layout with a radius of 10cm to ensure high-fidelity voice collection. Of course, the voice signal to be detected can also be collected by other methods, which are not limited in this embodiment.

[0095] In step 102, speech features of the speech signal are extracted.

[0096] In this step, the speech features of the speech signal may be extracted using Mel-frequency cepstral coefficients and linear prediction coding.

[0097] Among them, the Mel Frequency Cepstrum Coefficient (MFCC) first maps the linear frequency domain to the Mel frequency domain and then converts it to the cepstrum. MFCC does not depend on the nature of the signal, does not make any assumptions or restrictions on the input signal, and is suitable for the auditory model of the human ear.

[0098] The process of extracting features of the speech signal using Mel-frequency cepstral coefficients and linear prediction coding is as follows: Figure 2 As shown, Figure 2 Schematic diagram of a method for extracting features from a speech signal provided by an embodiment of the present invention, specifically comprising:

[0099] Step 201: Segment the speech signal into short time frames, each frame length is 25ms, and the frame shift is 10ms. The frame length and frame shift here are just examples, and are not limited to these in practical applications.

[0100] Among them, the speech signal is a time-varying non-stationary random process. It is difficult to obtain useful information by directly processing it. In this embodiment, considering that the speech signal changes slowly, it can be sliced ​​and divided into short time frames, that is, frame operation. The framed speech signal can be regarded as an approximate stationary random process. Generally, the time interval of the frame is usually 10 to 30ms, but it can also be appropriately adjusted according to actual needs, and this embodiment does not limit it.

[0101] In order to avoid excessive changes between two adjacent frames, there is an overlapping area between the two adjacent frames, which is called frame shift.

[0102] Step 202: Apply a Hamming window to each frame of signal.

[0103] The Hamming window is also called a cosine window or an improved raised cosine window. Its advantages are that the main lobe is widened and reduced, the side lobe is significantly reduced, and the spectrum leakage is reduced. Therefore, in order to reduce spectrum leakage, this embodiment usually uses a Hamming window. Each frame is multiplied by the Hamming window for use in the subsequent stage. The formula of the Hamming window is:

[0104]

[0105] Where n is the sample index in the window function and N is the length of the window function.

[0106] It should be noted that the windowing in this embodiment includes not only the Hamming window but also the rectangular window. The rectangular window is directly truncated, which is convenient for processing. The main lobe is relatively concentrated, but the side lobe is relatively high and has negative side lobes, which leads to high-frequency interference and leakage in the transformation, and even negative spectrum phenomenon. Therefore, it is usually not used, but in specific applications, the Hamming window or the rectangular window can also be used according to actual needs, and this embodiment does not limit it.

[0107] Step 203: Perform fast Fourier transform on each frame of the windowed signal to obtain frequency domain energy;

[0108] In this embodiment, the windowed signal of each frame is usually a speech signal in the time domain, which needs to be mapped to a linear frequency. This embodiment uses a fast Fourier transform to achieve conversion from the time domain to the frequency domain. Of course, a short-time Fourier transform may also be used, and this embodiment does not impose any restrictions.

[0109] Step 204: Use 26 triangular Mel filters to map the frequency domain energy to the Mel scale;

[0110] That is, the linear frequency is subjected to Mel filtering.

[0111] Step 205: taking the logarithm of the frequency domain energy passing through the triangular Mel filter bank;

[0112] The implementation process is already well known in the art and will not be described in detail here.

[0113] Step 206: Perform discrete cosine transform (DCT) on the frequency domain energy after taking the logarithm, and extract the first 13 coefficients as MFCC features;

[0114] In this step, the frequency domain energy after logarithm is discrete cosine transformed, and the first 13 coefficients are extracted as MFCC features, and the formula is:

[0115]

[0116] Where I is the output length of the filter bank and k is the index of the discrete cosine transform (DCT) coefficient.

[0117] Among them, DCT has the property of orthogonality, and it only calculates real numbers. The result of the log spectrum is extended by odd and even, and then FFT is performed to extract the real number part, which is the result of DCT.

[0118] In this embodiment, DCT can move the high-frequency components of the signal to low frequencies, and most of the signal energy is concentrated in low frequencies, that is, energy concentration characteristics. Therefore, DCT can remove the energy correlation of the outputs of each filter group.

[0119] Step 207: Calculate first-order difference and second-order difference coefficients to capture short-term changes;

[0120] In this embodiment, the speech is continuously changing, and each speech frame is not independent. This embodiment combines dynamic features with static features, which can greatly improve the recognition ability of the system.

[0121] In order to calculate the static features of MFCC of each frame, the static features are regarded as discrete variables, and the difference is usually used to describe the speed of change of MFCC over time, that is, the slope of MFCC over time.

[0122] The calculation method of the difference coefficient is:

[0123]

[0124] Among them, d i (n) is the nth coefficient of the first-order differential MFCC feature vector of the i-th frame of speech, c i (n) is the nth coefficient of the MFCC feature vector of the i-th frame of speech, and N is generally 2.

[0125] N-dimensional MFCC parameters (N / 3 MFCC coefficients + N / 3 first-order difference coefficients + N / 3 second-order difference coefficients) + short-time energy of the frame.

[0126] Therefore, in this embodiment, the first 13 coefficients (ie, MFCC coefficients), the first-order difference coefficients, and the second-order difference coefficients extracted after discrete cosine transform are used as MFCC features.

[0127] Step 208: Model the signal using an autoregressive model, and obtain linear predictive coding (LPC) coefficients by solving the Yule-Walker equation.

[0128] In this step, the signal is modeled using an autoregressive model. The order p of the model is determined according to the characteristics of the signal and the required accuracy. The LPC coefficient is obtained by solving the Yule-Walker equation, which is:

[0129]

[0130] Among them, R xx is the autocorrelation function, a i are the LPC model parameters.

[0131] That is to say, in this embodiment, Linear Predictive Coding (LPC) is a commonly used coding method. In principle, LPC generates the parameters of the vocal tract excitation and transfer function by analyzing the speech waveform. The coding of the sound waveform is actually converted into the coding of these parameters, which greatly reduces the amount of sound data.

[0132] In this step, advanced signal processing technology and machine learning algorithms are used to effectively distinguish and identify speech signals from different sources in complex environments, and maintain a high degree of accuracy even in multi-speaker scenarios.

[0133] In step 103, a model fingerprint is generated according to the speech features, where the model fingerprint is a feature set used to distinguish speech sources of different objects.

[0134] In this step, a convolutional neural network can be used to extract the speech features and generate a model fingerprint, wherein the structure of the convolutional neural network includes, in sequence: an input layer, a first convolutional layer, a first maximum pooling layer, a second convolutional layer, a second maximum pooling layer, a first fully connected layer, a dropout layer, a second fully connected layer, and an output layer.

[0135] The method of extracting the speech features using a convolutional neural network to generate a model fingerprint includes: performing two convolutions and maximum pooling on the input speech features using a convolutional neural network, and performing full connection, discarding, and full connection processing on the obtained feature vectors, and outputting the model fingerprint. Figure 3 As shown, Figure 3 A schematic diagram of generating a model fingerprint using a convolutional neural network provided by an embodiment of the present invention specifically includes:

[0136] Step 301: Input layer: Input the MFCC features into the input layer of the convolutional neural network;

[0137] In this step, the input layer receives the input MFCC features, wherein the feature vector dimension of each feature is 13 (MFCC coefficients) + 2 (first-order differences) + 2 (second-order differences) = 17.

[0138] Step 302: Convolutional layer 1: Use 64 filters of size 5×5, stride 1, and activation function is ReLU;

[0139] In this step, each filter of convolutional layer 1 slides on the input feature map to extract local features.

[0140] Step 303: Maximum pooling layer 1: the pooling window size is 2×2, and the step size is 2;

[0141] In this step, the maximum pooling layer 1 is used to reduce the feature dimension and extract important features while increasing the invariance of the model.

[0142] Step 304: Convolutional layer 2: Use 128 filters of size 3×3, stride 1, and activation function ReLU;

[0143] In this step, convolutional layer 2 further extracts more complex features.

[0144] Step 305: Max pooling layer 2: the pooling window size is 2×2, and the step size is 2;

[0145] In this step, the maximum pooling layer 2 is also used to reduce the feature dimension and extract important features while increasing the invariance of the model.

[0146] Step 306: Fully connected layer 1: contains 512 nodes, and the activation function is ReLU;

[0147] Step 307: Dropout layer: randomly drop 50% of the nodes to prevent overfitting;

[0148] In this step, the Dropout layer is used to temporarily discard neural network units from the network with a certain probability to prevent overfitting. For example, 50% of the nodes are randomly discarded to prevent overfitting.

[0149] Step 308: Fully connected layer 2: contains 256 nodes, and the activation function is ReLU;

[0150] Step 309: Output layer: Output model fingerprint, using Softmax as activation function.

[0151] Therefore, in this step, the model fingerprint generated based on the convolutional neural network enables the system to capture subtle sound characteristics from massive speech data.

[0152] In step 104, cluster analysis is performed on the model fingerprint to obtain a clustering result.

[0153] In this step, the K-means algorithm can be used to perform cluster analysis on the model fingerprint to obtain a clustering result. Figure 4 As shown, Figure 4 This is a schematic diagram of clustering analysis of model fingerprints provided by an embodiment of the present invention. Specifically, it includes:

[0154] Step 401: Initialization: randomly select a sample (a model fingerprint, the same below) from the data set (i.e., the model fingerprint) as the first cluster center;

[0155] Step 402: Repeat: Each time the next cluster center is selected, a weighted selection is performed based on the distance to the nearest existing cluster center;

[0156] Step 403: Iteration: repeatedly calculate the distance from each sample to each cluster center and assign it to the nearest cluster; update the cluster center to the mean of all samples in the cluster.

[0157] Specifically, in this embodiment, first, the K-Means++ algorithm is used to select the initial cluster center to optimize the clustering quality. K-Means++ selects the first cluster center as a random sample, and then selects the next cluster center according to the probability distribution of the distance from each sample to the selected cluster center;

[0158] Secondly, calculate the Euclidean distance from each sample to the selected cluster center, and then select the sample with the farthest distance as the new cluster center. The formula is:

[0159]

[0160] Among them, x i is the sample point, c j is the selected cluster center,

[0161] Specifically, for each sample, calculate its distance to all cluster centers and assign it to the nearest cluster center. The distance is calculated using the Euclidean distance formula:

[0162]

[0163] Among them, x is the sample feature vector and c is the cluster center vector.

[0164] Specifically, once all samples are assigned to cluster centers, the cluster center of each cluster is updated to the mean of all samples in the cluster. The update formula of the cluster center is:

[0165]

[0166] Among them, S j is assigned to the cluster center c j A collection of samples.

[0167] It should be noted that the k-means clustering algorithm is an iterative clustering analysis algorithm. Its implementation principle is to pre-divide the data into K groups, then randomly select K objects as the initial cluster centers, and then calculate the distance between each object and each seed cluster center, and assign each object to the cluster center closest to it. The cluster centers and the objects assigned to them represent a cluster. Every time a sample is assigned, the cluster center of the cluster will be recalculated based on the existing objects in the cluster. This process will be repeated until a certain termination condition is met. The termination condition can be that no (or a minimum number) of objects are reassigned to different clusters, no (or a minimum number) of cluster centers change again, and the sum of squared errors is locally minimized. Its specific implementation process is already a well-known technology in the field and will not be repeated here.

[0168] In this step, the model fingerprint clustering technology is introduced, and accurate speech classification and tracing are achieved through the efficient K-means clustering algorithm, thereby improving the accuracy and reliability of speech tracing.

[0169] In step 105, speech source tracing information of the speech signal is determined according to the clustering result.

[0170] In this step, first, the clustering result is matched with the model fingerprint in the model fingerprint library; secondly, the speech source information of the speech signal is determined according to the result of the feature matching.

[0171] The feature matching of the clustering result with the model fingerprint in the model fingerprint library includes: calculating the cosine similarity of the clustering result with the model fingerprint in the model fingerprint library; when the cosine similarity is greater than or equal to a set threshold, determining that the clustering result matches the feature of the model fingerprint in the model fingerprint library; and selecting the vector with the highest cosine similarity that matches the feature as the matching result. The set threshold can be set to any number between 0.7 and 0.8, including 0.7 or 0.8, etc. Of course, adaptive adjustment can be made according to actual needs, and this implementation does not limit it.

[0172] The feature matching in this embodiment uses cosine similarity to measure the similarity between feature vectors, which specifically includes:

[0173] Assume v i is the feature vector of the model fingerprint of the speech signal to be tested, u j is the feature vector of the model fingerprint in the model fingerprint library, then the cosine similarity is defined as:

[0174]

[0175] Among them, sjmjlarjty(v i ,u j ) is the cosine similarity, v i ·u j represents the vector dot product, ||v i ||2 and ||u j ||2 respectively represent the L2 norm of the vector.

[0176] In this embodiment, if the cosine similarity value similarity(v i ,u j )>0.75, then it is considered that v i and u j Match; otherwise, it is considered not to match.

[0177] For another example, for the speech feature vector A to be tested and each feature vector B in the model fingerprint library, their cosine similarity is calculated. The calculation formula of the cosine similarity is:

[0178]

[0179] Among them, A·B represents the dot product of the vector, ||A|| and ||B|| are the L2 norms of the vector respectively;

[0180] The dot product calculation formula is:

[0181]

[0182] Among them, A i and B i are the i-th elements of vectors A and B respectively.

[0183] In this embodiment, by setting a matching threshold, such as 0.75, if the calculated cosine similarity is greater than or equal to the threshold, it is considered that the speech to be tested matches a certain feature vector in the model fingerprint library. The matching process can traverse all feature vectors in the model fingerprint library and find the vector with the highest cosine similarity that matches the feature vector as the matching result.

[0184] In an embodiment of the present invention, a speech signal to be detected is obtained; speech features of the speech signal are extracted; a model fingerprint is generated based on the speech features, and the model fingerprint is a feature set used to distinguish speech sources of different objects; cluster analysis is performed on the model fingerprint to obtain a clustering result; and speech source tracing information of the speech signal is determined based on the clustering result. That is to say, in an embodiment of the present invention, a model fingerprint is generated based on the extracted speech features, so that the system can capture subtle sound characteristics from a large number of speech features, and cluster analysis is performed on the model fingerprint to achieve accurate speech classification and tracing, thereby improving the accuracy of speech source determination.

[0185] Optionally, in another embodiment, based on the above embodiment, the method may further include: preprocessing the acquired speech signal to obtain a preprocessed speech signal; extracting the speech features of the speech signal includes: extracting the speech features of the preprocessed speech signal.

[0186] In this step, the acquired speech signal can be subjected to preprocessing operations such as denoising, filtering and equalization processing to remove noise in the speech signal and optimize the speech signal. Among them, the preprocessing operation can use an adaptive filter to adjust the filter coefficient in real time through a minimum mean square error algorithm, and the equalization processing is usually processed using a finite impulse response filter, and its specific implementation process is already a well-known technology and will not be repeated here.

[0187] Optionally, in another embodiment, based on the above embodiment, the method may further include: performing feature vector normalization processing on the extracted speech features to obtain processed features.

[0188] Among them, in this step, the process of normalizing the feature vector of the speech feature is as follows: Figure 5 As shown, Figure 5 A schematic diagram of normalizing a feature vector of a speech feature provided by an embodiment of the present invention specifically includes:

[0189] Step 501: Calculate the average value of the feature vector;

[0190] Step 502: Calculate the standard deviation of the feature vector;

[0191] Step 503: Subtract the mean value from the feature vector and divide it by the standard deviation to obtain a standardized feature vector.

[0192] Specifically, let the mean value of the eigenvector X be μ; let the standard deviation of the eigenvector be σ; let the standardized eigenvector be z, use the numpy library in a programming language such as Python to calculate the mean and standard deviation, and apply these statistics to standardize the eigenvector. The specific formula is as follows:

[0193]

[0194] The purpose of standardization in this embodiment is to make the feature vector have zero mean and unit variance, which helps to improve the stability and accuracy of subsequent processing.

[0195] Its purpose is to normalize the vector or unit to facilitate subsequent processing. The specific implementation process is already a well-known technology to those skilled in the art and will not be described in detail here.

[0196] Optionally, in another embodiment, based on the above embodiment, the method may further include:

[0197] The voice source tracing information of the determined voice signal is displayed. Specifically, the voice source tracing information can be displayed on a user interaction interface or on a screen of a terminal, which is not limited in this embodiment.

[0198] Optionally, in another embodiment, based on the above embodiment, the method may further include: updating the model fingerprint in the model fingerprint library regularly or in real time. Specifically, it includes: collecting new voice samples in real time or regularly, extracting features from the new voice samples, generating model fingerprints based on the extracted features, and updating the generated model fingerprints to the model fingerprint library to improve the model fingerprints in the model fingerprint library.

[0199] In this embodiment, by regularly collecting new voice samples and extracting features from these samples, the system can continuously update and improve its model fingerprint library, ensuring that a high level of recognition rate can be maintained even when speech synthesis technology continues to evolve and new types of speech appear. This dynamic update mechanism not only enables the system to adapt to the development trend of speech technology, but also to cope with the diversity of speech signals in the real world, thereby ensuring the long-term stable operation and service quality of the system.

[0200] See also Figure 6 , is another flow chart of a method for determining a speech source provided by an embodiment of the present invention, the method comprising:

[0201] Step 601: Collect target generated speech signals through a high-precision microphone array.

[0202] Among them, the target generated speech signal is the speech signal to be detected.

[0203] High-precision microphone array, including a high-sensitivity microphone array consisting of 128 miniature microphones, each with a sampling rate of 48kHz and a bit depth of 24bit to ensure high-fidelity voice capture.

[0204] Step 602: pre-processing the collected speech signal, wherein the pre-processing includes: denoising, filtering, equalization and other operations.

[0205] Among them, this step can use an adaptive filter, and the filter coefficient can be adjusted in real time through a minimum mean square error algorithm, etc., and the equalization processing can use a finite impulse response filter, etc.

[0206] Step 603: extract key features from the preprocessed speech signal using Mel-frequency cepstral coefficients and linear predictive coding.

[0207] In this step, other methods may also be used to extract key features of the speech signal, which is not limited in this embodiment.

[0208] Step 604: perform feature vector normalization processing on the extracted key features to obtain processed features.

[0209] Step 605: Generate a model fingerprint based on the processed features, wherein the model fingerprint is generated using a convolutional neural network, including an input layer, two convolutional layers, two maximum pooling layers, two fully connected layers and a Dropout layer.

[0210] Step 606: performing cluster analysis on the generated model fingerprint to obtain a clustering result, wherein the cluster analysis may adopt a K-means algorithm or the like;

[0211] Step 607: Determine the speech source tracing result of the speech signal according to the clustering result and the model fingerprint in the model fingerprint library.

[0212] The determination method may include: calculating the cosine similarity between the clustering results and the model fingerprints in the model fingerprint library; when the cosine similarity is greater than or equal to a set threshold, determining that the clustering results match the characteristics of the model fingerprints in the model fingerprint library; and selecting the vector with the highest cosine similarity that matches the characteristics as the matching result. The set threshold may be set to 0.75, etc., but is not limited thereto in specific applications.

[0213] Step 608: Display the result of determining the speech source of the speech signal.

[0214] This step is optional, and the voice tracing result of the determined voice signal can be displayed through the user interaction interface.

[0215] Step 609: regularly collect new speech samples, perform feature extraction on the new speech samples, generate new model fingerprints, and update the model fingerprints in the model fingerprint library.

[0216] It should be noted that, in this embodiment, the specific implementation process of each step is detailed in the implementation process of the corresponding step above, and will not be repeated here.

[0217] In the embodiment of the present invention, by using advanced signal processing technology and machine learning algorithms, the invention can effectively distinguish and identify speech signals from different sources in complex environments, and can maintain a high degree of accuracy even in multi-speaker scenarios, overcoming the limitations of related technologies in this regard. Specifically, in the embodiment of the present invention, by using a variety of feature extraction methods including Mel-frequency cepstral coefficients and linear prediction coding, as well as model fingerprints generated based on convolutional neural networks, the system can capture subtle sound characteristics from massive amounts of speech data, and then achieve accurate speech classification and tracing through an efficient K-means clustering algorithm, significantly improving the accuracy and reliability of speech tracing.

[0218] It should be noted that, for the method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should know that this implementation disclosure is not limited by the order of the actions described, because according to the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the present invention.

[0219] Figure 7 : is a block diagram of a device for determining speech source tracing provided by an embodiment of the present invention, the device comprises: an acquisition module 701, an extraction module 702, a generation module 703, a clustering module 704 and a determination module 705, wherein:

[0220] An acquisition module 701 is used to acquire a speech signal to be detected;

[0221] An extraction module 702, used to extract speech features of the speech signal;

[0222] A generating module 703, configured to generate a model fingerprint according to the speech feature, wherein the model fingerprint is a feature set used to distinguish speech sources of different objects;

[0223] A clustering module 704 is used to perform cluster analysis on the model fingerprint to obtain a clustering result;

[0224] The determination module 705 is used to determine the speech source information of the speech signal according to the clustering result.

[0225] Optionally, in another embodiment, based on the above embodiment, the device further includes:

[0226] A preprocessing module, used to preprocess the acquired speech signal to obtain a preprocessed speech signal;

[0227] The extraction module is also used to extract the speech features of the pre-processed speech signal.

[0228] Optionally, in another embodiment, based on the above embodiment, the extraction module is specifically used to extract the speech features of the preprocessed speech signal by using Mel-frequency cepstral coefficients and linear prediction coding.

[0229] Optionally, in another embodiment, based on the above embodiment, the device further includes:

[0230] The standardization processing module is used to perform feature vector standardization processing on the extracted speech features to obtain processed features.

[0231] Optionally, in another embodiment, based on the above embodiment, the generation module is specifically used to use a convolutional neural network to extract features of the speech features and generate a model fingerprint, wherein the structure of the convolutional neural network includes, in sequence: an input layer, a first convolutional layer, a first maximum pooling layer, a second convolutional layer, a second maximum pooling layer, a first fully connected layer, a dropout layer, a second fully connected layer and an output layer.

[0232] Optionally, in another embodiment, based on the above-mentioned embodiment, the generation model is specifically used to use a convolutional neural network to perform two convolutions and maximum pooling on the input speech features, and fully connect, discard and fully connect the obtained feature vectors to output the model fingerprint.

[0233] Optionally, in another embodiment, based on the above embodiment, the clustering classification module is specifically used to perform clustering analysis on the model fingerprint using a K-means algorithm to obtain a clustering result.

[0234] Optionally, in another embodiment, based on the above embodiment, the determining module includes:

[0235] A matching module, used for performing feature matching between the clustering result and the model fingerprint in the model fingerprint library;

[0236] The source tracing determination module is used to determine the speech source tracing information of the speech signal according to the feature matching result.

[0237] Optionally, in another embodiment, based on the above embodiment, the matching module includes:

[0238] A calculation module, used for calculating the cosine similarity between the clustering results and the model fingerprints in the model fingerprint library;

[0239] A feature matching module, used for determining that the clustering result matches the feature of the model fingerprint in the model fingerprint library when the cosine similarity is greater than or equal to a set threshold;

[0240] The selection module is used to select the vector with the highest cosine similarity of the matching features as the matching result.

[0241] Optionally, in another embodiment, based on the above embodiment, the device further includes:

[0242] The display module is used to display the determined voice source tracing information of the voice signal.

[0243] Optionally, in another embodiment, based on the above embodiment, the device further includes:

[0244] The updating module is used to update the model fingerprint in the model fingerprint library regularly or in real time.

[0245] Optionally, an embodiment of the present invention further provides an electronic device, including:

[0246] processor;

[0247] a memory for storing instructions executable by the processor;

[0248] Wherein, the processor is configured to execute the instructions to implement the method for determining the voice source as described above.

[0249] Optionally, an embodiment of the present invention further provides a computer-readable storage medium, which, when instructions in the computer-readable storage medium are executed by a processor of an electronic device, enables the electronic device to execute the method for determining the voice source as described above.

[0250] Optionally, an embodiment of the present invention further provides a computer program product, including a computer program or instructions, which, when executed by a processor of an electronic device, implements the method for determining the voice source as described above.

[0251] In the embodiments of the present invention, the accuracy and reliability of speech tracing are significantly improved by introducing model fingerprint clustering technology. By utilizing advanced signal processing technology and machine learning algorithms, the invention can effectively distinguish and identify speech signals from different sources in complex environments, and can maintain a high degree of accuracy even in multi-speaker scenarios. It solves the problems of low accuracy and low efficiency in speech tracing technology in related technologies, and improves the ability to effectively distinguish different individual voices in multi-speaker scenarios, achieve accurate speech classification and tracing, and thus improve the accuracy of tracing.

[0252] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0253] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0254] Figure 8 800 is a block diagram of an electronic device 800 provided in an embodiment of the present invention. For example, the electronic device 800 may be a mobile terminal or a server, and the electronic device is a mobile terminal as an example for explanation in the embodiment of the present invention. For example, the electronic device 800 may be a mobile phone, a computer, a digital broadcast terminal, a message transceiver device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0255] Reference Figure 8 , the electronic device 800 may include one or more of the following components: a processing component 802 , a memory 804 , a power component 806 , a multimedia component 808 , an audio component 810 , an input / output (I / O) interface 812 , a sensor component 814 , and a communication component 816 .

[0256] The processing component 802 generally controls the overall operation of the electronic device 800, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 802 may include one or more modules to facilitate the interaction between the processing component 802 and other components. For example, the processing component 802 may include a multimedia module to facilitate the interaction between the multimedia component 808 and the processing component 802.

[0257] The memory 804 is configured to store various types of data to support operations on the device 800. Examples of such data include instructions for any application or method operating on the electronic device 800, contact data, phone book data, messages, pictures, videos, etc. The memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0258] The power supply component 806 provides power to the various components of the electronic device 800. The power supply component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device 800.

[0259] The multimedia component 808 includes a screen that provides an output interface between the electronic device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. When the device 800 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and the rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0260] The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC), and when the electronic device 800 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 804 or sent via the communication component 816. In some embodiments, the audio component 810 also includes a speaker for outputting audio signals.

[0261] I / O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: home button, volume button, start button, and lock button.

[0262] The sensor assembly 814 includes one or more sensors for providing various aspects of status assessment for the electronic device 800. For example, the sensor assembly 814 can detect the open / closed state of the device 800, the relative positioning of components, such as the display and keypad of the electronic device 800, and the sensor assembly 814 can also detect the position change of the electronic device 800 or a component of the electronic device 800, the presence or absence of contact between the user and the electronic device 800, the orientation or acceleration / deceleration of the electronic device 800, and the temperature change of the electronic device 800. The sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 814 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0263] The communication component 816 is configured to facilitate wired or wireless communication between the electronic device 800 and other devices. The electronic device 800 can access a wireless network based on a communication standard, such as WiFi, a carrier network (such as 2G, 3G, 4G or 5G), or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0264] In an embodiment, the electronic device 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to execute the method for determining the voice source shown above.

[0265] In an embodiment, a computer-readable storage medium is also provided, and when the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device 800 can execute the method for determining the voice source shown above. For example, the computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0266] In an embodiment, a computer program product is also provided, including a computer program or instructions. When the computer program or instructions are executed by the processor 820 of the electronic device 800, the electronic device 800 executes the above-mentioned method for determining the voice tracing.

[0267] Fig. 9 900 is a block diagram of a device 900 for determining a voice source provided by an embodiment of the present invention. For example, the device 900 may be provided as a server. Fig. 9 The apparatus 900 includes a processing component 922, which further includes one or more processors, and a memory resource represented by a memory 932 for storing instructions, such as an application, that can be executed by the processing component 922. The application stored in the memory 932 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 922 is configured to execute instructions to perform the above method.

[0268] The device 900 may also include a power supply component 926 configured to perform power management of the device 900, a wired or wireless network interface 950 configured to connect the device 900 to a network, and an input / output (I / O) interface 958. The device 900 may operate based on an operating system stored in the memory 932, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, or the like.

[0269] Those skilled in the art will readily appreciate other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses or adaptations of the present invention that follow the general principles of the present invention and include common knowledge or customary techniques in the art that are not disclosed by the present invention. The specification and examples are to be considered exemplary only, and the true scope and spirit of the present invention are indicated by the following claims.

[0270] It should be understood that the present invention is not limited to the exact construction that has been described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

Claims

1. A method for determining speech source, characterized in that: include: Acquire a speech signal to be detected; Extracting speech features of the speech signal; Generate a model fingerprint according to the speech feature, wherein the model fingerprint is a feature set used to distinguish speech sources of different objects; Performing cluster analysis on the model fingerprint to obtain a clustering result; The speech source tracing information of the speech signal is determined according to the clustering result.

2. The method for determining the source of speech according to claim 1, characterized in that: The method further comprises: Preprocessing the acquired speech signal to obtain a preprocessed speech signal; The extracting the speech feature of the speech signal includes: extracting the speech feature of the speech signal after preprocessing.

3. The method for determining the source of speech according to claim 1, characterized in that: The method further comprises: The extracted speech features are subjected to feature vector normalization processing to obtain processed features.

4. The method for determining the speech source according to claim 1, characterized in that: The generating a model fingerprint according to the speech feature comprises: The speech features are extracted using a convolutional neural network to generate a model fingerprint, wherein the structure of the convolutional neural network includes, in sequence: an input layer, a first convolutional layer, a first maximum pooling layer, a second convolutional layer, a second maximum pooling layer, a first fully connected layer, a dropout layer, a second fully connected layer, and an output layer.

5. The method for determining the voice source according to any one of claims 1 to 4, characterized in that: The determining the speech source tracing information of the speech signal according to the clustering result includes: Performing feature matching between the clustering result and the model fingerprint in the model fingerprint library; The speech source information of the speech signal is determined according to the feature matching result.

6. The method for determining the speech source according to claim 5, characterized in that: The performing feature matching on the clustering result and the model fingerprint in the model fingerprint library includes: Calculating the cosine similarity between the clustering results and the model fingerprints in the model fingerprint library; When the cosine similarity is greater than or equal to a set threshold, determining that the clustering result matches the characteristics of the model fingerprint in the model fingerprint library; The vector with the highest cosine similarity that matches the feature is selected as the matching result.

7. The method for determining the speech source according to claim 6, characterized in that: The method further comprises: The model fingerprints in the model fingerprint library are updated regularly or in real time.

8. A device for determining the source of speech, characterized in that: include: An acquisition module, used for acquiring a voice signal to be detected; An extraction module, used for extracting speech features of the speech signal; A generating module, used to generate a model fingerprint according to the speech feature, wherein the model fingerprint is a feature set used to distinguish speech sources of different objects; A clustering module, used to perform cluster analysis on the model fingerprint to obtain a clustering result; A determination module is used to determine the speech source information of the speech signal according to the clustering result.

9. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the method for determining the speech source as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the method for determining the speech source as described in any one of claims 1 to 7.

Images