Phoneme screening method and device, electronic equipment and readable storage medium

Abstract

The invention provides a phoneme screening method and device, electronic equipment and a readable storage medium, and the method comprises the steps: obtaining a to-be-detected voice stream, and determining the phoneme type of each phoneme included in each syllable in the to-be-detected voice stream according to the phoneme classification information corresponding to a target language to which the to-be-detected voice stream belongs; extracting acoustic information of each phoneme, performing dimension reduction processing and clustering processing on the acoustic information of each phoneme, and determining a plurality of clusters corresponding to each phoneme category; for each phoneme category, determining a phoneme stable interval corresponding to the phoneme category based on a plurality of clusters corresponding to the phoneme category; and based on the phoneme stable interval corresponding to each phoneme category, screening the to-be-detected voice stream, determining a plurality of target phonemes located in the phoneme stable interval in the to-be-detected voice stream, and performing voiceprint comparison analysis on the to-be-detected voice stream based on the determined plurality of target phonemes. In this way, the accuracy of voiceprint recognition and matching can be improved.

Description

Technical Field

[0001] The present application relates to the field of speech processing technology, and in particular to a phoneme screening method, device, electronic device and readable storage medium. Background Art

[0002] Before detecting the voice, it is generally necessary to obtain the voice stream to be detected, and perform voiceprint modeling or voiceprint detection on the voice stream to be detected.

[0003] In the related art, voiceprint recognition methods or voiceprint identification technologies that directly extract acoustic signals for modeling mostly focus on the acoustic characteristics of the extracted speech. The extracted speech is mostly based on speech streams and syllables for voiceprint modeling and voiceprint recognition. However, the voiceprint recognition process does not take into account the changes in the speaker's voice in different scenarios. The recognition and matching accuracy of voiceprints in complex scenarios is relatively poor. Summary of the invention

[0004] In view of this, the purpose of the present application is to provide a phoneme screening method, device, electronic device and readable storage medium. Before detecting the voice stream, phoneme stability analysis is performed based on the phonemes in the voice stream to determine the phoneme stability interval, so as to screen out multiple relatively stable target phonemes within the phoneme stability interval for subsequent voiceprint modeling analysis, so as to improve the accuracy of voiceprint recognition and matching.

[0005] In a first aspect, an embodiment of the present application provides a method for screening phonemes, the screening method comprising:

[0006] Acquire a speech stream to be detected, and divide the speech stream to be detected into a plurality of syllables;

[0007] Determine a plurality of phonemes contained in each syllable and a phoneme category of each phoneme according to the phoneme classification information corresponding to the target language to which the speech stream to be detected belongs;

[0008] Extracting acoustic information of each phoneme, performing dimensionality reduction and clustering processing on the acoustic information of each phoneme, and determining multiple clusters corresponding to each phoneme category;

[0009] For each phoneme category, based on a plurality of clusters corresponding to the phoneme category, determine a phoneme stability interval corresponding to the phoneme category;

[0010] Based on the phoneme stability interval corresponding to each phoneme category, the voice stream to be detected is screened, and multiple target phonemes located in the phoneme stability interval in the voice stream to be detected are determined, so as to perform voiceprint comparison analysis on the voice stream to be detected based on the determined multiple target phonemes.

[0011] In a possible implementation, the phoneme category of the phoneme is determined by the following steps:

[0012] Based on the phoneme classification information, determining the phoneme classification category to which each phoneme in each syllable belongs;

[0013] For each phoneme classification category, based on the voice environment information of the voice stream to be detected, the environment category to which each phoneme contained in the phoneme classification category belongs is determined; wherein the environment category to which the phoneme belongs is a subordinate category of the phoneme classification category to which it belongs.

[0014] In a possible implementation, the phonemes include vowel types and consonant types; and extracting acoustic information of each phoneme includes:

[0015] When the phoneme is a vowel type, extracting formant information of the phoneme;

[0016] When the phoneme is of a consonant type, the phoneme is analyzed to extract spectrum characteristics, energy level information, duration information, spectrum envelope information, spectrum tilt information, bandwidth, and glottal wave characteristics of the phoneme.

[0017] In a possible implementation, for each phoneme, the acoustic information of the phoneme is subjected to dimensionality reduction processing by the following steps:

[0018] For a plurality of acoustic features included in the acoustic information of the phoneme, each acoustic feature is centralized, and a covariance matrix after the centralization is calculated;

[0019] Determining a plurality of target eigenvectors based on the eigenvalues ​​and eigenvectors of the covariance matrix;

[0020] Each acoustic feature is projected according to the corresponding target feature vector to obtain the acoustic information of the reduced-dimensional phoneme.

[0021] In a possible implementation, for each phoneme category, the acoustic information of the phoneme is clustered by the following steps:

[0022] For each phoneme category, determine multiple cluster centers of the phoneme category based on the mean value of the phoneme category;

[0023] For each phoneme category, based on the distance between each phoneme and each determined cluster center under the phoneme category, the phonemes under the phoneme category are clustered to obtain a plurality of cluster clusters.

[0024] In a possible implementation, for each phoneme category, based on a plurality of clusters corresponding to the phoneme category, determining the phoneme stability interval corresponding to the phoneme category includes:

[0025] For each phoneme category, a cluster containing a number of phonemes greater than a preset phoneme number threshold in the cluster corresponding to the phoneme category is determined as a classification stability interval corresponding to the phoneme category;

[0026] For each phoneme category, a dimensionality reduction cluster analysis is performed on the environmental classification corresponding to each phoneme in the phoneme category to determine the environmental phoneme stability interval corresponding to the phoneme category.

[0027] In a possible embodiment, the screening method further comprises:

[0028] For each phoneme category, based on the acoustic information of the dimensionality reduction and clustering processing of each phoneme included in the phoneme category, determine the phoneme portrait corresponding to the phoneme category; wherein the phoneme portrait includes the phoneme stability interval and the phoneme fluctuation amplitude of the phoneme category;

[0029] Voiceprint comparison and analysis is performed based on the phoneme portrait corresponding to each phoneme category.

[0030] In a second aspect, an embodiment of the present application further provides a phoneme screening device, the screening device comprising:

[0031] A voice stream segmentation module, used for acquiring a voice stream to be detected and segmenting the voice stream to be detected into a plurality of syllables;

[0032] A phoneme classification extraction module, used to determine a plurality of phonemes contained in each syllable and a phoneme category of each phoneme according to the phoneme classification information corresponding to the target language to which the speech stream to be detected belongs;

[0033] The phoneme dimensionality reduction and clustering module is used to extract the acoustic information of each phoneme, and perform dimensionality reduction and clustering processing on the acoustic information of each phoneme to determine multiple clusters corresponding to each phoneme category;

[0034] A phoneme stability interval determination module, for each phoneme category, based on a plurality of clusters corresponding to the phoneme category, determines the phoneme stability interval corresponding to the phoneme category;

[0035] The phoneme screening module is used to screen the voice stream to be detected based on the phoneme stability interval corresponding to each phoneme category, determine multiple target phonemes in the voice stream to be detected that are located in the phoneme stability interval, and perform voiceprint comparison analysis on the voice stream to be detected based on the determined multiple target phonemes.

[0036] In a third aspect, an embodiment of the present disclosure further provides an electronic device, comprising: a processor, a storage medium and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the storage medium via the bus, and the processor executes the machine-readable instructions to perform the virtual scene editing method as described in any one of the first aspects.

[0037] In a fourth aspect, an embodiment of the present disclosure further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the method for editing a virtual scene as described in any one of the first aspects is executed.

[0038] The embodiments of the present application provide a method, device, electronic device and readable storage medium for screening phonemes, which obtain a speech stream to be detected and divide the speech stream to be detected into multiple syllables; determine multiple phonemes contained in each syllable and the phoneme category of each phoneme according to the phoneme classification information corresponding to the target language to which the speech stream to be detected belongs; extract the acoustic information of each phoneme, and perform dimensionality reduction and clustering processing on the acoustic information of each phoneme to determine multiple clustering clusters corresponding to each phoneme category; for each phoneme category, determine the phoneme stability interval corresponding to the phoneme category based on the multiple clustering clusters corresponding to the phoneme category; based on the phoneme stability interval corresponding to each phoneme category, screen the speech stream to be detected, determine multiple target phonemes in the speech stream to be detected that are located in the phoneme stability interval, and perform voiceprint comparison analysis on the speech stream to be detected based on the determined multiple target phonemes. In this way, before detecting the voice stream, phoneme stability analysis is performed based on the phonemes in the voice stream to determine the phoneme stability interval, so as to screen out multiple relatively stable target phonemes within the phoneme stability interval for subsequent voiceprint modeling analysis, so as to improve the accuracy of voiceprint recognition and matching.

[0039] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0041] Figure 1 A flowchart of a method for screening phonemes provided in an embodiment of the present application;

[0042] Figure 2 One of the structural schematic diagrams of a phoneme screening device provided in an embodiment of the present application;

[0043] Figure 3 A second structural schematic diagram of a phoneme screening device provided in an embodiment of the present application;

[0044] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0045] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application usually described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, each other embodiment obtained by those skilled in the art without making creative work belongs to the scope of protection of the present application.

[0046] First, the application scenarios to which the present application is applicable are introduced. The present application can be applied in the field of speech processing technology.

[0047] Before detecting the voice, it is generally necessary to obtain the voice stream to be detected, and perform voiceprint modeling or voiceprint detection on the voice stream to be detected.

[0048] Common voiceprint recognition methods or voiceprint identification technologies that directly extract acoustic signals for modeling focus more on the acoustic characteristics of the extracted speech. The extracted speech is mostly in units of speech streams and syllables, and lacks preprocessing and analysis of speech data at the phoneme level.

[0049] In isolated word speech recognition, the simplest and most effective method is to use the DTW (Dynamic Time Warping) algorithm. This algorithm solves the problem of template matching with different pronunciation lengths based on the idea of ​​dynamic programming (DP). It is an earlier and more classic algorithm in speech recognition. This method does not consider the variability and instability of the speaker's pronunciation, but directly inputs the voice data for algorithm comparison, which actually ignores the changes in the speaker's voice in different scenarios. The recognition and matching accuracy of voiceprints in complex scenarios is relatively poor.

[0050] Based on this, an embodiment of the present application provides a method for screening phonemes to improve the accuracy of voiceprint recognition and matching.

[0051] See also Figure 1 , Figure 1 This is a flow chart of a method for selecting phonemes provided in an embodiment of the present application. Figure 1 As shown in , the method for screening phonemes provided in the embodiment of the present application includes:

[0052] S101: Acquire a speech stream to be detected, and divide the speech stream to be detected into multiple syllables.

[0053] S102: Determine a plurality of phonemes included in each syllable and a phoneme category of each phoneme according to the phoneme classification information corresponding to the target language to which the speech stream to be detected belongs.

[0054] S103: extracting acoustic information of each phoneme, and performing dimensionality reduction and clustering processing on the acoustic information of each phoneme to determine a plurality of clusters corresponding to each phoneme category.

[0055] S104 . For each phoneme category, based on a plurality of clusters corresponding to the phoneme category, determine a phoneme stability interval corresponding to the phoneme category.

[0056] S105. Based on the phoneme stability interval corresponding to each phoneme category, the voice stream to be detected is screened to determine each target phoneme in the voice stream to be detected that is located in the phoneme stability interval, so as to perform voiceprint comparison analysis on the voice stream to be detected based on the determined multiple target phonemes.

[0057] A phoneme screening method provided in an embodiment of the present application performs phoneme stability analysis on the phonemes in the voice stream before detecting the voice stream, determines the phoneme stability interval, and screens out multiple relatively stable target phonemes within the phoneme stability interval for subsequent voiceprint modeling analysis, so as to improve the accuracy of voiceprint recognition and matching.

[0058] The following is a description of the exemplary steps of the embodiment of the present disclosure:

[0059] S101: Acquire a speech stream to be detected, and divide the speech stream to be detected into multiple syllables.

[0060] In the embodiments of the present application, before detecting the voice, it is generally necessary to obtain the voice stream to be detected, and perform voiceprint modeling or voiceprint detection on the voice stream to be detected. In the related art, the voiceprint recognition method or voiceprint identification technology that directly extracts acoustic signals for modeling focuses more on the acoustic characteristics of the extracted voice, and the extracted voice is mostly based on speech streams and syllables for voiceprint modeling and voiceprint recognition. However, in the related art, the voiceprint recognition process does not take into account the changes in the speaker's voice in different scenarios. The recognition and matching accuracy of voiceprints in complex scenarios is relatively poor.

[0061] In an embodiment of the present application, after obtaining the voice stream to be detected, it is necessary to segment the voice stream to be detected to obtain multiple discrete syllables. For syllables, different languages ​​correspond to different syllables. For example, if the language is Chinese, one syllable is one Chinese character; if the language is English, one syllable is one word.

[0062] Furthermore, after the speech stream to be detected is segmented into a plurality of syllables, the phonemes in the syllables may be scanned and analyzed to determine the plurality of phonemes contained in each syllable and the phoneme category of each phoneme.

[0063] S102: Determine a plurality of phonemes included in each syllable and a phoneme category of each phoneme according to the phoneme classification information corresponding to the target language to which the speech stream to be detected belongs.

[0064] Among them, phoneme is a basic concept in phonetics. It refers to the smallest speech unit divided according to the natural properties of speech, which can be divided into two categories: vowels and consonants; vowels and consonants can be further combined into syllables.

[0065] In one possible implementation, for each syllable, it is necessary to determine the multiple phonemes contained in each syllable and the phoneme category corresponding to each phoneme. Furthermore, for the entire speech stream to be detected, the phoneme set corresponding to the same phoneme category is determined to determine the phonemes contained in each phoneme category.

[0066] For example, if the audio stream to be detected is "Hahahaha, ah? Auntie", the syllable "ha" contains the phoneme vowel a and the consonant h; "ah" contains the phoneme vowel a; "ah" contains the phoneme vowel a, and when the vowel a is counted, it includes the vowel a in "ha", the vowel a in "ah" and the vowel a in "ah".

[0067] Specifically, the phoneme category of the phoneme is determined by the following steps:

[0068] a1: Based on the phoneme classification information, determine the phoneme classification category to which each phoneme in each syllable belongs.

[0069] a2: For each phoneme classification category, based on the voice environment information of the voice stream to be detected, determine the environment category to which each phoneme contained in the phoneme classification category belongs; wherein the environment category to which the phoneme belongs is a subordinate category of the phoneme classification category to which it belongs.

[0070] In a possible implementation, the phoneme classification category represents the category to which the phoneme belongs. For example, the phoneme classification category may include vowel a, consonant u, etc., and then the phoneme classification category to which each phoneme belongs is determined according to the corresponding phoneme category.

[0071] Furthermore, after the detected voice stream is classified, the phonemes can be extracted by category. The phonemes of the same category also differ when they appear in different voice environments. Therefore, the extracted phonemes need to be further divided in the phoneme category according to the voice environment information to determine the environment category to which each phoneme belongs, wherein the environment category to which the phoneme belongs is a subordinate category of the phoneme category to which it belongs.

[0072] For example, if vowel a can appear after consonants of different categories, and different consonants can affect vowel a through the synergistic sound change effect, then in the large category of vowel a, a should be divided into secondary subcategories according to the consonant category.

[0073] Furthermore, after determining to extract and classify the speech stream to be detected according to phoneme categories, acoustic information can be extracted for each phoneme. In order to reduce the excessive amount of computational effort in subsequent analysis of the phonemes caused by excessive acoustic information of the phonemes and reduce the efficiency of phoneme processing, after the acoustic features of each phoneme are extracted, the acoustic information of each phoneme can be subjected to dimensionality reduction processing.

[0074] S103: extracting acoustic information of each phoneme, and performing dimensionality reduction and clustering processing on the acoustic information of each phoneme to determine a plurality of clusters corresponding to each phoneme category.

[0075] In the embodiment of the present application, phonemes are divided into vowel types and consonant types. When extracting acoustic information from different types of phonemes, there are certain differences in the acoustic information that needs to be extracted.

[0076] Specifically, the step of "extracting acoustic information of each phoneme" includes:

[0077] b1: When the phoneme is a vowel type, extracting the formant information of the phoneme.

[0078] b2: When the phoneme is of consonant type, the phoneme is analyzed to extract spectrum characteristics, energy level information, duration information, spectrum envelope information, spectrum tilt information, bandwidth and glottal wave characteristics of the phoneme.

[0079] In a possible implementation, if the phoneme is a vowel type, the acoustic information of the phoneme generally mainly includes formant information, wherein the formant is a specific frequency that is amplified when the sound wave passes through the vocal tract and is the main frequency component in the speech signal.

[0080] Specifically, algorithms for extracting resonance peaks may include spectrum peak search, cepstrum, linear predictive coding (LPC), methods based on autocorrelation and cross-correlation, wideband spectrum analysis, multi-window spectrum estimation, and maximum likelihood estimation, etc.; the above algorithms estimate the frequency and bandwidth of resonance peaks and other useful signals by analyzing the spectrum, cepstrum, autocorrelation or cross-correlation of the speech signal, and using linear prediction models and probability models.

[0081] In another possible implementation, if the phoneme is of a consonant type, the acoustic information of the phoneme generally includes spectral characteristics, energy level, duration, spectral envelope, spectral tilt, bandwidth, and glottal wave characteristics.

[0082] Specifically, the extraction algorithm of acoustic information of consonant-type phonemes may include short-time Fourier transform (STFT) to obtain spectral features, linear predictive coding (LPC) and cepstral analysis for estimating spectral envelope, spectral centroid and spectral entropy for describing spectral characteristics and complexity, Mel-frequency cepstral coefficients (MFCC) and perceptual linear prediction (PLP) combining spectral analysis and auditory perception characteristics, and deep learning methods such as convolutional neural network (CNN) and recurrent neural network (RNN) to directly extract features from the original speech waveform.

[0083] In a possible implementation, after extracting acoustic information for each phoneme, the acoustic information of the phoneme may be subjected to dimensionality reduction processing to reduce excessive computational complexity in subsequent analysis of the phoneme due to excessive acoustic information of the phoneme, thereby improving the efficiency of phoneme processing.

[0084] Specifically, for each phoneme, the acoustic information of the phoneme is reduced in dimension through the following steps:

[0085] c1: for the multiple acoustic features included in the acoustic information of the phoneme, perform centralization processing on each acoustic feature and calculate the covariance matrix after centralization.

[0086] c2: Determine multiple target eigenvectors based on the eigenvalues ​​and eigenvectors of the covariance matrix.

[0087] c3: Project each acoustic feature according to the corresponding target feature vector to obtain the acoustic information of the reduced-dimensional phoneme.

[0088] In an embodiment of the present application, for the acoustic features of the extracted phonemes, the statistical method of principal component analysis (PCA) can be used to perform dimensionality reduction processing on the acoustic information. Specifically, the multiple acoustic features contained in the acoustic information of the obtained phonemes are centralized. The specific processing method is: for each acoustic feature, the characteristic mean of the acoustic feature is determined. Furthermore, for each acoustic feature, the characteristic value of the acoustic feature is subtracted from the corresponding characteristic mean to complete the centralization processing.

[0089] Furthermore, the covariance matrix of the multiple acoustic features after centralization is calculated. The covariance matrix can be used to represent the probability density of multidimensional random variables, and then the correlation between different acoustic features is captured through the covariance matrix.

[0090] Furthermore, the eigenvalues ​​and eigenvectors of the covariance matrix are solved; the eigenvectors are sorted in descending order according to the eigenvalues, and the eigenvectors corresponding to the first few largest eigenvalues ​​are selected as the target eigenvectors, and the acoustic features are projected onto the determined target eigenvectors to achieve dimensionality reduction.

[0091] In this way, through data dimensionality reduction, the various acoustic information in the phonemes can be reduced to two dimensionless dimensions, while reducing the number of variables in the data set and retaining the important information in the original acoustic information as much as possible.

[0092] In a possible implementation, a linear discriminant analysis method, a non-negative matrix factorization (NMF) method, or a t-SNE (t-Distributed Stochastic Neighbor Embedding) method may be used to perform dimensionality reduction processing on the acoustic features of the extracted phonemes.

[0093] Furthermore, after the acoustic information of the phonemes is subjected to dimensionality reduction processing, it is necessary to perform clustering processing on the multiple phonemes included in each phoneme category, and then determine the phoneme stability interval under each phoneme category based on the multiple clusters obtained by clustering.

[0094] Specifically, for each phoneme category, the acoustic information of the phoneme is clustered through the following steps:

[0095] d1: For each phoneme category, multiple cluster centers of the phoneme category are determined based on the mean value of the phoneme category.

[0096] d2: for each phoneme category, based on the distance between each phoneme and each determined cluster center under the phoneme category, the phonemes under the phoneme category are clustered to obtain multiple clusters.

[0097] In the embodiment of the present application, when clustering the phonemes under each phoneme category, K-means clustering can be used for clustering. K-means clustering iteratively distributes data points into K clusters and continuously updates the centers of these clusters to achieve the purpose of classifying similar data points into one category and separating data points of different categories.

[0098] In a possible implementation, a hierarchical clustering, DBSCAN, spectral clustering or other algorithms may be used to perform clustering processing on multiple phonemes included in each phoneme category.

[0099] In a possible implementation, for each phoneme category, after clustering, the phonemes under the phoneme category are clustered into different clusters, and then the phoneme stability interval of each phoneme category can be determined by analyzing each cluster.

[0100] S104 . For each phoneme category, based on a plurality of clusters corresponding to the phoneme category, determine a phoneme stability interval corresponding to the phoneme category.

[0101] In the embodiment of the present application, after clustering each phoneme category, it is necessary to determine multiple clusters under each phoneme category, and determine the cluster containing a large number of phonemes as a phoneme stable interval.

[0102] Specifically, the step of "for each phoneme category, determining the phoneme stability interval corresponding to the phoneme category based on a plurality of clusters corresponding to the phoneme category" includes:

[0103] e1: For each phoneme category, a cluster containing phonemes whose number is greater than a preset phoneme number threshold in the clusters corresponding to the phoneme category is determined as a classification stability interval corresponding to the phoneme category.

[0104] e2: For each phoneme category, a dimensionality reduction cluster analysis is performed on the environmental classification corresponding to each phoneme in the phoneme category to determine the environmental phoneme stability interval corresponding to the phoneme category.

[0105] In the embodiment of the present application, the preset phoneme quantity threshold can be set according to the clustering results and phoneme screening requirements, and is not specifically limited here.

[0106] In a possible implementation, if it is determined that the phonemes included in the clusters are larger than a preset phoneme number threshold, the phonemes are determined to be relatively stable phonemes.

[0107] Exemplarily, for the phoneme category of vowel a, which includes vowel a1, vowel a2, and vowel a3, when clustering the phoneme category of vowel a, cluster 1 and cluster 2 are generated, wherein the number of phonemes included in cluster 1 is greater than a preset phoneme number threshold, and the number of phonemes included in cluster 2 is less than the preset phoneme number threshold, that is, cluster 1 is a cluster with a larger number, and cluster 2 is a cluster with a larger number. Then, vowels a1 and vowel a3 included in cluster 1 are stable phonemes, and vowel a2 included in cluster 2 is an unstable phoneme.

[0108] In another possible implementation, for each phoneme category, the appearance environment is divided into different environment classifications, stable clusters are captured from each environment classification for analysis, and the acoustic information of the phoneme is subjected to secondary dimensionality reduction.

[0109] Exemplarily, the dimensionality reduction method is Gaussian Mixture Model (GMM). Each Gaussian component has its mean, covariance matrix and mixing coefficient, where the mean represents the center location of the data, the covariance matrix describes the shape and diffusion of the data, and the mixing coefficient represents the weight of each component in the overall distribution. GMM optimizes model parameters by iterative expectation maximization (EM) algorithm, including initialization parameters, E-step calculation of posterior probability and M-step update parameters until the parameters converge.

[0110] The Gaussian mixture model can be used to know the stable value range of a specific phoneme and the impact on stability in different voice environments. Then, the voice stream to be detected is screened according to the stable range of the phoneme, and the voice stream to be detected is analyzed through relatively stable phonemes.

[0111] S105. Based on the phoneme stability interval corresponding to each phoneme category, the voice stream to be detected is screened to determine a plurality of target phonemes in the voice stream to be detected that are located in the phoneme stability interval, and voiceprint comparison and analysis is performed on the voice stream to be detected based on the determined plurality of target phonemes.

[0112] In an embodiment of the present application, for the phoneme stability interval of each phoneme category, the frames corresponding to the target phonemes in the voice stream to be detected that are located in the phoneme stability interval are determined and retained, while the frames of unstable phonemes that are not located in the phoneme stability interval are eliminated, and the voice stream to be detected containing multiple stable phonemes is obtained for voiceprint comparison and analysis, thereby improving the accuracy of voiceprint recognition and voiceprint identification.

[0113] For the above example, it is determined that vowels a1 and a3 in the voice stream to be detected are stable phonemes, and vowel a2 is an unstable phoneme. Therefore, it is necessary to remove the frame where vowel a2 is located from the voice stream to be detected, and retain vowels a1 and a3 in the voice stream to be detected for subsequent voiceprint modeling analysis and voiceprint identification.

[0114] In another possible implementation, by analyzing the stable interval of the phonemes, a phoneme portrait corresponding to each phoneme category can be generated, and then voiceprint analysis and identification can be performed based on the phoneme portrait of each phoneme category.

[0115] Specifically, the screening method further comprises:

[0116] f1: For each phoneme category, determine the phoneme portrait corresponding to the phoneme category based on the acoustic information of dimensionality reduction and clustering processing of each phoneme included in the phoneme category; wherein the phoneme portrait includes the phoneme stability interval and phoneme fluctuation amplitude of the phoneme category.

[0117] f2: Perform voiceprint comparison analysis based on the phoneme portrait corresponding to each phoneme category.

[0118] In an embodiment of the present application, for each phoneme category, the phoneme stability interval and phoneme fluctuation amplitude of the phoneme category can be determined through analysis of the phonemes. Here, the phoneme fluctuation amplitude can be the distance fluctuation range between each phoneme in each stable cluster and the cluster center of the cluster.

[0119] In one possible implementation, the phoneme portraits corresponding to different speakers are quite different. Therefore, a phoneme portrait set of the constructed speaker (including the phoneme portrait of each phoneme) can be used to perform preliminary voiceprint identification on whether the detected speech is the sound of the corresponding speaker, which can improve the accuracy of the voiceprint identification.

[0120] A method for screening phonemes provided in an embodiment of the present application obtains a speech stream to be detected and divides the speech stream to be detected into multiple syllables; determines multiple phonemes contained in each syllable and the phoneme category of each phoneme according to the phoneme classification information corresponding to the target language to which the speech stream to be detected belongs; extracts the acoustic information of each phoneme, and performs dimensionality reduction and clustering processing on the acoustic information of each phoneme to determine multiple clustering clusters corresponding to each phoneme category; for each phoneme category, based on the multiple clustering clusters corresponding to the phoneme category, determines the phoneme stability interval corresponding to the phoneme category; based on the phoneme stability interval corresponding to each phoneme category, screens the speech stream to be detected, determines multiple target phonemes in the speech stream to be detected that are located in the phoneme stability interval, and performs voiceprint comparison and analysis on the speech stream to be detected based on the determined multiple target phonemes. In this way, before detecting the voice stream, phoneme stability analysis is performed based on the phonemes in the voice stream to determine the phoneme stability interval, so as to screen out multiple relatively stable target phonemes within the phoneme stability interval for subsequent voiceprint modeling analysis, so as to improve the accuracy of voiceprint recognition and matching.

[0121] Based on the same inventive concept, the embodiments of the present application also provide a phoneme screening device corresponding to the phoneme screening method. Since the principle of solving the problem by the device in the embodiments of the present application is similar to the above-mentioned phoneme screening method in the embodiments of the present application, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0122] See also Figure 2 , Figure 3 , Figure 2 This is one of the structural schematic diagrams of a phoneme screening device provided in an embodiment of the present application. Figure 3 This is a second structural diagram of a phoneme screening device provided in an embodiment of the present application. Figure 2 As shown in , the screening device 200 comprises:

[0123] A speech stream segmentation module 210 is used to obtain a speech stream to be detected and segment the speech stream to be detected into a plurality of syllables;

[0124] A phoneme classification extraction module 220, configured to determine a plurality of phonemes contained in each syllable and a phoneme category of each phoneme according to the phoneme classification information corresponding to the target language to which the speech stream to be detected belongs;

[0125] The phoneme dimension reduction clustering module 230 is used to extract the acoustic information of each phoneme, and perform dimension reduction and clustering processing on the acoustic information of each phoneme to determine a plurality of clusters corresponding to each phoneme category;

[0126] A phoneme stability interval determination module 240 is used to determine, for each phoneme category, a phoneme stability interval corresponding to the phoneme category based on a plurality of clusters corresponding to the phoneme category;

[0127] The phoneme screening module 250 is used to screen the voice stream to be detected based on the phoneme stability interval corresponding to each phoneme category, determine multiple target phonemes in the voice stream to be detected that are located in the phoneme stability interval, and perform voiceprint comparison analysis on the voice stream to be detected based on the determined multiple target phonemes.

[0128] In a possible implementation, the phoneme classification extraction module 220 is used to determine the phoneme category of the phoneme through the following steps:

[0129] Based on the phoneme classification information, determining the phoneme classification category to which each phoneme in each syllable belongs;

[0130] For each phoneme classification category, based on the voice environment information of the voice stream to be detected, the environment category to which each phoneme contained in the phoneme classification category belongs is determined; wherein the environment category to which the phoneme belongs is a subordinate category of the phoneme classification category to which it belongs.

[0131] In a possible implementation, the phonemes include vowel types and consonant types; when the phoneme dimensionality reduction clustering module 230 is used to extract acoustic information of each phoneme, the phoneme dimensionality reduction clustering module 230 is used to:

[0132] When the phoneme is a vowel type, extracting formant information of the phoneme;

[0133] When the phoneme is of a consonant type, the phoneme is analyzed to extract spectrum characteristics, energy level information, duration information, spectrum envelope information, spectrum tilt information, bandwidth, and glottal wave characteristics of the phoneme.

[0134] In a possible implementation, for each phoneme, the phoneme dimension reduction clustering module 230 is used to perform dimension reduction processing on the acoustic information of the phoneme through the following steps:

[0135] For a plurality of acoustic features included in the acoustic information of the phoneme, each acoustic feature is centralized, and a covariance matrix after the centralization is calculated;

[0136] Determining a plurality of target eigenvectors based on the eigenvalues ​​and eigenvectors of the covariance matrix;

[0137] Each acoustic feature is projected according to the corresponding target feature vector to obtain the acoustic information of the reduced-dimensional phoneme.

[0138] In a possible implementation, for each phoneme category, the phoneme dimension reduction clustering module 230 is used to perform clustering processing on the acoustic information of the phoneme through the following steps:

[0139] For each phoneme category, determine multiple cluster centers of the phoneme category based on the mean value of the phoneme category;

[0140] For each phoneme category, based on the distance between each phoneme and each determined cluster center under the phoneme category, the phonemes under the phoneme category are clustered to obtain a plurality of cluster clusters.

[0141] In a possible implementation, when the phoneme stability interval determination module 240 is used to determine the phoneme stability interval corresponding to each phoneme category based on a plurality of clusters corresponding to the phoneme category, the phoneme stability interval determination module 240 is used to:

[0142] For each phoneme category, a cluster containing a number of phonemes greater than a preset phoneme number threshold in the cluster corresponding to the phoneme category is determined as a classification stability interval corresponding to the phoneme category;

[0143] For each phoneme category, a dimensionality reduction cluster analysis is performed on the environmental classification corresponding to each phoneme in the phoneme category to determine the environmental phoneme stability interval corresponding to the phoneme category.

[0144] In a possible implementation, Figure 3 As shown, the screening device 200 further includes a voiceprint analysis module 260, and the voiceprint analysis module 260 is used to:

[0145] For each phoneme category, based on the acoustic information of the dimensionality reduction and clustering processing of each phoneme included in the phoneme category, determine the phoneme portrait corresponding to the phoneme category; wherein the phoneme portrait includes the phoneme stability interval and the phoneme fluctuation amplitude of the phoneme category;

[0146] Voiceprint comparison and analysis is performed based on the phoneme portrait corresponding to each phoneme category.

[0147] The embodiment of the present application provides a phoneme screening device, which obtains a speech stream to be detected and divides the speech stream to be detected into multiple syllables; determines multiple phonemes contained in each syllable and the phoneme category of each phoneme according to the phoneme classification information corresponding to the target language to which the speech stream to be detected belongs; extracts the acoustic information of each phoneme, and performs dimensionality reduction and clustering processing on the acoustic information of each phoneme to determine multiple clustering clusters corresponding to each phoneme category; for each phoneme category, based on the multiple clustering clusters corresponding to the phoneme category, determines the phoneme stability interval corresponding to the phoneme category; based on the phoneme stability interval corresponding to each phoneme category, screens the speech stream to be detected, determines multiple target phonemes in the speech stream to be detected that are located in the phoneme stability interval, and performs voiceprint comparison and analysis on the speech stream to be detected based on the determined multiple target phonemes. In this way, before detecting the voice stream, phoneme stability analysis is performed based on the phonemes in the voice stream to determine the phoneme stability interval, so as to screen out multiple relatively stable target phonemes within the phoneme stability interval for subsequent voiceprint modeling analysis, so as to improve the accuracy of voiceprint recognition and matching.

[0148] See also Figure 4 , Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 4 As shown in , the electronic device 400 includes a processor 410 , a memory 420 and a bus 430 .

[0149] The memory 420 stores machine-readable instructions executable by the processor 410. When the electronic device 400 is running, the processor 410 communicates with the memory 420 via the bus 430. When the machine-readable instructions are executed by the processor 410, the above-mentioned Figure 1 The specific implementation of the steps of the phoneme screening method in the method embodiment shown can be found in the method embodiment, and will not be repeated here.

[0150] The present application also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the computer program can execute the above-mentioned Figure 1 The specific implementation of the steps of the phoneme screening method in the method embodiment shown can be found in the method embodiment, and will not be repeated here.

[0151] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0152] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0153] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0154] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0155] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present application can essentially be embodied in the form of a software product, or in other words, the part that contributes to the prior art or the part of the technical solution. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0156] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present application, which are used to illustrate the technical solutions of the present application, rather than to limit them. The protection scope of the present application is not limited thereto. Although the present application is described in detail with reference to the above-mentioned embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-mentioned embodiments within the technical scope disclosed in the present application, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A method for screening phonemes, characterized in that: The screening method comprises: Acquire a speech stream to be detected, and divide the speech stream to be detected into a plurality of syllables; Determine a plurality of phonemes contained in each syllable and a phoneme category of each phoneme according to the phoneme classification information corresponding to the target language to which the speech stream to be detected belongs; Extracting acoustic information of each phoneme, performing dimensionality reduction and clustering processing on the acoustic information of each phoneme, and determining multiple clusters corresponding to each phoneme category; For each phoneme category, based on a plurality of clusters corresponding to the phoneme category, determine a phoneme stability interval corresponding to the phoneme category; Based on the phoneme stability interval corresponding to each phoneme category, the voice stream to be detected is screened, and multiple target phonemes located in the phoneme stability interval in the voice stream to be detected are determined, so as to perform voiceprint comparison analysis on the voice stream to be detected based on the determined multiple target phonemes.

2. The screening method according to claim 1, characterized in that The phoneme category of the phoneme is determined by the following steps: Based on the phoneme classification information, determining the phoneme classification category to which each phoneme in each syllable belongs; For each phoneme classification category, based on the voice environment information of the voice stream to be detected, the environment category to which each phoneme contained in the phoneme classification category belongs is determined; wherein the environment category to which the phoneme belongs is a subordinate category of the phoneme classification category to which it belongs.

3. The screening method according to claim 1, characterized in that The phonemes include vowel types and consonant types; and the extracting of acoustic information of each phoneme includes: When the phoneme is a vowel type, extracting formant information of the phoneme; When the phoneme is of a consonant type, the phoneme is analyzed to extract spectrum characteristics, energy level information, duration information, spectrum envelope information, spectrum tilt information, bandwidth, and glottal wave characteristics of the phoneme.

4. The screening method according to claim 1, characterized in that For each phoneme, the acoustic information of the phoneme is reduced in dimension through the following steps: For a plurality of acoustic features included in the acoustic information of the phoneme, each acoustic feature is centralized, and a covariance matrix after the centralization is calculated; Determining a plurality of target eigenvectors based on the eigenvalues ​​and eigenvectors of the covariance matrix; Each acoustic feature is projected according to the corresponding target feature vector to obtain the acoustic information of the reduced-dimensional phoneme.

5. The screening method according to claim 1, characterized in that For each phoneme category, the acoustic information of the phoneme is clustered by the following steps: For each phoneme category, determine multiple cluster centers of the phoneme category based on the mean value of the phoneme category; For each phoneme category, based on the distance between each phoneme and each determined cluster center under the phoneme category, the phonemes under the phoneme category are clustered to obtain a plurality of cluster clusters.

6. The screening method according to claim 1, characterized in that The method of determining, for each phoneme category, a phoneme stability interval corresponding to the phoneme category based on a plurality of clusters corresponding to the phoneme category, includes: For each phoneme category, a cluster containing a number of phonemes greater than a preset phoneme number threshold in the cluster corresponding to the phoneme category is determined as a classification stability interval corresponding to the phoneme category; For each phoneme category, a dimensionality reduction cluster analysis is performed on the environmental classification corresponding to each phoneme in the phoneme category to determine the environmental phoneme stability interval corresponding to the phoneme category.

7. The screening method according to claim 1, characterized in that The screening method further comprises: For each phoneme category, based on the acoustic information of the dimensionality reduction and clustering processing of each phoneme included in the phoneme category, determine the phoneme portrait corresponding to the phoneme category; wherein the phoneme portrait includes the phoneme stability interval and the phoneme fluctuation amplitude of the phoneme category; Voiceprint comparison and analysis is performed based on the phoneme portrait corresponding to each phoneme category.

8. A phoneme screening device, characterized in that: The screening device comprises: A voice stream segmentation module, used for acquiring a voice stream to be detected and segmenting the voice stream to be detected into a plurality of syllables; A phoneme classification extraction module, used to determine a plurality of phonemes contained in each syllable and a phoneme category of each phoneme according to the phoneme classification information corresponding to the target language to which the speech stream to be detected belongs; The phoneme dimensionality reduction and clustering module is used to extract the acoustic information of each phoneme, and perform dimensionality reduction and clustering processing on the acoustic information of each phoneme to determine multiple clusters corresponding to each phoneme category; A phoneme stability interval determination module, for determining, for each phoneme category, a phoneme stability interval corresponding to the phoneme category based on a plurality of clusters corresponding to the phoneme category; The phoneme screening module is used to screen the voice stream to be detected based on the phoneme stability interval corresponding to each phoneme category, determine multiple target phonemes in the voice stream to be detected that are located in the phoneme stability interval, and perform voiceprint comparison analysis on the voice stream to be detected based on the determined multiple target phonemes.

9. An electronic device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the memory via the bus, and the processor executes the machine-readable instructions to perform the steps of the phoneme screening method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed by a processor, executes the steps of the phoneme screening method according to any one of claims 1 to 7.

Images