A cross-lingual speaker recognition method and system based on speech fusion features

Abstract

The application provides a cross-lingual speaker recognition method and system based on speech fusion features, and relates to the technical field of speech processing. The method extracts voiced segments and integrates them into new speech segments by band-pass filtering the original speech signal. The pitch period and MFCC features in the new speech segments are extracted and concatenated to form a fusion feature vector. Multilingual speech segments of multiple testers are input, fusion features are extracted, and GMM models of each speaker are trained using K-means clustering and EM algorithm. The test speech segment is input, the fusion features are extracted, the likelihood probability score is calculated with the GMM model, compared with the preset threshold, and the tester identity is confirmed or rejected. The application can extract more stable features from speech signals, thereby improving the accuracy of cross-lingual speaker recognition, effectively dealing with speaker language mismatch situations, improving recognition performance, and having strong adaptability, especially in bilingual or multilingual application scenarios.

Description

Technical Field

[0001] This invention relates to the field of speech processing technology, and in particular to a cross-lingual speaker recognition method and system based on speech fusion features. Background Technology

[0002] Speaker recognition systems have wide and important applications in fields such as national defense and finance. Under conditions of sufficient and high-quality data, the system performs well; however, its performance deteriorates significantly when there is a mismatch between the registered and test voice. One serious mismatch is language mismatch, where the registered and test voices of the same speaker are in different languages, typically occurring in bilingual or multilingual populations. Considering that bilingual populations are prevalent not only in my country but also worldwide, cross-language speaker verification has broad application prospects.

[0003] When the registered and test voices are in different languages, the performance of speaker identification methods deteriorates significantly. This is largely attributed to the different distributions of acoustic features across languages. The main challenge in cross-language speaker identification lies in the unique linguistic factors inherent in different languages, such as phonemes, tones, and the relaxation of the vocal organs during pronunciation. To address this issue, there is a pressing need for a method that fuses fundamental tone parameters related to human auditory features (MFCC) with information about the human vocal tract. While the fundamental tone period is related to the speaker's vocal organs, pronunciation habits, and the text, statistically, the distribution of fundamental tone periods differs among speakers, which can be used to distinguish between them. Summary of the Invention

[0004] In view of this, the purpose of this invention is to propose a cross-lingual speaker recognition method and system based on speech fusion features. The aim is to improve the accuracy of cross-lingual speaker recognition by fusing features such as pitch period and Mel-Frequency Cepstral Coefficients (MFCCs) and combining them with the Teager Energy Operator (TEO) to extract more stable features from the speech signal. This method can effectively address speaker language mismatch situations and improve recognition performance, especially in bilingual or multilingual application scenarios, where it has strong adaptability.

[0005] To achieve the above objectives, the present invention provides the following technical solution:

[0006] To achieve the above objectives, in a first aspect, the present invention provides a cross-lingual speaker recognition method based on speech fusion features, comprising the following steps:

[0007] The original speech signal is bandpass filtered to extract voiced segments and integrate them into a new speech segment;

[0008] Extract pitch period and MFCC features from new speech segments and concatenate them to form a fused feature vector;

[0009] Input multilingual speech segments from multiple testers, extract fusion features, and train a Gaussian Mixture Model (GMM) for each speaker using K-means clustering and the EM algorithm.

[0010] Input a test speech segment, extract fusion features, calculate the likelihood probability score with the GMM model, compare it with a preset threshold, and confirm or reject the tester's identity.

[0011] As a further aspect of the present invention, bandpass filtering is performed on the original speech signal to extract voiced segments and integrate them into a new speech segment, including the following steps:

[0012] The original speech signal is filtered to remove 50Hz DC noise and high-frequency noise greater than 4kHz, and then bandpass filtered in the 60Hz to 4kHz frequency band.

[0013] Perform a Fast Fourier Transform (FFT) on the filtered signal and calculate the Teager Energy Operator (TEO).

[0014] Voiced segments are extracted based on a preset threshold, the corresponding speech frame number is taken, and the voiced segments are concatenated and integrated into a new speech segment, which is used as the original speech for extracting and fusing features.

[0015] As a further aspect of the present invention, the pitch period and MFCC features in the new speech segment are extracted and concatenated to form a fused feature vector, including the following steps:

[0016] Voiced segments are extracted and integrated by calculating the TEO value of the speech signal;

[0017] Extract pitch period and MFCC features from new speech segments;

[0018] The obtained 1-dimensional pitch period and MFCC are concatenated to form a new fused feature vector.

[0019] As a further aspect of the present invention, training a GMM model for each speaker includes the following steps:

[0020] Input multilingual speech segments from multiple testers and extract fusion features from each speaker's speech segment;

[0021] The K-means clustering method was used to cluster the multilingual speech data to obtain the initial GMM parameters for each speaker;

[0022] The GMM model for each speaker's speech segment is trained using the Expectation-Maximization Algorithm (EM) to fit the speech features of each speaker.

[0023] As a further aspect of the present invention, confirming or rejecting the identity of a tester includes the following steps:

[0024] During the testing phase, the tester's speech segment is input, and the tester's fusion features are extracted.

[0025] Calculate the likelihood probability score between the tester's speech features and the GMM model of the known speaker model;

[0026] The calculated likelihood probability score is compared with a set threshold. When the score is greater than the threshold, the test subject is considered a known speaker; when the score is less than the threshold, the test subject is rejected as a known speaker, and the recognition result is output.

[0027] Secondly, the present invention also provides a cross-lingual speaker recognition system based on speech fusion features, comprising the following components:

[0028] Voiced Segment Extraction Module: Used to extract and integrate voiced segments;

[0029] Fusion feature extraction module: used to extract pitch period and MFCC features and form a fusion feature vector;

[0030] GMM training module: Used for training Gaussian Mixture Models (GMM);

[0031] Speaker verification module: used to calculate the likelihood probability score between the test speech and the GMM model, and compare it with a preset threshold to confirm or reject the tester's identity.

[0032] As a further aspect of the present invention, the voiced segment extraction module includes:

[0033] Bandpass filter: Used for bandpass filtering of raw speech signals from 60Hz to 4kHz;

[0034] FFT calculation unit: used to perform Fast Fourier Transform (FFT) on the filtered signal;

[0035] TEO computational unit: used to compute the Teager energy operator (TEO);

[0036] Voiced Segment Integration Unit: Used to extract voiced segments according to a preset threshold and integrate the voiced segments into a new speech segment.

[0037] As a further aspect of the present invention, the fusion feature extraction module includes:

[0038] Pitch period extraction unit: used to extract pitch periods from new speech segments;

[0039] MFCC Extraction Unit: Used to extract MFCC features from new speech segments;

[0040] Feature fusion unit: used to concatenate the pitch period and MFCC features to form a fused feature vector.

[0041] As a further aspect of the present invention, the GMM training module includes:

[0042] Fusion Feature Extraction Unit: Used to extract fusion features from each speaker by inputting multilingual speech segments from multiple testers;

[0043] K-means clustering unit: used to initialize GMM parameters;

[0044] EM Algorithm Unit: Used to train the GMM model for each speaker.

[0045] As a further aspect of the present invention, the speaker confirmation module includes:

[0046] Fusion Feature Extraction Unit: Used to extract fusion features from input test speech segments;

[0047] Likelihood probability calculation unit: used to calculate the likelihood probability score between the test speech and the GMM model;

[0048] Threshold comparison unit: Used to compare with a preset threshold to confirm or reject the tester's identity.

[0049] Compared with existing technologies, the cross-lingual speaker recognition method and system based on speech fusion features proposed in this invention have the following advantages:

[0050] 1. This invention, by fusing multiple features including pitch period and Mel frequency cepstral coefficients (MFCC), can effectively reduce feature differences between different languages ​​and significantly improve the accuracy of cross-language speaker recognition. In bilingual or multilingual environments, the system can adapt well to the pronunciation habits and acoustic characteristics of different languages, overcoming the performance degradation caused by language mismatch.

[0051] 2. When faced with differences in pronunciation, phoneme variations, and tonal differences between different languages, the system of this invention, through the fusion of pitch period and MFCC features, makes the recognition system highly robust to changes in speech signals. Especially when there are significant differences in pronunciation habits across languages, it can still maintain high recognition accuracy, significantly improving the stability and reliability of the system.

[0052] 3. This method employs the efficient Teager energy operator and MFCC calculation during feature extraction, resulting in lower computational cost and improved real-time performance. This characteristic enables the method to function effectively in applications requiring rapid response, making it suitable for various real-time-critical fields such as security monitoring, financial transactions, and intelligent security. By combining pitch period with MFCC features and utilizing a Gaussian Mixture Model (GMM) for modeling, this invention can more accurately capture speaker-specific characteristics, particularly in multilingual environments, thus improving speaker recognition accuracy. This method achieves a balance across different languages, avoiding the performance drop issues common to traditional methods during language switching.

[0053] 4. This invention optimizes and adjusts parameter settings during the feature extraction process through experiments, enabling the system to adaptively adjust the feature extraction method based on different speech data, further improving adaptability and recognition performance in cross-linguistic environments. Furthermore, dynamic threshold adjustment allows the system to optimize recognition decisions based on real-time data, improving recognition flexibility and accuracy. This invention is applicable to identity authentication and security monitoring in multilingual environments, particularly suitable for applications in bilingual or multilingual regions. It also has broad application areas, especially in high-security fields such as national defense, finance, and public security, providing users with efficient and accurate speaker recognition technology support.

[0054] In summary, the cross-lingual speaker recognition method and system based on speech fusion features provided by this invention have excellent cross-lingual adaptability, robustness, real-time performance, and high-precision recognition capabilities. They can effectively improve speaker recognition performance in multilingual environments, adapt to a wider range of application scenarios, and have strong market competitiveness and application value.

[0055] These or other aspects of this application will become more apparent from the following description of embodiments. It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only, and are not intended to limit the application. Attached Figure Description

[0056] To more clearly illustrate the technical solutions in the embodiments of the present invention or related technologies, the accompanying drawings used in the description of the exemplary embodiments or related technologies will be briefly introduced below. The drawings are used to provide a further understanding of the present invention and constitute a part of the specification. They are used together with the embodiments of the present invention to explain the present invention and do not constitute a limitation thereof. In the drawings:

[0057] Figure 1 This is a flowchart of a cross-language speaker recognition method based on speech fusion features, according to an embodiment of the present invention.

[0058] Figure 2 This is a flowchart of the fusion feature process in a cross-language speaker recognition method based on speech fusion features according to an embodiment of the present invention.

[0059] Figure 3 This is a flowchart of the testing phase in a cross-language speaker recognition method based on speech fusion features according to an embodiment of the present invention. Detailed Implementation

[0060] The present application will now be further described in conjunction with the accompanying drawings and specific embodiments. It should be noted that, without conflict, the various embodiments or technical features described below can be arbitrarily combined to form new embodiments.

[0061] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be further described in detail below with reference to specific examples and the accompanying drawings. It should be understood that the specific embodiments described herein are merely illustrative and are not intended to limit the scope of this application.

[0062] It should be noted that all uses of "first" and "second" in the embodiments of the present invention are for the purpose of distinguishing two different entities or different parameters with the same name. Therefore, "first" and "second" are merely for convenience of expression and should not be construed as limiting the embodiments of the present invention. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, such as other steps or units inherent in a process, method, system, product, or device that includes a series of steps or units.

[0063] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0064] The flowchart shown in the attached diagram is for illustrative purposes only and does not necessarily include all content and operations / steps, nor does it necessarily have to be performed in the order described. For example, some operations / steps can be broken down, combined, or partially merged, so the actual execution order may change depending on the actual situation.

[0065] The following detailed description of some embodiments of this application is provided in conjunction with the accompanying drawings. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0066] The challenge of cross-linguistic speaker recognition lies primarily in the unique linguistic factors inherent in different languages, such as phonemes, tones, and the tension and relaxation of the articulatory organs during pronunciation. To address this issue, this invention proposes a cross-linguistic speaker recognition method based on speech fusion features. The pitch period is related to the speaker's articulatory organs, pronunciation habits, and the text itself; however, statistically, the pitch period distribution differs among different speakers, which can be used to distinguish between them. The Teager energy operator offers excellent temporal resolution and is simple and fast, enabling real-time tracking of the time-varying components of the speech signal. A more accurate instruction table is used to obtain a more accurate threshold h.

[0067] See Figure 1 As shown, an embodiment of the present invention provides a cross-lingual speaker recognition method based on speech fusion features, the method comprising the following steps:

[0068] Step 1: Perform bandpass filtering on the original speech signal, extract voiced segments, and integrate them into a new speech segment;

[0069] Step 2: Extract the pitch period and MFCC features from the new speech segment, and concatenate them to form a fused feature vector;

[0070] Step 3: Input multilingual speech segments from multiple testers, extract fusion features, and train a Gaussian Mixture Model (GMM) for each speaker using K-means clustering and the EM algorithm.

[0071] Step 4: Input the test speech segment, extract the fusion features, calculate the likelihood probability score with the GMM model, compare it with the preset threshold, and confirm or reject the tester's identity.

[0072] In this invention, voiced segments are extracted from the original speech, concatenated to form a new speech segment, and the pitch period and MFCC are extracted from the new speech segment. The obtained pitch period and MFCC are then used to form a fusion feature F. t ; Calculate F t With the assumed speaker GMM model parameters λ i Likelihood probability score The result is compared with a threshold; if the result is greater than the threshold, the test subject is considered the speaker; otherwise, the test subject is rejected, and a confirmation result is output. Here, the GMM model parameter λ... i The method of obtaining it is:

[0073] Input N testers' multilingual voice recordings;

[0074] Calculate and extract fusion features of multilingual speech for each speaker;

[0075] The initialization parameters λ0 of the speaker model are obtained using the K-means clustering method.

[0076] Based on the initialization parameter λ0, the GMM model with a mixture degree M = 128 for each speaker is calculated using the EM algorithm. L ={λ1, ...,λ i , …, λ N}, λ i For the parameters of the GMM model of the i-th tester, output G. L G L This is a set containing all GMM model parameters for all testers.

[0077] In an embodiment of the present invention, the extraction and integration of voiced segments in step one involves bandpass filtering the original speech signal s(n), extracting voiced segments, and integrating them into a new speech segment, including the following steps:

[0078] Step 1: Filter out 50Hz DC noise and high-frequency noise greater than 4kHz from the original speech signal, and perform bandpass filtering processing using a bandpass filter H(n) in the 60Hz–4kHz frequency band; where:

[0079] x(n)=s(n)×H(n), n=1,2,…,N;

[0080] In the formula, x(n) is a signal with a limited frequency band; s(n) is the original speech signal; H(n) is a bandpass filter with a frequency band of 60Hz-4kHz, used to perform bandpass filtering on the original speech signal s(n) to filter out noise outside the specified frequency range.

[0081] Step 2: Perform a Fast Fourier Transform (FFT) on the filtered signal and calculate the Teager Energy Operator (TEO); specifically, perform a Fast Fourier Transform on the finite-bandwidth signal X(n) to obtain:

[0082]

[0083] In the formula, X(k) is the complex value corresponding to the k-th frequency point in the frequency domain after performing a Fast Fourier Transform (FFT) on the signal X(n) with a finite frequency band, representing the amplitude and phase information of that frequency point; N is the total number of sampling points of the discrete signal X(n); j is the imaginary unit, used to represent complex number operations in the Fourier transform; k is the frequency point index in the frequency domain, ranging from 1 to N, corresponding to different frequency components;

[0084] When calculating the Teager energy operator, it can effectively track the instantaneous energy of a signal, making it useful for time-varying speech signal analysis. For a finite-bandwidth signal x(n), the operator can be approximated as:

[0085] φ[x(n)]=[x(n)] 2 -x(n+1)x(n-1)

[0086] In the formula, φ[x(n)] is the result of the Teager energy operator calculation for the finite-band signal X(n), used to track the instantaneous energy of the signal; x(n+1) is the sampled value of the signal X(n) at time n+1; x(n-1) is the sampled value of the signal X(n) at time n-1;

[0087] TEO envelope extraction calculates the envelope using three adjacent sampling points of the detected waveform. It offers excellent time resolution and is simple, fast, and can track the time-varying components of speech signals in real time. The calculation formula for this step is:

[0088] t(k)=φ[X(k)]

[0089] In the formula, t(k) is the result obtained after calculating the Teager energy operator on the frequency domain signal X(k) after the fast Fourier transform, which is used for subsequent operations such as extracting voiced segments.

[0090] Step 3: Extract voiced segments according to the preset threshold, take the speech frame number corresponding to the voiced segments, and concatenate the voiced segments to integrate them into a new speech segment, which is used as the original speech for extracting and fusing features.

[0091] Among them, the voiced segment v(k) is selected. Since the voiced sound t(k) is greater than the unvoiced sound and the silence, a threshold h can be used to extract the voiced segment:

[0092]

[0093] The voiced threshold is set to h = 0.02. The speech frame number corresponding to the voiced segment is taken, and the voiced segments are concatenated together to form a new speech segment, which is used as the original speech for extracting fusion features.

[0094] In the embodiment, in step two, see... Figure 2 As shown, the pitch period and MFCC features are extracted from the new speech segment and concatenated to form a fused feature vector, including the following steps:

[0095] Voiced segments are extracted and integrated by calculating the TEO value of the speech signal;

[0096] Extract pitch period and MFCC features from new speech segments;

[0097] The obtained 1-dimensional pitch period and MFCC are concatenated to form a new fused feature vector.

[0098] Step three involves training a GMM model for each speaker, including the following steps:

[0099] Input multilingual speech segments from multiple testers and extract fusion features from each speaker's speech segment;

[0100] The K-means clustering method was used to cluster the multilingual speech data to obtain the initial GMM parameters for each speaker;

[0101] The GMM model for each speaker's speech segment is trained using the Expectation-Maximization Algorithm (EM) to fit the speech features of each speaker.

[0102] In this embodiment, after obtaining the fused features, they are applied to the training of the model, wherein N testers' speech segments S in English / Korean / Japanese, etc., are input:

[0103] S={s i}, i = 1, 2, ..., N;

[0104] l∈{English,Korean,Japenese,Mongolian}

[0105] In the formula, s i Let N be the set of all speech segments of the i-th tester, where i ranges from 1 to N and N is the number of testers; Let j represent the speech segment of the i-th tester in the l-th language, where l represents the language and j represents the number of frames in the speech segment of the i-th tester in the l-th language.

[0106] 1. Calculate and extract the fusion feature parameters of multilingual speech for each speaker.

[0107] F = {F1, F2, ..., F} i F n};F i ={f1, ..., f t}, where T is the number of frames; F is the set of multilingual speech fusion features from all testers; F n F represents the fusion feature set of the multilingual speech of the nth tester; i f is the fusion feature set of the multilingual speech of the i-th tester; t For the i-th tester, the feature of the t-th frame in the fused feature set is t, where t represents the frame number, ranging from 1 to T, and T is the frame number;

[0108] 2. The initialization parameters λ0 of the speaker model are obtained using the K-means clustering method;

[0109] 3. Based on the initialization parameter λ0, the GMM model with a mixing degree M = 128 for each speaker is calculated using the EM algorithm. L={λ1, ...,λ i , …, λ N}, λ i For the parameters of the GMM model of the i-th tester, output G. L .

[0110] The model validation method is as follows:

[0111] Input a Chinese speech segment s from a tester t GMM model with N speaker models L Threshold (Threshold = 20);

[0112] 1. Calculate and extract the fusion feature parameter F of the test subject's Chinese speech. t ;

[0113] 2. Calculate F t With the assumed speaker GMM model parameters λ i Likelihood probability score The result is compared with a threshold. If the result is greater than the threshold, the test subject is considered the speaker; otherwise, the test subject is rejected, and the confirmation result is output.

[0114] See Figure 3 As shown, in this embodiment, the Gaussian Mixture Model (GMM) is used as the main feature model for cross-lingual speaker recognition. The speaker's GMM model is trained using the K-means clustering method and the EM algorithm. In the testing phase of this invention, the maximum likelihood function method is used to calculate the score between the test speech and the GMM model, and then the score is compared with a threshold to obtain the decision result.

[0115] In this embodiment, step four, confirming or rejecting the tester's identity, includes the following steps:

[0116] During the testing phase, the tester's speech segment is input, and the tester's fusion features are extracted.

[0117] Calculate the likelihood probability score between the tester's speech features and the GMM model of the known speaker model;

[0118] The calculated likelihood probability score is compared with a set threshold. When the score is greater than the threshold, the test subject is considered a known speaker; when the score is less than the threshold, the test subject is rejected as a known speaker, and the recognition result is output.

[0119] This invention effectively reduces feature differences between different languages ​​and significantly improves the accuracy of cross-linguistic speaker recognition by fusing multiple features, including pitch period and Mel frequency cepstral coefficients (MFCC). In bilingual or multilingual environments, it can adapt well to the pronunciation habits and acoustic characteristics of different languages, overcoming the performance degradation caused by language mismatch.

[0120] The method of this invention employs the efficient Teager energy operator and MFCC calculation method during feature extraction, resulting in a smaller computational load and thus improving the real-time performance of the entire process. This characteristic enables the method to function effectively in applications requiring rapid response, making it suitable for various fields with high real-time requirements, such as security monitoring, financial transactions, and intelligent security. By combining pitch period with MFCC features and using a Gaussian mixture model (GMM) for modeling, this invention can more accurately capture the speaker's personalized features, especially in multilingual environments, improving speaker recognition accuracy. This method achieves a balance across different languages, avoiding the performance drop problem of traditional methods when switching languages.

[0121] It should be noted that the above figures are merely illustrative of the processes included in the method according to exemplary embodiments of the present invention, and are not intended to be limiting. It is readily understood that the processes shown in the above figures do not indicate or limit the temporal order of these processes. Furthermore, it is readily understood that these processes may, for example, be executed synchronously or asynchronously in multiple modules.

[0122] It should be understood that although the above description follows a certain order, these steps are not necessarily executed in that order. Unless otherwise expressly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, some steps in this embodiment may include multiple steps or multiple stages, which are not necessarily completed at the same time, but may be executed at different times. The execution order of these steps or stages is not necessarily sequential, but may be performed alternately or in turn with other steps or at least a portion of steps or stages in other steps.

[0123] Embodiments of the present invention also provide a cross-lingual speaker recognition system based on speech fusion features. This system is used to execute the aforementioned cross-lingual speaker recognition method. The cross-lingual speaker recognition system includes:

[0124] Voiced Segment Extraction Module: Used to extract and integrate voiced segments;

[0125] Fusion feature extraction module: used to extract pitch period and MFCC features and form a fusion feature vector;

[0126] GMM training module: Used for training Gaussian Mixture Models (GMM);

[0127] Speaker verification module: used to calculate the likelihood probability score between the test speech and the GMM model, and compare it with a preset threshold to confirm or reject the tester's identity.

[0128] The voiced segment extraction module includes:

[0129] Bandpass filter: Used for bandpass filtering of raw speech signals from 60Hz to 4kHz;

[0130] FFT calculation unit: used to perform Fast Fourier Transform (FFT) on the filtered signal;

[0131] TEO computational unit: used to compute the Teager energy operator (TEO);

[0132] Voiced Segment Integration Unit: Used to extract voiced segments according to a preset threshold and integrate the voiced segments into a new speech segment.

[0133] The fusion feature extraction module includes:

[0134] Pitch period extraction unit: used to extract pitch periods from new speech segments;

[0135] MFCC Extraction Unit: Used to extract MFCC features from new speech segments;

[0136] Feature fusion unit: used to concatenate the pitch period and MFCC features to form a fused feature vector.

[0137] The GMM training module includes:

[0138] Fusion Feature Extraction Unit: Used to extract fusion features from each speaker by inputting multilingual speech segments from multiple testers;

[0139] K-means clustering unit: used to initialize GMM parameters;

[0140] EM Algorithm Unit: Used to train the GMM model for each speaker.

[0141] The speaker confirmation module includes:

[0142] Fusion Feature Extraction Unit: Used to extract fusion features from input test speech segments;

[0143] Likelihood probability calculation unit: used to calculate the likelihood probability score between the test speech and the GMM model;

[0144] Threshold comparison unit: Used to compare with a preset threshold to confirm or reject the tester's identity.

[0145] This invention optimizes and adjusts parameter settings during the feature extraction process through experiments, enabling the system to adaptively adjust the feature extraction method based on different speech data, further improving adaptability and recognition performance in cross-linguistic environments. Furthermore, dynamic threshold adjustment allows the system to optimize recognition decisions based on real-time data, enhancing recognition flexibility and accuracy. This invention is applicable to identity authentication and security monitoring in multilingual environments, and is particularly suitable for applications in bilingual or multilingual regions. Simultaneously, this invention has a wide range of applications, especially in high-security fields such as national defense, finance, and public security, providing users with efficient and accurate speaker recognition technology support.

[0146] When faced with differences in pronunciation, phoneme variations, and tonal differences across languages, the system of this invention, through the fusion of pitch period and MFCC features, makes the recognition system highly robust to changes in speech signals. Especially when there are significant differences in pronunciation habits across languages, it can still maintain high recognition accuracy, significantly improving the stability and reliability of the system.

[0147] Through the detailed steps described above, the cross-lingual speaker recognition system based on speech fusion features of the present invention is used to execute the steps of the cross-lingual speaker recognition method based on speech fusion features in the above embodiments, which will not be repeated here. The cross-lingual speaker recognition method and system based on speech fusion features provided by the present invention have excellent cross-lingual adaptability, robustness, real-time performance, and high-precision recognition capabilities. It can effectively improve speaker recognition performance in multilingual environments, adapt to a wider range of application scenarios, and has strong market competitiveness and application value.

[0148] The above are exemplary embodiments disclosed in this invention. However, it should be noted that various changes and modifications can be made without departing from the scope of the embodiments of this invention as defined by the claims. The functions, steps, and / or actions of the methods according to the disclosed embodiments described herein do not need to be performed in any particular order. Furthermore, although the elements disclosed in the embodiments of this invention may be described or claimed individually, they may be understood as multiple unless explicitly limited to a singular number.

[0149] It should be understood that, as used herein, the singular form "a" is intended to include the plural form as well, unless the context clearly supports an exception. It should also be understood that, as used herein, "and / or" refers to any and all possible combinations of one or more of the associatedly listed items. The embodiment numbers disclosed above are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0150] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of the invention (including the claims) is limited to these examples. Within the framework of the invention, technical features of the above embodiments or different embodiments can be combined, and many other variations of different aspects of the invention exist, which are not provided in the details for the sake of brevity. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the invention should be included within the protection scope of the invention.

Claims

1. A cross-lingual speaker recognition method based on speech fusion features, characterized in that, The method includes the following steps: The original speech signal is bandpass filtered to extract voiced segments and integrate them into a new speech segment; Extract pitch period and MFCC features from new speech segments and concatenate them to form a fused feature vector; Input multilingual speech segments from multiple testers, extract fusion features, and train the GMM model for each speaker using K-means clustering and the EM algorithm; Input a test speech segment, extract fusion features, calculate the likelihood probability score with the GMM model, compare it with a preset threshold, and confirm or reject the tester's identity. The process of extracting pitch period and MFCC features from new speech segments and concatenating them to form a fused feature vector includes the following steps: Voiced segments are extracted and integrated by calculating the TEO value of the speech signal; Extract pitch period and MFCC features from new speech segments; The obtained 1-dimensional pitch period and MFCC are concatenated to form a new fused feature vector.

2. The cross-lingual speaker recognition method based on speech fusion features as described in claim 1, characterized in that, Bandpass filtering is performed on the original speech signal to extract voiced segments and integrate them into a new speech segment. This includes the following steps: The original speech signal is filtered to remove 50Hz DC noise and high-frequency noise greater than 4kHz, and then bandpass filtered in the 60Hz~4kHz frequency band. Perform a Fast Fourier Transform on the filtered signal and calculate the Teager energy operator; Voiced segments are extracted based on a preset threshold, the corresponding speech frame number is taken, and the voiced segments are concatenated and integrated into a new speech segment, which is used as the original speech for extracting and fusing features.

3. The cross-lingual speaker recognition method based on speech fusion features as described in claim 1, characterized in that, Training a GMM model for each speaker involves the following steps: Input multilingual speech segments from multiple testers and extract fusion features from each speaker's speech segment; The K-means clustering method was used to cluster the multilingual speech data to obtain the initial GMM parameters for each speaker; The GMM model for each speaker's speech segment is trained using the EM algorithm to fit the speech features of each speaker.

4. The cross-lingual speaker recognition method based on speech fusion features as described in claim 3, characterized in that, Confirming or rejecting a test taker's identity includes the following steps: During the testing phase, the tester's speech segment is input, and the tester's fusion features are extracted. Calculate the likelihood probability score between the tester's speech features and the GMM model of the known speaker model; The calculated likelihood probability score is compared with a set threshold. When the score is greater than the threshold, the test subject is considered a known speaker; when the score is less than the threshold, the test subject is rejected as a known speaker, and the recognition result is output.

5. A cross-lingual speaker recognition system based on speech fusion features, characterized in that, For performing the cross-lingual speaker recognition method based on speech fusion features as described in any one of claims 1-4, the cross-lingual speaker recognition system comprises: Voiced Segment Extraction Module: Used to extract and integrate voiced segments; Fusion Feature Extraction Module: Used to extract pitch period and MFCC features and form a fused feature vector; it includes the following steps: Voiced segments are extracted and integrated by calculating the TEO value of the speech signal; Extract pitch period and MFCC features from new speech segments; The 1-dimensional pitch period and MFCC are concatenated to form a new fused feature vector; GMM training module: used to train Gaussian mixture models; Speaker verification module: used to calculate the likelihood probability score between the test speech and the GMM model, and compare it with a preset threshold to confirm or reject the tester's identity.

6. The cross-lingual speaker recognition system based on speech fusion features as described in claim 5, characterized in that, The voiced segment extraction module includes: Bandpass filter: Used for bandpass filtering of the original speech signal from 60Hz to 4kHz; FFT calculation unit: used to perform fast Fourier transform on the filtered signal; TEO computational unit: used to compute the Teager energy operator; Voiced Segment Integration Unit: Used to extract voiced segments according to a preset threshold and integrate the voiced segments into a new speech segment.

7. The cross-lingual speaker recognition system based on speech fusion features as described in claim 5, characterized in that, The fusion feature extraction module includes: Pitch period extraction unit: used to extract pitch periods from new speech segments; MFCC Extraction Unit: Used to extract MFCC features from new speech segments; Feature fusion unit: used to concatenate the pitch period and MFCC features to form a fused feature vector.

8. The cross-lingual speaker recognition system based on speech fusion features as described in claim 5, characterized in that, The GMM training module includes: Fusion Feature Extraction Unit: Used to extract fusion features from each speaker by inputting multilingual speech segments from multiple testers; K-means clustering unit: used to initialize GMM parameters; EM Algorithm Unit: Used to train the GMM model for each speaker.

9. The cross-lingual speaker recognition system based on speech fusion features as described in claim 5, characterized in that, The speaker confirmation module includes: Fusion Feature Extraction Unit: Used to extract fusion features from input test speech segments; Likelihood probability calculation unit: used to calculate the likelihood probability score between the test speech and the GMM model; Threshold comparison unit: Used to compare with a preset threshold to confirm or reject the tester's identity.

Images