A Phoneme Selection Method and Device for Voiceprint Recognition

Abstract

The present invention discloses a phoneme selection method and device for voiceprint recognition, which includes obtaining a voice database and a label corresponding to each voice data; obtaining the phoneme sequence of each voice data and the boundary position of each phoneme; calculating the benefit value of each phoneme class for the voiceprint recognition task; constructing a reward function based on the phoneme sequence contained in the voice data to be recognized and the corresponding benefit value, and defining the state and possible actions; calculating the optimal effective phoneme path with the goal of maximizing the total reward value; and calculating the voiceprint recognition result using the effective phoneme path. The present invention automatically and dynamically calculates the optimal effective phoneme combination based on the content of the sentence to be recognized, and optimizes the recognition rate by considering the global benefit based on the principle of achieving the highest overall benefit; and for longer voices to be recognized, the recognition efficiency and accuracy are improved by selecting strongly distinguishing phonemes and ignoring weakly distinguishing phonemes.

Description

Technical Field

[0001] The present invention relates to the fields of speech signal processing, voiceprint recognition technology, and reinforcement learning technology, and in particular to a phoneme selection method and device for voiceprint recognition. Background Art

[0002] Voiceprint information, as an important biometric, is an effective means of user authentication. Voiceprint recognition identifies the speaker's identity based on a given voice signal. Voiceprints can also be further leveraged to analyze and identify speaker attributes, offering broad application prospects, particularly in security, smart devices, and personalized recommendations. Text-independent voiceprint recognition, as it is not restricted by the textual content of the voice signal, is adaptable to a wider range of application scenarios. However, compared to text-dependent voiceprint recognition, it is more susceptible to text variations, resulting in reduced recognition performance. Therefore, phoneme-based voiceprint recognition systems, by modeling a single minimal speech unit, can effectively mitigate the negative impact of text variations on recognition performance. However, for speech data from different targets, the highly discriminative phoneme classes may not converge. Furthermore, the introduction of non-discriminative phoneme units can negatively impact the performance of voiceprint recognition systems. Therefore, automatically selecting the optimal phoneme combination for the sentence to be recognized would effectively improve the performance of voiceprint recognition systems. However, currently, no method exists for automatically and dynamically finding the optimal phoneme set for the sentence to be recognized.

[0003] Currently, voiceprint recognition systems that use phoneme units for modeling typically include all phonemes contained in the sentence to be recognized in their calculations, or predefine a valid phoneme set, such as phonemes corresponding to numbers or partial phoneme classes. All sentences to be recognized from any target source are calculated using the same predefined valid phoneme set. These methods often have the following problems:

[0004] 1) Ignoring the uniqueness of speech data from different targets, it fails to capture the most distinctive phoneme units, resulting in poor voiceprint recognition results;

[0005] 2) For longer recognition sentences, the recognition results of all phonemes contained in the sentence to be recognized are calculated separately, which not only reduces the computational efficiency of the system, but also the participation of weakly distinguishable phonemes may have a negative impact on system performance. Summary of the Invention

[0006] The purpose of the present invention is to provide a phoneme selection method and device for voiceprint recognition, in order to solve the shortcomings of the existing phoneme-based voiceprint recognition technology proposed in the above background technology, including the lack of an efficient method for screening valuable highly discriminative phonemes, and the inability to dynamically select the optimal phoneme combination for voiceprint recognition for the sentence to be recognized, resulting in the recognition effect not being able to achieve the optimal problem.

[0007] To achieve the above object, the present invention provides the following technical solutions:

[0008] A phoneme selection method for voiceprint recognition, comprising:

[0009] Get the voice database and the label corresponding to each voice data;

[0010] Obtain the phoneme sequence of each voice data and the boundary position of each phoneme;

[0011] Calculate the benefit of each phoneme class for the voiceprint recognition task;

[0012] For the speech data to be recognized, a reward function is constructed based on the phoneme sequence contained in the speech data to be recognized and the corresponding benefit value, and the state and possible actions are defined;

[0013] With the goal of maximizing the total reward value, the reinforcement learning algorithm is used to calculate the optimal effective phoneme path;

[0014] Calculate the voiceprint recognition results using the effective phoneme path.

[0015] A further technical solution is: the voice database and the labels corresponding to each voice data are obtained, and the corresponding labels are obtained according to the specific task type. The labels at least include: speaker identity labels, emotion labels and attribute labels.

[0016] A further technical solution is: in the obtaining of the phoneme sequence of each voice data and the boundary position of each phoneme, the phoneme sequence is a phoneme category marked according to the order in which the phonemes appear in the voice signal, and the phoneme category is a phoneme category defined by linguistics or a minimum speech unit defined by an unsupervised learning method;

[0017] Specific methods for obtaining the phoneme sequence of each voice data include obtaining the phoneme sequence by manual annotation or by using voice recognition or phoneme recognition.

[0018] A further technical solution is: a method for obtaining the boundary position of each phoneme is: using a forced alignment algorithm to calculate the boundary position information of each phoneme contained in each voice data in the voice signal.

[0019] A further technical solution is: the benefit value of each phoneme class for the voiceprint recognition task is calculated by:

[0020] Using the data of each phoneme class to train a phoneme-limited model, and calculating the recognition accuracy of the model for the corresponding phoneme segment as the benefit value of the phoneme class;

[0021] Alternatively, a benefit value can be defined for each phoneme class using statistics or prior knowledge.

[0022] A further technical solution is: after calculating the benefit value of each phoneme class for the voiceprint recognition task, the benefit value of each phoneme class is also corrected and normalized.

[0023] A further technical solution is: for the voice data to be recognized, defining the state and the actions that can be taken includes:

[0024] Define ignore and output behavior for a single-level phoneme set;

[0025] Defines merge, output, and ignore behavior for multi-level phoneme collections.

[0026] A further technical solution is: the use of a reinforcement learning algorithm to calculate the optimal effective phoneme path includes: adopting an online reinforcement learning algorithm, constructing a reward table, calculating the optimal path from the first phoneme state to the last phoneme state through a Q-learning algorithm, and obtaining the optimal phoneme class set for the sentence to be recognized through the path.

[0027] A further technical solution is: the method for calculating the voiceprint recognition result by using the effective phoneme path includes:

[0028] Recognition is performed using the fusion of multi-phoneme model scores;

[0029] Or adopt the method of multi-phoneme feature fusion for recognition.

[0030] To achieve the above object, the present invention further provides the following technical solutions:

[0031] A phoneme selection device for voiceprint recognition, comprising:

[0032] Data unit, obtains the voice database and the label corresponding to each voice data;

[0033] A phoneme detection unit, which obtains the phoneme sequence of each voice data and the boundary position of each phoneme;

[0034] A benefit calculation unit, used to calculate the benefit value of each phoneme class for the voiceprint recognition task;

[0035] A reward mechanism unit is used to construct a reward function for the speech data to be recognized based on the phoneme sequence contained in the speech data to be recognized and the corresponding benefit value, and to define the state and possible actions;

[0036] A reinforcement learning unit is used to calculate the optimal effective phoneme path using a reinforcement learning algorithm with the goal of maximizing the total reward value;

[0037] The voiceprint recognition unit is used to calculate the voiceprint recognition result using the effective phoneme path.

[0038] Compared with the prior art, the present invention has the following beneficial effects:

[0039] 1) The phoneme selection method provided by the present invention can automatically and dynamically calculate the optimal effective phoneme combination based on the specific content of each sentence to be recognized, thereby improving the recognition performance of voiceprint recognition;

[0040] 2) The phoneme selection method provided by the present invention uses reinforcement learning, based on the principle of achieving the highest overall benefit, to optimize the recognition rate and improve performance by considering the global benefit;

[0041] 3) For longer speech to be recognized, the phoneme selection method provided by the present invention can save computing time and improve recognition efficiency and accuracy by selecting strongly distinguishing phonemes and ignoring weakly distinguishing phonemes. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 Schematic diagram of a flow chart of a phoneme selection method for voiceprint recognition according to an embodiment of the present invention;

[0043] Figure 2 Schematic diagram of the functional modules of a phoneme selection device for voiceprint recognition in an embodiment of the present invention. DETAILED DESCRIPTION

[0044] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0045] Embodiment 1;

[0046] like Figure 1 As shown, the present invention provides a phoneme selection method for voiceprint recognition, which specifically includes the following steps:

[0047] Step 1: Get the voice database and the label corresponding to each voice data.

[0048] Corresponding labels can be obtained according to the specific task type. The obtained labels include but are not limited to: speaker identity labels, emotion labels, attribute labels, etc.

[0049] In a specific embodiment, the downstream task is to perform speaker recognition, and therefore, speech data and a speaker label corresponding to each speech are obtained.

[0050] Step 2: Obtain the phoneme sequence of each voice data and the boundary position of each phoneme.

[0051] When obtaining the phoneme sequence of each voice data and the boundary position of each phoneme, the phoneme sequence is specifically to mark the phoneme category according to the order in which the phonemes appear in the voice signal. In this embodiment, the phoneme category can adopt the phoneme category defined by linguistics, or the minimum speech unit defined by methods such as unsupervised learning.

[0052] The method for obtaining the phoneme sequence of each voice data can be to obtain the phoneme sequence by manual annotation, or to obtain the phoneme sequence by voice recognition, phoneme recognition, etc.

[0053] Furthermore, after the phoneme sequence is labeled, only a single-level phoneme set may be considered, or a multi-level phoneme set consisting of multiple levels of speech units may be considered.

[0054] In this specific embodiment, a phoneme set defined by phonetics is used, with a total of 65 phoneme categories for the Chinese corpus. For each sentence in the speech database, speech recognition methods and a phoneme mapping table are used to obtain a phoneme sequence. This sequence corresponds to the order of occurrence of the phonemes in the speech data. A forced alignment algorithm is used to calculate the boundary position information of each phoneme contained in each speech data item. To obtain richer information, different combinations of phonemes are considered and a multi-level phoneme category is generated.

[0055] Step 3: Calculate the benefit value of each phoneme class for the voiceprint recognition task.

[0056] Calculate the benefit of each phoneme class for the voiceprint recognition task. The specific method can be:

[0057] Use the data of each phoneme class to train a phoneme-limited model, and calculate the recognition accuracy of the model for the corresponding phoneme segment as the benefit value of the phoneme;

[0058] Alternatively, a benefit value may be defined for each phoneme using statistics or prior knowledge.

[0059] In order to further improve the performance, after completing the calculation of the benefit value of each phoneme class for the voiceprint recognition task, the benefit value of each phoneme class can be corrected, normalized, and other processing.

[0060] In a specific embodiment, a training dataset is used to model each phoneme class, generate a corresponding speaker recognition model, and calculate the recognition rate of each phoneme class for the speaker recognition task on the validation dataset. The recognition rates of all phoneme classes are normalized and used as the corresponding benefit value for each phoneme class.

[0061] Step 4: For the speech data to be recognized, construct a reward function based on the phoneme sequence contained in the speech data to be recognized and the corresponding benefit value, and define the state and possible actions.

[0062] The defined states and valid actions that can be taken are:

[0063] For a single-level phoneme set, you can define ignore or output; for a multi-level phoneme set, you can define behaviors such as merge, output, and ignore.

[0064] In a specific embodiment, the state is defined as the phoneme category in the current sentence. Since multiple levels of phonemes are considered, multiple continuous phonemes can be regarded as a combined class, so the effective actions that can be taken include merging, outputting, and ignoring. Among them, merging means that the current phoneme class is not recognized as an independent phoneme, but continues to enter the next state to form a combined phoneme class; outputting means that the current phoneme class is used as an independent phoneme for recognition; ignoring means that the current phoneme class is neither used as an independent phoneme for recognition nor combined with the next phoneme to form a higher-level phoneme class, and directly jumps to the next phoneme class state.

[0065] Step 5: With the goal of maximizing the total reward value, use the reinforcement learning algorithm to calculate the optimal effective phoneme path.

[0066] The optimal effective phoneme path is calculated using a reinforcement learning algorithm. The reinforcement learning algorithm can first be trained using offline reinforcement learning or an online reinforcement learning algorithm.

[0067] In a specific embodiment, an online reinforcement learning algorithm is used to construct a reward table, and the optimal path from the first phoneme state to the last phoneme state is calculated through the Q-learning algorithm, and the optimal phoneme class set for the sentence to be recognized is obtained through the path.

[0068] Step 6: Calculate the voiceprint recognition result using the valid phoneme path.

[0069] After confirming the valid phoneme path, voiceprint recognition can be performed by using the multi-phoneme model score fusion method or the multi-phoneme feature fusion method.

[0070] In a specific embodiment, based on the optimal set of phoneme classes corresponding to the sentence to be recognized, each segment corresponding to each phoneme class is independently recognized using the corresponding phoneme model. Then, using a score fusion approach, the registered speaker with the highest score is selected as the recognition result. If an unregistered speaker exists, a score threshold is used to determine whether they are unregistered speakers.

[0071] The present invention provides a phoneme selection method for voiceprint recognition that automatically and dynamically calculates the optimal effective phoneme combination based on the specific content of each sentence to be recognized, thereby improving the method's recognition performance. Furthermore, through a reinforcement learning algorithm, based on the principle of maximizing overall returns, this method considers global benefits to optimize the system's recognition rate, thereby improving recognition efficiency and accuracy.

[0072] Embodiment 2;

[0073] like Figure 2 FIG. 1 is a phoneme selection device for voiceprint recognition according to the present invention, comprising:

[0074] Data unit, obtains the voice database and the label corresponding to each voice data;

[0075] A phoneme detection unit, which obtains the phoneme sequence of each voice data and the boundary position of each phoneme;

[0076] A benefit calculation unit, used to calculate the benefit value of each phoneme class for the voiceprint recognition task;

[0077] The reward mechanism unit is used to construct a reward function based on the phoneme sequence and corresponding benefit value of the recognized speech data, and is used to define the state and possible actions;

[0078] A reinforcement learning unit is used to calculate the optimal effective phoneme path using a reinforcement learning algorithm with the goal of maximizing the total reward value;

[0079] The voiceprint recognition unit is used to calculate the voiceprint recognition result using the effective phoneme path.

[0080] It should be noted that the various units in this embodiment are logical in nature. In a specific implementation process, one unit can be split into multiple units, and multiple units can also be combined into one unit.

[0081] According to the second embodiment of the present invention, a phoneme selection device for voiceprint recognition is provided. This device can automatically and dynamically calculate the optimal effective phoneme combination based on the specific content of each sentence to be recognized, thereby improving the device's recognition performance. Furthermore, through a reinforcement learning algorithm, based on the principle of maximizing overall benefits, this algorithm considers global benefits to optimize the system's recognition rate, thereby improving the device's performance, recognition efficiency, and accuracy.

[0082] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A phoneme selection method for voiceprint recognition, characterized in that: include: Get the voice database and the label corresponding to each voice data; Obtain the phoneme sequence of each voice data and the boundary position of each phoneme; Calculate the benefit of each phoneme class for the voiceprint recognition task; For the speech data to be recognized, a reward function is constructed based on the phoneme sequence contained in the speech data to be recognized and the corresponding benefit value, and the state and possible actions are defined; With the goal of maximizing the total reward value, the reinforcement learning algorithm is used to calculate the optimal effective phoneme path; The method of calculating the optimal effective phoneme path using a reinforcement learning algorithm includes: using an online reinforcement learning algorithm to construct a reward table, calculating the optimal path from the first phoneme state to the last phoneme state using a Q-learning algorithm, and obtaining the optimal phoneme class set for the sentence to be recognized through the path; Calculate the voiceprint recognition results using the effective phoneme path.

2. The phoneme selection method for voiceprint recognition according to claim 1, characterized in that: The speech database and the labels corresponding to each speech data are obtained, and the corresponding labels are obtained according to the specific task type. The labels at least include: speaker identity labels, emotion labels and attribute labels.

3. The phoneme selection method for voiceprint recognition according to claim 1, characterized in that: In the step of obtaining a phoneme sequence for each piece of speech data and a boundary position of each phoneme, the phoneme sequence is a phoneme category marked according to the order in which the phonemes appear in the speech signal, and the phoneme category is a phoneme category defined by linguistics or a minimum speech unit defined by an unsupervised learning method; Specific methods for obtaining the phoneme sequence of each voice data include obtaining the phoneme sequence by manual annotation or by using voice recognition or phoneme recognition.

4. The phoneme selection method for voiceprint recognition according to claim 3, characterized in that: The method for obtaining the boundary position of each phoneme is: using a forced alignment algorithm to calculate and obtain the boundary position information of each phoneme contained in each piece of speech data in the speech signal.

5. The phoneme selection method for voiceprint recognition according to claim 3, characterized in that: The specific method for calculating the benefit value of each phoneme class for the voiceprint recognition task is as follows: Using the data of each phoneme class to train a phoneme-limited model, and calculating the recognition accuracy of the model for the corresponding phoneme segment as the benefit value of the phoneme class; Alternatively, a benefit value can be defined for each phoneme class using statistics or prior knowledge.

6. The phoneme selection method for voiceprint recognition according to claim 5, characterized in that: After calculating the benefit value of each phoneme class for the voiceprint recognition task, the benefit value of each phoneme class is further corrected and normalized.

7. The phoneme selection method for voiceprint recognition according to claim 1, characterized in that: The definition of states and possible actions for the speech data to be recognized includes: Define ignore and output behavior for a single-level phoneme set; Defines merge, output, and ignore behavior for multi-level phoneme collections.

8. The phoneme selection method for voiceprint recognition according to claim 1, characterized in that: The method for calculating the voiceprint recognition result by using the effective phoneme path includes: Recognition is performed using the fusion of multi-phoneme model scores; Or adopt the method of multi-phoneme feature fusion for recognition.

9. A phoneme selection device for voiceprint recognition, characterized in that: include: Data unit, obtains the voice database and the label corresponding to each voice data; A phoneme detection unit, which obtains the phoneme sequence of each voice data and the boundary position of each phoneme; A benefit calculation unit, used to calculate the benefit value of each phoneme class for the voiceprint recognition task; A reward mechanism unit is used to construct a reward function for the speech data to be recognized based on the phoneme sequence contained in the speech data to be recognized and the corresponding benefit value, and to define the state and possible actions; A reinforcement learning unit is used to calculate the optimal effective phoneme path using a reinforcement learning algorithm with the goal of maximizing the total reward value; The method of calculating the optimal effective phoneme path using a reinforcement learning algorithm includes: using an online reinforcement learning algorithm to construct a reward table, calculating the optimal path from the first phoneme state to the last phoneme state using a Q-learning algorithm, and obtaining the optimal phoneme class set for the sentence to be recognized through the path; The voiceprint recognition unit is used to calculate the voiceprint recognition result using the effective phoneme path.

Images