Voiceprint recognition method, system, storage medium and program product

Abstract

The present invention relates to a voiceprint recognition method, system, computer storage medium, and computer program product based on an offline self-training neural training network. When performing voiceprint feature vector comparison, the present invention calculates the cosine distance between the input voiceprint feature vector and the stored voiceprint feature vector, and sets a cosine distance acceptance threshold and a cosine distance rejection threshold to screen out qualified stored voiceprint feature vectors as reference voiceprint feature vectors. The reference voiceprint feature vectors are then further compared with the input feature vectors using a habit classification fully connected neural network, thereby screening out voiceprint feature vectors that conform to user habits and determining control parameter values. The present invention effectively adapts to the different operating habits of the same person at different times, improves the accuracy of voiceprint recognition, and effectively enhances the user's intelligent control experience.

Description

Technical Field

[0001] The present invention relates to the technical field of voiceprint recognition, and in particular to a voiceprint recognition method, system, computer storage medium, and computer program product based on an offline self-training neural network. Background Art

[0002] With the development of information technology and the popularization of the Internet, more and more applications require human identity recognition. Traditional needs include the need for various online accounts, online payment, access control, etc., and with the promotion of the application of the Internet and artificial intelligence, different identity recognition methods and systems are needed according to the habits and characteristics of different people, such as fingerprint, face, and voiceprint recognition. Among them, voiceprint recognition is a type of biometric technology. By processing the voice, an identity vector indicating the identity information of the voice inputter can be generated. The similarity between the identity vectors of the two voices can be calculated to determine whether the inputter of the two voices is the same user.

[0003] However, due to factors such as time and environment, people's voiceprints will change. If they are not updated in a timely manner, the recognition efficiency will decrease, thereby affecting the personalized control through voiceprints.

[0004] In view of this, the designers of the present invention have conducted in-depth research and development on the problems existing in voiceprint recognition, and thus produced this case. Summary of the Invention

[0005] The purpose of the present invention is to provide a voiceprint recognition method, system, computer storage medium and computer program product based on an offline self-training neural training network, which improves the voiceprint recognition efficiency by updating the association between the user's voiceprint and its commonly used settings in real time, effectively enhancing the user's intelligent control experience.

[0006] To achieve the above object, the technical solution adopted by the present invention is:

[0007] A voiceprint recognition method based on an offline self-training neural network, comprising:

[0008] S11, collecting input voice electrical signals, slicing the input voice electrical signals, and processing the MFCC to generate an audio segment MFCC vector;

[0009] S12: Input the MFCC vector of the audio clip into the recurrent neural network for isolated word recognition, extract the isolated word, and compare it with the required wake-up word. If the comparison is inconsistent, return to S11; if the comparison is consistent, input the MFCC vector of the audio clip into the recurrent neural network for voiceprint extraction to perform voiceprint recognition preprocessing and generate an input voiceprint feature vector;

[0010] The voiceprint recognition method further includes:

[0011] S13. Select an uncompared stored voiceprint feature vector from the stored voiceprint feature vectors as the voiceprint feature vector to be compared, calculate the cosine distance between the voiceprint feature vector to be compared and the input voiceprint feature vector using the cosine distance formula, and record the calculated distance. Repeat step S13 until there are no uncompared voiceprint feature vectors in the stored voiceprint feature vectors.

[0012] S14. Determine whether the recorded maximum cosine distance is greater than the cosine distance acceptance threshold. If so, use all stored voiceprint feature vectors whose cosine distances are greater than the cosine distance acceptance threshold as reference voiceprint feature vectors, and then enter the habit prediction control parameter setting sub-process; otherwise, enter S15.

[0013] S15. Determine whether the recorded maximum cosine distance is greater than the cosine distance rejection threshold. If so, use all stored voiceprint feature vectors whose cosine distances are greater than the cosine distance rejection threshold as control voiceprint feature vectors, and then enter the habit prediction control parameter value setting sub-process.

[0014] S16. After the sub-process ends, the current control parameter value is associated with the current voiceprint feature vector and the current time, and stored; if the user modifies the control parameter value at this time, the preset value of the input voiceprint feature vector is additionally recorded;

[0015] The sub-process for setting custom predictive control parameter values ​​is as follows:

[0016] S21, inputting the input voiceprint feature vector and the corresponding current time into the habit classification fully connected neural network to generate a comparison habit classification vector;

[0017] S22, obtaining all control voiceprint feature vectors and corresponding vector association times and inputting them into a habit classification fully connected neural network to generate a control habit classification vector;

[0018] S23, calculating the Euclidean distance between the comparison habit classification vector and all control habit classification vectors;

[0019] S24. Setting the current control parameter to the control parameter associated with the reference voiceprint feature vector corresponding to the minimum Euclidean distance;

[0020] S25, end;

[0021] The voiceprint recognition method also includes a real-time training process of a habit classification fully connected neural network, which is as follows:

[0022] S51. Obtain a portion of the stored voiceprint feature vectors and the corresponding associated time as a training set input, and the remaining stored voiceprint feature vectors and their associated time as a validation set. Use triple loss as the loss function and use weighted summation to calculate the Euclidean distance between the two sets of control parameter values.

[0023] S52. Determine whether the average Euclidean distance error of the prediction results of the habit classification fully connected neural network on the training set and the validation set is less than a preset error threshold. If not, randomly select a vector a from the training set, set p to any vector with the same control parameter value, and set n to any vector with a different control parameter value, with the preset inter-class distance being m. Calculate the loss using the loss calculation formula, and reversely train the habit classification fully connected neural network.

[0024] The loss is calculated as follows:

[0025]

[0026] Repeat step S52 until the average Euclidean distance error of the prediction results of the habit classification fully connected neural network on the training set and the validation set is less than a preset error threshold;

[0027] S53, end.

[0028] In S15, if the recorded maximum cosine distance is not greater than the cosine distance rejection threshold, the sub-process of setting the control parameter value according to the voiceprint feature vector close to the gender is entered, which is as follows:

[0029] S31. Calculate the cosine distance between the input voiceprint feature vector and the stored multiple groups of male voiceprint feature vectors, and output the minimum cosine distance and the corresponding age of the voiceprint inputter;

[0030] S32. Calculate the cosine distance between the input voiceprint feature vector and the stored multiple groups of female voiceprint feature vectors, and output the minimum cosine distance and the corresponding age of the voiceprint inputter;

[0031] S33, determining whether the minimum cosine distances of multiple groups of male voiceprint feature vectors are smaller than the minimum cosine distances of multiple groups of female voiceprint feature vectors; if so, setting the current control parameter to the default control parameter value for males of the corresponding age; otherwise, setting the current control parameter to the default control parameter value for females of the corresponding age;

[0032] S34, end.

[0033] The voiceprint recognition method further includes a process of removing redundant voiceprint feature vectors, which is as follows:

[0034] S41. Obtain all stored voiceprint feature vectors and their creation time;

[0035] S42. Calculate the duplicate score of each stored voiceprint feature vector;

[0036] Specifically, the cosine distance between the stored voiceprint feature vector and the rest of the stored voiceprint feature vectors is calculated. If all cosine distances are not less than the cosine distance threshold, the duplicate item score is 0; if there is a cosine distance less than the cosine distance threshold, the duplicate item score is negative. The more cosine distances less than the cosine distance threshold, the lower the duplicate item score.

[0037] S43. Calculate the time score of each stored voiceprint feature vector. The score created at the current moment is 0. The earlier the creation time, the lower the score.

[0038] S44. Calculate the error score of each stored voiceprint feature vector. If the vector has been modified after being preset, the score is positive; if it has not been modified, the score is 0.

[0039] S45. Perform a weighted summation of the duplicate item, time item, and error item of each stored voiceprint feature vector to obtain an importance score for the stored voiceprint feature vector;

[0040] S46. Select multiple items of the pre-set value with the highest importance score from all stored voiceprint feature vectors, and replace the original stored voiceprint feature vectors with these voiceprint feature vectors as new stored voiceprint feature vectors;

[0041] S47, end.

[0042] A voiceprint recognition system based on an offline self-training neural network, comprising:

[0043] An acquisition module, used for acquiring input voice electrical signals;

[0044] The MFCC processing module is used to slice and perform MFCC processing on the input speech electrical signal to generate the MFCC vector of the audio segment;

[0045] The wake-up module is used to input the MFCC vector of the audio clip into the recurrent neural network for isolated word recognition, extract the isolated word and compare it with the required wake-up word. If the comparison is consistent, wake up;

[0046] The voiceprint extraction module is used to input the MFCC vector of the audio clip into the voiceprint extraction recurrent neural network for voiceprint recognition preprocessing after waking up, and generate the input voiceprint feature vector;

[0047] The voiceprint recognition system further includes:

[0048] The voiceprint comparison module is used to compare all stored voiceprint feature vectors with the input voiceprint feature vector, calculate the cosine distance between the two and record it; when the maximum cosine distance recorded is greater than the cosine distance acceptance threshold, all stored voiceprint feature vectors with cosine distance greater than the cosine distance acceptance threshold are used as control voiceprint feature vectors; when the maximum cosine distance recorded is not greater than the cosine distance acceptance threshold but greater than the cosine distance rejection threshold, all stored voiceprint feature vectors with cosine distance greater than the cosine distance rejection threshold are used as control voiceprint feature vectors;

[0049] a habit prediction control module, configured to be activated when a comparison voiceprint feature vector exists; the habit prediction control module is configured to input the input voiceprint feature vector and the corresponding association time into a habit classification fully connected neural network to generate a comparison habit classification vector; and further configured to input the comparison voiceprint feature vector and the corresponding association time into the habit classification fully connected neural network to generate a comparison habit classification vector; and then calculate the Euclidean distance between the comparison habit classification vector and all comparison habit classification vectors, and set the current control parameter value to the comparison voiceprint feature vector corresponding to the minimum Euclidean distance.

[0050] A storage module, configured to associate and store the current control parameter value with the current input voiceprint feature vector and its corresponding time;

[0051] A habit classification fully connected neural network real-time training module is used to obtain stored voiceprint feature vectors and corresponding association times, and perform real-time training of the habit classification fully connected neural network; specifically, a portion of the stored voiceprint feature vectors and the corresponding association time are obtained as training set input, and the remaining stored voiceprint feature vectors and their association time are used as validation set, triple loss is used as the loss function, and weighted sum is used to calculate the Euclidean distance between the two sets of control parameter values; and it is determined whether the average Euclidean distance error of the prediction results of the habit classification fully connected neural network on the training set and the validation set is less than a preset error threshold. If not, a vector a is randomly selected from the training set, and any one of the vectors with the same control parameter value is set as p, and any one of the vectors with different control parameter values ​​is set as n, and the class distance preset value is m; the loss calculation formula is used to calculate this loss, and the habit classification fully connected neural network is reversely trained until the average Euclidean distance error of the prediction results of the habit classification fully connected neural network on the training set and the validation set is less than the preset error threshold;

[0052] The loss is calculated as follows:

[0053] .

[0054] The voiceprint recognition system further includes a proximity gender control module;

[0055] When the maximum cosine distance recorded in the voiceprint comparison module is not greater than the cosine distance rejection threshold, the proximity gender control module is started; the proximity gender control module is used to calculate the cosine distance between the input voiceprint feature vector and the stored multiple groups of male voiceprint feature vectors, and output the minimum cosine distance therein and the corresponding age of the voiceprint inputter; it is also used to calculate the cosine distance between the input voiceprint feature vector and the stored multiple groups of female voiceprint feature vectors, and output the minimum cosine distance therein and the corresponding age of the voiceprint inputter; then it is determined whether the minimum value of the cosine distance of the multiple groups of male voiceprint feature vectors is less than the minimum value of the cosine distance of the multiple groups of female voiceprint feature vectors. If so, the current control parameter is set to the default control parameter value for males of the corresponding age; otherwise, the current control parameter is set to the default control parameter value for females of the corresponding age.

[0056] The voiceprint recognition system also includes a voiceprint removal module, which is used to regularly update the stored voiceprint feature vector, including

[0057] A duplicate score calculation unit is used to calculate the duplicate score of each stored voiceprint feature vector. Specifically, the cosine distance between the stored voiceprint feature vector and the remaining stored voiceprint feature vectors is calculated. If all cosine distances are not less than the cosine distance threshold, the duplicate score is 0; if there is a cosine distance less than the cosine distance threshold, the duplicate score is negative. The more cosine distances less than the cosine distance threshold, the lower the duplicate score.

[0058] The time item score calculation unit is used to calculate the time score of each stored voiceprint feature vector. The score created at the current moment is 0, and the score of the item created earlier is lower;

[0059] An error item score calculation unit is used to calculate the error item score of each stored voiceprint feature vector. If the vector is modified after the preset, the error item score is positive, and if it is not modified, the error item score is 0;

[0060] An importance score calculation unit calculates the importance score of each stored voiceprint feature vector by performing a weighted sum of the duplicate item, time item, and error item of each stored voiceprint feature vector;

[0061] The updating unit is used to select multiple items of the preset value with the highest importance score from all stored voiceprint feature vectors, and replace the original stored voiceprint feature vectors with these voiceprint feature vectors as new stored voiceprint feature vectors.

[0062] A computer-readable storage medium stores instructions, which, when executed on a terminal device, enable the terminal device to execute the voiceprint recognition method as described above.

[0063] A computer software program product, when running on a terminal device, enables the terminal device to execute the voiceprint recognition method as described above.

[0064] After adopting the above scheme, when performing voiceprint feature vector comparison, the present invention calculates the cosine distance between the input voiceprint feature vector and the stored voiceprint feature vector, and by setting a cosine distance acceptance threshold and a cosine distance rejection threshold, screens out qualified stored voiceprint feature vectors as reference voiceprint feature vectors. The reference voiceprint feature vectors are then further compared with the input feature vectors using a habit classification fully connected neural network, thereby screening out voiceprint feature vectors that conform to the user's habits and determining the control parameter values. Each input voiceprint feature vector and its time are stored. The present invention also provides a real-time training process for the habit classification fully connected neural network, which is periodically initiated to train the habit classification fully connected neural network. During training, it trains based on the stored voiceprint feature vectors (including the user's recent voiceprint feature information), effectively fitting the different operating habits of the same person at different times, improving the accuracy of voiceprint recognition, and effectively enhancing the user's intelligent control experience.

[0065] Furthermore, the present invention regularly updates stored voiceprint feature vectors. During this update, an importance score is calculated by combining the duplication score, time score, and error score of each stored voiceprint feature vector. These feature vectors with higher importance scores are then selected and used as new stored data to replace the original data. This regular removal of voiceprint feature vectors improves the accuracy of the training data used by the habit classification fully connected neural network, thereby improving the accuracy of voiceprint recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 It is the main flow chart of the present invention;

[0067] Figure 2 Setting sub-flowchart for custom predictive control parameter values ​​of the present invention;

[0068] Figure 3 This is a sub-flowchart for setting control parameter values ​​according to the voiceprint feature vector being close to gender in the present invention;

[0069] Figure 4 This is a flowchart of offline training of a fully connected neural network for habit classification according to the present invention;

[0070] Figure 5 This is a flow chart of voiceprint feature removal according to the present invention. DETAILED DESCRIPTION

[0071] In order to more clearly understand the above-mentioned objects, features and advantages of the present invention, the present invention is described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present invention and the features therein can be combined with each other without conflict.

[0072] The following description sets forth numerous specific details to facilitate a thorough understanding of the present invention. The embodiments described are merely some, not all, of the present invention. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without inventive effort are intended to fall within the scope of protection of the present invention.

[0073] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in this specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention.

[0074] The terms "first," "second," and "third," etc., in the specification and claims of the present invention and the accompanying drawings are used to distinguish between different items, not to describe a specific order. Furthermore, the terms "comprise," "comprising," and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or elements is not limited to the listed steps or elements but may optionally include steps or elements not listed, or may optionally include other steps or elements inherent to the process, method, product, or apparatus.

[0075] like Figure 1-4 As shown, the present invention discloses a voiceprint recognition method based on an offline self-training neural network, which includes the following steps:

[0076] S11, collecting input voice electrical signals, slicing the input voice electrical signals, and processing the MFCC to generate an audio segment MFCC vector;

[0077] S12: Input the MFCC vector of the audio clip into the recurrent neural network for isolated word recognition, extract the isolated word, and compare it with the required wake-up word. If the comparison is inconsistent, return to S11; if the comparison is consistent, input the MFCC vector of the audio clip into the recurrent neural network for voiceprint extraction to perform voiceprint recognition preprocessing and generate an input voiceprint feature vector;

[0078] S13. Select an uncompared stored voiceprint feature vector from the stored voiceprint feature vectors as the voiceprint feature vector to be compared, calculate the cosine distance between the voiceprint feature vector to be compared and the input voiceprint feature vector using the cosine distance formula, and record the calculated distance. Repeat step S13 until there are no uncompared voiceprint feature vectors in the stored voiceprint feature vectors.

[0079] S14. Determine whether the recorded maximum cosine distance is greater than the cosine distance acceptance threshold. If so, use all stored voiceprint feature vectors whose cosine distances are greater than the cosine distance acceptance threshold as reference voiceprint feature vectors, and then enter the habit prediction control parameter setting sub-process; otherwise, enter S15.

[0080] S15. Determine whether the recorded maximum cosine distance is greater than the cosine distance rejection threshold. If so, use all stored voiceprint feature vectors whose cosine distances are greater than the cosine distance rejection threshold as control voiceprint feature vectors, and then enter the habit prediction control parameter value setting sub-process.

[0081] S16. After the sub-process ends, the current control parameter value is associated with the current voiceprint feature vector and the current time, and stored; if the user modifies the control parameter value at this time, the input voiceprint feature vector preset value is additionally recorded and modified;

[0082] The sub-process for setting custom predictive control parameter values ​​is as follows:

[0083] S21, inputting the input voiceprint feature vector and the corresponding current time into the habit classification fully connected neural network to generate a comparison habit classification vector;

[0084] S22, obtaining all control voiceprint feature vectors and corresponding vector association times and inputting them into a habit classification fully connected neural network to generate a control habit classification vector;

[0085] S23, calculating the Euclidean distance between the comparison habit classification vector and all control habit classification vectors;

[0086] S24. Setting the current control parameter to the control parameter associated with the reference voiceprint feature vector corresponding to the minimum Euclidean distance;

[0087] S25, end.

[0088] In S15, if the recorded maximum cosine distance is not greater than the cosine distance rejection threshold, the user's voiceprint feature vector cannot be identified. Generally, no output is performed. However, in this case, the present invention enters the sub-process of setting control parameter values ​​based on the proximity of the voiceprint feature vector to gender, that is, judging whether the user's voiceprint feature vector is male or female, and then setting the control parameters to the corresponding gender. The sub-process of setting control parameter values ​​based on the proximity of the voiceprint feature vector to gender is as follows:

[0089] S31. Calculate the cosine distance between the input voiceprint feature vector and the stored multiple groups of male voiceprint feature vectors, and output the minimum cosine distance and the corresponding age of the voiceprint inputter;

[0090] S32. Calculate the cosine distance between the input voiceprint feature vector and the stored multiple groups of female voiceprint feature vectors, and output the minimum cosine distance and the corresponding age of the voiceprint inputter;

[0091] S33, determining whether the minimum cosine distances of multiple groups of male voiceprint feature vectors are smaller than the minimum cosine distances of multiple groups of female voiceprint feature vectors; if so, setting the current control parameter to the default control parameter value for males of the corresponding age; otherwise, setting the current control parameter to the default control parameter value for females of the corresponding age;

[0092] S34, end.

[0093] The user's voiceprint will change with time and environment, so in order to improve the recognition accuracy, the present invention sets up a real-time training process for habit classification fully connected neural network and starts it regularly for real-time training. The details are as follows:

[0094] S51. Obtain a portion of the stored voiceprint feature vectors and the corresponding associated time as the training set input, and the remaining stored voiceprint feature vectors and their associated time as the validation set. Use triple loss as the loss function and use weighted summation to calculate the Euclidean distance between the two sets of control parameter values. S52. Determine whether the average Euclidean distance error of the prediction results of the habit classification fully connected neural network on the training set and the validation set is less than a preset error threshold. If not, randomly select a vector a from the training set, randomly select one from the vectors with the same control parameter value as it and set it as p, and randomly select one from the vectors with different control parameter values ​​as it and set it as n. The preset class distance is m; use the loss calculation formula to calculate this loss, and reversely train the habit classification fully connected neural network.

[0095] The loss is calculated as follows:

[0096]

[0097] Repeat step S52 until the average Euclidean distance error of the prediction results of the habit classification fully connected neural network on the training set and the validation set is less than a preset error threshold;

[0098] S53, end.

[0099] Because the user's voiceprint features will change with the passage of time and the environment, the more recently stored voiceprint features are more valuable for reference, while the earlier stored voiceprint feature vectors have lower reference value. Therefore, the present invention adds a voiceprint recognition method that also includes a process for removing redundant voiceprint feature vectors. The process for removing redundant voiceprint feature vectors is started regularly or when the storage limit is reached, and the stored features are updated to remove voiceprint feature vectors of low importance. Figure 5 As shown, the details are as follows:

[0100] S41. Obtain all stored voiceprint feature vectors and their creation time;

[0101] S42. Calculate the duplicate score of each stored voiceprint feature vector;

[0102] Specifically, the cosine distance between the stored voiceprint feature vector and the rest of the stored voiceprint feature vectors is calculated. If all cosine distances are not less than the cosine distance threshold, the duplicate item score is 0; if there is a cosine distance less than the cosine distance threshold, the duplicate item score is negative. The more cosine distances less than the cosine distance threshold, the lower the duplicate item score.

[0103] S43. Calculate the time score of each voiceprint feature vector. The score created at the current moment is 0. The earlier the creation time, the lower the score.

[0104] S44. Calculate the error score of each voiceprint feature vector. If the vector has been modified after being preset, the score is positive; if it has not been modified, the score is 0.

[0105] S45. Perform a weighted summation of the duplicate item, time item, and error item of each stored voiceprint feature vector to obtain an importance score for the stored voiceprint feature vector;

[0106] S46. Select multiple items of the pre-set value with the highest importance score from all stored voiceprint feature vectors, and replace the original stored voiceprint feature vectors with these voiceprint feature vectors as new stored voiceprint feature vectors;

[0107] S47, end.

[0108] Based on the same inventive concept, the present invention also discloses a voiceprint recognition system based on an offline self-training neural network, which includes:

[0109] An acquisition module, used for acquiring input voice electrical signals;

[0110] The MFCC processing module is used to slice and perform MFCC processing on the input speech electrical signal to generate the MFCC vector of the audio segment;

[0111] The wake-up module is used to input the MFCC vector of the audio clip into the recurrent neural network for isolated word recognition, extract the isolated word and compare it with the required wake-up word. If the comparison is consistent, wake up;

[0112] The voiceprint extraction module is used to input the MFCC vector of the audio clip into the voiceprint extraction recurrent neural network for voiceprint recognition preprocessing after waking up, and generate the input voiceprint feature vector;

[0113] The voiceprint comparison module is used to compare all stored voiceprint feature vectors with the input voiceprint feature vector, calculate the cosine distance between the two and record it; when the maximum cosine distance recorded is greater than the cosine distance acceptance threshold, all stored voiceprint feature vectors with cosine distance greater than the cosine distance acceptance threshold are used as control voiceprint feature vectors; when the maximum cosine distance recorded is not greater than the cosine distance acceptance threshold but greater than the cosine distance rejection threshold, all stored voiceprint feature vectors with cosine distance greater than the cosine distance rejection threshold are used as control voiceprint feature vectors;

[0114] a habit prediction control module, configured to be activated when a comparison voiceprint feature vector exists; the habit prediction control module is configured to input the input voiceprint feature vector and the corresponding association time into a habit classification fully connected neural network to generate a comparison habit classification vector; and further configured to input the comparison voiceprint feature vector and the corresponding association time into the habit classification fully connected neural network to generate a comparison habit classification vector; and then calculate the Euclidean distance between the comparison habit classification vector and all comparison habit classification vectors, and set the current control parameter value to the comparison voiceprint feature vector corresponding to the minimum Euclidean distance.

[0115] a storage module, configured to associate and store the current control parameter value with the current input voiceprint feature vector and its corresponding time; and

[0116] A habit classification fully connected neural network real-time training module is used to obtain stored voiceprint feature vectors and corresponding association times, and perform real-time training of the habit classification fully connected neural network; specifically, a portion of the stored voiceprint feature vectors and the corresponding association time are obtained as training set input, and the remaining stored voiceprint feature vectors and their association time are used as validation set, triple loss is used as the loss function, and weighted sum is used to calculate the Euclidean distance between the two sets of control parameter values; and it is determined whether the average Euclidean distance error of the prediction results of the habit classification fully connected neural network on the training set and the validation set is less than a preset error threshold. If not, a vector a is randomly selected from the training set, and any one of the vectors with the same control parameter value is set as p, and any one of the vectors with different control parameter values ​​is set as n, and the class distance preset value is m; the loss calculation formula is used to calculate this loss, and the habit classification fully connected neural network is reversely trained until the average Euclidean distance error of the prediction results of the habit classification fully connected neural network on the training set and the validation set is less than the preset error threshold;

[0117] The loss is calculated as follows:

[0118] .

[0119] The voiceprint recognition system further includes a proximity gender control module;

[0120] When the maximum cosine distance recorded in the voiceprint comparison module is not greater than the cosine distance rejection threshold, the proximity gender control module is started; the proximity gender control module is used to calculate the cosine distance between the input voiceprint feature vector and the stored multiple groups of male voiceprint feature vectors, and output the minimum cosine distance therein and the corresponding age of the voiceprint inputter; it is also used to calculate the cosine distance between the input voiceprint feature vector and the stored multiple groups of female voiceprint feature vectors, and output the minimum cosine distance therein and the corresponding age of the voiceprint inputter; then it is determined whether the minimum value of the cosine distance of the multiple groups of male voiceprint feature vectors is less than the minimum value of the cosine distance of the multiple groups of female voiceprint feature vectors. If so, the current control parameter is set to the default control parameter value for males of the corresponding age; otherwise, the current control parameter is set to the default control parameter value for females of the corresponding age.

[0121] The voiceprint recognition system also includes a voiceprint removal module, which is used to update the stored voiceprint feature vector at a fixed time or when the storage limit is reached.

[0122] A duplicate score calculation unit is used to calculate the duplicate score of each stored voiceprint feature vector. Specifically, the cosine distance between the stored voiceprint feature vector and the remaining stored voiceprint feature vectors is calculated. If all cosine distances are not less than the cosine distance threshold, the duplicate score is 0; if there is a cosine distance less than the cosine distance threshold, the duplicate score is negative. The more cosine distances less than the cosine distance threshold, the lower the duplicate score.

[0123] The time item score calculation unit is used to calculate the time score of each stored voiceprint feature vector. The score created at the current moment is 0, and the score of the item created earlier is lower;

[0124] An error item score calculation unit is used to calculate the error item score of each stored voiceprint feature vector. If the vector is modified after the preset, the error item score is positive, and if it is not modified, the error item score is 0;

[0125] An importance score calculation unit calculates the importance score of each stored voiceprint feature vector by performing a weighted sum of the duplicate item, time item, and error item of each stored voiceprint feature vector;

[0126] The updating unit is used to select multiple items of the preset value with the highest importance score from all stored voiceprint feature vectors, and replace the original stored voiceprint feature vectors with these voiceprint feature vectors as new stored voiceprint feature vectors.

[0127] The present invention also discloses a computer-readable storage medium, in which instructions are stored. When the instructions are executed on a terminal device, the terminal device executes the voiceprint recognition method as described above.

[0128] The present invention also discloses a computer software program product, which, when running on a terminal device, enables the terminal device to execute the voiceprint recognition method as described above.

[0129] When performing voiceprint feature vector comparison, the present invention calculates the cosine distance between the input voiceprint feature vector and the stored voiceprint feature vector, and by setting a cosine distance acceptance threshold and a cosine distance rejection threshold, screens out qualified stored voiceprint feature vectors as reference voiceprint feature vectors. The reference voiceprint feature vectors are then further compared with the input feature vectors using a habit classification fully connected neural network, thereby screening out voiceprint feature vectors that conform to the user's habits and determining the control parameter values. Each input voiceprint feature vector and its time are stored, and the present invention also provides a real-time training process for the habit classification fully connected neural network, which is periodically initiated to train the habit classification fully connected neural network. During training, the process is based on the stored voiceprint feature vectors (which contain the user's recent voiceprint feature information), effectively fitting the different operating habits of the same person at different times, improving the accuracy of voiceprint recognition, and effectively enhancing the user's intelligent control experience.

[0130] Furthermore, the present invention regularly updates stored voiceprint feature vectors. During this update, an importance score is calculated by combining the duplication score, time score, and error score of each stored voiceprint feature vector. These feature vectors with higher importance scores are then selected and used as new stored data to replace the original data. This regular removal of voiceprint feature vectors improves the accuracy of the training data used by the habit classification fully connected neural network, thereby improving the accuracy of voiceprint recognition.

[0131] The module referred to in the present invention refers to a series of computer program segments that can be executed by at least one processor and can complete fixed functions, which are stored in a memory. In some embodiments, the functions of each module will be described in detail in subsequent embodiments.

[0132] The server includes: a database, a memory, at least one processor, a computer program stored in the memory and executable on the at least one processor, and at least one communication bus. When the at least one processor executes the computer program, the steps of the aforementioned embodiment of the semi-supervised training method for a neural network are implemented.

[0133] Exemplarily, the computer program may be divided into one or more modules / units, which are stored in the memory and executed by the at least one processor to implement the present invention. The one or more modules / units may be a series of computer program instruction segments capable of performing specific functions, and the instruction segments are used to describe the execution process of the computer program 34 in the server 3.

[0134] A server is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions. Its hardware includes but is not limited to microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0135] Those skilled in the art will understand that the schematic diagram is only an example of a server and does not constitute a limitation on the server. The server may include more or fewer components than shown in the diagram, or a combination of certain components, or different components. For example, the server may also include input and output devices, network access devices, buses, etc.

[0136] The database is a repository on the server that organizes, stores, and manages data according to a data structure. Databases are generally classified into three types: hierarchical databases, network databases, and relational databases. In this embodiment, the database is used to store the sample data, etc.

[0137] The at least one processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA), other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the server and connects various parts of the entire server using various interfaces and lines.

[0138] The memory can be used to store the computer programs and / or modules / units. The processor implements the various functions of the server by running or executing the computer programs and / or modules / units stored in the memory and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application required for a function (such as a sound playback function, an image playback function, etc.); the data storage area can store data created based on the use of the server (such as audio data, a phone book, etc.). In addition, the memory can include a high-speed random access memory and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, at least one disk storage device, a flash memory device, or other volatile solid-state storage device.

[0139] The memory stores program codes, and the at least one processor can call the program codes stored in the memory to execute related functions.

[0140] If the module / unit integrated in the server is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the process in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, it can implement the steps of the above-mentioned various method embodiments. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0141] Although not shown, the server may further include a power supply (such as a battery) for supplying power to the various components. Preferably, the power supply may be logically connected to the at least one processor via a power management system, thereby enabling the power management system to manage charging, discharging, and power consumption. The power supply may further include any components such as one or more DC or AC power supplies, a recharging system, a power failure detection circuit, a power converter or inverter, and a power status indicator. The server may further include a Bluetooth module, a Wi-Fi module, etc., which will not be described in detail here.

[0142] The above description is merely an embodiment of the present invention and does not limit the technical scope of the present invention. Therefore, any minor modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention are still within the scope of the technical solution of the present invention.

Claims

1. A voiceprint recognition method based on an offline self-training neural network, comprising: S11, collecting input voice electrical signals, slicing the input voice electrical signals, and processing the MFCC to generate an audio segment MFCC vector; S12: Input the MFCC vector of the audio clip into the recurrent neural network for isolated word recognition, extract the isolated word, and compare it with the required wake-up word. If the comparison is inconsistent, return to S11; if the comparison is consistent, input the MFCC vector of the audio clip into the recurrent neural network for voiceprint extraction to perform voiceprint recognition preprocessing and generate an input voiceprint feature vector; It is characterized in that: the voiceprint recognition method further includes: S13. Select an uncompared stored voiceprint feature vector from the stored voiceprint feature vectors as the voiceprint feature vector to be compared, calculate the cosine distance between the voiceprint feature vector to be compared and the input voiceprint feature vector using the cosine distance formula, and record the calculated distance. Repeat step S13 until there are no uncompared voiceprint feature vectors in the stored voiceprint feature vectors. S14. Determine whether the recorded maximum cosine distance is greater than the cosine distance acceptance threshold. If so, use all stored voiceprint feature vectors whose cosine distances are greater than the cosine distance acceptance threshold as reference voiceprint feature vectors, and then enter the habit prediction control parameter setting sub-process; otherwise, enter S15. S15. Determine whether the recorded maximum cosine distance is greater than the cosine distance rejection threshold. If so, use all stored voiceprint feature vectors whose cosine distances are greater than the cosine distance rejection threshold as control voiceprint feature vectors, and then enter the habit prediction control parameter value setting sub-process. S16. When the sub-process ends, the current control parameter value is associated with the input voiceprint feature vector and the current time, and stored; if the user modifies the control parameter value at this time, the preset value of the input voiceprint feature vector is additionally recorded; The above-mentioned custom predictive control parameter value setting sub-process is as follows: S21, inputting the input voiceprint feature vector and the corresponding current time into the habit classification fully connected neural network to generate a comparison habit classification vector; S22, obtaining all control voiceprint feature vectors and corresponding vector association times and inputting them into a habit classification fully connected neural network to generate a control habit classification vector; S23, calculating the Euclidean distance between the comparison habit classification vector and all control habit classification vectors; S24. Setting the current control parameter to the control parameter associated with the reference voiceprint feature vector corresponding to the minimum Euclidean distance; S25, end; The voiceprint recognition method also includes a real-time training process of a habit classification fully connected neural network; The real-time training process of the habit classification fully connected neural network is as follows: S51. Obtain a portion of the stored voiceprint feature vectors and the corresponding associated time as a training set input, and the remaining stored voiceprint feature vectors and their associated time as a validation set. Use triple loss as the loss function and use weighted summation to calculate the Euclidean distance between the two sets of control parameter values. S52. Determine whether the average Euclidean distance error of the prediction results of the habit classification fully connected neural network on the training set and the validation set is less than a preset error threshold. If not, randomly select a vector a from the training set, set p to any vector with the same control parameter value, and set n to any vector with a different control parameter value, with the preset inter-class distance being m. Calculate the loss using the loss calculation formula, and reversely train the habit classification fully connected neural network. The loss is calculated as follows: Repeat step S52 until the average Euclidean distance error of the prediction results of the habit classification fully connected neural network on the training set and the validation set is less than a preset error threshold; S53, end.

2. The voiceprint recognition method based on offline self-training neural network according to claim 1, characterized in that: In S15, if the recorded maximum cosine distance is not greater than the cosine distance rejection threshold, the sub-process of setting the control parameter value according to the voiceprint feature vector close to the gender is entered, which is as follows: S31. Calculate the cosine distance between the input voiceprint feature vector and the stored multiple groups of male voiceprint feature vectors, and output the minimum cosine distance and the corresponding age of the voiceprint inputter; S32. Calculate the cosine distance between the input voiceprint feature vector and the stored multiple groups of female voiceprint feature vectors, and output the minimum cosine distance and the corresponding age of the voiceprint inputter; S33, determining whether the minimum cosine distances of multiple groups of male voiceprint feature vectors are smaller than the minimum cosine distances of multiple groups of female voiceprint feature vectors; if so, setting the current control parameter to the default control parameter value for males of the corresponding age; otherwise, setting the current control parameter to the default control parameter value for females of the corresponding age; S34, end.

3. The voiceprint recognition method based on offline self-training neural network according to claim 1, characterized in that: The voiceprint recognition method further includes a process of removing redundant voiceprint feature vectors, which is as follows: S41. Obtain all stored voiceprint feature vectors and their creation time; S42. Calculate the duplicate score of each stored voiceprint feature vector; Specifically, the cosine distance between the stored voiceprint feature vector and the rest of the stored voiceprint feature vectors is calculated. If all cosine distances are not less than the cosine distance threshold, the duplicate item score is 0; if there is a cosine distance less than the cosine distance threshold, the duplicate item score is negative. The more cosine distances less than the cosine distance threshold, the lower the duplicate item score. S43. Calculate the time score of each stored voiceprint feature vector. The score created at the current moment is 0. The earlier the creation time, the lower the score. S44. Calculate the error score of each stored voiceprint feature vector. If the vector has been modified after being preset, the score is positive; if it has not been modified, the score is 0. S45. Perform a weighted summation of the duplicate item, time item, and error item of each stored voiceprint feature vector to obtain an importance score for the stored voiceprint feature vector; S46. Select multiple items of the pre-set value with the highest importance score from all stored voiceprint feature vectors, and replace the original stored voiceprint feature vectors with these voiceprint feature vectors as new stored voiceprint feature vectors; S47, end.

4. A voiceprint recognition system based on an offline self-training neural network, comprising: An acquisition module, used for acquiring input voice electrical signals; The MFCC processing module is used to slice and perform MFCC processing on the input speech electrical signal to generate the MFCC vector of the audio segment; The wake-up module is used to input the MFCC vector of the audio clip into the recurrent neural network for isolated word recognition, extract the isolated word and compare it with the required wake-up word. If the comparison is consistent, wake up; The voiceprint extraction module is used to input the MFCC vector of the audio clip into the voiceprint extraction recurrent neural network for voiceprint recognition preprocessing after waking up, and generate the input voiceprint feature vector; Characterized in that: the voiceprint recognition system further includes: The voiceprint comparison module is used to compare all stored voiceprint feature vectors with the input voiceprint feature vector, calculate the cosine distance between the two and record it; when the maximum cosine distance recorded is greater than the cosine distance acceptance threshold, all stored voiceprint feature vectors with cosine distance greater than the cosine distance acceptance threshold are used as control voiceprint feature vectors; when the maximum cosine distance recorded is not greater than the cosine distance acceptance threshold but greater than the cosine distance rejection threshold, all stored voiceprint feature vectors with cosine distance greater than the cosine distance rejection threshold are used as control voiceprint feature vectors; a habit prediction control module, configured to be activated when a comparison voiceprint feature vector exists; the habit prediction control module is configured to input the input voiceprint feature vector and the corresponding association time into a habit classification fully connected neural network to generate a comparison habit classification vector; and further configured to input the comparison voiceprint feature vector and the corresponding association time into the habit classification fully connected neural network to generate a comparison habit classification vector; and then calculate the Euclidean distance between the comparison habit classification vector and all comparison habit classification vectors, and set the current control parameter value to the comparison voiceprint feature vector corresponding to the minimum Euclidean distance. A storage module, configured to associate and store the current control parameter value with the current input voiceprint feature vector and its corresponding time; A habit classification fully connected neural network real-time training module, which is used to obtain and store voiceprint feature vectors and corresponding association times, and perform real-time training of the habit classification fully connected neural network; The real-time training of the habit classification fully connected neural network real-time training module is as follows: a portion of the stored voiceprint feature vectors and the corresponding associated time are obtained as the training set input, and the remaining stored voiceprint feature vectors and their associated time are used as the validation set, triple loss is used as the loss function, and the Euclidean distance between the two sets of control parameter values ​​is calculated using weighted summation; and it is determined whether the average Euclidean distance error of the prediction results of the habit classification fully connected neural network on the training set and the validation set is less than the preset error threshold. If not, a vector a is randomly selected from the training set, and any one of the vectors with the same control parameter value is set as p, and any one of the vectors with different control parameter values ​​is set as n, and the class distance preset value is m; the loss calculation formula is used to calculate this loss, and the habit classification fully connected neural network is reversely trained until the average Euclidean distance error of the prediction results of the habit classification fully connected neural network on the training set and the validation set is less than the preset error threshold; The loss is calculated as follows: 。 5. The voiceprint recognition system based on offline self-training neural network according to claim 4, characterized in that: The voiceprint recognition system further includes a proximity gender control module; When the maximum cosine distance recorded in the voiceprint comparison module is not greater than the cosine distance rejection threshold, the proximity gender control module is started; the proximity gender control module is used to calculate the cosine distance between the input voiceprint feature vector and the stored multiple groups of male voiceprint feature vectors, and output the minimum cosine distance therein and the corresponding age of the voiceprint inputter; it is also used to calculate the cosine distance between the input voiceprint feature vector and the stored multiple groups of female voiceprint feature vectors, and output the minimum cosine distance therein and the corresponding age of the voiceprint inputter; then it is determined whether the minimum value of the cosine distance of the multiple groups of male voiceprint feature vectors is less than the minimum value of the cosine distance of the multiple groups of female voiceprint feature vectors. If so, the current control parameter is set to the default control parameter value for males of the corresponding age; otherwise, the current control parameter is set to the default control parameter value for females of the corresponding age.

6. The voiceprint recognition system based on offline self-training neural network according to claim 4, characterized in that: The voiceprint recognition system also includes a voiceprint removal module, which is used to regularly update the stored voiceprint feature vector, including A duplicate score calculation unit is used to calculate the duplicate score of each stored voiceprint feature vector. Specifically, the cosine distance between the stored voiceprint feature vector and the remaining stored voiceprint feature vectors is calculated. If all cosine distances are not less than the cosine distance threshold, the duplicate score is 0; if there is a cosine distance less than the cosine distance threshold, the duplicate score is negative. The more cosine distances less than the cosine distance threshold, the lower the duplicate score. The time item score calculation unit is used to calculate the time score of each stored voiceprint feature vector. The score created at the current moment is 0, and the score of the item created earlier is lower; An error item score calculation unit is used to calculate the error item score of each stored voiceprint feature vector. If the vector is modified after the preset, the error item score is positive, and if it is not modified, the error item score is 0; An importance score calculation unit calculates the importance score of each stored voiceprint feature vector by performing a weighted sum of the duplicate item, time item, and error item of each stored voiceprint feature vector; The updating unit is used to select multiple items of the preset value with the highest importance score from all stored voiceprint feature vectors, and replace the original stored voiceprint feature vectors with these voiceprint feature vectors as new stored voiceprint feature vectors.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, and when the instructions are executed on a terminal device, the terminal device executes the voiceprint recognition method according to any one of claims 1 to 3.

8. A computer software program product, characterized in that: When the computer software program product is run on a terminal device, the terminal device executes the voiceprint recognition method according to any one of claims 1 to 3.

Images