Voiceprint generation method, verification method, identification device and storage medium

Abstract

The present invention provides a voiceprint generation method, a verification method, an identification device and a storage medium. The voiceprint generation method includes: obtaining multiple positive samples, the positive samples representing the first voiceprint of the first voice that has passed the voiceprint verification for the same target user within a historical period; updating the reference voiceprint according to the multiple positive samples and the registered voiceprint of the target user, wherein the registered voiceprint represents the reference voiceprint of the registered voice that the target user first recorded for the electronic device and is used for similarity calculation in voiceprint verification; and updating the similarity threshold of the voiceprint verification according to the updated reference voiceprint.

Description

Technical Field

[0001] The present invention relates to the field of speech processing technology, and in particular to a voiceprint generation method, a verification method, an identification device and a storage medium. Background Art

[0002] Voiceprint recognition technology compares the voiceprint stored in the voiceprint management device with the collected voiceprint to determine the user's identity information.

[0003] However, the user's voiceprint may change over time. For example, if the user has a cold, the voiceprint of the wake-up voice issued by the user is quite different from the stored voiceprint, causing the terminal device to be unable to recognize the user's voice, affecting the user's experience. Summary of the invention

[0004] In view of this, the present invention provides a voiceprint generation method, a verification method, an identification device and a storage medium.

[0005] According to a first aspect of the present invention, there is provided a voiceprint generation method, comprising: obtaining a plurality of positive samples, the positive samples representing a first voiceprint of a first speech that has passed voiceprint verification within a historical period for the same target user; updating a baseline voiceprint based on the plurality of positive samples and a registered voiceprint of the target user, wherein the registered voiceprint represents a voiceprint of a registered voice that is first recorded by the target user for an electronic device, and the baseline voiceprint is used for similarity calculation in voiceprint verification; and updating a similarity threshold for voiceprint verification based on the updated baseline voiceprint.

[0006] According to an embodiment of the present invention, a baseline voiceprint is updated based on multiple positive samples and a registered voiceprint of a target user, including: determining a first similarity between each positive sample and the baseline voiceprint at a corresponding historical moment to obtain multiple first similarities, wherein the multiple first similarities correspond one-to-one to multiple positive samples; calculating an average value of the first similarities corresponding to the multiple positive samples as the historical voiceprint similarity; determining multiple target positive samples from the multiple positive samples based on the historical voiceprint similarity, wherein the target positive sample represents a positive sample having a deviation less than a maximum threshold, and the deviation is the degree of deviation between the first similarity of the positive sample and the historical voiceprint similarity; and updating the baseline voiceprint based on the multiple target positive samples and the registered voiceprint of the target user.

[0007] According to an embodiment of the present invention, the method also includes: when it is determined that the number of positive samples in the historical period is greater than a preset threshold, deleting some positive samples so that the number of positive samples meets the preset threshold; and / or when it is determined that the number of positive samples in the historical period is less than the preset threshold, extending the time length of the historical period so that the number of positive samples meets the preset threshold.

[0008] According to an embodiment of the present invention, a baseline voiceprint is updated based on multiple positive samples and a registered voiceprint of a target user, including: determining average values ​​of different voiceprint features based on voiceprint vectors of different dimensions of multiple positive samples, where voiceprint vectors of different dimensions represent the data volume of different voiceprint features; synthesizing the current voiceprint of the target user based on multiple average values; and updating the baseline voiceprint based on the registered voiceprint and current voiceprint of the target user.

[0009] According to an embodiment of the present invention, updating the baseline voiceprint according to the registered voiceprint and the current voiceprint of the target user includes: obtaining the weight values ​​of the registered voiceprint and the current voiceprint of the target user respectively; and determining the baseline voiceprint according to the weight value, the registered voiceprint and the current voiceprint.

[0010] According to an embodiment of the present invention, the weight value of the registered voiceprint is not less than the weight value of the current voiceprint.

[0011] According to an embodiment of the present invention, updating the similarity threshold of voiceprint verification according to the updated baseline voiceprint includes: obtaining multiple negative samples within a historical period, wherein the negative samples represent a second voiceprint determined for a second speech of a target user that fails voiceprint verification; based on multiple negative samples and multiple positive samples within the historical period, respectively determining the F1 values ​​of multiple candidate thresholds, wherein the F1 value of each candidate threshold represents the F1 value of a binary classification model under the candidate threshold; and updating the similarity threshold to the value of the candidate threshold with the largest F1 value.

[0012] The second aspect of the present invention provides a voiceprint verification method, comprising: in response to receiving a third voice to be verified, determining a third voiceprint of the third voice; determining a fourth similarity between the third voiceprint and an updated baseline voiceprint; and when the fourth similarity is greater than an updated similarity threshold, the voiceprint verification of the target user is passed; wherein the updated baseline voiceprint and the updated similarity threshold are both generated according to the above-mentioned voiceprint generation method.

[0013] The third aspect of the present invention provides a voiceprint recognition device, comprising: one or more processors; a memory for storing one or more programs, wherein when the one or more programs are executed by one or more processors, the one or more processors execute the above-mentioned voiceprint generation method or voiceprint verification method.

[0014] The fourth aspect of the present invention further provides a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, causes the processor to execute the above-mentioned voiceprint generation method or voiceprint verification method.

[0015] The fifth aspect of the present invention further provides a computer program product, including a computer program, which implements the above-mentioned voiceprint generation method or voiceprint verification method when executed by a processor.

[0016] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The above and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:

[0018] Figure 1 A schematic diagram schematically shows an application scenario provided by an embodiment of the present invention;

[0019] Figure 2 A flowchart of a method for generating a voiceprint according to an embodiment of the present invention is schematically shown;

[0020] Figure 3 A flowchart of a method for generating a voiceprint according to an embodiment of the present invention is schematically shown;

[0021] Figure 4 A schematic diagram schematically shows a method for determining a maximum threshold according to an embodiment of the present invention;

[0022] Figure 5 A flowchart of a method for determining a maximum threshold according to an embodiment of the present invention is schematically shown;

[0023] Figure 6 A flowchart of a method for generating a voiceprint according to another embodiment of the present invention is schematically shown;

[0024] Figure 7 A flowchart of a method for generating a voiceprint according to another embodiment of the present invention is schematically shown;

[0025] Figure 8 A schematic diagram schematically shows a method for generating a voiceprint according to another embodiment of the present invention;

[0026] Fig. 9 A schematic diagram schematically shows a method for determining a minimum similarity threshold according to yet another embodiment of the present invention;

[0027] Fig.10 The following schematically shows a judgment flow chart of a voiceprint verification method according to an embodiment of the present invention;

[0028] Fig.11 A structural block diagram of a voiceprint generating device according to an embodiment of the present invention is schematically shown;

[0029] Fig.12 A structural block diagram of a voiceprint verification device according to an embodiment of the present invention is schematically shown;

[0030] Fig.13 A block diagram of an electronic device suitable for implementing a voiceprint generation or voiceprint verification method according to an embodiment of the present invention is schematically shown. DETAILED DESCRIPTION

[0031] Below, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present invention. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of embodiments of the present invention. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of concepts of the present invention.

[0032] The terms used herein are only for describing specific embodiments and are not intended to limit the present invention. The terms "comprise", "include", etc. used herein indicate the existence of features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0033] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0034] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0035] The embodiments of the present invention provide a voiceprint generation method, a verification method, an identification device and a storage medium. Before introducing the technical solution provided by the embodiments of the present invention, the related technologies involved in the present invention are first described.

[0036] Voiceprint is a form of expression of the characteristics of sound. Since the human vocal organs actually differ in size, shape and function, the sounds produced by different people have different characteristics. Therefore, voiceprint as a basic feature to achieve human identity recognition is irreplaceable and stable, making voiceprint recognition technology widely used in various fields of information networks.

[0037] Usually, sound can be analyzed by electroacoustic instruments to display the sound wave spectrum that carries speech information. The sound wave characteristics (such as wavelength, frequency, intensity, rhythm, etc.) of the sound wave spectrum corresponding to different sounds may be different. The sound wave characteristics of the sound wave spectrum corresponding to the sound can be called the voiceprint of the sound.

[0038] Voiceprint recognition technology compares the voiceprint stored in the voiceprint management device with the collected voiceprint to determine the user's identity information.

[0039] For example, smart devices with voice assistants. For example, mobile phones, smart glasses, smart watches, smart speakers, etc. basically support voice assistants and support voice wake-up. Due to security issues, voice wake-up needs to support voiceprint verification. Only the owner's own voice can wake up his own device to prevent the device from being woken up by non-owners.

[0040] When a user uses a smart device for voice wake-up, the following steps are included. Step 1, the user registers the wake-up voiceprint. That is, the user can say the wake-up word multiple times, obtain multiple initial voiceprints, and calculate the average of the multiple initial voiceprints to obtain the registered voiceprint; or the registered audio contains a long audio with multiple wake-up words, and the registered voiceprint is calculated. Step 2, the user says the wake-up audio containing the wake-up word. Step 3, if the smart device is voice-wake up, determine the wake-up voiceprint of the user's wake-up audio. Step 4, calculate the similarity between the wake-up voiceprint and the registered voiceprint. Step 5, if the similarity is greater than the preset threshold, it is considered to be the owner's wake-up command, and the device wakes up, otherwise the wake-up is ignored.

[0041] Therefore, the setting of the similarity threshold for wake-up voiceprint verification is critical. If the threshold is too small, it is easy for non-owners to wake up the device. On the contrary, if the similarity threshold is too large, the owner's voice may not be able to wake up the device.

[0042] Because the user's voiceprint may change over time. For example, if the user has a cold or is sick and his voice is affected, the voiceprint of the wake-up voice issued by the user is quite different from the stored voiceprint, which may cause the smart device to fail to recognize the user's voice and cause a high wake-up failure rate. If you want to wake up accurately, you need to re-record a voiceprint after the voice changes, which is a more troublesome operation. This affects the user experience.

[0043] Before further describing the embodiments of the present invention in detail, the nouns and terms involved in the embodiments of the present invention are described. The nouns and terms involved in the embodiments of the present invention are subject to the following interpretations.

[0044] Voice voiceprint is to encode a piece of audio into a vector. The similarity of voiceprint vectors encoded from audios of different people is relatively small, that is, the voiceprints are different. The similarity of voiceprint encodings obtained from multiple audios of each person is relatively high, that is, the voiceprints are relatively close. Voiceprint encoding is mostly based on neural networks, such as CNN (Convolutional Neural Networks) / LSTM (Long-Stort Term Memory) / BERT (Bidirectional Encoder Representation from Transformers, pre-trained language representation model), etc. The input is audio or audio features. For example, FBANK (filter bank, a type of audio feature) and MFCC (Mel Frequency Cepstral Coefficents, Mel Frequency Cepstral Coefficents, are cepstral parameters extracted in the Mel scale frequency domain, which is a feature widely used in automatic speech and speaker recognition), and the output is a high-dimensional vector, such as 64 / 128 / 256 dimensions.

[0045] The F1 value is a comprehensive consideration of precision and recall. It provides a single metric to balance the performance of precision and recall. The higher the F1 value, the better the model performs in both precision and recall. Precision measures the proportion of samples that are truly positive among the results predicted by the model as positive samples. In other words, it reflects how accurate the model is when predicting positive. Recall measures the proportion of samples that the model successfully predicts as positive among all true positive samples. It reflects the model's ability to find all positive samples.

[0046] An embodiment of the present invention provides a voiceprint generation method, including: obtaining multiple positive samples, the positive samples representing a first voiceprint of a first speech that has passed voiceprint verification within a historical period for the same target user; updating a baseline voiceprint based on the multiple positive samples and the registered voiceprint of the target user, wherein the registered voiceprint represents a voiceprint of a registered voice that is first recorded by the target user for an electronic device, and the baseline voiceprint is used for similarity calculation in voiceprint verification; and updating a similarity threshold for voiceprint verification based on the updated baseline voiceprint.

[0047] An embodiment of the present invention also provides a voiceprint verification method, comprising: in response to receiving a third voice to be verified, determining a third voiceprint of the third voice; determining a fourth similarity between the third voiceprint and an updated baseline voiceprint; and when the fourth similarity is greater than an updated similarity threshold, the voiceprint verification of the target user is passed; wherein the updated baseline voiceprint and the updated similarity threshold are both generated according to the above-mentioned voiceprint generation method.

[0048] Figure 1 A schematic diagram schematically shows an application scenario provided by an embodiment of the present invention.

[0049] like Figure 1 As shown, the application scenario according to this embodiment may include user A and a smart terminal device 100, and the smart terminal device 100 may include a voiceprint registration function and a voiceprint verification function.

[0050] Based on the voiceprint registration function, the smart terminal device 100 can receive the voice information of user A, extract the corresponding voiceprint features according to the voice information of user A, and generate the corresponding registration voiceprint of user A according to the extracted voiceprint features.

[0051] Based on the voiceprint verification function, when the smart terminal device 100 detects the voice information of a certain speaker, it can extract the corresponding voiceprint features according to the voice information of the speaker, and compare the extracted voiceprint features with the registered voiceprint generated in the voiceprint registration to determine whether the current speaker is the user A corresponding to the registered voiceprint in the voiceprint registration. For example, when the smart terminal device 100 detects that the voice emitted by user A is "Xiao C Xiao C", the smart terminal device 100 can wake up the voice in the above manner, and then identify whether the speaker is user A. When the speaker is identified as user A, the smart terminal device 100 is awakened and replies "I am here". When the speaker is identified as not user A, the smart terminal device 100 may not respond.

[0052] The following Figure 2~Figure 9 The voiceprint generation method of the embodiment of the present invention is described in detail.

[0053] Figure 2 The flowchart of the voiceprint generation method according to an embodiment of the present invention is schematically shown.

[0054] like Figure 2 As shown, the voiceprint generation method of this embodiment includes operations S210 to S230.

[0055] In operation S210, a plurality of positive samples are obtained, where the positive samples represent a first voiceprint of a first speech verified by voiceprint within a historical period for the same target user.

[0056] Exemplarily, in response to an instruction to update the voiceprint of the electronic device at the current moment, a plurality of positive samples in a historical period before the current moment are obtained.

[0057] The instruction for updating the voiceprint of the electronic device at the current moment may be an instruction formed after the user actively triggers the voiceprint update button. It may also be that the electronic device automatically triggers the voiceprint update at a preset time, for example, the electronic device automatically triggers the voiceprint update at 00:00 every day. It may also be that the number of wake-ups of the electronic device is equal to the preset threshold during the time from the N-1th voiceprint update to the Nth voiceprint update, for example, if the total number of wake-ups of the electronic device reaches 100 times since the last voiceprint update of the electronic device, a new voiceprint update of the electronic device is automatically triggered.

[0058] The historical period can be a preset time period, for example, multiple positive samples within 10 days before the current moment. The historical period can also be a historical period defined according to the number of positive samples. For example, 10 positive samples are obtained, and 10 positive samples are generated for the target user within 5 days before the current moment. Therefore, the historical period can be the historical period that meets the preset number of positive samples at the current moment. The historical period can also be the time from the first power-on of the electronic device to the current moment. The embodiment of the present invention does not specifically limit the setting of the historical period, and it can be set according to the actual application.

[0059] A positive sample can be the voiceprint corresponding to the voice of the target user that wakes up the electronic device and passes the authentication. For example, at time a in the historical period, user A sends the first voice to the electronic device: "Xiao C Xiao C", which can wake up the electronic device (the wake-up word corresponding to the electronic device is Xiao C Xiao C), and the similarity between the first voiceprint corresponding to the first voice of user A: "Xiao C Xiao C" and the reference voiceprint of user A on the electronic device at time a is greater than the similarity threshold at time a, then the authentication is passed.

[0060] In operation S220, a reference voiceprint is updated according to the plurality of positive samples and the registered voiceprint of the target user, wherein the registered voiceprint represents the voiceprint of the registered voice first recorded by the target user for the electronic device, and the reference voiceprint is used for similarity calculation in voiceprint verification.

[0061] Exemplarily, the registered voiceprint can be the voiceprint formed by the registration voice that the target user first enters for the electronic device. Here, the registered voiceprint can be regarded as the initial value of the reference voiceprint. Before the first voiceprint update, the electronic device uses the initial reference voiceprint as a reference to verify the user's voice. After the first electronic device voiceprint update instruction is triggered, the initial reference voiceprint is updated to obtain the first updated reference voiceprint. Before the second voiceprint update is performed, the electronic device uses the first updated reference voiceprint as a reference to verify the user's voice. Each time the voiceprint update instruction is triggered, the voiceprint is updated on the basis of the registered voiceprint.

[0062] In operation S230, the similarity threshold for voiceprint verification is updated according to the updated reference voiceprint.

[0063] Exemplarily, in the case of updating the reference voiceprint, the similarity threshold of voiceprint verification is also updated synchronously. The similarity threshold of voiceprint verification is re-determined based on the updated reference voiceprint. For example, before the first voiceprint update, the electronic device uses the initial reference voiceprint (i.e., the registered voiceprint) and the initial similarity threshold as a reference to perform voiceprint verification on the user's voice. If the similarity between the voiceprint of user A's voice and the registered voiceprint is greater than or equal to the initial similarity threshold, it indicates that the voiceprint verification of user A has passed. After the instruction for the first electronic device voiceprint update is triggered, the registered voiceprint is updated to obtain the reference voiceprint after the first update, and the initial similarity threshold of the electronic device voiceprint verification is updated using the reference voiceprint after the first update to obtain the similarity threshold after the first update. If the similarity between the voiceprint of user A's voice and the reference voiceprint after the first update is greater than or equal to the similarity threshold after the first update, it indicates that the voiceprint verification of user A has passed.

[0064] It should be noted that, for the update of the reference voiceprint and the similarity threshold, the Mth reference voiceprint / similarity threshold may be the same as the M-1th reference voiceprint / similarity threshold, or may be different.

[0065] It is understandable that this method can dynamically update the reference voiceprint and similarity threshold for voiceprint verification without the user having to re-record the wake-up voiceprint. When the user's voiceprint changes over time, the reference voiceprint and similarity threshold can also be adjusted accordingly. This ensures that when the owner's voice changes, the electronic device can also perform accurate voiceprint verification based on the voiceprint formed by the changed voice, recognize the user's voice, and improve the user experience.

[0066] Figure 3 The flowchart of the voiceprint generation method according to an embodiment of the present invention is schematically shown. Figure 4 The following is a schematic diagram of a method for determining a maximum threshold according to an embodiment of the present invention.

[0067] As described above, in operation S220, the reference voiceprint is updated according to the plurality of positive samples and the registered voiceprint of the target user. Figure 3 As shown, the operation may further include operation S221a to operation S224a.

[0068] In operation S221a, a first similarity between each positive sample and a reference voiceprint at a corresponding historical moment is determined to obtain a plurality of first similarities, and the plurality of first similarities correspond one-to-one to the plurality of positive samples.

[0069] In operation S222a, an average value of the first similarities corresponding to the plurality of positive samples is calculated as the historical voiceprint similarity.

[0070] In operation S223a, multiple target positive samples are determined from multiple positive samples according to the historical voiceprint similarity, wherein the target positive sample represents a positive sample with a deviation less than a maximum threshold, and the deviation is the degree of deviation between the first similarity of the positive sample and the historical voiceprint similarity.

[0071] In operation S224a, the reference voiceprint is updated according to the plurality of target positive samples and the registered voiceprint of the target user.

[0072] Exemplarily, in operation S221a, a positive sample and a reference voiceprint at a corresponding historical moment can be understood as a reference voiceprint used when performing voiceprint verification on the positive sample. For example, the reference voiceprint at a historical moment corresponding to the positive sample used in the first voiceprint update is the registered voiceprint, and the reference voiceprint at a historical moment corresponding to the positive sample used in the second voiceprint update is the reference voiceprint after the first update.

[0073] In a possible implementation, since the similarity between the voiceprint of the speech to be verified and the current reference voiceprint needs to be calculated during voiceprint verification, the similarity between the voiceprint of the speech to be verified and the current reference voiceprint can be saved after the voiceprint verification of the speech to be verified passes. In the subsequent process of updating the reference voiceprint and the similarity threshold, the voiceprint of the speech to be verified will be used as a positive sample, so that the first similarity between it and the reference voiceprint at the corresponding historical moment can be directly read, eliminating the process of calculating the first similarity during the update process.

[0074] In operation S222a, the historical voiceprint similarity can be understood as the average value of the sum of the first similarities between multiple voiceprints that have been verified by voice and the corresponding benchmark voiceprints at historical moments stored in the electronic device before the voiceprint update instruction is triggered. Here, the selected multiple first similarities can be all the first similarities stored in the electronic device before each voiceprint update instruction is triggered. It can also be the first similarities stored in the electronic device within a preset period before each voiceprint update instruction is triggered, for example, the first similarities stored in the electronic device within 10 days before the voiceprint update instruction is triggered.

[0075] In operation S223a, the target positive sample can be understood as the sample corresponding to the case where the first similarity of the positive sample deviates from the historical voiceprint similarity to a lesser extent than the maximum threshold. The first similarities between multiple positive samples and the reference voiceprint at the corresponding historical moment can be regarded as discrete points, and the historical voiceprint similarity can be regarded as an average line. Points that deviate far from the average line are discarded and cannot be selected as target positive samples.

[0076] The deviation can be calculated according to the following formula:

[0077] Deviation = |First similarity - Historical voiceprint similarity | / Historical voiceprint similarity*100.

[0078] The maximum threshold may be determined by referring to the following operations S310 to S330, which will not be described in detail here.

[0079] In operation S224a, after the instruction to update the voiceprint of the electronic device is triggered, the historical voiceprint similarity can be calculated based on all the first similarities saved in the electronic device, and then the deviations between all the first similarities and the historical voiceprint similarities are calculated respectively. If the deviation is less than the maximum threshold, the positive sample corresponding to the first similarity is used as the target positive sample; if the deviation is greater than the maximum threshold, the positive sample corresponding to the first similarity is discarded. The initial registered voiceprint is updated according to the selected target positive sample to obtain an updated reference voiceprint.

[0080] Exemplarily, before the first voiceprint update, when the similarity between the voiceprint corresponding to the voice emitted by the electronic device during the voiceprint verification of user A and the initial reference voiceprint is greater than the similarity threshold at that time, the voice is the first voice, the voiceprint is the first voiceprint (i.e., the positive sample), and the similarity between the voiceprint and the initial reference voiceprint is the first similarity. For example, before the first voiceprint update, the first similarity obtained with the initial reference voiceprint as a reference may be: 0.8, 0.9, 0.8, 0.9, 0.4.

[0081] Before the second voiceprint update and after the first voiceprint update, if the similarity between the voiceprint corresponding to the voice emitted by the electronic device during the voiceprint verification of user A and the reference voiceprint after the first update is greater than the similarity threshold at that time, then the voiceprint is the first voiceprint (i.e., a positive sample), and the similarity between the voiceprint and the reference voiceprint after the first update is the first similarity. For example, before the second voiceprint update and after the first voiceprint update, the first similarity obtained by taking the reference voiceprint after the first update as a reference may be: 0.9, 0.7, 0.6, 0.6.

[0082] That is, before the N+1th voiceprint update, when the electronic device performs voiceprint verification, it calculates the similarity between the voiceprint of the user's voice and the reference voiceprint after the Nth update (i.e., the reference voiceprint at the corresponding historical moment) to obtain the first similarity. Thus, multiple first similarities are formed before the N+1th voiceprint update.

[0083] Taking the above two voiceprint updates as an example, after the third voiceprint update instruction is triggered, the first similarities of multiple positive samples saved in the electronic device are: 0.8, 0.9, 0.8, 0.9, 0.4, 0.9, 0.7, 0.6, 0.6. Therefore, the average of multiple first similarities can be calculated to obtain the historical voiceprint similarity (0.78). That is, (0.8+0.9+0.8+0.9+0.40.9+0.7+0.6+0.6) / 8=0.73.

[0084] In the process of selecting the target positive sample, for the multiple first similarities before the instruction that triggers the third voiceprint update (for example, 0.8, 0.9, 0.8, 0.9, 0.4, 0.9, 0.7, 0.6, 0.6), if the first similarity deviates from the historical voiceprint similarity calculated after the instruction that triggers the third voiceprint update by a large margin (that is, the deviation is greater than the maximum threshold), it is considered that the first similarity is abnormal, and the voiceprint corresponding to the first similarity does not need to be used as the target positive sample. If the first similarity deviates from the historical voiceprint similarity calculated after the instruction that triggers the third voiceprint update by a small margin (the deviation is less than the maximum threshold), the voiceprint corresponding to the first similarity can be used as the target positive sample. For example, the maximum threshold is set to 30%. Assuming that the first similarity of a positive sample is 0.4 and the historical voiceprint similarity is 0.73, the deviation of the positive sample = |0.4-0.73| / 0.73=45%, 45%>30%, then the voiceprint corresponding to 0.4 can be considered as an abnormal point and cannot be used as a target positive sample to participate in subsequent voiceprint updates.

[0085] For example, refer to Figure 4, at different moments in the process of using the electronic device, if the voiceprint verification is passed, a first similarity is obtained. Therefore, when the user wakes up the electronic device and the verification is passed, a first similarity corresponds to it. When the user wakes up the electronic device multiple times and the verification is passed, multiple first similarities are formed. After each instruction to trigger the update of the electronic device voiceprint, the multiple first similarities saved in the electronic device before the corresponding moment of triggering the voiceprint update can be averaged to obtain the historical voiceprint similarity. For example, if the electronic device voiceprint update is triggered at time T2, the historical voiceprint similarity corresponding to time T2 is calculated by averaging the multiple first similarities saved by the electronic device before time T2. In the process of the user using the electronic device, each time (i.e., at different moments) the electronic device voiceprint update is triggered, a historical voiceprint similarity corresponds to it. Usually, the value of the similarity is between 0-1. According to the relationship between time and difference (1-first similarity), the broken line (i.e., actual line) of the first similarity at different moments and the curve (average line) of the historical voiceprint similarity at different moments are obtained. If the difference value at a certain point deviates greatly from the average line (which may include the difference value being much higher than the average line or the difference value being much lower than the average line), the difference value corresponding to the point is called a discrete point, and the voiceprint corresponding to the point cannot be used as a target positive sample.

[0086] It can be understood that after selecting multiple positive samples in the historical period before the current moment, multiple target positive samples are determined according to the degree of deviation of the first similarity from the historical voiceprint similarity, thereby removing discrete points in the voiceprint verification. Using the target positive sample to update the reference voiceprint is conducive to making the update of the reference voiceprint more accurate, thereby ensuring the accuracy of the voiceprint verification.

[0087] In other embodiments, the calculation of the first similarity may also be performed by calculating the first similarity between each positive sample and the registered voiceprint to obtain multiple first similarities, that is, the first similarity is always calculated with the registered voiceprint as a reference.

[0088] Alternatively, multiple first similarities can be obtained by calculating the first similarity between each positive sample and the current baseline voiceprint. For example, after the second voiceprint update is triggered at time T2, the current baseline voiceprint of the electronic device is the baseline voiceprint after the first update. The first similarities between the voiceprints of the 10 positive samples within 10 days before time T2 and the baseline voiceprint after the first update are calculated respectively, and the multiple first similarities are averaged to obtain the historical voiceprint similarity corresponding to time T2. After the third voiceprint update is triggered at time T3, the current baseline voiceprint of the electronic device is the baseline voiceprint after the second update. The first similarities between the voiceprints of the 10 positive samples within 10 days before time T3 and the baseline voiceprint after the second update are calculated respectively, and the multiple first similarities are averaged to obtain the historical voiceprint similarity corresponding to time T3.

[0089] Figure 5 The flowchart of a method for determining a maximum threshold according to an embodiment of the present invention is schematically shown.

[0090] As described above, for the maximum threshold in operation S223a. In one possible implementation, Figure 5 As shown, determining the maximum threshold may include operations S310 to S330.

[0091] In operation S310, voiceprint sample pairs of multiple sample users are obtained, and the voiceprint sample pairs include normal voiceprint similarity and abnormal voiceprint similarity. The normal voiceprint similarity represents the similarity between the voiceprint of the sample user's voice verified by voiceprint under normal circumstances and the registered voiceprint of the sample user, and the abnormal voiceprint similarity represents the similarity between the voiceprint of the sample user's voice verified by voiceprint under abnormal circumstances and the registered voiceprint of the sample user.

[0092] In operation S320, the sample deviation between the normal voiceprint similarity and the abnormal voiceprint similarity in each voiceprint sample pair is determined.

[0093] In operation S330, a maximum threshold is determined according to the deviations of the multiple samples, where the maximum threshold represents the average maximum deviation between the abnormal voiceprint similarity and the normal voiceprint similarity.

[0094] Exemplarily, the maximum threshold is predetermined before the device leaves the factory. For example, before setting the maximum threshold, a normal voiceprint that passes the voiceprint verification when the sample user's voice is normal is collected. The normal voiceprint can be the voiceprint corresponding to the voice emitted by the sample user under normal conditions, and the voiceprint corresponding to the registered voiceprint of the sample user has a similarity greater than the preset similarity threshold. And an abnormal voiceprint that passes the voiceprint verification when the sample user's voice is abnormal is collected. The abnormal voiceprint can be the voiceprint corresponding to the voice emitted by the sample user under abnormal conditions, and the voiceprint corresponding to the registered voiceprint of the sample user has a similarity greater than the preset similarity threshold. Thus, a voiceprint sample pair of the sample user is obtained. It should be noted that the abnormal sound here can include sound changes in various situations, such as a cold, excessive use of the voice, etc. Therefore, for the same sample user, multiple voiceprint sample pairs corresponding to the sample user can be obtained.

[0095] Collect voiceprint sample pairs of multiple sample users. According to the calculation formula of sample deviation:

[0096] Sample deviation = 100*|Abnormal voiceprint similarity-Normal voiceprint similarity| / Normal voiceprint similarity. First, calculate the deviation of each voiceprint sample pair of different sample users, and then average the deviations of multiple voiceprint sample pairs of the user to obtain the average deviation of the user. Determine the maximum threshold from the average deviations of multiple users. For example, the average deviation of sample user 1 is 30%, the average deviation of sample user 2 is 28%, and the average deviation of sample user 3 is 31%. Then 31% is used as the maximum threshold.

[0097] In other embodiments, the maximum threshold may be set according to different abnormal situations, and the maximum threshold corresponding to each abnormal situation may be determined. The user may select the maximum threshold according to the abnormal situation he or she is in. For example, the maximum threshold may be determined when the user has a cold, or the maximum threshold may be determined when the user overuses his or her voice.

[0098] In some embodiments, the voiceprint generation method of the embodiment of the present invention further includes: when it is determined that the number of positive samples in the historical period is greater than a preset threshold, deleting some positive samples so that the number of positive samples meets the preset threshold.

[0099] For example, if the number of positive samples selected each time when the voiceprint is updated is 100 (i.e., the preset threshold), if the voiceprint update is triggered at time T3, and within 10 days before time T3 (historical period), the number of voiceprints that the electronic device is awakened by the user and passed verification is 120, then the number of positive samples exceeds the preset threshold. At this time, 100 positive samples can be randomly selected from the historical period to meet the preset threshold.

[0100] In other embodiments, the voiceprint generation method of the embodiment of the present invention further includes: when it is determined that the number of positive samples in the historical period is less than a preset threshold, extending the time length of the historical period so that the number of positive samples meets the preset threshold.

[0101] For example, if the number of positive samples selected each time when the voiceprint is updated is 100 (i.e., the preset threshold), if the voiceprint update is triggered at time T3, within 10 days before time T3 (historical period), the number of voiceprints that are awakened by the user and verified by the electronic device is 90, then the number of positive samples does not meet the preset threshold. At this time, the historical period can be extended to 11 days. If the number of positive samples within 11 days is 95, which still does not meet the preset threshold, the historical period will continue to be extended to 12 days until the number of positive samples is greater than or equal to 100.

[0102] It is understandable that limiting the calculation threshold (preset threshold) of the number of positive samples can not only ensure the accuracy of voiceprint updates and reduce the amount of calculations, but also prevent deviations in voiceprint updates. In other embodiments, negative samples represent the second voiceprint determined for the second voice of the target user that has not passed the voiceprint verification. Negative samples can be the voiceprints corresponding to the voice of the target user that wakes up the electronic device but fails the voiceprint authentication. If the number of positive samples exceeds the preset threshold and the number of negative samples is lower than the preset threshold at the same time in the historical period, the number of positive samples can be deleted and the number of negative samples can be supplemented according to the above method.

[0103] It is understandable that, in the case of screening target positive samples from positive samples, if the number of target positive samples in the historical period is greater than the preset threshold, some target positive samples are deleted so that the number of target positive samples meets the preset threshold. If the number of target positive samples is less than the preset threshold, the length of the historical period is extended so that the number of target positive samples meets the preset threshold.

[0104] For example, if for voiceprint update, the number of target positive samples selected each time is 100 (i.e., the preset threshold). If the voiceprint update is triggered at time T3, within 10 days before time T3 (historical period), the number of voiceprints that the electronic device is awakened by the user and verified is 100 (indicating positive samples), that is, there are 100 first voiceprints and 100 first similarities. The average value of these 100 first similarities is calculated to obtain the historical voiceprint similarity corresponding to time T3. The deviation between these 100 first similarities and the historical voiceprint similarity corresponding to time T3 is calculated to obtain the deviation of these 100 positive samples. If the deviation of 90 positive samples among these 100 positive samples is greater than or less than the maximum threshold, these 90 positive samples are used as target positive samples. At this time, the number of target positive samples is less than the preset threshold of 100, and the number of target positive samples does not meet the preset threshold. At this time, the historical period can be extended to 11 days. If the number of target positive samples within 11 days is 95 and does not meet the preset threshold, the historical period is further extended to 12 days until the number of target positive samples is greater than or equal to 100. It can be understood that in the case where there is a process of screening target positive samples from positive samples, the process of processing positive samples can be replaced by the process of processing target positive samples, and the specific process will not be repeated.

[0105] Figure 6 The following schematically shows a flow chart of a method for generating a voiceprint according to another embodiment of the present invention.

[0106] As described above, in operation S220, the reference voiceprint is updated according to the plurality of positive samples and the registered voiceprint of the target user. Figure 6 As shown, the operation may further include operation S221b to operation S223b.

[0107] In operation S221b, the average values ​​of different voiceprint features are determined according to the voiceprint vectors of different dimensions of the multiple positive samples, and the voiceprint vectors of different dimensions represent the data volume of different voiceprint features.

[0108] In operation S222b, the current voiceprint of the target user is synthesized based on the multiple average values.

[0109] In operation S223b, the reference voiceprint is updated according to the registered voiceprint and the current voiceprint of the target user.

[0110] Exemplarily, when 100 positive samples are selected, the voiceprint of the positive sample corresponds to a 64-dimensional vector. The vectors from the 1st dimension to the 64th dimension corresponding to the 100 positive samples are added respectively, and the average value from the 1st dimension to the 64th dimension is calculated. Then the average values ​​from the 1st dimension to the 64th dimension are synthesized to obtain the current voiceprint. Here, the current voiceprint can be understood as the voiceprint of the user over a period of time (historical period). For example, if the user's voiceprint has changed during this period of time, the current voiceprint can be a comprehensive voiceprint of the user after the normal voice changes to the abnormal voice. The voiceprints of normal sound and abnormal sound are merged to obtain a voiceprint that fits the change of the user's voice. Finally, the current voiceprint and the registered voiceprint are used to update the baseline voiceprint.

[0111] It can be understood that by averaging the vectors of multiple dimensions of each positive sample and then synthesizing the current voiceprint, the amount of calculation is greatly reduced compared to using multiple positive samples to calculate the similarity of voiceprints, obtaining multiple similarities, taking the average, and then comparing with the similarity threshold to determine whether it is a registered user.

[0112] It is understandable that, in the case where there is a process of screening target positive samples from positive samples, Figure 6 The process of processing the positive samples shown can be replaced by the process of processing the target positive samples, and the specific process will not be repeated.

[0113] Figure 7 The following schematically shows a flow chart of a method for generating a voiceprint according to another embodiment of the present invention.

[0114] As described above, in operation S223b, the reference voiceprint is updated according to the registered voiceprint and the current voiceprint of the target user. Figure 7 As shown, the operation may further include operation S2231 to operation S2232.

[0115] In operation S2231, the weight values ​​of the registered voiceprint and the current voiceprint of the target user are respectively obtained.

[0116] In operation S2232, a reference voice print is determined according to the weight value, the registered voice print, and the current voice print.

[0117] Exemplarily, the updated reference voiceprint may be calculated according to the following calculation formula.

[0118] Vnew = α*registered voiceprint + (1-α) *current voiceprint.

[0119] Among them, Vnew represents the baseline voiceprint after the Nth update, and α is a floating point value between 0.5 and 1. The larger the value, the greater the proportion of the registered voiceprint, and vice versa.

[0120] It is understandable that since the voiceprint of the user during registration is the most reliable, the voiceprint at the time of registration must be used as the basic reference data in all subsequent voiceprint verifications, so that the user's new voiceprint data is not easy to deviate from the original registered voiceprint data, ensuring the security of voiceprint verification.

[0121] In some embodiments, the weight value of the registered voiceprint is not less than the weight value of the current voiceprint.

[0122] For example, it is necessary to ensure that α is always not less than (1-α). Since the voiceprint of the user during registration is the most reliable, the registered voiceprint must be set to a ratio not less than the user's changed voiceprint (i.e., the current voiceprint) in all subsequent voiceprint verifications, thereby further making it difficult for the user's new voiceprint data to deviate from the original registered voiceprint data, ensuring the security of voiceprint verification.

[0123] It is understandable that, in the case where there is a process of screening target positive samples from positive samples, Figure 7 The process of processing the positive samples shown can be replaced by the process of processing the target positive samples, and the specific process will not be repeated.

[0124] Figure 8 A schematic diagram of a voiceprint generation method according to another embodiment of the present invention is schematically shown.

[0125] As described above, in operation S230, the similarity threshold of voiceprint verification is updated according to the updated reference voiceprint. Figure 8 As shown, the operation may further include operation S231 to operation S233.

[0126] In operation S231, a plurality of negative samples within a historical period are obtained, wherein the negative samples represent a second voiceprint determined for a second speech of a target user that fails voiceprint verification.

[0127] In operation S232, based on multiple negative samples and multiple positive samples in the historical period, F1 values ​​of multiple candidate thresholds are respectively determined, wherein the F1 value of each candidate threshold represents the F1 value of the binary classification model under the candidate threshold.

[0128] In operation S233, the similarity threshold is updated to the value of the candidate threshold with the largest F1 value.

[0129] For example, a negative sample may be a voiceprint corresponding to a user's voice that wakes up the electronic device but fails authentication. For example, user A sends a second voice to the electronic device: "Xiao C Xiao C", which can wake up the electronic device (the wake-up word corresponding to the electronic device is Xiao C Xiao C), but the similarity between the second voiceprint corresponding to the second voice of user A: "Xiao C Xiao C" and the reference voiceprint of user A on the electronic device is less than the similarity threshold at that time, and the authentication fails.

[0130] It is understandable that in binary classification, the model usually determines whether a sample is a positive sample or a negative sample based on a certain probability threshold (i.e., candidate threshold). Different candidate thresholds may lead to different precision and recall rates, resulting in different F1 values. By calculating the F1 values ​​under different candidate thresholds, the best classification threshold can be compared and selected.

[0131] Exemplarily, before the first voiceprint update, the electronic device uses the initial similarity threshold (i.e., the similarity threshold set before the device leaves the factory) as a reference to perform voiceprint verification on the user's voice. If the similarity between the voiceprint of user A's voice and the registered voiceprint is greater than or equal to the initial similarity threshold, it indicates that the voiceprint verification of user A has passed. After the instruction to update the voiceprint of the electronic device for the first time is triggered, the initial baseline voiceprint is updated to obtain the baseline voiceprint after the first update, the initial similarity threshold is updated to obtain the similarity threshold after the first update, and the similarity threshold after the first update is used as a reference to perform voiceprint verification on the user's voice. If the similarity between the voiceprint of user A's voice and the baseline voiceprint after the first update is greater than or equal to the similarity threshold after the first update, it indicates that the voiceprint verification of user A has passed.

[0132] For example, when updating the similarity threshold, 100 positive samples and 100 negative samples within 10 days before the current time are obtained. The negative sample can be the second voiceprint corresponding to the second voice that wakes up the electronic device by voice but fails the voiceprint verification.

[0133] Automatically traverse the positive and negative samples, calculate the second similarity between the first voiceprint of the positive sample and the updated baseline voiceprint, and the third similarity between the second voiceprint of the negative sample and the updated baseline voiceprint. Set multiple candidate thresholds, run the binary classification model for each candidate threshold, and calculate the F1 value corresponding to each candidate threshold. For example, set candidate threshold T1=0.9, candidate threshold T2=0.8, and candidate threshold T3=0.7 respectively. After running the binary classification model for each candidate threshold, calculate the corresponding F1 value. Assuming that the precision (Precision) corresponding to candidate threshold T1=0.9 is 0.85, and the recall rate (Recall) is 0.90, then the F1 value = 2 × (Precision × Recall) / (Precision + Recall) = 0.875. Similarly, it is calculated that when candidate threshold T2=0.8, the F1 value is 0.857, and when candidate threshold T3=0.7, the F1 value is 0.645. According to the F1 value, the candidate threshold T1 (F1 value is 0.875) performs best, so the similarity threshold can be updated to 0.9. The similarity threshold after the first update is 0.85. After the second trigger voiceprint update, if the similarity threshold after the second update is calculated to be 0.9, the similarity threshold after the first update 0.85 is updated to 0.9.

[0134] The candidate threshold can be taken in sequence between 0 and 1 according to the set step size, such as 0.01, 0.02, ..., 1. The threshold value is taken according to the set step size and verified by the binary classification model. It is also possible to randomly take values ​​between 0 and 1, for example, set the values ​​between 0 and 1 to be 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1. Randomly select between these 10 values, take 0.5 for the first verification, take 0.8 for the second verification, and so on, until all 10 numbers are verified. It is also possible to start from 0.5. If the F1 value of the first verification is greater than 0.8, it is considered that the values ​​between 0.1-0.4 are not suitable as candidate thresholds, and 0.6 is taken for the second verification, until the F1 value greater than 0.8 is verified.

[0135] If the user's voice has changed in the historical period, the updated similarity threshold will be smaller than the previous similarity threshold, which means that the verification requirements are relaxed, but it cannot be too small to avoid risks. Therefore, a reasonable similarity threshold should be set. If it is lower than the similarity threshold, the verification is considered to have failed.

[0136] It is understandable that, in the case where there is a process of screening target positive samples from positive samples, Figure 8 The process of processing the positive samples shown can be replaced by the process of processing the target positive samples, and the specific process will not be repeated.

[0137] In the case of abnormal user voice, the updated similarity threshold is usually smaller than the similarity threshold before updating. This can be understood as relaxing the verification requirements for voiceprint verification. However, if the updated similarity threshold is too small, it is easy to cause verification errors, for example, the wake-up voice verification made by someone other than the user of the electronic device will pass. Therefore, a minimum similarity threshold should be set. If the similarity threshold obtained based on the updated reference voiceprint is lower than the minimum similarity threshold, the minimum similarity threshold will be used as the updated similarity threshold. For the design method of the minimum similarity threshold, combined with Fig. 9 The details are as follows:

[0138] Fig. 9 The following is a schematic diagram of a method for determining a minimum similarity threshold according to another embodiment of the present invention.

[0139] As described above, the updated similarity threshold is greater than the minimum similarity threshold. In one possible implementation, Fig. 9 As shown, determining the minimum similarity threshold may include operations S410 to S420.

[0140] In operation S410, abnormal voiceprint similarities of multiple sample users are obtained, where the abnormal voiceprint similarity represents the similarity between the voiceprint of the sample user's speech verified by voiceprint when the voice of the sample user is abnormal and the registered voiceprint of the sample user.

[0141] In operation S420, a minimum similarity threshold is determined according to the similarities of the plurality of abnormal voiceprints. For example, the similarities of the abnormal voiceprints may refer to the above operations S310 to S330, which will not be described in detail here.

[0142] In operation S420, a minimum value is determined from multiple abnormal voiceprint similarities, that is, the minimum value of abnormal voiceprint similarities. The average value of the minimum abnormal voiceprint similarities corresponding to multiple sample users is calculated, and the average value of the minimum abnormal voiceprint similarities of multiple sample users is used as the minimum similarity threshold.

[0143] For example, the minimum value of abnormal voiceprint similarity of sample user 1 is 50%, the minimum value of abnormal voiceprint similarity of sample user 2 is 60%, and the minimum value of abnormal voiceprint similarity of sample user 3 is 54%. Calculate (50%+60%+54%) / 3=54.7%. 54.7% is used as the minimum similarity threshold. That is, the updated similarity threshold must be greater than the minimum similarity threshold.

[0144] In other embodiments, for setting the minimum similarity threshold, the minimum similarity threshold corresponding to each abnormal situation may be determined according to different abnormal situations. The user may select the minimum similarity threshold according to the abnormal situation he / she is in. For example, the minimum similarity threshold may be determined when the user has a cold, or the minimum similarity threshold may be determined when the user overuses his / her voice.

[0145] The following will be based on Figure 1 The scene described by Fig.10 The voiceprint verification method of the disclosed embodiment is described in detail.

[0146] Fig.10 The following schematically shows a judgment flow chart of a voiceprint verification method according to an embodiment of the present invention.

[0147] like Fig.10 As shown, the voiceprint verification method of this embodiment includes operations S510 to S550.

[0148] In operation S510, in response to receiving a third voice to be verified, a third voiceprint of the third voice is determined.

[0149] In operation S520, a fourth similarity between the third voice print and the updated reference voice print is determined.

[0150] In operation S530, it is determined whether the fourth similarity is greater than the updated similarity threshold.

[0151] In operation S540, when the fourth similarity is greater than or equal to the updated similarity threshold, the voiceprint verification of the target user is successful.

[0152] In operation S550, when the fourth similarity is less than the updated similarity threshold, the voiceprint verification of the target user fails.

[0153] Exemplarily, after the third voiceprint update, the third updated reference voiceprint and the third updated similarity threshold corresponding to the third updated reference voiceprint are obtained. The latest voice of the user is collected by the voice collection device of the electronic device, and the third voiceprint of the latest voice is determined. Then, the fourth similarity between the third voiceprint and the third updated reference voiceprint is calculated. If the fourth similarity is greater than or equal to the third updated similarity threshold, the voiceprint verification is passed; if the fourth similarity is less than the third updated similarity threshold, the voiceprint verification is not passed.

[0154] Based on the above voiceprint generation method, the present invention also provides a voiceprint generation device. Fig.11 The device is described in detail.

[0155] Fig.11The structure block diagram of the voiceprint generating device according to an embodiment of the present invention is schematically shown.

[0156] like Fig.11 As shown, the voiceprint generating device 600 of this embodiment includes an acquisition module 610 , a first updating module 620 and a second updating module 630 .

[0157] The acquisition module 610 is used to acquire multiple positive samples, where the positive samples represent the first voiceprint of the first speech verified by voiceprint for the same target user in a historical period. In one embodiment, the acquisition module 610 can be used to perform the operation S210 described above, which will not be repeated here.

[0158] The first updating module 620 is used to update the reference voiceprint according to the plurality of positive samples and the registered voiceprint of the target user, wherein the registered voiceprint represents the voiceprint of the registered voice first recorded by the target user for the electronic device, and the reference voiceprint is used for similarity calculation in voiceprint verification. In one embodiment, the first updating module 620 can be used to perform the operation S220 described above, which will not be described in detail here.

[0159] The second updating module 630 is used to update the similarity threshold of the voiceprint verification according to the updated reference voiceprint. In one embodiment, the second updating module 630 can be used to perform the operation S230 described above, which will not be described in detail here.

[0160] According to an embodiment of the present invention, any multiple modules in the acquisition module 610, the first update module 620, and the second update module 630 can be combined into one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present invention, at least one of the acquisition module 610, the first update module 620, and the second update module 630 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application-specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware, and firmware, or in any appropriate combination of any of them. Alternatively, at least one of the acquisition module 610, the first update module 620, and the second update module 630 can be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding function can be executed.

[0161] Based on the above voiceprint verification method, the present invention also provides a voiceprint verification device. Fig.12 The device is described in detail.

[0162] Fig.12 The structure block diagram of the voiceprint verification device according to an embodiment of the present invention is schematically shown.

[0163] like Fig.12 As shown, the voiceprint verification device 700 of this embodiment includes a first determination module 710 , a second determination module 720 and a verification module 730 .

[0164] The first determination module 710 is used to determine the third voiceprint of the third voice in response to receiving the third voice to be verified. In one embodiment, the first determination module 710 can be used to perform the operation S510 described above, which will not be repeated here.

[0165] The second determination module 720 is used to determine a fourth similarity between the third voiceprint and the updated reference voiceprint. In one embodiment, the second determination module 720 can be used to perform the operation S520 described above, which will not be described in detail here.

[0166] The verification module 730 is used to verify that the target user's voiceprint verification passes when the fourth similarity is greater than the updated similarity threshold, and to verify that the target user's voiceprint verification fails when the fourth similarity is less than the updated similarity threshold. In one embodiment, the verification module 730 can be used to perform the operations S530-S550 described above, which will not be repeated here.

[0167] According to an embodiment of the present invention, any multiple modules of the first determination module 710, the second determination module 720 and the verification module 730 can be combined into one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present invention, at least one of the first determination module 710, the second determination module 720 and the verification module 730 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware and firmware or in any appropriate combination of any of them. Alternatively, at least one of the first determination module 710, the second determination module 720 and the verification module 730 can be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding function can be executed.

[0168] Fig.13A block diagram of an electronic device suitable for implementing a voiceprint generation or voiceprint verification method according to an embodiment of the present invention is schematically shown.

[0169] like Fig.13 As shown, the electronic device 800 according to an embodiment of the present invention includes a processor 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from a storage part 808 to a random access memory (RAM) 803. The processor 801 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 801 may also include an onboard memory for caching purposes. The processor 801 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0170] In RAM 803, various programs and data required for the operation of electronic device 800 are stored. Processor 801, ROM 802 and RAM 803 are connected to each other via bus 804. Processor 801 performs various operations of the method flow according to the embodiment of the present invention by executing the programs in ROM 802 and / or RAM 803. It should be noted that the program can also be stored in one or more memories other than ROM 802 and RAM 803. Processor 801 can also perform various operations of the method flow according to the embodiment of the present invention by executing the programs stored in one or more memories.

[0171] According to an embodiment of the present invention, the electronic device 800 may further include an input / output (I / O) interface 805, which is also connected to the bus 804. The electronic device 800 may further include one or more of the following components connected to the I / O interface 805: an input portion 806 including a keyboard, a mouse, etc.; an output portion 807 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 808 including a hard disk, etc.; and a communication portion 909 including a network interface card such as a LAN card, a modem, etc. The communication portion 909 performs communication processing via a network such as the Internet. A drive 810 is also connected to the I / O interface 805 as needed. A removable medium 811, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 810 as needed, so that a computer program read therefrom is installed into the storage portion 808 as needed.

[0172] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiment; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present invention is implemented.

[0173] According to an embodiment of the present invention, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, it may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device. For example, according to an embodiment of the present invention, the computer-readable storage medium may include the ROM 802 and / or RAM 803 described above and / or one or more memories other than ROM 802 and RAM 803.

[0174] The embodiment of the present invention also includes a computer program product, which includes a computer program, and the computer program contains program code for executing the method shown in the flowchart. When the computer program product is run in a computer system, the program code is used to enable the computer system to implement the voiceprint generation or voiceprint verification method provided by the embodiment of the present invention.

[0175] The computer program executes the above functions defined in the system / device of the embodiment of the present invention when the processor 801 executes the computer program. According to the embodiment of the present invention, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0176] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 909, and / or installed from the removable medium 811. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0177] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 909, and / or installed from the removable medium 811. When the computer program is executed by the processor 801, the above functions defined in the system of the embodiment of the present invention are performed. According to the embodiment of the present invention, the system, device, means, module, unit, etc. described above can be implemented by a computer program module.

[0178] According to an embodiment of the present invention, the program code for executing the computer program provided by the embodiment of the present invention can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).

[0179] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0180] It will be appreciated by those skilled in the art that the features described in the various embodiments of the present invention may be combined and / or combined in various ways, even if such combinations and / or combinations are not explicitly described in the present invention. In particular, without departing from the spirit and teachings of the present invention, the features described in the various embodiments of the present invention may be combined and / or combined in various ways. All of these combinations and / or combinations fall within the scope of the present invention.

[0181] The embodiments of the present invention are described above. However, these embodiments are only for the purpose of illustration, and are not intended to limit the scope of the present invention. Although each embodiment is described above, it does not mean that the measures in each embodiment cannot be used in combination advantageously. Without departing from the scope of the present invention, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present invention.

Claims

1. A voiceprint generation method, characterized in that: include: Acquire multiple positive samples, where the positive samples represent first voiceprints of first speech verified by voiceprints for the same target user within a historical period; updating a reference voiceprint according to the plurality of positive samples and the registered voiceprint of the target user, wherein the registered voiceprint represents a voiceprint according to the registered voice recorded by the target user for the electronic device for the first time, and the reference voiceprint is used for similarity calculation in the voiceprint verification; Updating the similarity threshold of the voiceprint verification according to the updated reference voiceprint; Among them, updating the baseline voiceprint according to the multiple positive samples and the registered voiceprint of the target user includes: determining the first similarity between each of the positive samples and the baseline voiceprint at the corresponding historical moment to obtain multiple first similarities, and the multiple first similarities correspond to the multiple positive samples one-to-one; calculating the average value of the first similarities corresponding to the multiple positive samples as the historical voiceprint similarity; determining multiple target positive samples from the multiple positive samples according to the historical voiceprint similarity, wherein the target positive samples represent positive samples with a deviation less than a maximum threshold, and the deviation is the degree of deviation between the first similarity of the positive sample and the historical voiceprint similarity; updating the baseline voiceprint according to the multiple target positive samples and the registered voiceprint of the target user.

2. The method according to claim 1, characterized in that in, The method further comprises: When it is determined that the number of positive samples in the historical period is greater than a preset threshold, deleting some of the positive samples so that the number of positive samples meets the preset threshold; and / or When it is determined that the number of positive samples in the historical period is less than the preset threshold, the time length of the historical period is extended so that the number of positive samples meets the preset threshold.

3. The method according to any one of claims 1 to 2, characterized in that: in, The updating of the reference voiceprint according to the plurality of positive samples and the registered voiceprint of the target user comprises: Determine the average values ​​of different voiceprint features according to the voiceprint vectors of different dimensions of the plurality of positive samples, wherein the voiceprint vectors of different dimensions represent the data amounts of different voiceprint features; synthesizing the current voiceprint of the target user according to the multiple average values; The reference voiceprint is updated according to the registered voiceprint of the target user and the current voiceprint.

4. The method according to claim 3, characterized in that in, The updating of the reference voiceprint according to the registered voiceprint of the target user and the current voiceprint includes: Respectively obtaining weight values ​​of the registered voiceprint and the current voiceprint of the target user; The reference voiceprint is determined according to the weight value, the registered voiceprint and the current voiceprint.

5. The method according to claim 4, characterized in that in, The weight value of the registered voiceprint is not less than the weight value of the current voiceprint.

6. The method according to claim 1, wherein: The updating of the similarity threshold of the voiceprint verification according to the updated reference voiceprint includes: Acquire a plurality of negative samples within the historical period, wherein the negative samples represent a second voiceprint determined for a second speech of the target user that fails voiceprint verification; Based on the multiple negative samples and the multiple positive samples in the historical period, respectively determine the F1 values ​​of multiple candidate thresholds, wherein the F1 value of each candidate threshold represents the F1 value of the binary classification model under the candidate threshold; The similarity threshold is updated to the value of the candidate threshold with the largest F1 value.

7. A voiceprint verification method, characterized in that: include: In response to receiving a third voice to be verified, determining a third voiceprint of the third voice; Determining a fourth similarity between the third voiceprint and the updated reference voiceprint; When the fourth similarity is greater than the updated similarity threshold, the voiceprint verification of the target user is passed; The updated reference voiceprint and the updated similarity threshold are both generated according to the method according to any one of claims 1-6.

8. A voiceprint recognition device, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors execute the method according to any one of claims 1 to 6 or 7.

9. A computer-readable storage medium, characterized in that: Executable instructions are stored thereon, and when the instructions are executed by a processor, the processor executes the method according to any one of claims 1 to 6 or 7.

Images