Method, device and computer equipment for verifying voiceprint model recognition effect
By considering the population distribution of business scenarios and multiple verification data in the voiceprint model verification, combined with the success rate of business identification, the problem of inaccurate verification results in the existing technology is solved, and the accuracy of evaluating the recognition effect of voiceprint model is improved.
Patent Information
- Application Number
- CN202011485999.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-12-16
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2040-12-16
AI Technical Summary
The existing voiceprint model verification methods fail to fully consider non-model factors, resulting in inaccurate verification results, especially in different application scenarios, the impact of different population distributions has not been considered.
By obtaining population distribution data related to business scenarios, collecting multiple sets of registered audio and verification audio, performing at least two voiceprint verifications, calculating the test recognition success rate, and combining the business recognition success rate, weighted summing is performed to evaluate the voiceprint model recognition effect.
The accuracy of voiceprint model verification is improved, and the effectiveness of actual business scenarios is truly reflected, and the model effect deviation caused by non-model factors is eliminated.
Smart Images

Figure CN112599137B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of artificial intelligence technology, and in particular to a method, device, computer equipment and storage medium for verifying the recognition effect of a voiceprint model. Background Art
[0002] Voiceprints have the advantages of being easy to obtain and having low recognition costs, so the application of voiceprint recognition is becoming more and more widespread. Voiceprint recognition is to convert sound signals into electrical signals and then use computers to recognize them. Specifically, a voiceprint model is trained using a neural network to recognize the sound. The recognition effect of the voiceprint model has a decisive influence on the application of voiceprints.
[0003] After the voiceprint model is trained, it is usually verified to evaluate whether it can be put into use. In the traditional solution, pre-collected audio files are used, and 3 audios in each group are used as voiceprint registration audios, and 1 audio is used as verification audio to verify the false alarm rate of the voiceprint (the audios in the same group are recorded by the same person. When the voiceprint is recognized, it should be judged as the person himself. If it is judged as not the person, it will be counted into the false alarm rate), and other audios outside the group are used to verify the false alarm rate (the audios in different groups are recorded by different people. When the voiceprint is recognized, it should be judged as not the person himself. If it is judged as the person himself, it will be counted into the false alarm rate). Based on these two indicators, the accuracy and reliability of the voiceprint model are verified.
[0004] This method only considers the impact of the model itself on the accuracy of voiceprint verification during verification. In actual applications, there are other influencing factors, such as differences in population distribution in different application scenarios. Therefore, this verification method does not take into account the impact of non-model factors on voiceprint recognition, resulting in inaccurate verification results. Summary of the invention
[0005] Based on this, it is necessary to provide a method, device, computer equipment and storage medium for verifying voiceprint model recognition effect which can improve verification accuracy in response to the above technical problems.
[0006] A method for verifying the recognition effect of a voiceprint model, the method comprising:
[0007] Obtain a verification data set, wherein the population distribution of the test users in the verification data set is related to the business scenario, the verification data set includes the registration audio and verification audio of the test users, and each test user has at least two sets of verification audio;
[0008] Calling the voiceprint model, and performing at least two voiceprint verifications on each test user according to the registration audio and verification audio of each test user in the verification data set, to obtain a test recognition success rate;
[0009] According to the test recognition success rate, a verification result of the voiceprint model recognition effect is obtained.
[0010] In one embodiment, the calling of the voiceprint model, performing at least two voiceprint verifications on each test user according to the registration audio and verification audio of each test user in the verification data set, and obtaining the test recognition success rate includes:
[0011] Calling the voiceprint model, performing at least two voiceprint verifications on each test user according to the registration audio and verification audio of each test user in the verification data set, and obtaining a first recognition success rate, a second recognition success rate, and a recognition success rate;
[0012] The first recognition success rate, the second recognition success rate and the recognition success rate are weighted and summed to obtain a test recognition success rate; wherein the weight of the first recognition success rate is greater than the weight of the second recognition success rate and the weight of the recognition success rate.
[0013] In one embodiment, obtaining a verification data set includes:
[0014] Obtain the population distribution of business scenarios;
[0015] Determine the number of test users of each gender and age group according to the population distribution, and obtain target test users;
[0016] At least three sets of registration audio and at least two sets of verification audio of each target test user are collected to obtain a verification data set.
[0017] In one embodiment, the method further comprises:
[0018] According to the actual business scenario, the number of users who have successfully been identified among those with normal audio is verified, and the business recognition success rate under normal audio conditions is obtained;
[0019] The obtaining a verification result of the voiceprint model recognition effect according to the test recognition success rate includes: obtaining a verification result of the voiceprint model recognition effect according to the test recognition success rate and the business recognition success rate.
[0020] In one embodiment, according to an actual service scenario, the number of users with normal audio being successfully identified is verified to obtain a service identification success rate under normal audio conditions, including:
[0021] Obtain verification audio data of users in actual business, analyze the verification audio data, and obtain users whose verification audio is normal;
[0022] Obtain the number of users who have been successfully identified among users whose audio is normal in actual business verification;
[0023] The service recognition success rate under normal audio conditions is calculated based on the number of users whose audio is verified to be normal and the number of users who are successfully identified among the users whose audio is verified to be normal.
[0024] In one embodiment, obtaining verification audio data of a user in an actual business, analyzing the verification audio data, and obtaining a user whose verification audio is normal includes:
[0025] Detect and verify the signal-to-noise ratio of audio data to obtain high-quality sound data;
[0026] Perform frequency analysis and speech recognition on high-quality sound signals to obtain effective sound signals;
[0027] The sound signals of people who log in and register differently from the valid sound signals are eliminated to obtain users whose audio is verified to be normal.
[0028] In one embodiment, according to the test recognition success rate and the business recognition success rate, a verification result of the voiceprint model recognition effect is obtained, including:
[0029] Determine the weights of the test recognition success rate and the business recognition success rate according to the online time of the application and the number of users who have verified that the audio is normal;
[0030] Weighting the test recognition success rate and the business recognition success rate to obtain a score of the voiceprint model;
[0031] The score is used to verify the recognition effect of the voiceprint model.
[0032] A device for verifying the recognition effect of a voiceprint model, the device comprising:
[0033] A verification data acquisition module is used to acquire a verification data set, wherein the population distribution of the test users in the verification data set is related to the business scenario, and the verification data set includes the registration audio and verification audio of the test users, and each test user has at least two sets of verification audio;
[0034] The recognition and analysis module is used to call the voiceprint model, and perform at least two voiceprint verifications on each test user according to the registration audio and verification audio of each test user in the verification data set to obtain a test recognition success rate;
[0035] An evaluation module is used to obtain a verification result of the voiceprint model recognition effect according to the test recognition success rate.
[0036] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:
[0037] Obtain a verification data set, wherein the population distribution of the test users in the verification data set is related to the business scenario, the verification data set includes the registration audio and verification audio of the test users, and each test user has at least two sets of verification audio;
[0038] Calling the voiceprint model, and performing at least two voiceprint verifications on each test user according to the registration audio and verification audio of each test user in the verification data set, to obtain a test recognition success rate;
[0039] According to the test recognition success rate, a verification result of the voiceprint model recognition effect is obtained.
[0040] A computer storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the following steps are implemented:
[0041] Obtain a verification data set, wherein the population distribution of the test users in the verification data set is related to the business scenario, the verification data set includes the registration audio and verification audio of the test users, and each test user has at least two sets of verification audio;
[0042] Calling the voiceprint model, and performing at least two voiceprint verifications on each test user according to the registration audio and verification audio of each test user in the verification data set, to obtain a test recognition success rate;
[0043] According to the test recognition success rate, a verification result of the voiceprint model recognition effect is obtained.
[0044] The above-mentioned method, device, computer equipment and storage medium for verifying the recognition effect of the voiceprint model verify that the population distribution of the test users in the data set is related to the business scenario, call the voiceprint model, and perform at least two voiceprint verifications on each test user according to the registration audio and verification audio of each test user in the verification data set to obtain the test recognition success rate. On the one hand, the data source used to verify the voiceprint model can reflect the population distribution of the business scenario and can truly reflect the usage of the actual business scenario. On the other hand, at least two login verifications are performed during the login test, which meets the user's usage intention. Therefore, the verification result of the voiceprint model recognition effect obtained by this method can truly reflect the usage effect of the actual business scenario, eliminate the model effect deviation caused by non-model factors, and improve the verification accuracy. BRIEF DESCRIPTION OF THE DRAWINGS
[0045] Figure 1 This is an application scenario diagram of a method for verifying the recognition effect of a voiceprint model in an embodiment;
[0046] Figure 2 A flowchart of a method for verifying the recognition effect of a voiceprint model in one embodiment;
[0047] Figure 3 A flowchart of a method for verifying the recognition effect of a voiceprint model in another embodiment;
[0048] Figure 4 A schematic diagram of an analysis of audio verification in one embodiment;
[0049] Figure 5 A flowchart of steps for obtaining a service identification success rate in one embodiment;
[0050] Figure 6 A structural block diagram of a device for verifying the recognition effect of a voiceprint model in one embodiment;
[0051] Figure 7 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION
[0052] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.
[0053] The method, device, computer equipment and storage medium for verifying the recognition effect of the voiceprint model provided in this application can be applied to Figure 1 In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The application of the terminal 102 supports voiceprint verification. The terminal collects the user's voice signal, sends the sound signal to the server, and the server calls the voiceprint model for identification and authentication. The server also verifies the recognition effect of the voiceprint model. Specifically, the server obtains a verification data set, the population distribution of the test users in the verification data set is related to the business scenario, the verification data set includes the registration audio and verification audio of the test users, and each test user has at least two sets of verification audio; calls the voiceprint model, and performs at least two voiceprint verifications on each test user according to the registration audio and verification audio of each test user in the verification data set to obtain a test recognition success rate; according to the test recognition success rate, obtains the verification result of the recognition effect of the voiceprint model.
[0054] The terminal 102 may be, but is not limited to, various personal computers, laptop computers, smart phones, tablet computers, and portable wearable devices, and the server 104 may be implemented as an independent server or a server cluster consisting of multiple servers.
[0055] In one embodiment, Figure 2 As shown, a method for verifying the recognition effect of the voiceprint model is provided, and the method is applied to Figure 1The server in the example is used to illustrate the following steps:
[0056] Step 202, obtaining a verification data set, wherein the population distribution of the test users in the verification data set is related to the business scenario, and the verification data set includes the registration audio and verification audio of the test users, and each test user has at least two sets of verification audio.
[0057] Oftentimes, voiceprint recognition will be affected by the combined influence of external factors and personal factors, resulting in deviations in the model effect. This part of the influence needs to be excluded as much as possible during the evaluation. Therefore, for the selection of the test population, it is necessary to introduce scenario classification to improve the test data source for the evaluation. In this application, the population distribution of test users in the verification data set is related to the business scenario. Specifically, the population distribution of test users can be determined based on the business scenario of the application to which the voiceprint model is applied, so that the test users in the verification data set used for testing conform to the population distribution of the actual business scenario, thereby closely realizing the use effect of the voiceprint model in the business scenario. For example, for a learning application, the user group is mainly students, and the main population distribution is in the youth group. The proportion of youth groups is increased in the test users. For a financial management application, the user group is mainly young and middle-aged, and the proportion of young and middle-aged people is increased in the test users.
[0058] The verification data set includes the registration audio and verification audio of each test user. There can be at least three groups of registration audio and at least two groups of verification audio. The registration audio simulates the audio data when the user registers and is a reference for verification. The verification audio simulates the audio data submitted by the user in the user scenario of using voiceprint recognition, such as the audio data submitted during login verification. The audio content of the registration audio and verification audio can be determined according to the actual business.
[0059] Specifically, the step of obtaining a verification data set includes: obtaining the population distribution of the business scenario; determining the number of test users of each gender and age group according to the population distribution to obtain target test users; collecting at least three groups of registration audios and at least two groups of verification audios of each target test user to obtain a verification data set.
[0060] Specifically, the original audio data required for voiceprint verification is obtained in a targeted manner according to the division of gender and age groups. It is divided into male and female according to gender, and into childhood, adolescence, adulthood, and the elderly according to age groups. Finally, audio files of various gender and age combinations are combined to obtain a verification data set.
[0061] In an application scenario, the main user group of a financial management application is young adults aged 25-54. As shown in Table 1, among the test users of the validation dataset, the number of test users in the age group of 25-54 accounts for the largest proportion.
[0062] Table 1 Distribution of test users of the validation dataset
[0063]
[0064]
[0065] The scenario of the application reflects the distribution of the target audience of the application. The test population of this application is determined based on the scenario of the application, so that the verification data set used for testing can reflect the distribution of the population in the business scenario, thereby improving the reliability of the verification effect.
[0066] Step 204, calling the voiceprint model, and performing at least two voiceprint verifications on each test user according to the registration audio and verification audio of each test user in the verification data set, to obtain a test recognition success rate.
[0067] Specifically, the trained voiceprint model is called to perform two login verifications on each test user. The test recognition success rate is the ratio of successful user recognition in the verification data set obtained by verifying the voiceprint model with the verification data set.
[0068] Among them, the calling of the voiceprint model, performing at least two voiceprint verifications on each test user according to the registration audio and verification audio of each test user in the verification data set, and obtaining a test recognition success rate, includes: calling the voiceprint model, performing at least two voiceprint verifications on each test user according to the registration audio and verification audio of each test user in the verification data set, and obtaining a primary recognition success rate, a secondary recognition success rate, and a recognition success rate; performing weighted summation of the primary recognition success rate, the secondary recognition success rate, and the recognition success rate to obtain a test recognition success rate; wherein the weight of the primary recognition success rate is greater than the weight of the secondary recognition success rate and the weight of the recognition success rate.
[0069] Assume that each test user has N sets of verification audio, where N ≥ 2. Then the first success rate is the number of users who succeeded in all N voiceprint recognitions / the number of test users, the second recognition success rate is the number of users who failed in only one voiceprint recognition / the number of test users, and the login success rate is the number of users who succeeded in one or more voiceprint recognitions / the number of test users.
[0070] Each test user in the verification data set undergoes at least two voiceprint recognition verifications. Based on the number of users who successfully perform N voiceprint recognitions, the number of users who fail only once in N voiceprint recognitions, and the number of users who succeed once or more in N voiceprint recognitions, the first recognition success rate, second recognition success rate, and recognition success rate are obtained respectively.
[0071] The first recognition success rate, second recognition success rate and login success rate have different weights, among which the first recognition success rate has the highest weight, the second recognition success rate has the second highest weight, and the recognition success rate has the lowest weight. The sum of the three is 1.
[0072] Test recognition success rate = A*first recognition success rate + B*second recognition success rate + C*login success rate
[0073] Step 206: obtaining a verification result of the voiceprint model recognition effect according to the test recognition success rate.
[0074] In the actual user use process, after a voiceprint recognition fails, the user will usually try a second voiceprint verification, but multiple voiceprint failures will affect the user experience, and the user's willingness to use the voiceprint verification product will be greatly reduced. In the embodiment of the present application, by performing at least two voiceprint verifications on the test user, the recognition effect is comprehensively evaluated by the first recognition success rate and the second recognition success rate, which actually reflects the user's usage habits and reflects whether the user is willing to use the voiceprint product. Therefore, this indicator verification method is a more business-oriented indicator and has a strong guiding significance.
[0075] The above-mentioned method for verifying the recognition effect of the voiceprint model verifies that the population distribution of the test users in the data set is related to the business scenario, calls the voiceprint model, and performs at least two voiceprint verifications on each test user based on the registration audio and verification audio of each test user in the verification data set to obtain the test recognition success rate. On the one hand, the data source used to verify the voiceprint model can reflect the population distribution of the business scenario and can truly reflect the usage of the actual business scenario. On the other hand, at least two login verifications are performed during the login test, which meets the user's usage intention. Therefore, the verification result of the voiceprint model recognition effect obtained by this method can truly reflect the usage effect of the actual business scenario, eliminates the model effect deviation caused by non-model factors, and improves the verification accuracy. In another embodiment, Figure 3 As shown, the method also includes:
[0076] Step 302, obtaining a verification data set, wherein the population distribution of the test users in the verification data set is related to the business scenario, and the verification data set includes the registration audio and verification audio of the test users, and each test user has at least two sets of verification audio.
[0077] Step 304, calling the voiceprint model, and performing at least two voiceprint verifications on each test user according to the registration audio and verification audio of each test user in the verification data set to obtain a test recognition success rate.
[0078] Step 306 , according to the actual business scenario, verify the number of users who are successfully identified among users with normal audio, and obtain the business recognition success rate under normal audio conditions.
[0079] Normal verification audio means that in actual business scenarios, there is no abnormality in the verification audio submitted for voiceprint recognition verification. Specifically, the verification audio can be analyzed to evaluate whether it is abnormal. Abnormal situations include poor audio quality, invalid data, login and note that the person is not the same, etc.
[0080] In actual business scenarios, voiceprint verification failures may be caused by the sound itself, such as poor audio quality, login and registration are not the same, etc. These are verification failures caused by sound problems, not voiceprint model effect problems. Figure 4 As shown in the figure, the verification audio data analysis in an actual business scenario shows that only 54% of the verification audio is normal, and other abnormal sound quality problems will also lead to login failures. Only the successful login of users with normal login voices can truly reflect the recognition effect of the voiceprint model. Therefore, when evaluating the voiceprint model in an actual business scenario, users with voice quality problems should be excluded, and only the data of users with normal verification audio should be considered.
[0081] Specifically, Figure 5 As shown, the steps of obtaining the business identification success rate include:
[0082] Step 502, obtaining verification audio data of users in actual business, analyzing the verification audio data, and obtaining users whose verification audio is normal.
[0083] Specifically, the signal-to-noise ratio of the audio data is detected and verified to obtain high-quality sound data; the high-quality sound signal is subjected to frequency analysis and speech recognition to obtain a valid sound signal; the sound signals of people who are not the same person logging in and registering are eliminated from the valid sound signals to obtain users with normal verified audio.
[0084] Among them, the signal-to-noise ratio refers to the ratio of the signal to the noise in an electronic device or electronic system. The signal here refers to the electronic signal from outside the device that needs to be processed by this device, and the noise refers to the irregular additional signal (or information) that does not exist in the original signal after passing through the device, and this signal does not change with the change of the original signal. The larger the signal-to-noise ratio, the smaller the noise mixed in the signal, and the higher the sound quality of the sound playback, otherwise the opposite. Through signal-to-noise ratio analysis, sound signals with large noise can be eliminated.
[0085] Spectral analysis is a technique that decomposes complex signals into simpler signals. Finding out the information of a signal at different frequencies (such as amplitude, power, intensity or phase, etc.) is called spectrum analysis. Spectrum refers to the representation of a time domain signal in the frequency domain, which can be obtained by Fourier transforming the signal. The spectrum can indicate which frequencies of sine waves a signal is composed of, and can also show the size and phase of each frequency sine wave. By analyzing the frequency spectrum of the sound, the audio blank and the sound signal with small sound can be eliminated in the audio data for verification.
[0086] During login verification, the input content is usually certain, and the verification audio data can be identified to eliminate audio signals with invalid audio content or dialect.
[0087] Among the valid sound signals, there may be some sound signals from people who are not the same person logging in and registering. These sound signals can be identified manually.
[0088] Step 504: Obtain the number of users who have been successfully identified among users whose audio is normal in actual business verification.
[0089] For example, 1,600 out of 1,700 users with normal verification audio logged in successfully.
[0090] Step 506 , calculating the service recognition success rate in the case of normal audio according to the number of users whose audio has been verified to be normal and the number of users whose audio has been successfully recognized among the users whose audio has been verified to be normal.
[0091] Specifically, the service recognition success rate is the number of users who have successfully identified the users with normal audio verification / the number of users with normal audio verification. Taking the above example, the service recognition success rate is 1600 / 1700. The number of users with normal audio verification excludes the audio data of poor audio quality, invalid audio, and audio data of different people logging in and registering in the actual login service, which can eliminate the evaluation of the voiceprint model recognition effect due to non-model reasons.
[0092] After step 306, the method further includes:
[0093] Step 308: obtaining a verification result of the voiceprint model recognition effect according to the test recognition success rate and the business recognition success rate.
[0094] For voiceprint models that are not online, only the test recognition success rate can be used as the evaluation criterion for the recognition effect of the voiceprint model.
[0095] For the voiceprint models that have been launched, the test recognition success rate and business recognition success rate can be combined as the evaluation criteria for the voiceprint model recognition effect.
[0096] Specifically, the weights of the test recognition success rate and the business recognition success rate are determined based on the online time of the application and the number of users who have verified that the audio is normal; the test recognition success rate and the business recognition success rate are weighted to obtain the score of the voiceprint model, and the verification result of the voiceprint model recognition effect is obtained based on the score.
[0097] Specifically, for voiceprint models that have been online for a short time, since there is less actual business data, the test data can be used as the main evaluation benchmark, and a higher weight can be set for the test recognition success rate. For voiceprint models that have been online for a long time, since there is more actual business data, the actual business data can be used as the main evaluation benchmark, and a higher weight can be set for the business recognition success rate. The test recognition success rate and the business recognition success rate are weighted to obtain the score of the voiceprint model. The higher the score, the better the recognition effect of the voiceprint model.
[0098] This method of verifying the recognition effect of the voiceprint model improves the test data source for evaluation according to scene classification during testing, eliminating the combined influence of external factors and personal factors that lead to model effect deviation. For the online model, the effect of the online voiceprint model is evaluated by analyzing the sound, eliminating abnormal sounds, and excluding factors that are not model problems.
[0099] It should be understood that although Figure 2-3 ,and Figure 5 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 2-3 ,and Figure 5 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.
[0100] In one embodiment, Figure 6 As shown, a device for verifying the recognition effect of a voiceprint model is provided, comprising:
[0101] The verification data acquisition module 602 is used to obtain a verification data set, in which the population distribution of the test users is related to the business scenario. The verification data set includes the registration audio and verification audio of the test users, and each test user has at least two sets of verification audio.
[0102] The recognition analysis module 604 is used to call the voiceprint model, and perform at least two voiceprint verifications on each test user according to the registration audio and verification audio of each test user in the verification data set to obtain a test recognition success rate.
[0103] The evaluation module 606 is used to obtain the verification result of the voiceprint model recognition effect according to the test recognition success rate.
[0104] The above-mentioned device for verifying the recognition effect of the voiceprint model verifies that the population distribution of the test users in the data set is related to the business scenario, calls the voiceprint model, and performs at least two voiceprint verifications on each test user according to the registration audio and verification audio of each test user in the verification data set to obtain the test recognition success rate. On the one hand, the data source used to verify the voiceprint model can reflect the population distribution of the business scenario and can truly reflect the usage of the actual business scenario. On the other hand, at least two login verifications are performed during the login test, which meets the user's usage intention. Therefore, the verification result of the voiceprint model recognition effect obtained by this method can truly reflect the usage effect of the actual business scenario, eliminate the model effect deviation caused by non-model factors, and improve the verification accuracy.
[0105] In another embodiment, the identification analysis module includes:
[0106] The recognition module is used to call the voiceprint model, and perform at least two voiceprint verifications on each test user according to the registration audio and verification audio of each test user in the verification data set to obtain the first recognition success rate, the second recognition success rate and the recognition success rate.
[0107] The analysis module is used to perform weighted summation on the first recognition success rate, the second recognition success rate and the recognition success rate to obtain a test recognition success rate; wherein the weight of the first recognition success rate is greater than the weight of the second recognition success rate and the weight of the recognition success rate.
[0108] In another embodiment, a verification data acquisition module is used to obtain the population distribution of a business scenario; determine the number of test users of each gender and age group based on the population distribution to obtain target test users; collect at least three groups of registration audios and at least two groups of verification audios from each target test user to obtain a verification data set.
[0109] In another embodiment, the apparatus further comprises:
[0110] The verification analysis module is used to verify the number of users who have been successfully identified among users with normal audio according to actual business scenarios, and obtain the business recognition success rate under normal audio conditions.
[0111] The evaluation module is used to obtain the verification result of the voiceprint model recognition effect according to the test recognition success rate and the business recognition success rate.
[0112] In another embodiment, the validation analysis includes:
[0113] The audio analysis module is used to obtain the verification audio data of users in actual business, analyze the verification audio data, and obtain users whose verification audio is normal.
[0114] The verification and identification module is used to obtain the number of users who have been successfully identified among users whose audio verification is normal in actual business;
[0115] The business analysis module is used to calculate the business recognition success rate under normal audio conditions based on the number of users whose audio has been verified to be normal and the number of users whose audio has been successfully recognized among the users whose audio has been verified to be normal.
[0116] In another embodiment, the audio analysis module is used to detect and verify the signal-to-noise ratio of audio data to obtain high-quality sound data; perform frequency analysis and voice recognition on the high-quality sound signal to obtain a valid sound signal; and eliminate the sound signals of people who are not the same person logging in and registering from the valid sound signals to obtain users with normal verified audio.
[0117] In another embodiment, the evaluation module comprises:
[0118] The weight determination module is used to determine the weights of the test recognition success rate and the business recognition success rate according to the online time of the application and the number of users who have verified that the audio is normal.
[0119] The weighted processing module is used to weight the test recognition success rate and the business recognition success rate to obtain the score of the voiceprint model.
[0120] An evaluation and analysis module is used to verify the recognition effect of the voiceprint model based on the score.
[0121] The specific definition of the device for verifying the recognition effect of the voiceprint model can be found in the definition of the method for verifying the recognition effect of the voiceprint model above, which will not be repeated here. The various modules in the above-mentioned device for verifying the recognition effect of the voiceprint model A can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.
[0122] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 7As shown. The computer device includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store audio data. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for verifying the recognition effect of a voiceprint model is implemented.
[0123] Those skilled in the art will understand that Figure 7 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.
[0124] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:
[0125] Obtain a verification data set, wherein the population distribution of the test users in the verification data set is related to the business scenario, the verification data set includes the registration audio and verification audio of the test users, and each test user has at least two sets of verification audio;
[0126] Calling the voiceprint model, and performing at least two voiceprint verifications on each test user according to the registration audio and verification audio of each test user in the verification data set, to obtain a test recognition success rate;
[0127] According to the test recognition success rate, a verification result of the voiceprint model recognition effect is obtained.
[0128] In one embodiment, the calling of the voiceprint model, performing at least two voiceprint verifications on each test user according to the registration audio and verification audio of each test user in the verification data set, and obtaining the test recognition success rate includes:
[0129] Calling the voiceprint model, performing at least two voiceprint verifications on each test user according to the registration audio and verification audio of each test user in the verification data set, and obtaining a first recognition success rate, a second recognition success rate, and a recognition success rate;
[0130] The first recognition success rate, the second recognition success rate and the recognition success rate are weighted and summed to obtain a test recognition success rate; wherein the weight of the first recognition success rate is greater than the weight of the second recognition success rate and the weight of the recognition success rate.
[0131] In one embodiment, obtaining a verification data set includes:
[0132] Obtain the population distribution of business scenarios;
[0133] Determine the number of test users of each gender and age group according to the population distribution, and obtain target test users;
[0134] At least three sets of registration audio and at least two sets of verification audio of each target test user are collected to obtain a verification data set.
[0135] In one embodiment, the processor implements the following steps when executing the computer program:
[0136] According to the actual business scenario, the number of users who have successfully been identified among those with normal audio is verified, and the business recognition success rate under normal audio conditions is obtained;
[0137] The obtaining a verification result of the voiceprint model recognition effect according to the test recognition success rate includes: obtaining a verification result of the voiceprint model recognition effect according to the test recognition success rate and the business recognition success rate.
[0138] In one embodiment, according to an actual service scenario, the number of users with normal audio being successfully identified is verified to obtain a service identification success rate under normal audio conditions, including:
[0139] Obtain verification audio data of users in actual business, analyze the verification audio data, and obtain users whose verification audio is normal;
[0140] Obtain the number of users who have been successfully identified among users whose audio is normal in actual business verification;
[0141] The service recognition success rate under normal audio conditions is calculated based on the number of users whose audio is verified to be normal and the number of users who are successfully identified among the users whose audio is verified to be normal.
[0142] In one embodiment, obtaining verification audio data of a user in an actual business, analyzing the verification audio data, and obtaining a user whose verification audio is normal includes:
[0143] Detect and verify the signal-to-noise ratio of audio data to obtain high-quality sound data;
[0144] Perform frequency analysis and speech recognition on high-quality sound signals to obtain effective sound signals;
[0145] The sound signals of people who log in and register differently from the valid sound signals are eliminated to obtain users whose audio is verified to be normal.
[0146] In one embodiment, the verification result of the voiceprint model recognition effect is obtained according to the test recognition success rate and the business recognition success rate, including:
[0147] Determine the weights of the test recognition success rate and the business recognition success rate according to the online time of the application and the number of users who have verified that the audio is normal;
[0148] Weighting the test recognition success rate and the business recognition success rate to obtain a score of the voiceprint model;
[0149] The score is used to verify the recognition effect of the voiceprint model.
[0150] In one embodiment, a computer storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:
[0151] Obtain a verification data set, wherein the population distribution of the test users in the verification data set is related to the business scenario, the verification data set includes the registration audio and verification audio of the test users, and each test user has at least two sets of verification audio;
[0152] Calling the voiceprint model, and performing at least two voiceprint verifications on each test user according to the registration audio and verification audio of each test user in the verification data set, to obtain a test recognition success rate;
[0153] According to the test recognition success rate, a verification result of the voiceprint model recognition effect is obtained.
[0154] In one embodiment, the calling of the voiceprint model, performing at least two voiceprint verifications on each test user according to the registration audio and verification audio of each test user in the verification data set, and obtaining the test recognition success rate includes:
[0155] Calling the voiceprint model, performing at least two voiceprint verifications on each test user according to the registration audio and verification audio of each test user in the verification data set, and obtaining a first recognition success rate, a second recognition success rate, and a recognition success rate;
[0156] The first recognition success rate, the second recognition success rate and the recognition success rate are weighted and summed to obtain a test recognition success rate; wherein the weight of the first recognition success rate is greater than the weight of the second recognition success rate and the weight of the recognition success rate.
[0157] In one embodiment, obtaining a verification data set includes:
[0158] Obtain the population distribution of business scenarios;
[0159] Determine the number of test users of each gender and age group according to the population distribution, and obtain target test users;
[0160] At least three sets of registration audio and at least two sets of verification audio of each target test user are collected to obtain a verification data set.
[0161] In one embodiment, the processor implements the following steps when executing the computer program:
[0162] According to the actual business scenario, the number of users who have successfully been identified among those with normal audio is verified, and the business recognition success rate under normal audio conditions is obtained;
[0163] The obtaining a verification result of the voiceprint model recognition effect according to the test recognition success rate includes: obtaining a verification result of the voiceprint model recognition effect according to the test recognition success rate and the business recognition success rate.
[0164] In one embodiment, according to an actual service scenario, the number of users with normal audio being successfully identified is verified to obtain a service identification success rate under normal audio conditions, including:
[0165] Obtain verification audio data of users in actual business, analyze the verification audio data, and obtain users whose verification audio is normal;
[0166] Obtain the number of users who have been successfully identified among users whose audio is normal in actual business verification;
[0167] The service recognition success rate under normal audio conditions is calculated based on the number of users whose audio is verified to be normal and the number of users who are successfully identified among the users whose audio is verified to be normal.
[0168] In one embodiment, obtaining verification audio data of a user in an actual business, analyzing the verification audio data, and obtaining a user whose verification audio is normal includes:
[0169] Detect and verify the signal-to-noise ratio of audio data to obtain high-quality sound data;
[0170] Perform frequency analysis and speech recognition on high-quality sound signals to obtain effective sound signals;
[0171] The sound signals of people who log in and register differently from the valid sound signals are eliminated to obtain users whose audio is verified to be normal.
[0172] In one embodiment, the verification result of the voiceprint model recognition effect is obtained according to the test recognition success rate and the business recognition success rate, including:
[0173] Determine the weights of the test recognition success rate and the business recognition success rate according to the online time of the application and the number of users who have verified that the audio is normal;
[0174] Weighting the test recognition success rate and the business recognition success rate to obtain a score of the voiceprint model;
[0175] The score is used to verify the recognition effect of the voiceprint model.
[0176] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
[0177] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0178] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.
Claims
1. A method for verifying the recognition effect of a voiceprint model, the method comprising: Obtain a verification data set, wherein the population distribution of the test users in the verification data set is related to the business scenario, the verification data set includes the registration audio and verification audio of the test users, and each test user has at least two sets of verification audio; Calling a voiceprint model, and performing at least two voiceprint verifications on each test user according to the registration audio and verification audio of each test user in the verification data set to obtain a test recognition success rate; the calling of the voiceprint model, and performing at least two voiceprint verifications on each test user according to the registration audio and verification audio of each test user in the verification data set to obtain a test recognition success rate, including: calling a voiceprint model, and performing at least two voiceprint verifications on each test user according to the registration audio and verification audio of each test user in the verification data set to obtain a first recognition success rate, a second recognition success rate, and a recognition success rate, wherein the first recognition success rate is the number of users whose N voiceprint recognitions are all successful, the second recognition success rate is the number of users whose N voiceprint recognitions fail only once, and the recognition success rate is the number of users whose N voiceprint recognitions have one or more successes; performing weighted summation of the first recognition success rate, the second recognition success rate, and the recognition success rate to obtain a test recognition success rate; wherein the weight of the first recognition success rate is greater than the weight of the second recognition success rate and the weight of the recognition success rate; According to the test recognition success rate, a verification result of the voiceprint model recognition effect is obtained.
2. The method according to claim 1, characterized in that The obtaining of the verification data set comprises: Obtain the population distribution of business scenarios; Determine the number of test users of each gender and age group according to the population distribution, and obtain target test users; Collect at least three sets of registration audio and at least two sets of verification audio from each target test user to obtain a verification data set.
3. The method according to claim 1, characterized in that: The method further comprises: According to the actual business scenario, the number of users who have successfully been identified among those with normal audio is verified, and the business recognition success rate under normal audio conditions is obtained; The obtaining a verification result of the voiceprint model recognition effect according to the test recognition success rate includes: obtaining a verification result of the voiceprint model recognition effect according to the test recognition success rate and the business recognition success rate.
4. The method according to claim 3, characterized in that According to the actual business scenario, the number of users who have successfully been identified among users with normal audio is verified, and the business recognition success rate under normal audio conditions is obtained, including: Obtain verification audio data of users in actual business, analyze the verification audio data, and obtain users whose verification audio is normal; Obtain the number of users who have been successfully identified among users whose audio is normal in actual business verification; The service recognition success rate under normal audio conditions is calculated based on the number of users whose audio is verified to be normal and the number of users who are successfully identified among the users whose audio is verified to be normal.
5. The method according to claim 4, characterized in that Obtain the verification audio data of users in actual business, analyze the verification audio data, and obtain users whose verification audio is normal, including: Detect and verify the signal-to-noise ratio of audio data to obtain high-quality sound data; Perform frequency analysis and speech recognition on high-quality sound signals to obtain effective sound signals; The sound signals of people who log in and register differently from the valid sound signals are eliminated to obtain users whose audio is verified to be normal.
6. The method according to claim 3, characterized in that According to the test recognition success rate and the business recognition success rate, a verification result of the voiceprint model recognition effect is obtained, including: Determine the weights of the test recognition success rate and the business recognition success rate according to the online time of the application and the number of users who have verified that the audio is normal; Weighting the test recognition success rate and the business recognition success rate to obtain a score of the voiceprint model; The score is used to verify the recognition effect of the voiceprint model.
7. A device for verifying the recognition effect of a voiceprint model, characterized in that: The device comprises: A verification data acquisition module is used to acquire a verification data set, wherein the population distribution of the test users in the verification data set is related to the business scenario, and the verification data set includes the registration audio and verification audio of the test users, and each test user has at least two sets of verification audio; The recognition analysis module is used to call the voiceprint model, and perform at least two voiceprint verifications on each test user according to the registration audio and verification audio of each test user in the verification data set to obtain a test recognition success rate; the calling of the voiceprint model, and performing at least two voiceprint verifications on each test user according to the registration audio and verification audio of each test user in the verification data set to obtain a test recognition success rate, including: calling the voiceprint model, and performing at least two voiceprint verifications on each test user according to the registration audio and verification audio of each test user in the verification data set to obtain a first recognition success rate, a second recognition success rate, and a recognition success rate, wherein the first recognition success rate is the number of users whose N voiceprint recognitions are all successful, the second recognition success rate is the number of users whose N voiceprint recognitions fail only once, and the recognition success rate is the number of users whose N voiceprint recognitions have one or more successes; performing weighted summation of the first recognition success rate, the second recognition success rate, and the recognition success rate to obtain a test recognition success rate; wherein the weight of the first recognition success rate is greater than the weight of the second recognition success rate and the weight of the recognition success rate; An evaluation module is used to obtain a verification result of the voiceprint model recognition effect according to the test recognition success rate.
8. The device according to claim 7, characterized in that The verification data acquisition module is used to obtain the population distribution of the business scenario; determine the number of test users of each gender and age group according to the population distribution to obtain target test users; collect at least three groups of registration audio and at least two groups of verification audio from each target test user to obtain a verification data set.
9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.
10. A computer storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.
