A voiceprint qualification assessment method, device, equipment and storage medium

Abstract

The present invention provides a voiceprint qualification assessment method, device, equipment and storage medium, wherein the voiceprint qualification assessment method includes: obtaining a sample set corresponding to an assessment level from sample sets corresponding to several levels, the sample sets corresponding to several levels are constructed based on several samples, the sample set corresponding to one level includes samples belonging to the level among several samples, the discrimination difficulty of samples belonging to different levels is different, each of the several samples includes sample data and sample data, and the assessment level is one of several levels; using samples in the sample set corresponding to the assessment level, testing whether the qualification of the voiceprint comparison engine to be assessed reaches the assessment level. The assessment process of the voiceprint qualification assessment method provided by the present invention is relatively simple, the assessment process is relatively standardized, the operability is strong, the credibility of the assessment result is high, and the assessment results of different voiceprint comparison engines can be directly compared.

Description

Technical Field

[0001] The present invention relates to the technical field of voiceprint qualification assessment, and in particular to a voiceprint qualification assessment method, device, equipment and storage medium. Background Art

[0002] Generally, fingerprints, irises, DNA and handwriting can be used as valid credentials to identify an individual. Compared with individual identification based on these traditional individual biometrics, individual identification based on voiceprint features is an emerging research field. In recent years, with the in-depth research and development of related technologies, individual identification technology based on voiceprint features has received more and more attention from relevant researchers. With the accumulation of data and research development in the field of voiceprint recognition, the accuracy of individual identification based on voiceprint features is getting higher and higher.

[0003] Individual identity recognition technology based on voiceprint features refers to the judgment of whether two voiceprint features are homologous (i.e., whether the two voiceprint features are the voiceprint features of the same individual) by comparing voiceprint features. For example, voiceprint features are extracted from a voice sample (i.e., sample data) obtained from a crime scene, and the voiceprint features are compared with the voiceprint features extracted from the suspect's voice sample (i.e., sample data) to determine whether the two voiceprint features are homologous. If the two voiceprint features are homologous, it means that the voice sample obtained from the crime scene is the suspect's voice sample.

[0004] At present, in the vast majority of domestic court speaker recognition cases, voice identification experts are expected to give a clear conclusion of "the same person" or "not the same person". However, due to the limitations of various subjective and objective conditions, such as the recording environment and conditions, voice evidence extraction, storage conditions and methods, the time interval between inspection and identification, and the limitations of inspection equipment and methods, there will be a certain degree of difference between the criminal sample and the suspect sample, which determines that it is almost impossible for the appraiser to determine the same or negate the exclusion to achieve 100% confirmation. This objective reality has led to the development of many voiceprint matching engines.

[0005] Nowadays, there are many voiceprint matching engines on the market. However, the matching quality of these voiceprint matching engines varies greatly. In order to know the matching quality of these voiceprint matching engines, there is an urgent need for a method to authenticate the qualifications of voiceprint matching engines. Summary of the invention

[0006] In view of this, the present invention provides a voiceprint qualification assessment method, device, equipment and storage medium for performing qualification assessment on a voiceprint comparison engine. The technical solution is as follows:

[0007] A voiceprint qualification assessment method, comprising:

[0008] From the sample sets corresponding to the several levels, a sample set corresponding to the assessment level is obtained; wherein the sample sets corresponding to the several levels are constructed based on the several samples, the sample set corresponding to a level includes the samples belonging to the level among the several samples, the difficulty of distinguishing samples belonging to different levels is different, each of the several samples includes sample data and sample data, and the assessment level is one of the several levels;

[0009] The samples in the sample set corresponding to the assessment level are used to test whether the qualification of the voiceprint comparison engine to be assessed reaches the assessment level.

[0010] Optionally, the process of constructing the sample sets corresponding to the several levels includes:

[0011] Obtain a number of samples;

[0012] Determine the discrimination difficulty representation values ​​corresponding to the plurality of samples respectively, wherein the discrimination difficulty representation values ​​can represent the discrimination difficulty of the corresponding samples;

[0013] Determining, from the plurality of levels, the levels to which the plurality of samples respectively belong according to the discrimination difficulty representation values ​​respectively corresponding to the plurality of samples;

[0014] The samples belonging to the same level are grouped into sample sets to obtain sample sets corresponding to the several levels respectively.

[0015] Optionally, determining the discrimination difficulty representation values ​​corresponding to the plurality of samples respectively includes:

[0016] For the target sample whose discrimination difficulty representation value is to be determined among the plurality of samples:

[0017] Based on a standard voiceprint extraction model, voiceprint features are extracted from the sample data and sample data included in the target sample to obtain a voiceprint feature pair corresponding to the target sample;

[0018] Calculating the similarity of the voiceprint feature pair corresponding to the target sample to obtain the voiceprint feature similarity corresponding to the target sample;

[0019] According to the similarity of the voiceprint features corresponding to the target sample, the discrimination difficulty representation value corresponding to the target sample is determined.

[0020] Optionally, determining the discrimination difficulty representation value corresponding to the target sample according to the similarity of the voiceprint features corresponding to the target sample includes:

[0021] Based on a preset linear mapping relationship, the voiceprint feature similarity corresponding to the target sample is mapped to a value that can characterize the discrimination difficulty of the target sample as the discrimination difficulty characterization value corresponding to the target sample.

[0022] Optionally, the plurality of samples include positive samples and negative samples, the sample data and sample data included in the positive samples are from the same subject, and the sample data and sample data included in the negative samples are from different subjects;

[0023] The mapping of the voiceprint feature similarity corresponding to the target sample to a value capable of representing the difficulty of distinguishing the target sample based on a preset linear mapping relationship includes:

[0024] If the target sample is a positive sample, based on a first linear mapping relationship preset for the positive sample, the voiceprint feature similarity of the target sample is mapped to a value that can characterize the difficulty of distinguishing the target sample;

[0025] If the target sample is a negative sample, based on a second linear mapping relationship preset for the negative sample, the voiceprint feature similarity of the target sample is mapped to a value that can characterize the difficulty of distinguishing the target sample.

[0026] Optionally, the several levels respectively correspond to a range of values ​​representing the difficulty of discrimination;

[0027] The step of determining the levels to which the samples belong respectively from the levels according to the discrimination difficulty representation values ​​respectively corresponding to the samples includes:

[0028] For the target sample whose level is to be determined among the several samples:

[0029] Determine, from the discrimination difficulty representation value ranges corresponding to the plurality of levels, a discrimination difficulty representation value range in which the discrimination difficulty representation value corresponding to the target sample lies, as a target discrimination difficulty representation value range;

[0030] The level corresponding to the target discrimination difficulty characterization value range is determined as the level to which the target sample belongs.

[0031] Optionally, the using of samples in the sample set corresponding to the assessment level to test whether the qualification of the voiceprint comparison engine to be assessed reaches the assessment level includes:

[0032] Obtaining a specified number of samples from the sample set corresponding to the assessment level as test samples to obtain a specified number of test samples;

[0033] Based on the voiceprint comparison engine, determine whether the sample data and the sample data contained in each test sample are from the same object, and obtain the determination results corresponding to the specified number of test samples respectively;

[0034] Determine the discrimination accuracy of the voiceprint comparison engine according to the discrimination results corresponding to the specified number of test samples and the sample labels corresponding to the specified number of test samples, wherein the sample labels are used to indicate whether the sample data and the sample data contained in the corresponding test samples are from the same object;

[0035] According to the discrimination accuracy of the voiceprint comparison engine, it is determined whether the qualification of the voiceprint comparison engine reaches the assessment level.

[0036] A voiceprint qualification assessment device, comprising: a sample set acquisition module and a voiceprint qualification assessment module;

[0037] The sample set acquisition module is used to acquire a sample set corresponding to an assessment level from sample sets corresponding to several levels; wherein the sample sets corresponding to the several levels are constructed based on several samples, and the sample set corresponding to a level includes samples belonging to the level among the several samples, and samples belonging to different levels have different difficulty levels in distinguishing, and each sample among the several samples includes sample data and sample data, and the assessment level is one of the several levels;

[0038] The voiceprint qualification assessment module is used to use samples in the sample set corresponding to the assessment level to test whether the qualification of the voiceprint comparison engine to be assessed reaches the assessment level.

[0039] A voiceprint qualification assessment device, comprising: a memory and a processor;

[0040] The memory is used to store programs;

[0041] The processor is used to execute the program to implement each step of the voiceprint qualification assessment method described in any one of the above items.

[0042] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements each step of the voiceprint qualification assessment method described in any one of the above items.

[0043] The voiceprint qualification assessment method, device, equipment and storage medium provided by the present invention first obtains a sample set corresponding to the assessment level from the sample sets corresponding to the several levels, and then uses the samples in the sample set corresponding to the assessment level to test whether the qualification of the voiceprint comparison engine to be assessed reaches the assessment level. The voiceprint qualification assessment method provided by the embodiment of the present invention can determine the qualification of the voiceprint comparison engine, and the determined qualification is the specific level reached, not simply whether it meets the standard. The voiceprint qualification assessment method provided by the present invention is a unified method for assessing different voiceprint comparison engines. The assessment results given by the voiceprint qualification assessment method for different voiceprint comparison engines can be directly compared; the voiceprint qualification assessment method provided by the present invention uses the same batch of samples (i.e., sample sets corresponding to several levels) to assess different voiceprint comparison engines, avoiding the disturbance caused by different samples to the assessment, so that the assessment result is fair and reliable; the voiceprint qualification assessment method provided by the present invention divides the samples into levels according to the difficulty of discrimination, and uses the sample set corresponding to the assessment level to test whether the qualification of the voiceprint comparison engine reaches the assessment level. This assessment method throws away the constraints of commonly used evaluation indicators and is more operable. In summary, the assessment process of the voiceprint qualification assessment method provided by the present invention is relatively simple, the assessment process is relatively standardized, the operability is strong, the credibility of the assessment results is high, and the assessment results of different voiceprint comparison engines can be directly compared. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0045] Figure 1 A flow chart of a voiceprint qualification assessment method provided by an embodiment of the present invention;

[0046] Figure 2 A schematic diagram of a process of constructing sample sets corresponding to a plurality of levels based on a plurality of samples provided in an embodiment of the present invention;

[0047] Figure 3 A schematic diagram of using samples in a sample set corresponding to an assessment level provided in an embodiment of the present invention to test whether the qualification of a voiceprint comparison engine to be assessed meets the assessment level;

[0048] Figure 4 A schematic diagram of the structure of a voiceprint qualification assessment device provided in an embodiment of the present invention;

[0049] Figure 5A schematic diagram of the structure of a voiceprint qualification assessment device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0050] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0051] In order to be able to conduct qualification appraisal of the voiceprint comparison engine, the inventor of this case conducted research. The initial idea was to use a voiceprint qualification assessment method based on evaluation indicators to conduct qualification appraisal of the voiceprint comparison engine. The general idea of ​​this method is:

[0052] First, several test samples (each test sample includes sample data and sample data) are input into the voiceprint comparison engine to be assessed to obtain the discrimination results corresponding to the several test samples, and then one or more of the following indicators are determined according to the discrimination results corresponding to the several test samples: accuracy rate, false alarm rate, missed alarm rate, equal error rate, recall rate, precision, etc. Finally, whether the voiceprint comparison engine meets the standards is determined according to the determined indicators.

[0053] Among them, accuracy refers to the ratio of positive and negative samples that are correctly classified, and its calculation method is as follows:

[0054]

[0055] N in the formula TP Indicates the number of positive samples (including the sample data and the sample data from the same object) that are correctly classified, N TN Indicates the number of negative samples (including sample data and sample data from different objects) that are correctly classified, N FP Indicates the number of negative samples identified as positive samples, N FN Indicates the number of positive samples that are identified as negative samples. The higher the accuracy, the better the performance of the voiceprint matching engine.

[0056] The false alarm rate is also called the false acceptance rate. In the voiceprint comparison problem, if the sample data and the sample data contained in a sample come from different objects, but are judged by the voiceprint comparison engine to be from the same object, then it is a false acceptance case. The false acceptance rate refers to the proportion of false acceptance cases in all heterogeneous matching cases, and its calculation method is as follows:

[0057]

[0058] The false negative rate is also called the false rejection rate. In the voiceprint comparison problem, if the sample data and the sample data contained in a sample come from the same object, but are judged by the voiceprint comparison engine as coming from different objects, it is a false rejection case. The false rejection rate refers to the proportion of false rejection cases in all similar matching cases, and its calculation method is as follows:

[0059]

[0060] It can be seen that the smaller the false alarm rate and missed alarm rate, the better the performance of the voiceprint matching engine.

[0061] The equal error rate is a relative balance point between the false alarm rate and the missed alarm rate. In an accept / reject binary classification system, a threshold is usually set, and an acceptance decision is made only when the score exceeds this value. By adjusting the threshold, the false alarm rate and the missed alarm rate can be balanced according to business needs. When a high threshold is set, the score requirement for the system to make an acceptance decision is stricter, the false alarm rate is reduced, and the missed alarm rate is increased; when a low threshold is set, the score requirement for the system to make an acceptance decision is looser, the false alarm rate is increased, and the missed alarm rate is reduced. In different application scenarios, adjusting different thresholds can balance security and ease of use. Adjust the threshold so that the false alarm rate is equal to the missed alarm rate. The values ​​of the false alarm rate and the missed alarm rate at this time are called equal error rates.

[0062] The inventors of this case studied the above-mentioned voiceprint qualification assessment method based on evaluation indicators and found that: the evaluation indicators used by different voiceprint comparison engines are usually different. For example, the indicator used by voiceprint comparison engine A is the equal error rate, and the indicator used by voiceprint comparison engine B is the false alarm rate. The assessment results of different voiceprint comparison engines cannot be directly compared, and have no reference value for actual comparison; different comparison engines use different test samples, which leads to that even if different comparison engines use the same evaluation indicators, the final assessment results cannot be directly compared; the voiceprint qualification assessment method based on evaluation indicators can only give a conclusion on whether the standards are met, but there is no reliable conclusion on the specific performance level of the voiceprint recognition engine to be assessed.

[0063] In view of the many defects of the voiceprint qualification assessment method based on evaluation indicators, the inventor of this case continued to conduct research, and through continuous research, finally proposed a voiceprint qualification assessment method with better effect, which perfectly overcomes the defects of the voiceprint qualification assessment method based on evaluation indicators.

[0064] The voiceprint qualification assessment method provided by the present invention can be applied to electronic devices with processing capabilities, which can be servers on the network side or terminals used by users, wherein the server can be a single server or a server cluster composed of multiple servers, or a cloud computing server center, and the server can include a processor, a memory, and a network interface, etc. The terminal can be, but is not limited to, a PC, a notebook, a smart phone, etc. The server on the network side or the terminal used by the user side can perform voiceprint qualification assessment on the voiceprint comparison engine to be assessed according to the voiceprint qualification assessment method provided by the present invention.

[0065] Those skilled in the art should understand that the above-mentioned terminals and servers are only examples, and other existing or future terminals or servers, if applicable to the present invention, should also be included in the protection scope of the present invention and are included here by reference.

[0066] Next, the voiceprint qualification assessment method provided by the present invention is introduced through the following embodiments.

[0067] First embodiment

[0068] See also Figure 1 , which shows a flow chart of a voiceprint qualification assessment method provided by an embodiment of the present invention, which may include:

[0069] Step S101: Obtain a sample set corresponding to an assessment level from sample sets corresponding to several levels.

[0070] Among them, the sample sets corresponding to several levels are constructed based on several samples. The sample set corresponding to a level includes samples belonging to this level among several samples. The difficulty of distinguishing samples belonging to different levels is different. Each sample among the several samples includes sample data and sample data, and the assessment level is one of the several levels.

[0071] Exemplarily, the several levels include level 1, level 2, level 3, level 4, and level 5, wherein level 1 is the lowest level and level 5 is the highest level. The sample sets corresponding to level 1, level 2, level 3, level 4, and level 5 are constructed based on several samples. The sample set corresponding to level 1 includes samples belonging to level 1 among several samples, the sample set corresponding to level 2 includes samples belonging to level 2 among several samples, the sample set corresponding to level 3 includes samples belonging to level 3 among several samples, the sample set corresponding to level 4 includes samples belonging to level 4 among several samples, and the sample set corresponding to level 5 includes samples belonging to level 6 among several samples. The sample set corresponding to level five includes several samples belonging to level five. The difficulty of distinguishing samples belonging to different levels is different. The higher the level, the greater the difficulty of distinguishing samples, and the lower the level, the smaller the difficulty of distinguishing samples. For example, the difficulty of distinguishing samples belonging to level five is greater than that of samples belonging to level four, the difficulty of distinguishing samples belonging to level four is greater than that of samples belonging to level three, the difficulty of distinguishing samples belonging to level three is greater than that of samples belonging to level two, and the difficulty of distinguishing samples belonging to level two is greater than that of samples belonging to level one.

[0072] Step S102: using samples in the sample set corresponding to the assessment level to test whether the qualification of the voiceprint comparison engine to be assessed meets the assessment level.

[0073] Exemplarily, the several levels include level one, level two, level three, level four, and level five. The voiceprint comparison engine to be assessed pre-applies for level five qualification, that is, the assessment level is level five. Then, samples in the sample set corresponding to level five are used to test whether the voiceprint comparison engine's qualifications reach level five.

[0074] When using samples in the sample set corresponding to the assessment level to test whether the qualification of the voiceprint comparison engine reaches the assessment level, some samples can be obtained from the sample set corresponding to the assessment level as test samples, and the obtained test samples are used to test whether the qualification of the voiceprint comparison engine reaches the assessment level. Of course, this embodiment is not limited to this. For example, all samples in the sample set corresponding to the assessment level can also be used to test whether the qualification of the voiceprint comparison engine reaches the assessment level.

[0075] It should be noted that, for different voiceprint comparison engines, the present invention uses the same batch of samples (i.e., sample sets corresponding to several levels) for assessment. For example, for voiceprint comparison engine A, the sample set corresponding to its assessment level a is obtained from the sample sets corresponding to several levels, and the samples in the sample set corresponding to assessment level a are used to test whether the qualification of voiceprint comparison engine A reaches the assessment level a. For voiceprint comparison engine B, the sample set corresponding to its assessment level b is obtained from the sample sets corresponding to several levels, and the samples in the sample set corresponding to assessment level b are used to test whether the qualification of voiceprint comparison engine B reaches the assessment level b.

[0076] The voiceprint qualification assessment method provided in the embodiment of the present invention first obtains a sample set corresponding to an assessment level from sample sets corresponding to several levels, and then uses samples in the sample set corresponding to the assessment level to test whether the qualification of the voiceprint comparison engine to be assessed reaches the assessment level. The voiceprint qualification assessment method provided in the embodiment of the present invention can determine the qualification of the voiceprint comparison engine, and the determined qualification is not simply whether it meets the standard, but can give the specific level reached.

[0077] The voiceprint qualification assessment method provided by the embodiment of the present invention has the following advantages: first, the voiceprint qualification assessment method provided by the embodiment of the present invention is a unified method for assessing different voiceprint comparison engines, and the assessment results given by the voiceprint qualification assessment method for different voiceprint comparison engines can be directly compared; second, the voiceprint qualification assessment method provided by the embodiment of the present invention uses the same batch of samples (i.e., sample sets corresponding to several levels) to assess different voiceprint comparison engines, avoiding the disturbance caused by different samples to the assessment, so that the assessment results are fair and reliable; third, the voiceprint qualification assessment method provided by the embodiment of the present invention divides the samples into levels according to the difficulty of discrimination, and uses the samples in the sample set corresponding to the assessment level to test whether the qualification of the voiceprint comparison engine reaches the assessment level. This assessment method throws away the constraints of evaluation indicators and is more operable. In summary, the assessment process of the voiceprint qualification assessment method provided by the embodiment of the present invention is relatively simple, the assessment process is relatively standardized, the operability is strong, the credibility of the assessment results is high, and the assessment results of different voiceprint comparison engines can be directly compared.

[0078] Second embodiment

[0079] The above embodiment mentioned that the sample sets corresponding to the several levels are constructed based on the several samples. This embodiment introduces the specific implementation process of constructing the sample sets corresponding to the several levels based on the several samples.

[0080] See also Figure 2 , shows a schematic diagram of a process of constructing sample sets corresponding to several levels based on several samples, which may include:

[0081] Step S201: Obtain several samples.

[0082] Each sample includes sample data and sample data.

[0083] It should be noted that the sample data contained in the plurality of samples are voice data of relatively good quality and whose identity of the subject is clearly known. The voice data of relatively good quality may be voice data that meets conditions such as a high signal-to-noise ratio, a certain length of effective language duration, and balanced frequency response of a recording channel.

[0084] Unlike sample data, the voice quality of each inspection data contained in several samples may be uneven. This is because, in actual applications, the voice quality of the inspection data cannot be guaranteed due to factors such as the recording environment and conditions, the extraction, preservation conditions and methods of voice evidence, etc.

[0085] Preferably, the plurality of samples obtained in step S201 may include positive samples and negative samples, wherein the specimen data and sample data included in the positive samples are from the same subject, and the specimen data and sample data included in the negative samples are from different subjects.

[0086] Specifically, the process of obtaining a number of samples may include: obtaining sample data corresponding to M objects, and specimen data corresponding to N objects; combining the sample data corresponding to the M objects and the specimen data corresponding to the N objects in pairs to obtain M*N data pairs, each data pair being a sample, that is, obtaining M*N samples.

[0087] Optionally, the M objects and the N objects may include the same object and different objects, so that the samples finally obtained include both positive samples and negative samples. It should be noted that the object identity of each sample data and each inspection material data obtained is clearly known. After obtaining M*N samples, the sample labels corresponding to the M*N samples can be obtained. For example, the sample label of the positive sample is "1" and the sample label of the negative sample is "0".

[0088] Exemplarily, sample data a of object A, sample data b of object B, and sample data c of object C are obtained, and sample data b′ of object B, sample data d of object D, and sample data c′ of object C are obtained. Then, the above 3 sample data and 3 sample data are combined in pairs to obtain 9 samples (a, b′), (a, d), (a, c′), (b, b′), (b, d), (b, c′), (c, b′), (c, d), (c, c′), among which (a, b′), (a, d), (a, c′), (b, d), (b, c′), (c, b′), (c, d) are negative samples, and the corresponding sample labels are all "0", and (b, b′), (c, c′) are positive samples, and the corresponding sample labels are all "1".

[0089] Step S202: Determine the discrimination difficulty representation values ​​corresponding to a plurality of sample pairs.

[0090] Among them, the discrimination difficulty representation value can represent the discrimination difficulty of the corresponding sample.

[0091] The process of determining the discrimination difficulty representation values ​​corresponding to a plurality of sample pairs may include:

[0092] For each sample x in a number of samples i(i=1,2,…,M*N,M*N is the total number of samples), execute:

[0093] Step a1: Based on the standard voiceprint extraction model, sample x i The included sample data and sample data are used to extract voiceprint features and obtain sample x i The corresponding voiceprint feature pair.

[0094] In this embodiment, the “standard voiceprint extraction model” refers to a voiceprint extraction model with better performance.

[0095] Step a2: Calculate sample x i The similarity of the corresponding voiceprint feature pair is obtained to obtain the sample x i The corresponding voiceprint feature similarity.

[0096] Calculate sample x i The similarity of the corresponding voiceprint feature pair, that is, calculating the sample x i The similarity between the two voiceprint features contained in the corresponding voiceprint feature pair.

[0097] There are many ways to calculate the similarity of two voiceprint features. For example, the cosine distance between the two voiceprint features can be calculated, and the calculated cosine distance is used as the similarity of the two voiceprint features. The calculation formula of the cosine distance cosθ is as follows:

[0098]

[0099] This embodiment is not limited to using cosine distance to measure the similarity of two voiceprint features, and other distances may be used, such as Euclidean distance, Minkowski distance, Mahalanobis distance, etc. For ease of description, the cosine distance is used as an example for the following description.

[0100] Step a3: According to sample x i The corresponding voiceprint feature similarity determines the sample x i The corresponding discrimination difficulty representation value.

[0101] Specifically, based on the preset linear mapping relationship, the sample x i The corresponding voiceprint feature similarity is mapped to be able to characterize sample x i The value of the difficulty of discrimination, as the sample x i The corresponding discrimination difficulty representation value.

[0102] More specifically, if the sample x i is a positive sample, then based on the first linear mapping relationship preset for the positive sample, the sample x i The voiceprint feature similarity is mapped to be able to characterize the sample x i The value of the difficulty of discrimination, as the sample x iThe corresponding discrimination difficulty representation value; if the sample x i is a negative sample, then based on the second linear mapping relationship preset for negative samples, the sample x i The voiceprint feature similarity is mapped to be able to characterize the sample x i The value of the difficulty of discrimination, as the sample x i The corresponding discrimination difficulty representation value.

[0103] Exemplarily, the similarity of two voiceprint features is measured by the cosine distance of the two voiceprint features, and the first linear mapping relationship preset for the positive sample and the second linear mapping relationship preset for the negative sample can be respectively:

[0104]

[0105]

[0106] Among them, CS represents the similarity of the voiceprint features corresponding to the sample, and SCORE represents the discrimination difficulty representation value corresponding to the sample.

[0107] Optionally, for positive samples, several sets of relationships in Table 1 can be preset, and the first linear mapping relationship can be determined based on the several sets of relationships in Table 1. For negative samples, several sets of relationships in Table 2 can be preset, and the second linear mapping relationship can be determined based on the several sets of relationships in Table 2.

[0108] Table 1 Relationship between feature similarity and discrimination difficulty representation value for positive samples

[0109] Feature similarity (cosine distance) Difficulty characterization value (score) -1 0 -0.5 25 0 50 0.5 75 1 100

[0110] Table 2 Relationship between feature similarity and discrimination difficulty representation value for negative samples

[0111] Feature similarity (cosine distance) Difficulty characterization value (score) -1 100 -0.5 75 0 50 0.5 25 1 0

[0112] For positive samples, if the feature similarity (cosine distance) is 1, the score is 100, and if the feature similarity (cosine distance) is -1, the score is 0. When the feature similarity (cosine distance) is between -1 and 1, the score is linearly distributed. It should be noted that for positive samples, the greater the corresponding feature similarity, the greater the score, and the lower the difficulty of discrimination. Conversely, the smaller the corresponding feature similarity, the smaller the score, and the greater the difficulty of discrimination.

[0113] For negative samples, if the feature similarity (cosine distance) is -1, the score is 100, and if the feature similarity (cosine distance) is 1, the score is 0. It should be noted that for negative samples, the greater the corresponding feature similarity, the smaller the score, and the greater the difficulty of discrimination. Conversely, the smaller the corresponding feature similarity, the larger the score, and the smaller the difficulty of discrimination.

[0114] Regardless of whether it is a positive sample or a negative sample, the smaller the score, the greater the difficulty of discrimination, and the larger the score, the smaller the difficulty of discrimination.

[0115] Step S203: According to the discrimination difficulty representation values ​​corresponding to the samples, the levels to which the samples belong are determined from the set levels to obtain sample sets corresponding to the levels.

[0116] Optionally, several levels correspond to respective ranges of values ​​representing the difficulty of discrimination.

[0117] The process of determining the levels to which the samples belong from the set levels according to the discrimination difficulty representation values ​​corresponding to the samples includes:

[0118] For each sample x in a number of samples i (i=1,2,…,M*N,M*N is the total number of samples), execute:

[0119] Step b1: Determine the sample x from the range of discrimination difficulty representation values ​​corresponding to several levels. i The discrimination difficulty representation value range in which the corresponding discrimination difficulty representation value lies is used as the target discrimination difficulty representation value range.

[0120] Exemplarily, the several levels include level 1, level 2, level 3, level 4, and level 5, wherein level 1 is the lowest level and level 5 is the highest level. The ranges of discrimination difficulty representation values ​​(such as score ranges) corresponding to the five levels are shown in the following table:

[0121] Table 3 The range of discrimination difficulty representation values ​​corresponding to several levels

[0122] grade Difficulty characterization value range (score range) Level 5 0≤score<20 Level 4 20≤score<40 Level 3 40≤score<60 Level 2 60≤score<80 Level 1 80≤score≤100

[0123] Assume that the sample x i The corresponding discrimination difficulty representation value (such as score) is 20. From the above table, we can see that 20 is within the discrimination difficulty representation value range (score range) of 0≤score<20. Therefore, 0≤score<20 is determined as the target discrimination difficulty representation value range. Assume that sample x i The corresponding discrimination difficulty representation value (such as score) is 50. From the above table, it can be seen that 50 is within the discrimination difficulty representation value range (score range) of 40≤score<60. Therefore, 40≤score<60 is determined as the target discrimination difficulty representation value range.

[0124] Step b2: Determine the level corresponding to the target discrimination difficulty representation value range as sample x i The level to which it belongs.

[0125] For example, if the target discrimination difficulty representation value range is 0≤score<20 in the above table, then determine the sample x i The level is level 5. If the target discrimination difficulty representation value range is 20≤score<40 in the above table, then determine the sample x i The level is level 4. If the target discrimination difficulty representation value range is 40≤score<60 in the above table, then determine the sample x i The level is level 3. If the target discrimination difficulty representation value range is 60≤score<80 in the above table, then determine the sample x i The level is level 2. If the target discrimination difficulty representation value range is 80≤score≤100 in the above table, then determine the sample x i The level it belongs to is level one.

[0126] Through the above process, the levels to which the samples belong respectively can be obtained.

[0127] Step S204: samples belonging to the same level are grouped into sample sets to obtain sample sets corresponding to several levels.

[0128] For example, the levels include level 1, level 2, level 3, level 4, and level 5, and the samples include x1, x2, x3, x4, x5, x6, x7, x8, x9, x10, x11, x12, x13, x14, x15, x16, x17, x18, x19, x20, x21, x22, x23, x24, x25, x26, x 10 、x 11 、x 12 、x 13 、x 14 、x 15 , assuming that x1 is determined to be level 1, x2 is determined to be level 2, x3 is determined to be level 1, x4 is determined to be level 3, x5 is determined to be level 5, x6 is determined to be level 4, x7 is determined to be level 2, x8 is determined to be level 3, x9 is determined to be level 4, x 10 The level is level 2, x 11 The level is level 1, x 12 The level is level 5, x 13 The level is level 3, x 14 The level is level 2, x 15 The level it belongs to is level 4, then the sample set corresponding to level 1 {x1, x3, x 11}, the sample set corresponding to level 2 {x2, x7, x 10 、x 14}, the sample set corresponding to level three {x4, x8, x 13}, the sample set corresponding to level 4 {x6, x9, x 15} and the sample set corresponding to level 5 {x5, x 12}.

[0129] Through the method provided in this embodiment, sample sets corresponding to several levels can be constructed.

[0130] Third embodiment

[0131] This embodiment introduces the specific implementation process of "step S102: using samples in the sample set corresponding to the assessment level to test whether the qualification of the voiceprint comparison engine to be assessed meets the assessment level" in the above embodiment.

[0132] See also Figure 3 , showing a flow chart of using samples in a sample set corresponding to an assessment level to test whether the qualification of a voiceprint comparison engine to be assessed meets the assessment level, which may include:

[0133] Step S301: obtaining a specified number of samples from a sample set corresponding to an assessment level as test samples, so as to obtain a specified number of test samples.

[0134] If the total number of samples in the sample set corresponding to the assessment level is represented as n, then the specified number may be n, or may be less than n. It should be noted that the number of test samples may be set according to specific circumstances.

[0135] Optionally, if the specified number is m, and m is less than n, the following strategy can be used to obtain test samples from the sample set corresponding to the assessment level: first, arrange the samples in the sample set corresponding to the assessment level in order of the discrimination difficulty representation value (such as score) from large to small, or from small to large, and then take a sample every n / m samples.

[0136] Step S302: Based on the voiceprint comparison engine, determine whether the sample data and the sample data contained in each test sample are from the same object, and obtain the determination results corresponding to the specified number of test samples.

[0137] Specifically, each test sample is input into the voiceprint comparison engine, which first extracts voiceprint features from the specimen data and sample data contained in the input test sample, respectively, and then calculates the similarity between the two extracted voiceprint features. Finally, the engine determines whether the specimen data and sample data contained in the test sample are from the same object or from different objects based on the calculated similarity, and then outputs the determination result.

[0138] Step S303: Determine the discrimination accuracy of the voiceprint comparison engine according to the discrimination results corresponding to the specified number of test samples and the sample labels corresponding to the specified number of test samples.

[0139] The sample label is used to indicate whether the specimen data and sample data contained in the corresponding test sample are from the same object.

[0140] It should be noted that, for a test sample, when the judgment result of the test sample indicates that the specimen data and sample data contained in the test sample are from the same object, if the sample label corresponding to the test sample indicates that the specimen data and sample data contained in the test sample are from the same object, then it can be determined that the voiceprint comparison engine has correctly judged the test sample; if the sample label corresponding to the test sample indicates that the specimen data and sample data contained in the test sample are from different objects, then it can be determined that the voiceprint comparison engine has incorrectly judged the test sample; when the judgment result of the test sample indicates that the specimen data and sample data contained in the test sample are from different objects, if the sample label corresponding to the test sample indicates that the specimen data and sample data contained in the test sample are from the same object, then it can be determined that the voiceprint comparison engine has incorrectly judged the test sample; if the sample label corresponding to the test sample indicates that the specimen data and sample data contained in the test sample are from different objects, then it can be determined that the voiceprint comparison engine has correctly judged the test sample.

[0141] If the specified number is m, and the voiceprint comparison engine correctly judges p test samples out of m test samples, then the judgment accuracy of the voiceprint comparison engine can be determined to be p / m.

[0142] Step S304: Determine whether the qualifications of the voiceprint comparison engine meet the assessment level based on the judgment accuracy of the voiceprint comparison engine.

[0143] Specifically, it can be determined whether the discrimination accuracy of the voiceprint comparison engine is greater than or equal to a preset discrimination accuracy threshold. If so, it is determined that the qualifications of the voiceprint comparison engine have reached the assessment level. If not, it is determined that the qualifications of the voiceprint comparison engine have not reached the assessment level.

[0144] Exemplarily, the assessment level is level five. If the discrimination accuracy of the voiceprint comparison engine is greater than or equal to the preset discrimination accuracy threshold, it can be determined that the qualification of the voiceprint comparison engine reaches level five. If the discrimination accuracy of the voiceprint comparison engine is less than the preset discrimination accuracy threshold, it is determined that the qualification of the voiceprint comparison engine does not reach level five.

[0145] It should be noted that if the qualifications of the voiceprint comparison engine do not meet the assessment level, you can apply for a lower level. For example, if the assessment level of the voiceprint comparison engine is level five (apply for level five qualifications), after assessment, the qualifications of the voiceprint comparison engine do not meet level five, you can apply for level four qualifications.

[0146] Fourth embodiment

[0147] An embodiment of the present invention further provides a voiceprint qualification assessment device. The voiceprint qualification assessment device provided by the embodiment of the present invention is described below. The voiceprint qualification assessment device described below and the voiceprint qualification assessment method described above can refer to each other.

[0148] See also Figure 4 , shows a schematic diagram of the structure of a voiceprint qualification assessment device provided in an embodiment of the present invention, which may include: a sample set acquisition module 401 and a voiceprint qualification assessment module 402.

[0149] The sample set acquisition module 401 is used to acquire a sample set corresponding to an assessment level from sample sets corresponding to several levels.

[0150] Among them, the sample sets corresponding to several levels are constructed based on several samples. The sample set corresponding to a level includes samples belonging to this level among several samples. The difficulty of distinguishing samples belonging to different levels is different. Each sample among the several samples includes sample data and sample data, and the assessment level is one of the several levels.

[0151] The voiceprint qualification assessment module 402 is used to use samples in the sample set corresponding to the assessment level to test whether the qualification of the voiceprint comparison engine to be assessed meets the assessment level.

[0152] Optionally, the voiceprint qualification assessment device provided by the embodiment of the present invention may further include: a sample set construction module.

[0153] The sample set construction module may include: a sample acquisition submodule, a discrimination difficulty characterization value determination submodule, a level determination submodule and a sample set construction submodule.

[0154] The sample acquisition submodule is used to acquire a number of samples.

[0155] The discrimination difficulty characterization value determination submodule is used to determine the discrimination difficulty characterization values ​​corresponding to the plurality of samples respectively, wherein the discrimination difficulty characterization values ​​can characterize the discrimination difficulty of the corresponding samples.

[0156] The level determination submodule is used to determine the levels to which the samples belong respectively from the levels according to the discrimination difficulty representation values ​​respectively corresponding to the samples.

[0157] The sample set construction submodule is used to group samples belonging to the same level into sample sets to obtain sample sets corresponding to the several levels respectively.

[0158] Optionally, when determining the discrimination difficulty representation values ​​corresponding to the plurality of samples, the discrimination difficulty representation value determination submodule is specifically used to:

[0159] For the target sample whose discrimination difficulty representation value is to be determined among the plurality of samples:

[0160] Based on a standard voiceprint extraction model, voiceprint features are extracted from the sample data and sample data included in the target sample to obtain a voiceprint feature pair corresponding to the target sample;

[0161] Calculating the similarity of the voiceprint feature pair corresponding to the target sample to obtain the voiceprint feature similarity corresponding to the target sample;

[0162] According to the similarity of the voiceprint features corresponding to the target sample, the discrimination difficulty representation value corresponding to the target sample is determined.

[0163] Optionally, when the discrimination difficulty representation value determining submodule determines the discrimination difficulty representation value corresponding to the target sample according to the voiceprint feature similarity corresponding to the target sample, it is specifically used to:

[0164] Based on a preset linear mapping relationship, the voiceprint feature similarity corresponding to the target sample is mapped to a value that can characterize the discrimination difficulty of the target sample as the discrimination difficulty characterization value corresponding to the target sample.

[0165] Optionally, the plurality of samples include positive samples and negative samples, the sample data and sample data included in the positive samples are from the same subject, and the sample data and sample data included in the negative samples are from different subjects;

[0166] The discrimination difficulty characterization value determination submodule maps the voiceprint feature similarity corresponding to the target sample to a value that can characterize the discrimination difficulty of the target sample based on a preset linear mapping relationship, specifically for:

[0167] If the target sample is a positive sample, based on a first linear mapping relationship preset for the positive sample, the voiceprint feature similarity of the target sample is mapped to a value that can characterize the difficulty of distinguishing the target sample;

[0168] If the target sample is a negative sample, based on a second linear mapping relationship preset for the negative sample, the voiceprint feature similarity of the target sample is mapped to a value that can characterize the difficulty of distinguishing the target sample.

[0169] Optionally, the several levels respectively correspond to a range of values ​​representing the difficulty of discrimination;

[0170] When the level determination submodule determines the levels to which the samples belong respectively from the levels according to the discrimination difficulty representation values ​​respectively corresponding to the samples, the level determination submodule is specifically used to:

[0171] For the target sample whose level is to be determined among the several samples:

[0172] Determine, from the discrimination difficulty representation value ranges corresponding to the plurality of levels, a discrimination difficulty representation value range in which the discrimination difficulty representation value corresponding to the target sample lies, as a target discrimination difficulty representation value range;

[0173] The level corresponding to the target discrimination difficulty characterization value range is determined as the level to which the target sample belongs.

[0174] Optionally, when the voiceprint qualification assessment module 402 uses the samples in the sample set corresponding to the assessment level to test whether the qualification of the voiceprint comparison engine to be assessed reaches the assessment level, it is specifically used to:

[0175] Obtaining a specified number of samples from the sample set corresponding to the assessment level as test samples to obtain a specified number of test samples;

[0176] Based on the voiceprint comparison engine, determine whether the sample data and the sample data contained in each test sample are from the same object, and obtain the determination results corresponding to the specified number of test samples respectively;

[0177] Determine the discrimination accuracy of the voiceprint comparison engine according to the discrimination results corresponding to the specified number of test samples and the sample labels corresponding to the specified number of test samples, wherein the sample labels are used to indicate whether the sample data and the sample data contained in the corresponding test samples are from the same object;

[0178] According to the discrimination accuracy of the voiceprint comparison engine, it is determined whether the qualification of the voiceprint comparison engine reaches the assessment level.

[0179] The voiceprint qualification assessment device provided in the embodiment of the present invention first obtains a sample set corresponding to an assessment level from sample sets corresponding to several levels, and then uses samples in the sample set corresponding to the assessment level to test whether the qualification of the voiceprint comparison engine to be assessed reaches the assessment level. The voiceprint qualification assessment device provided in the embodiment of the present invention can determine the qualification of the voiceprint comparison engine, and the determined qualification is not simply whether it meets the standard, but can give the specific level reached.

[0180] Fifth embodiment

[0181] The embodiment of the present invention also provides a voiceprint qualification assessment device, see Figure 5 , showing a schematic diagram of the structure of the voiceprint qualification assessment device, the voiceprint qualification assessment device may include: at least one processor 501, at least one communication interface 502, at least one memory 503 and at least one communication bus 504;

[0182] In the embodiment of the present invention, the number of the processor 501, the communication interface 502, the memory 503, and the communication bus 504 is at least one, and the processor 501, the communication interface 502, and the memory 503 communicate with each other through the communication bus 504;

[0183] The processor 501 may be a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention;

[0184] The memory 503 may include a high-speed RAM memory, and may also include a non-volatile memory, such as at least one disk memory;

[0185] The memory stores a program, and the processor can call the program stored in the memory, wherein the program is used to:

[0186] From the sample sets corresponding to the several levels, a sample set corresponding to the assessment level is obtained; wherein the sample sets corresponding to the several levels are constructed based on the several samples, the sample set corresponding to a level includes the samples belonging to the level among the several samples, the difficulty of distinguishing samples belonging to different levels is different, each of the several samples includes sample data and sample data, and the assessment level is one of the several levels;

[0187] The samples in the sample set corresponding to the assessment level are used to test whether the qualification of the voiceprint comparison engine to be assessed reaches the assessment level.

[0188] Optionally, the detailed functions and extended functions of the program may refer to the above description.

[0189] Sixth embodiment

[0190] An embodiment of the present invention further provides a readable storage medium, which may store a program suitable for execution by a processor, wherein the program is used to:

[0191] From the sample sets corresponding to the several levels, a sample set corresponding to the assessment level is obtained; wherein the sample sets corresponding to the several levels are constructed based on the several samples, the sample set corresponding to a level includes the samples belonging to the level among the several samples, the difficulty of distinguishing samples belonging to different levels is different, each of the several samples includes sample data and sample data, and the assessment level is one of the several levels;

[0192] The samples in the sample set corresponding to the assessment level are used to test whether the qualification of the voiceprint comparison engine to be assessed reaches the assessment level.

[0193] Optionally, the detailed functions and extended functions of the program may refer to the above description.

[0194] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0195] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0196] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A voiceprint qualification assessment method, characterized in that: include: From the sample sets corresponding to the several levels, a sample set corresponding to the assessment level is obtained; wherein the sample sets corresponding to the several levels are constructed based on the several samples, the sample set corresponding to a level includes the samples belonging to the level among the several samples, the difficulty of distinguishing samples belonging to different levels is different, each of the several samples includes sample data and sample data, and the assessment level is one of the several levels; Using samples in the sample set corresponding to the assessment level to test whether the qualification of the voiceprint comparison engine to be assessed meets the assessment level; The process of constructing the sample sets corresponding to the several levels includes: Obtain a number of samples; Determine the discrimination difficulty representation values ​​corresponding to the plurality of samples respectively, wherein the discrimination difficulty representation values ​​can represent the discrimination difficulty of the corresponding samples; Determining, from the plurality of levels, the levels to which the plurality of samples respectively belong according to the discrimination difficulty representation values ​​respectively corresponding to the plurality of samples; The samples belonging to the same level are grouped into sample sets to obtain sample sets corresponding to the several levels respectively; The using of samples in the sample set corresponding to the assessment level to test whether the qualification of the voiceprint comparison engine to be assessed reaches the assessment level includes: Obtaining a specified number of samples from the sample set corresponding to the assessment level as test samples to obtain a specified number of test samples; Based on the voiceprint comparison engine, determine whether the sample data and the sample data contained in each test sample are from the same object, and obtain the determination results corresponding to the specified number of test samples respectively; Determine the discrimination accuracy of the voiceprint comparison engine according to the discrimination results corresponding to the specified number of test samples and the sample labels corresponding to the specified number of test samples, wherein the sample labels are used to indicate whether the sample data and the sample data contained in the corresponding test samples are from the same object; According to the discrimination accuracy of the voiceprint comparison engine, it is determined whether the qualification of the voiceprint comparison engine reaches the assessment level.

2. The voiceprint qualification assessment method according to claim 1, characterized in that: The determining of the discrimination difficulty representation values ​​corresponding to the plurality of samples respectively includes: For the target sample whose discrimination difficulty representation value is to be determined among the plurality of samples: Based on a standard voiceprint extraction model, voiceprint features are extracted from the sample data and sample data included in the target sample to obtain a voiceprint feature pair corresponding to the target sample; Calculating the similarity of the voiceprint feature pair corresponding to the target sample to obtain the voiceprint feature similarity corresponding to the target sample; According to the similarity of the voiceprint features corresponding to the target sample, the discrimination difficulty representation value corresponding to the target sample is determined.

3. The voiceprint qualification assessment method according to claim 2, characterized in that: The determining, according to the similarity of the voiceprint features corresponding to the target sample, a discrimination difficulty representation value corresponding to the target sample comprises: Based on a preset linear mapping relationship, the voiceprint feature similarity corresponding to the target sample is mapped to a value that can characterize the discrimination difficulty of the target sample as the discrimination difficulty characterization value corresponding to the target sample.

4. The voiceprint qualification assessment method according to claim 3, characterized in that: The plurality of samples include positive samples and negative samples, the sample data and sample data included in the positive samples are from the same subject, and the sample data and sample data included in the negative samples are from different subjects; The mapping of the voiceprint feature similarity corresponding to the target sample to a value capable of representing the difficulty of distinguishing the target sample based on a preset linear mapping relationship includes: If the target sample is a positive sample, based on a first linear mapping relationship preset for the positive sample, the voiceprint feature similarity of the target sample is mapped to a value that can characterize the difficulty of distinguishing the target sample; If the target sample is a negative sample, based on a second linear mapping relationship preset for the negative sample, the voiceprint feature similarity of the target sample is mapped to a value that can characterize the difficulty of distinguishing the target sample.

5. The voiceprint qualification assessment method according to claim 1, characterized in that: The several levels respectively correspond to a range of discrimination difficulty representation values; The step of determining the levels to which the samples belong respectively from the levels according to the discrimination difficulty representation values ​​respectively corresponding to the samples includes: For the target sample whose level is to be determined among the several samples: Determine, from the discrimination difficulty representation value ranges corresponding to the plurality of levels, a discrimination difficulty representation value range in which the discrimination difficulty representation value corresponding to the target sample lies, as a target discrimination difficulty representation value range; The level corresponding to the target discrimination difficulty characterization value range is determined as the level to which the target sample belongs.

6. A voiceprint qualification assessment device, characterized in that: include: Sample set acquisition module and voiceprint qualification assessment module; The sample set acquisition module is used to acquire a sample set corresponding to an assessment level from sample sets corresponding to several levels; wherein the sample sets corresponding to the several levels are constructed based on several samples, and the sample set corresponding to a level includes samples belonging to the level among the several samples, and samples belonging to different levels have different difficulty levels in distinguishing, and each sample among the several samples includes sample data and sample data, and the assessment level is one of the several levels; The voiceprint qualification assessment module is used to use samples in the sample set corresponding to the assessment level to test whether the qualification of the voiceprint comparison engine to be assessed reaches the assessment level; The process of constructing the sample sets corresponding to the several levels includes: Acquire a number of samples; determine the discrimination difficulty characterization values ​​corresponding to the samples, wherein the discrimination difficulty characterization values ​​can characterize the discrimination difficulty of the corresponding samples; determine the levels to which the samples belong from the levels according to the discrimination difficulty characterization values ​​corresponding to the samples; form a sample set by combining samples belonging to the same level, and obtain sample sets corresponding to the levels; The voiceprint qualification assessment module is specifically used for: A specified number of samples are obtained from the sample set corresponding to the assessment level as test samples to obtain a specified number of test samples; based on the voiceprint comparison engine, whether the specimen data and the sample data contained in each test sample are from the same object is determined to obtain the determination results corresponding to the specified number of test samples; according to the determination results corresponding to the specified number of test samples and the sample labels corresponding to the specified number of test samples, the determination accuracy of the voiceprint comparison engine is determined, wherein the sample label is used to indicate whether the specimen data and the sample data contained in the corresponding test sample are from the same object; according to the determination accuracy of the voiceprint comparison engine, it is determined whether the qualification of the voiceprint comparison engine reaches the assessment level.

7. A voiceprint qualification assessment device, characterized in that: include: Memory and processor; The memory is used to store programs; The processor is used to execute the program to implement each step of the voiceprint qualification assessment method as described in any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, each step of the voiceprint qualification assessment method as described in any one of claims 1 to 5 is implemented.

Images