Voiceprint recognition system evaluation method and apparatus, storage medium, and electronic device

Abstract

A voiceprint recognition system evaluation method and apparatus, a storage medium, and an electronic device. The voiceprint recognition system evaluation method comprises: acquiring a target voiceprint recognition system to be evaluated and an adversarial sample to be optimized, and determining a recognition type of the target voiceprint recognition system (S100); inputting said adversarial sample into the target voiceprint recognition system to obtain a recognition result of the target voiceprint recognition system for said adversarial sample (S102); on the basis of the recognition result, adjusting said adversarial sample by using a preset mode corresponding to the recognition type so as to obtain a voice adversarial sample (S104); using the voice adversarial sample to attack the target voiceprint recognition system (S106); and on the basis of an attack result of the voice adversarial sample on the target voiceprint recognition system, determining an evaluation result of the target voiceprint recognition system (S108).

Description

A method, device, storage medium and electronic device for evaluating a voiceprint recognition system Technical Field

[0001] This specification relates to the field of computer technology, and in particular to a voiceprint recognition system evaluation method, device, storage medium, and electronic device. Background Art

[0002] Voiceprint recognition is now widely used in various fields. One of the most common applications is to verify the user's identity by analyzing the user's voice. For example, users can unlock mobile phones, computers, and other terminal devices by voice, wake up voice assistants by speaking commands, and give instructions.

[0003] However, with the continuous development of science and technology, the security threats faced by voiceprint recognition technology are also increasing. At present, voice synthesis technology is also very advanced. Using this technology, attackers can completely synthesize the user's voice and impersonate the user, posing a huge threat to the user's privacy and interests.

[0004] To protect user privacy and interests, it is crucial for voiceprint recognition systems to effectively defend against attacks and sabotage. However, there is currently no mature method to evaluate the security of voiceprint recognition systems, making it impossible to accurately determine the security level of voiceprint recognition systems.

[0005] Therefore, how to effectively evaluate the security of voiceprint recognition systems is an urgent problem to be solved.

[0006] Summary of the Invention

[0007] This specification provides a voiceprint recognition system evaluation method, device, storage medium and electronic device to at least partially solve the above-mentioned problems existing in the related art.

[0008] This specification adopts the following technical solution: This specification provides a voiceprint recognition system evaluation method, including: obtaining a target voiceprint recognition system to be evaluated and an adversarial sample to be optimized, and determining the recognition type of the target voiceprint recognition system; inputting the adversarial sample to be optimized into the target voiceprint recognition system to obtain the recognition result of the target voiceprint recognition system on the adversarial sample to be optimized; according to the recognition result, adjusting the adversarial sample to be optimized in a preset manner corresponding to the recognition type to obtain a voice adversarial sample; using the voice adversarial sample to attack the target voiceprint recognition system; and determining the evaluation result of the target voiceprint recognition system based on the attack result of the voice adversarial sample on the target voiceprint recognition system.

[0009] Optionally, the recognition type includes at least one of open-set speaker recognition, closed-set speaker recognition, and speaker confirmation, wherein the open-set speaker recognition is used to identify whether the original speaker is a user in the open-set registered speaker group, the closed-set speaker recognition is used to identify whether the original speaker is a user in the closed-set registered speaker group, and the speaker confirmation is used to identify whether the original speaker is a registered speaker.

[0010] Optionally, the recognition type is open-set speaker recognition, and the attack type of the speech adversarial sample to be obtained is that an unrelated speaker impersonates the first designated user in the open-set registered speaker group; the adversarial sample to be optimized is adjusted in a preset manner corresponding to the recognition type, specifically including: adjusting the adversarial sample to be optimized with the highest probability of the adversarial sample to be identified as the first designated user in the target voiceprint recognition system as the optimization target.

[0011] Optionally, the recognition type is open-set speaker recognition, and the attack type of the speech adversarial sample to be obtained is a direct attack by an irrelevant speaker; the adversarial sample to be optimized is adjusted in a preset manner corresponding to the recognition type, specifically including: adjusting the adversarial sample to be optimized with the probability that the adversarial sample to be optimized is identified as any user in the open-set registered speaker group in the target voiceprint recognition system being no greater than a specified threshold as the optimization goal.

[0012] Optionally, the recognition type is open-set speaker recognition, and the attack type of the speech adversarial sample to be obtained is that the second designated user in the open-set registered speaker group impersonates any other user in the open-set registered speaker group; the adversarial sample to be optimized is adjusted in a preset manner corresponding to the recognition type, specifically including: adjusting the adversarial sample to be optimized with the lowest probability of the adversarial sample to be identified as the second designated user in the target voiceprint recognition system and the highest probability of being identified as any other user as the optimization goal.

[0013] Optionally, the recognition type is open-set speaker recognition, and the attack type of the speech adversarial sample to be obtained is an unrelated speaker impersonating any user in the open-set registered speaker group; the adversarial sample to be optimized is adjusted in a preset manner corresponding to the recognition type, specifically including: adjusting the adversarial sample to be optimized with the highest probability of the adversarial sample to be identified as any user in the open-set registered speaker group in the target voiceprint recognition system as the optimization target.

[0014] Optionally, the recognition type is closed-set speaker recognition, and the attack type of the speech adversarial sample to be obtained is that any user in the closed-set registered speaker group impersonates a third designated user in the closed-set registered speaker group; the adversarial sample to be optimized is adjusted in a preset manner corresponding to the recognition type, specifically including: adjusting the adversarial sample to be optimized with the highest probability of the adversarial sample to be identified as the third designated user in the target voiceprint recognition system as the optimization target.

[0015] Optionally, the recognition type is closed-set speaker recognition, and the attack type of the speech adversarial sample to be obtained is that the fourth designated user in the closed-set registered speaker group impersonates any other user in the closed-set registered speaker group; the adversarial sample to be optimized is adjusted in a preset manner corresponding to the recognition type, specifically including: adjusting the adversarial sample to be optimized with the lowest probability of the adversarial sample to be identified as the fourth designated user in the target voiceprint recognition system and the highest probability of being identified as any other user in the closed-set registered speaker group as the optimization goal.

[0016] Optionally, the recognition type is speaker confirmation, and the attack type of the speech adversarial sample to be obtained is the registered speaker test; the adversarial sample to be optimized is adjusted in a preset manner corresponding to the recognition type, specifically including: adjusting the adversarial sample to be optimized with the probability of the adversarial sample to be identified as the registered speaker in the target voiceprint recognition system as the lowest optimization goal.

[0017] Optionally, the recognition type is speaker confirmation, and the attack type of the speech adversarial sample to be obtained is that an unrelated speaker impersonates the registered speaker; the adversarial sample to be optimized is adjusted in a preset manner corresponding to the recognition type, specifically including: adjusting the adversarial sample to be optimized with the highest probability of the adversarial sample to be identified as the registered speaker in the target voiceprint recognition system as the optimization target.

[0018] Optionally, the adversarial sample to be optimized is adjusted in a preset manner corresponding to the recognition type to obtain a speech adversarial sample, specifically including: obtaining a discrete sequence of the adversarial sample to be optimized; adjusting the size of each numerical value in the discrete sequence of the adversarial sample to be optimized in a preset manner corresponding to the recognition type to obtain a target discrete sequence; and determining the speech adversarial sample based on the target discrete sequence.

[0019] Optionally, using the adversarial sample to attack the target voiceprint recognition system specifically includes: scrambling the adversarial sample in a preset scrambling manner to obtain a simulated physical adversarial sample; and using the simulated physical adversarial sample to attack the target voiceprint recognition system.

[0020] Optionally, the scrambling method includes at least one of environmental noise interference, reverberation interference, and hardware interference.

[0021] Optionally, the evaluation result includes an adversarial success rate and an adversarial overhead; the evaluation result of the target voiceprint recognition system is determined based on the attack result of the voice adversarial sample on the target voiceprint recognition system, specifically including: determining the attack success rate based on the number of voice adversarial samples that are incorrectly identified in the recognition result of the target voiceprint recognition system and the total number of voice adversarial samples input into the target voiceprint recognition system; determining the attack overhead based on the cost of generating the voice adversarial sample and the number of queries for the voice adversarial sample in the target voiceprint recognition system.

[0022] The present specification provides a voiceprint recognition system evaluation device, comprising: an acquisition module, used to acquire a target voiceprint recognition system to be evaluated and an adversarial sample to be optimized, and determine the recognition type of the target voiceprint recognition system; an input module, used to input the adversarial sample to be optimized into the target voiceprint recognition system, and obtain the recognition result of the target voiceprint recognition system on the adversarial sample to be optimized; an optimization module, used to adjust the adversarial sample to be optimized according to the recognition result, using a preset method corresponding to the recognition type, to obtain a voice adversarial sample; an attack module, used to attack the target voiceprint recognition system using the voice adversarial sample; and a determination module, used to determine the evaluation result of the target voiceprint recognition system according to the attack result of the voice adversarial sample on the target voiceprint recognition system.

[0023] This specification provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned voiceprint recognition system evaluation method is implemented.

[0024] This specification provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the above-mentioned voiceprint recognition system evaluation method is implemented.

[0025] At least one of the above-mentioned technical solutions adopted in this specification can achieve the following beneficial effects: in the voiceprint recognition system evaluation method provided in this specification, a target voiceprint recognition system to be evaluated and an adversarial sample to be optimized are obtained, and the recognition type of the target voiceprint recognition system is determined; the adversarial sample to be optimized is input into the target voiceprint recognition system to obtain a recognition result of the target voiceprint recognition system on the adversarial sample to be optimized; according to the recognition result, the adversarial sample to be optimized is adjusted in a preset manner corresponding to the recognition type to obtain a voice adversarial sample; the voice adversarial sample is used to attack the target voiceprint recognition system; and according to the attack result of the voice adversarial sample on the target voiceprint recognition system, the evaluation result of the target voiceprint recognition system is determined.

[0026] When using the voiceprint recognition system evaluation method provided in this specification to evaluate a target voiceprint recognition system, different methods can be used to generate voice adversarial samples for scenarios corresponding to various attack methods under different recognition types. Ultimately, the voice adversarial samples are used to attack the target voiceprint recognition system and obtain the final evaluation results. This method can cover a variety of different attack methods when evaluating the target voiceprint recognition system, and obtain accurate and comprehensive evaluation results. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The drawings described herein are used to provide a further understanding of this specification and constitute a part of this specification. The exemplary embodiments and descriptions of this specification are used to explain this specification and do not constitute an improper limitation of this specification. In the drawings:

[0028] FIG1 is a flow chart of a method for evaluating a voiceprint recognition system provided in this specification;

[0029] FIG2 is a schematic diagram of a process of audio signal propagation through a physical channel provided in this specification;

[0030] FIG3 is a schematic diagram of a voiceprint recognition system evaluation device provided in this specification;

[0031] FIG4 is a schematic diagram of an electronic device provided in this specification corresponding to FIG1 . DETAILED DESCRIPTION

[0032] Typically, voiceprint recognition systems are used to analyze voices and verify the identity of the voice. Although voiceprint recognition systems have good security performance in most cases, they can be deceived by attackers who conduct special research and design to target a specific voiceprint recognition system.

[0033] Unlike the human ear, which relies on timbre to identify sound, computers convert sound into data such as waveforms and sequences, and analyze this data to determine the source of the sound. This can lead to voiceprint recognition systems being deceived in some scenarios, resulting in misidentification. For example, a specially processed audio segment may sound like the voice of user A to the human ear, but be identified as person B by the voiceprint recognition system.

[0034] Since voiceprint recognition systems are usually used in scenarios such as user identity authentication and are closely related to the privacy and interests of users, whether the voiceprint recognition system can effectively perceive and defend against attacks from attackers, that is, the security performance of the voiceprint recognition system is crucial.

[0035] In order to more accurately obtain the security performance of the voiceprint recognition system when facing attacks, this manual provides a comprehensive and reliable voiceprint recognition system evaluation method.

[0036] To make the objectives, technical solutions, and advantages of this specification more clear, the following will clearly and completely describe the technical solutions of this specification in conjunction with the specific embodiments of this specification and the corresponding drawings. Obviously, the embodiments described are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without making any creative efforts are within the scope of protection of this specification.

[0037] The technical solutions provided by the embodiments of this specification are described in detail below with reference to the accompanying drawings.

[0038] FIG1 is a flow chart of a voiceprint recognition system evaluation method provided in this specification, including the following steps S100 to S108.

[0039] S100: Obtain a target voiceprint recognition system to be evaluated and an adversarial sample to be optimized, and determine the recognition type of the target voiceprint recognition system.

[0040] In this specification, the execution entity used to implement the voiceprint recognition system evaluation method may refer to a designated device such as a server set up on the business platform. For the sake of convenience of description, this specification only takes the server as the execution entity as an example to illustrate a voiceprint recognition system evaluation method provided in this specification.

[0041] The voiceprint recognition system evaluation method provided in this specification can be applied to any voiceprint recognition system. Therefore, in this step, the target voiceprint recognition system to be evaluated can be obtained. The target voiceprint recognition system can have a variety of structures, and this specification does not impose specific restrictions on this. Furthermore, when evaluating a voiceprint recognition system, it is necessary to simulate the process of attacking the voiceprint recognition system. Therefore, in this step, adversarial samples used to simulate attacks on the voiceprint recognition system can also be obtained. These adversarial samples can be audio data. Since the adversarial samples have not yet undergone any optimization or adjustment in this step, they are considered to be adversarial samples to be optimized.

[0042] Furthermore, in actual applications, there may be multiple voiceprint recognition systems with different recognition types, depending on the different recognition methods used by the voiceprint recognition system. Specifically, the recognition type includes at least one of open-set speaker recognition, closed-set speaker recognition, and speaker verification.

[0043] Open-set speaker recognition is used to identify whether the original speaker is a user in an open set of registered speakers. It is often used in voice control of smart homes, for example. The original speaker is the speaker to whom a speech segment originally belongs, and the open set of registered speakers is a collection of users whose identities can be verified. In open-set speaker recognition, the original speaker can be any user.

[0044] Closed-set speaker recognition is used to identify whether the original speaker is a member of a closed set of registered speakers. It is often used in organizations such as criminal investigation. The original speaker is the speaker to whom a piece of speech originally belongs, and the closed-set registered speaker group is a collection of users whose identities can be verified. In closed-set speaker recognition, the original speaker is guaranteed to be a member of the closed set of registered speakers.

[0045] Speaker verification is used to identify whether the original speaker is a registered speaker, and is commonly used in scenarios such as user account login. The original speaker is the speaker to whom a speech originally belongs, and the registered speaker is a user whose identity can be verified. In speaker verification recognition, there is only one registered speaker.

[0046] In this step, the recognition type of the acquired target voiceprint recognition system can be determined, and different processing can be performed according to different recognition types in subsequent steps.

[0047] S102: Inputting the adversarial sample to be optimized into the target voiceprint recognition system to obtain a recognition result of the target voiceprint recognition system on the adversarial sample to be optimized.

[0048] In this step, the adversarial sample to be optimized obtained in step S100 can be input into the target voiceprint recognition system to obtain the recognition result of the target voiceprint recognition system for the adversarial sample to be optimized. The recognition result can be expressed in various forms, including but not limited to confidence level, probability, etc., which are not specifically limited in this specification.

[0049] For different recognition types, the recognition results output by the target voiceprint recognition system will also vary. Taking the recognition result as confidence as an example, when the recognition type is open-set speaker recognition, the recognition result can be a vector, where each component represents the confidence that the adversarial sample to be optimized comes from each different user in the open-set registered speaker group; similarly, when the recognition type is closed-set speaker recognition, the recognition result can also be a vector, where each component represents the confidence that the adversarial sample to be optimized comes from each different user in the closed-set registered speaker group; and when the recognition type is speaker confirmation, the recognition result can be a scalar, representing the confidence that the adversarial sample to be optimized comes from the registered speaker.

[0050] Furthermore, different voiceprint recognition systems of different recognition types also have different ways of making final judgments based on the recognition results. Let's take the recognition result of the confidence level in the above example. In the case where the recognition type is open-set speaker recognition, there will be a preset threshold. When the value of a component in the recognition result is the largest and not less than the preset threshold, the adversarial sample to be optimized will be judged as coming from the user corresponding to the component; in the case where the recognition type is closed-set speaker recognition, there is no threshold. When the value of a component in the recognition result is the largest, the adversarial sample to be optimized will be judged as coming from the user corresponding to the component; in the case where the recognition type is speaker confirmation, there is also a preset threshold. When the confidence level is not less than the preset threshold, the adversarial sample to be optimized can be judged as coming from the registered speaker.

[0051] In this step, it is only necessary to obtain the recognition result of the target voiceprint recognition system for the optimized adversarial sample, and it is not necessary to obtain the final judgment made by the target voiceprint recognition system on the identity of the optimized adversarial sample.

[0052] S104: According to the recognition result, the adversarial sample to be optimized is adjusted in a preset manner corresponding to the recognition type to obtain a speech adversarial sample.

[0053] In this step, the optimized adversarial sample can be optimized and adjusted based on the recognition result of the target voiceprint recognition system on the optimized adversarial sample in step S102 to obtain the final voice adversarial sample. In addition, for different recognition types of the target voiceprint recognition system determined in step S100, the optimized adversarial sample can be adjusted using different preset methods corresponding to each recognition type.

[0054] Furthermore, under each different recognition type, there will also be a variety of different attack scenarios. For example, for the recognition type of open-set speaker recognition, there may be an attack in which the original speaker is an unrelated speaker who does not belong to the open-set registered speaker group and impersonates a designated user in the open-set registered speaker group; there may also be an attack in which the original speaker is one of the users in the open-set registered speaker group and impersonates another user in the open-set registered speaker group. Based on this, this specification provides a method for optimizing the optimized adversarial samples for various different situations under various different recognition types, so as to evaluate the security performance of the target voiceprint recognition system when facing various different situations. Specifically, this specification presets eight different ways to optimize the optimized adversarial samples for scenarios when voiceprint recognition systems of different recognition types are subjected to different attacks. Each optimization method will be introduced in turn below, taking the recognition result output by the target voiceprint recognition system as the probability.

[0055] In each different way, there is a different loss function for adjusting the adversarial sample to be optimized. There are some common representations in each loss function, which are explained here. In the voiceprint recognition system evaluation method provided in this specification, L represents the loss function, x represents the adversarial sample to be optimized, S(x) represents a component in the recognition result; s represents the original speaker, that is, the speaker to whom the adversarial sample to be optimized originally belongs, t represents the target speaker, that is, the speaker to whom the adversarial sample to be optimized hopes to be identified after adjustment, G represents the registered speaker group; θ represents the threshold, which can be set according to needs; σ represents the softmax function, Represents the sigmoid function. Both the softmax function and the sigmoid function are used to convert the recognition results into probabilities when the recognition results are not probabilities.

[0056] First, consider open-set speaker recognition. In open-set speaker recognition, the original speaker can be any user, who may be from the open-set registered speaker group or a user outside the group. Therefore, in open-set speaker recognition, a threshold needs to be set in advance, and the identity will only be confirmed if the recognition result of the voiceprint recognition system is not less than the threshold. Among all the recognition types, the application scenario of open-set speaker recognition is the most complex, and the corresponding types of situations are also more. This manual lists the optimization methods for the adversarial samples to be optimized corresponding to four different situations under the open-set speaker recognition recognition type.

[0057] When the recognition type is open-set speaker recognition, and the attack type of the speech adversarial sample to be obtained is an unrelated speaker impersonating the first designated user in the open-set registered speaker group, the probability that the adversarial sample to be optimized is recognized as the first designated user in the target voiceprint recognition system can be the highest as the optimization target, and the adversarial sample to be optimized can be adjusted.

[0058] In the scenario of attacking the voiceprint recognition system, the most common attack is from an irrelevant speaker. In the voiceprint recognition system evaluation method provided in this manual, the irrelevant speaker is a user whose identity cannot be verified. The irrelevant speaker impersonating the first designated user in the open set of registered speakers can be regarded as a targeted attack, which means that during the attack, the voice belonging to the irrelevant speaker, that is, the adversarial sample to be optimized, is adjusted to a certain extent, so that the adjusted voice adversarial sample is identified by the target voiceprint recognition system as belonging to the first designated user. Under this attack type, the following formula can be used to optimize the adversarial sample to be optimized: L CE (x,t)=-log[σ(S(x))] t L M (x,t)=max i∈G,i≠t [S(x)] i -[S(x)] t L1(x,t)=-[S(x)] t L2(x,t)=max{θ,max i∈G,i≠t [S(x)] i}-[S(x)] t

[0059] Where t represents the target speaker, which is the first designated user in this example, i represents any user in the open set of registered speakers except the first designated user, and G represents the registered speaker group, which is the open set of registered speakers in this example. In the above formula, [S(x)] t Indicates the probability that the adversarial sample to be optimized belongs to the first specified user in the recognition result, [S(x)] i Indicates the probability that the adversarial example to be optimized in the recognition result belongs to any other user in the open set of registered speakers. In practical applications, at least one of the above loss functions can be selected for use, and the adversarial example to be optimized is adjusted to minimize the loss function, that is, to maximize the probability that the adversarial example to be optimized is recognized as the first designated user in the target voiceprint recognition system.

[0060] When the recognition type is open-set speaker recognition and the attack type of the speech adversarial sample to be obtained is a direct attack by an irrelevant speaker, the probability that the adversarial sample to be optimized is identified as any user in the open-set registered speaker group in the target voiceprint recognition system is no greater than a specified threshold as the optimization goal, and the adversarial sample to be optimized is adjusted.

[0061] In the scenario of attacking a voiceprint recognition system, a direct attack by an unrelated speaker can be considered an attack launched directly by the unrelated speaker's voice without any modification. Generally speaking, this type of attack is difficult to deceive the voiceprint recognition system. It is often used to test whether the voiceprint recognition system can achieve basic judgment. Under this type of attack, the following formula can be used to optimize the adversarial sample: L3(x,t)=max i∈G [S(x)] i -θ

[0062] Where s represents the original speaker, which is an unrelated speaker in this case, i represents any user in the open set of registered speakers, and G represents the registered speaker group, which is the open set of registered speakers in this case. In the above formula, [S(x)] s Indicates the probability that the adversarial sample to be optimized belongs to an irrelevant speaker in the recognition result, [S(x)] i Indicates the probability that the adversarial sample to be optimized in the recognition result belongs to any user in the open set of registered speakers. In practical applications, at least one of the above loss functions can be selected for use, and the optimization goal is to minimize the loss function value, that is, the probability that the adversarial sample to be optimized is recognized as any user in the open set of registered speakers in the target voiceprint recognition system is no greater than a specified threshold, and the adversarial sample to be optimized is adjusted.

[0063] When the recognition type is open-set speaker recognition, and the attack type of the speech adversarial sample to be obtained is that the second designated user in the open-set registered speaker group impersonates any other user in the open-set registered speaker group, the probability of the adversarial sample to be optimized being identified as the second designated user in the target voiceprint recognition system can be the lowest, and the probability of being identified as any other user can be the highest as the optimization goal, and the adversarial sample to be optimized can be adjusted.

[0064] In the scenario of attacking the voiceprint recognition system, users who are also in the open set of registered speakers may also become the initiators of the attack. An attack by a second designated user in the open set of registered speakers impersonating any other user in the open set of registered speakers can be considered an untargeted attack. Under this type of attack, the following formula can be used to optimize the adversarial sample to be optimized:

[0065] Where s represents the original speaker, which is the second designated user in this example, i represents any user in the open set of registered speakers except the second designated user, and G represents the registered speaker group, which is the open set of registered speakers in this example. In the above formula, [S(x)] s Indicates the probability that the adversarial sample to be optimized belongs to the second specified user in the recognition result, [S(x)] i Indicates the probability that the adversarial example to be optimized in the recognition result belongs to any user in the open set of registered speakers, excluding the second designated user. In practical applications, at least one of the aforementioned loss functions can be selected for use, and the optimization goal is to minimize the loss function, that is, to minimize the probability that the adversarial example to be optimized is recognized as the second designated user by the target voiceprint recognition system, and maximize the probability that it is recognized as any other user. The adversarial example to be optimized is then adjusted.

[0066] When the recognition type is open-set speaker recognition, and the attack type of the speech adversarial sample to be obtained is that an unrelated speaker impersonates any user in the open-set registered speaker group, the probability that the adversarial sample to be optimized is recognized as any user in the open-set registered speaker group in the target voiceprint recognition system can be the highest as the optimization target, and the adversarial sample to be optimized can be adjusted.

[0067] In the scenario of attacking the voiceprint recognition system, in addition to users in the open set of registered speakers, unrelated speakers may also launch untargeted attacks. In other words, an attack is launched by any user in the open set of registered speakers. Under this type of attack, the following formula can be used to optimize the adversarial sample:

[0068] Where i represents any user in the open set of registered speakers, G represents the registered speaker group, which is the open set of registered speakers in this example. In the above formula, [S(x)] i Indicates the probability that the adversarial sample to be optimized in the recognition result belongs to any user in the open set of registered speakers. In practical applications, at least one of the above loss functions can be selected for use, and the optimization goal is to minimize the loss function value, that is, to maximize the probability that the adversarial sample to be optimized is recognized as any user in the open set of registered speakers by the target voiceprint recognition system, and adjust the adversarial sample to be optimized.

[0069] On the other hand, consider closed-set speaker recognition. In closed-set speaker recognition, the original speaker must be one of the users in the closed-set registered speaker group; in other words, the original speaker must be a user whose identity can be verified. Therefore, in closed-set speaker recognition, there is no threshold; the user identity corresponding to the highest-probability component in the recognition result is directly determined as the original speaker's identity. This specification lists optimization methods for adversarial examples to be optimized for two different scenarios under closed-set speaker recognition.

[0070] When the recognition type is closed-set speaker recognition, and the attack type of the speech adversarial sample to be obtained is that any user in the closed-set registered speaker group impersonates the third designated user in the closed-set registered speaker group, the probability that the adversarial sample to be optimized is recognized as the third designated user in the target voiceprint recognition system can be the highest as the optimization target, and the adversarial sample to be optimized can be adjusted.

[0071] In the scenario of attacking the voiceprint recognition system, a user in the closed set of registered speakers may impersonate a fixed user in the closed set of registered speakers, that is, the third designated user. Under this attack type, the following formula can be used to optimize the adversarial sample: L CE (x,t)=-log[σ(S(x))] t L M (x,t)=max i∈g,i≠t [S(x)] i -[S(x)] t L1(x,t)=-[S(x)] t

[0072] Where t represents the target speaker, which is the third designated user in this example, i represents any user in the closed set of registered speakers except the third designated user, and G represents the registered speaker group, which is the closed set of registered speakers in this example. In the above formula, [S(x)] t Indicates the probability that the adversarial sample to be optimized belongs to the third specified user in the recognition result, [S(x)] i Indicates the probability that the adversarial sample to be optimized in the recognition result belongs to any user in the closed set of registered speakers, excluding the third designated user. In practical applications, at least one of the above loss functions can be selected for use, and the optimization goal is to minimize the loss function, that is, to maximize the probability that the adversarial sample to be optimized is recognized as the third designated user in the target voiceprint recognition system, and adjust the adversarial sample to be optimized.

[0073] When the recognition type is closed-set speaker recognition, and the attack type of the speech adversarial sample to be obtained is that the fourth designated user in the closed-set registered speaker group impersonates any other user in the closed-set registered speaker group, the probability of the adversarial sample to be optimized being identified as the fourth designated user in the target voiceprint recognition system can be the lowest, and the probability of being identified as any other user in the closed-set registered speaker group can be the highest as the optimization goal, and the adversarial sample to be optimized can be adjusted.

[0074] In the scenario of attacking the voiceprint recognition system, contrary to the previous embodiment, the fourth designated user in the closed set of registered speakers may also impersonate any other user in the closed set of registered speakers to attack. In this attack type, the following formula can be used to optimize the adversarial sample to be optimized:

[0075] Where s represents the original speaker, which is the fourth designated user in this example, i represents any user in the closed set of registered speakers except the fourth designated user, and G represents the registered speaker group, which is the closed set of registered speakers in this example. In the above formula, [S(x)] s Indicates the probability that the adversarial sample to be optimized belongs to the fourth designated user in the recognition result, [S(x)] i Indicates the probability that the adversarial example to be optimized in the recognition result belongs to any user in the closed set of registered speakers except the fourth designated user. In practical applications, at least one of the above loss functions can be selected for use, and the optimization goal is to minimize the loss function, that is, to minimize the probability that the adversarial example to be optimized is recognized as the fourth designated user by the target voiceprint recognition system, and maximize the probability that it is recognized as any other user in the closed set of registered speakers. The adversarial example to be optimized is then adjusted.

[0076] Finally, consider speaker verification. In speaker verification, the original speaker can be any user, while the registered speaker is unique, meaning there is only one user whose identity can be verified. Therefore, speaker verification also uses a threshold, and the recognition result combined with the threshold determines whether the original speaker is the registered speaker. This specification lists optimization methods for adversarial examples to be optimized for two different situations in speaker verification.

[0077] When the recognition type is speaker confirmation and the attack type of the speech adversarial sample to be obtained is a registered speaker test, the probability that the adversarial sample to be optimized is identified as the registered speaker in the target voiceprint recognition system can be minimized as the optimization target, and the adversarial sample to be optimized can be adjusted.

[0078] In scenarios where voiceprint recognition systems are attacked, speaker testing is typically used to test whether the target voiceprint recognition system can achieve basic judgment, that is, whether it can identify the registered speaker. In the speaker testing scenario, the original speaker is the registered speaker itself. In this attack type, the following formula can be used to optimize the adversarial sample: L B (x, t) = S(x) - θ

[0079] In the above formula, S(x) represents the recognition result, that is, the probability that the adversarial sample to be optimized belongs to the registered speaker. In practical applications, at least one of the above loss functions can be selected for use, and the optimization goal is to minimize the value of the loss function, that is, to minimize the probability that the adversarial sample to be optimized is recognized as the registered speaker in the target voiceprint recognition system, and adjust the adversarial sample to be optimized.

[0080] When the recognition type is speaker confirmation and the attack type of the speech adversarial sample to be obtained is an unrelated speaker impersonating the registered speaker, the probability that the adversarial sample to be optimized is recognized as the registered speaker in the target voiceprint recognition system can be the highest as the optimization target, and the adversarial sample to be optimized can be adjusted.

[0081] In the scenario of attacking voiceprint recognition systems, one of the most common attack methods is for an unrelated speaker to impersonate the registered speaker under speaker confirmation. In this type of attack, the following formula can be used to optimize the adversarial sample:

[0082] In the above formula, S(x) represents the recognition result, that is, the probability that the adversarial sample to be optimized belongs to the registered speaker. In practical applications, at least one of the above loss functions can be selected for use, and the optimization goal is to minimize the loss function value, that is, to maximize the probability that the adversarial sample to be optimized is recognized as the registered speaker in the target voiceprint recognition system, and adjust the adversarial sample to be optimized.

[0083] For scenarios under various attack methods, this manual provides several loss functions for adjusting the adversarial samples to be optimized. For each scenario, the loss function works in the same way, that is, the adversarial samples to be optimized are adjusted with the loss function as small as possible as the optimization goal. In addition, each loss function in each scenario can be used alone, or some of them can be selected for use, or all loss functions can be used together, and the optimization effect of the adversarial samples to be optimized can be achieved. This manual does not impose specific restrictions on this. It should be noted that in most cases, only one iteration, that is, only solving the loss function once, cannot directly obtain the speech adversarial samples that meet the requirements. Multiple iterations, that is, repeatedly executing steps S102 to S104, are required to obtain the speech adversarial samples that meet the requirements.

[0084] Furthermore, since the above-mentioned loss functions use the recognition results of the target voiceprint recognition system, in the process of solving the above-mentioned loss functions, whether the specific structure and parameters of the target voiceprint recognition system are understood will affect the solution process of the loss function to a certain extent.

[0085] When the target voiceprint recognition system is a white box, that is, the structural parameters of the target voiceprint recognition system are clearly known, the gradient descent method based on backpropagation can be used to solve the above loss functions. When the target voiceprint recognition system is a black box, that is, the structural parameters of the target voiceprint recognition system are unknown, two different situations can be considered depending on whether the query access budget for the target voiceprint recognition system is sufficient: when the query access budget is sufficient, the gradient descent method based on gradient estimation and the genetic algorithm that does not rely on gradients can be used to solve the above loss functions; when the query access budget is limited, the transferability of adversarial samples can be utilized to first generate voice adversarial samples using a white box method on other white box voiceprint recognition systems, and then migrate the voice adversarial samples to the black box target voiceprint recognition system for use.

[0086] Preferably, there are various specific implementations for optimizing the adversarial sample to be optimized. This specification provides a specific embodiment for reference. Specifically, a discrete sequence of the adversarial sample to be optimized can be obtained; the values ​​of each item in the discrete sequence of the adversarial sample to be optimized can be adjusted using a preset method corresponding to the recognition type to obtain a target discrete sequence; and the speech adversarial sample can be determined based on the target discrete sequence.

[0087] Typically, in a discrete sequence of adversarial examples to be optimized in audio contexts, each numerical value represents the magnitude of a specific sound component. By adjusting the adversarial examples to be optimized in this way, we can maximize the preservation of the original timbre of the adversarial examples, making the optimized speech adversarial examples deceptive to voiceprint recognition systems. To the human ear, there's virtually no difference between the adversarial examples to be optimized and the speech adversarial examples.

[0088] S106: Using the voice adversarial sample to attack the target voiceprint recognition system.

[0089] After optimizing the adversarial sample to be optimized in step S104 and obtaining a voice adversarial sample, the voice adversarial sample can be used to attack the target voiceprint recognition system in this step. Specifically, the voice adversarial sample can be input into the target voiceprint recognition system to achieve the attack on the target voiceprint recognition system.

[0090] In actual application scenarios, the voice adversarial samples input into the target voiceprint recognition system may exist in two different forms: one is directly input through the Application Programming Interface (API), that is, through the API channel input; the other is output through an audio playback device and then received by the target voiceprint recognition system, that is, through a physical channel input. Among them, directly inputting the voice adversarial samples into the target voiceprint recognition system can realize API channel input, while physical channel input is mostly manually played using audio playback devices in actual applications, which is more cumbersome. Therefore, in this method, a simulated physical adversarial sample is generated by scrambling to simulate the physical channel input.

[0091] Specifically, the adversarial sample can be scrambled using a preset scrambling method to obtain a simulated physical adversarial sample, which is then used to attack the target voiceprint recognition system. The scrambling method includes at least one of environmental noise interference, reverberation interference, and hardware interference.

[0092] Figure 2 illustrates the audio transmission process over a physical channel. As shown in Figure 2, audio transmission over a physical channel is typically subject to interference from three types of noise: hardware noise, reverberation, and environmental noise. Hardware noise originates from the non-uniform frequency response of hardware, such as players and microphones, and can enhance some frequency components of the signal passing through the hardware while attenuating others. Reverberation primarily arises from multipath propagation, where a linearly propagating signal and reflected signals overlap at the receiving device. Environmental noise originates from other sound sources besides the adversarial voice.

[0093] For the three different types of noise mentioned above, this method provides different transformation functions to scramble the speech adversarial samples. For environmental noise interference, the transformation function is designed to be the additive sum of speech and noise. Among them, the noise type may include but is not limited to Gaussian white noise, music noise, mixed noise of multi-speaker speech and noise in daily life (such as engine roar, bird calls, etc.), and the size of the noise is controlled by the signal-to-noise ratio of the noise and the speech adversarial sample. For reverberation interference, the transformation function is designed to be the convolution of the room excitation response and the speech adversarial sample. The room excitation response measures the reverberation of a space and is related to the absorption coefficient of the material of the objects in the space and the relative position of the receiving and playing devices. For hardware interference, the transformation function is designed to be the convolution of the hardware's excitation response and the speech adversarial sample. The hardware's excitation response measures the degree of the hardware's own non-uniform frequency response, and also represents the degree of interference.

[0094] Therefore, the input of the simulated physical channel can be realized by means of a transformation function instead of manual playback.

[0095] In actual applications, you can choose to use API channel input, simulated physical channel input, or a combination of the two based on specific needs. This manual does not impose specific restrictions on this.

[0096] S108: Determine an evaluation result of the target voiceprint recognition system according to the attack result of the voice adversarial sample on the target voiceprint recognition system.

[0097] After using the speech adversarial samples to attack the target voiceprint recognition system in step S106, the attack results can be obtained and used to determine the evaluation results of the target voiceprint recognition system. Generally, the attack results can refer to whether the target voiceprint recognition system correctly identifies each speech adversarial sample.

[0098] Furthermore, the evaluation results may include the adversarial success rate and the adversarial overhead. When determining the evaluation results, the attack success rate may be determined based on the number of voice adversarial samples that are incorrectly identified in the recognition results of the target voiceprint recognition system and the total number of voice adversarial samples input into the target voiceprint recognition system; the attack overhead may be determined based on the cost of generating the voice adversarial samples and the number of queries for the voice adversarial samples in the target voiceprint recognition system.

[0099] The attack success rate is used to represent the probability that the target voiceprint recognition system will fail to defend itself against an attack. A higher attack success rate indicates a lower security level for the target voiceprint recognition system, and vice versa. The attack cost is used to represent the cost of attacking the target voiceprint recognition system. A higher attack cost indicates a higher cost of attacking the target voiceprint recognition system, and vice versa.

[0100] The cost of generating a speech adversarial sample generally refers to the number of iterations required to solve the loss function during the process of adjusting the adversarial sample to be optimized. The higher the number of iterations, the higher the cost of generating the speech adversarial sample. The number of queries for the speech adversarial sample in the target voiceprint recognition system refers to the total number of recognition results obtained in the target voiceprint recognition system during the process of generating the speech adversarial sample. The number of iterations required to generate the speech adversarial sample and the number of queries for the speech adversarial sample in the target voiceprint recognition system together determine the attack cost. The attack cost is positively correlated with the number of iterations and queries; the higher the number of iterations and queries, the higher the attack cost.

[0101] It should be noted that the attack success rate is usually obtained based on the attack results of multiple speech adversarial samples, while the attack cost is obtained based on each speech adversarial sample separately.

[0102] When using the voiceprint recognition system evaluation method provided in this specification to evaluate a target voiceprint recognition system, different methods can be used to generate voice adversarial samples for scenarios corresponding to various attack methods under different recognition types. Ultimately, the voice adversarial samples are used to attack the target voiceprint recognition system and obtain the final evaluation results. This method can cover a variety of different attack methods when evaluating the target voiceprint recognition system, and obtain accurate and comprehensive evaluation results.

[0103] The above are one or more implementations of the voiceprint recognition system evaluation method of this specification. Based on the same idea, this specification also provides a corresponding voiceprint recognition system evaluation device, as shown in Figure 3.

[0104] Figure 3 is a schematic diagram of a voiceprint recognition system evaluation device provided in this specification, including: an acquisition module 200, used to obtain a target voiceprint recognition system to be evaluated and an adversarial sample to be optimized, and determine the recognition type of the target voiceprint recognition system; an input module 202, used to input the adversarial sample to be optimized into the target voiceprint recognition system, and obtain the recognition result of the target voiceprint recognition system on the adversarial sample to be optimized; an optimization module 204, used to adjust the adversarial sample to be optimized according to the recognition result, using a preset method corresponding to the recognition type, to obtain a voice adversarial sample; an attack module 206, used to attack the target voiceprint recognition system using the voice adversarial sample; a determination module 208, used to determine the evaluation result of the target voiceprint recognition system according to the attack result of the voice adversarial sample on the target voiceprint recognition system.

[0105] Optionally, the recognition type includes at least one of open-set speaker recognition, closed-set speaker recognition, and speaker confirmation, wherein the open-set speaker recognition is used to identify whether the original speaker is a user in the open-set registered speaker group, the closed-set speaker recognition is used to identify whether the original speaker is a user in the closed-set registered speaker group, and the speaker confirmation is used to identify whether the original speaker is a registered speaker.

[0106] Optionally, the recognition type is open-set speaker recognition, and the attack type of the speech adversarial sample to be obtained is an unrelated speaker impersonating the first designated user in the open-set registered speaker group; the optimization module 204 is specifically used to adjust the adversarial sample to be optimized with the highest probability of the adversarial sample to be identified as the first designated user in the target voiceprint recognition system as the optimization target.

[0107] Optionally, the recognition type is open-set speaker recognition, and the attack type of the speech adversarial sample to be obtained is a direct attack by an irrelevant speaker; the optimization module 204 is specifically used to adjust the adversarial sample to be optimized with the optimization goal that the probability that the adversarial sample to be optimized is identified as any user in the open-set registered speaker group in the target voiceprint recognition system is no more than a specified threshold.

[0108] Optionally, the recognition type is open-set speaker recognition, and the attack type of the speech adversarial sample to be obtained is that the second designated user in the open-set registered speaker group impersonates any other user in the open-set registered speaker group; the optimization module 204 is specifically used to adjust the adversarial sample to be optimized with the optimization goal of having the probability of the adversarial sample to be optimized being identified as the second designated user in the target voiceprint recognition system being the lowest and the probability of being identified as any other user being the highest.

[0109] Optionally, the recognition type is open-set speaker recognition, and the attack type of the speech adversarial sample to be obtained is that an unrelated speaker impersonates any user in the open-set registered speaker group; the optimization module 204 is specifically used to adjust the adversarial sample to be optimized with the highest probability of the adversarial sample to be identified as any user in the open-set registered speaker group in the target voiceprint recognition system as the optimization target.

[0110] Optionally, the recognition type is closed-set speaker recognition, and the attack type of the speech adversarial sample to be obtained is that any user in the closed-set registered speaker group impersonates a third designated user in the closed-set registered speaker group; the optimization module 204 is specifically used to adjust the adversarial sample to be optimized with the highest probability of the adversarial sample to be identified as the third designated user in the target voiceprint recognition system as the optimization target.

[0111] Optionally, the recognition type is closed-set speaker recognition, and the attack type of the speech adversarial sample to be obtained is that the fourth designated user in the closed-set registered speaker group impersonates any other user in the closed-set registered speaker group; the optimization module 204 is specifically used to adjust the adversarial sample to be optimized with the optimization goal of having the lowest probability of the adversarial sample to be identified as the fourth designated user in the target voiceprint recognition system and the highest probability of being identified as any other user in the closed-set registered speaker group.

[0112] Optionally, the recognition type is speaker confirmation, and the attack type of the speech adversarial sample to be obtained is the registered speaker test; the optimization module 204 is specifically used to adjust the adversarial sample to be optimized with the probability of the adversarial sample to be optimized being identified as the registered speaker in the target voiceprint recognition system as the lowest optimization goal.

[0113] Optionally, the recognition type is speaker confirmation, and the attack type of the speech adversarial sample to be obtained is that an unrelated speaker impersonates the registered speaker; the optimization module 204 is specifically used to adjust the adversarial sample to be optimized with the highest probability of the adversarial sample to be identified as the registered speaker in the target voiceprint recognition system as the optimization target.

[0114] Optionally, the optimization module 204 is specifically used to obtain the discrete sequence of the adversarial sample to be optimized; adjust the size of each numerical value in the discrete sequence of the adversarial sample to be optimized using a preset method corresponding to the recognition type to obtain a target discrete sequence; and determine the speech adversarial sample based on the target discrete sequence.

[0115] Optionally, the attack module 206 is specifically configured to scramble the adversarial sample in a preset scrambling manner to obtain a simulated physical adversarial sample; and use the simulated physical adversarial sample to attack the target voiceprint recognition system.

[0116] Optionally, the scrambling method includes at least one of environmental noise interference, reverberation interference, and hardware interference.

[0117] Optionally, the evaluation result includes the adversarial success rate and the adversarial overhead; the determination module 208 is specifically used to determine the attack success rate based on the number of speech adversarial samples that are incorrectly identified in the recognition results of the target voiceprint recognition system and the total number of speech adversarial samples input into the target voiceprint recognition system; and determine the attack overhead based on the cost of generating the speech adversarial samples and the number of queries for the speech adversarial samples in the target voiceprint recognition system.

[0118] This specification also provides a computer-readable storage medium, which stores a computer program. The computer program can be used to execute a voiceprint recognition system evaluation method provided in FIG1 above.

[0119] This specification also provides a schematic structural diagram of an electronic device corresponding to Figure 1, as shown in Figure 4. As shown in Figure 4, at the hardware level, the electronic device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory, and of course may also include hardware required for other services. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to implement the voiceprint recognition system evaluation method described in Figure 1 above. Of course, in addition to software implementation, this specification does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc., that is, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0120] In the 1990s, technological improvements could be clearly distinguished as either hardware improvements (for example, improvements to circuit structures like diodes, transistors, and switches) or software improvements (improvements to process flows). However, with the advancement of technology, many process flow improvements today can now be considered direct improvements to hardware circuit structures. Designers almost always create the corresponding hardware circuit structure by programming the improved process flow into the hardware circuit. Therefore, it cannot be said that a process flow improvement cannot be implemented using hardware modules. For example, a programmable logic device (PLD), such as a field programmable gate array (FPGA), is an integrated circuit whose logical function is determined by user programming. Designers can "integrate" a digital system on a PLD by programming it themselves, without having to hire a chip manufacturer to design and manufacture a dedicated integrated circuit chip. Moreover, nowadays, instead of manually fabricating integrated circuit chips, this programming is mostly done using "logic compiler" software. This is similar to the software compiler used when developing programs. Before compilation, the original code must also be written in a specific programming language, called a hardware description language (HDL). There is not just one HDL, but many, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art will also understand that by simply programming the method flow in one of these hardware description languages ​​and then programming it into an integrated circuit, a hardware circuit that implements the logic method flow can be easily obtained.

[0121] The controller can be implemented in any suitable manner. For example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also know that in addition to implementing the controller in a purely computer-readable program code format, the controller can be implemented in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, such a controller can be considered a hardware component, and the devices included therein for implementing various functions can also be considered as structures within the hardware component. Or even, the devices for implementing various functions can be considered as both software modules that implement the method and structures within the hardware component.

[0122] The systems, devices, modules, or units described in the above embodiments may be implemented by computer chips or entities, or by products having certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0123] For the convenience of description, the above devices are described as being divided into various units according to their functions. Of course, when implementing this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0124] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems, or computer program products. Therefore, this specification may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0125] This specification is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of this specification. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0126] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a product including an instruction device that implements the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0127] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0128] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0129] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0130] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0131] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0132] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems, or computer program products. Thus, this specification may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0133] This specification may be described in the general context of computer-executable instructions, such as program modules, executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, and the like that perform specific tasks or implement specific abstract data types. This specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media, including storage devices.

[0134] The various embodiments in this specification are described in a progressive manner. Similar parts between the various embodiments can be referred to in conjunction with each other. Each embodiment focuses on the differences between the other embodiments. In particular, the system embodiments are generally similar to the method embodiments, so the description is relatively simple. For relevant parts, refer to the description of the method embodiments.

[0135] The foregoing is merely an example of the present invention and is not intended to limit the present invention. Various modifications and variations are possible within the scope of the present invention. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be included within the scope of the claims of the present invention.

Claims

1. A voiceprint recognition system evaluation method, comprising: Obtaining a target voiceprint recognition system to be evaluated and an adversarial sample to be optimized, and determining a recognition type of the target voiceprint recognition system; Inputting the adversarial sample to be optimized into the target voiceprint recognition system to obtain a recognition result of the adversarial sample to be optimized by the target voiceprint recognition system; According to the recognition result, the adversarial sample to be optimized is adjusted in a preset manner corresponding to the recognition type to obtain a speech adversarial sample; Using the voice adversarial sample to attack the target voiceprint recognition system; An evaluation result of the target voiceprint recognition system is determined according to the attack result of the speech adversarial sample on the target voiceprint recognition system.

2. The method according to claim 1, wherein the recognition type comprises at least one of open-set speaker recognition, closed-set speaker recognition, and speaker verification, wherein: The open-set speaker recognition is used to identify whether the original speaker is a user in the open-set registered speaker group, the closed-set speaker recognition is used to identify whether the original speaker is a user in the closed-set registered speaker group, and the speaker confirmation is used to identify whether the original speaker is a registered speaker.

3. The method according to claim 2, wherein the recognition type is open-set speaker recognition, and the attack type of the speech adversarial sample to be obtained is that an unrelated speaker impersonates the first designated user in the open-set registered speaker group; The adversarial sample to be optimized is adjusted in a preset manner corresponding to the recognition type, specifically including: The probability that the adversarial sample to be optimized is recognized as the first designated user in the target voiceprint recognition system is the highest as an optimization goal, and the adversarial sample to be optimized is adjusted.

4. The method according to claim 2, wherein the recognition type is open set speaker recognition, and the attack type of the speech adversarial sample to be obtained is a direct attack by an irrelevant speaker; The adversarial sample to be optimized is adjusted in a preset manner corresponding to the recognition type, specifically including: The adversarial sample to be optimized is adjusted with the probability that the adversarial sample to be optimized is recognized as any user in the open set of registered speakers in the target voiceprint recognition system being no greater than a specified threshold as an optimization goal.

5. The method of claim 2, wherein the recognition type is open-set speaker recognition, and the attack type of the speech adversarial sample to be obtained is that a second designated user in the open-set registered speaker group impersonates any other user in the open-set registered speaker group; The adversarial sample to be optimized is adjusted in a preset manner corresponding to the recognition type, specifically including: The probability that the adversarial sample to be optimized is identified as the second designated user in the target voiceprint recognition system is the lowest, and the probability that it is identified as any other user is the highest as the optimization goal, This adjustment is made.

6. The method according to claim 2, wherein the recognition type is open-set speaker recognition, and the attack type of the speech adversarial sample to be obtained is an unrelated speaker impersonating any user in the open-set registered speaker group; The adversarial sample to be optimized is adjusted in a preset manner corresponding to the recognition type, specifically including: The probability that the adversarial sample to be optimized is recognized as any user in the open set of registered speakers in the target voiceprint recognition system is the highest as an optimization goal, and the adversarial sample to be optimized is adjusted.

7. The method according to claim 2, wherein the recognition type is closed-set speaker recognition, and the attack type of the speech adversarial sample to be obtained is that any user in the closed-set registered speaker group impersonates a third designated user in the closed-set registered speaker group; The adversarial sample to be optimized is adjusted in a preset manner corresponding to the recognition type, specifically including: The probability that the adversarial sample to be optimized is recognized as the third designated user in the target voiceprint recognition system is the highest as an optimization goal, and the adversarial sample to be optimized is adjusted.

8. The method of claim 2, wherein the recognition type is closed-set speaker recognition, and the attack type of the speech adversarial sample to be obtained is that a fourth designated user in the closed-set registered speaker group impersonates any other user in the closed-set registered speaker group; The adversarial sample to be optimized is adjusted in a preset manner corresponding to the recognition type, specifically including: The adversarial sample to be optimized is adjusted with the lowest probability of being identified as the fourth designated user in the target voiceprint recognition system and the highest probability of being identified as any other user in the closed set of registered speakers as the optimization goal.

9. The method according to claim 2, wherein the recognition type is speaker confirmation, and the attack type of the speech adversarial sample to be obtained is the registered speaker test; The adversarial sample to be optimized is adjusted in a preset manner corresponding to the recognition type, specifically including: The probability that the adversarial sample to be optimized is recognized as the registered speaker in the target voiceprint recognition system is minimized as an optimization goal, and the adversarial sample to be optimized is adjusted.

10. The method according to claim 2, wherein the recognition type is speaker confirmation, and the attack type of the speech adversarial sample to be obtained is an unrelated speaker impersonating the registered speaker; The adversarial sample to be optimized is adjusted in a preset manner corresponding to the recognition type, specifically including: The probability that the adversarial sample to be optimized is recognized as the registered speaker in the target voiceprint recognition system is the highest as the optimization goal, and the adversarial sample to be optimized is adjusted.

11. The method according to claim 1, wherein the adversarial sample to be optimized is adjusted in a preset manner corresponding to the recognition type to obtain a speech adversarial sample, specifically comprising: Obtaining a discrete sequence of adversarial samples to be optimized; The values ​​of each item in the discrete sequence of the adversarial sample to be optimized are adjusted in a preset manner corresponding to the recognition type to obtain a target discrete sequence; The speech adversarial sample is determined according to the target discrete sequence.

12. The method according to claim 1, using the adversarial sample to attack the target voiceprint recognition system, specifically comprising: Scrambling the adversarial sample in a preset scrambling manner to obtain a simulated physical adversarial sample; The target voiceprint recognition system is attacked using the simulated physical adversarial sample.

13. The method according to claim 12, wherein the scrambling method comprises at least one of environmental noise interference, reverberation interference, and hardware interference.

14. The method according to claim 1, wherein the evaluation result includes a confrontation success rate and a confrontation cost; Determining an evaluation result of the target voiceprint recognition system according to the attack result of the voice adversarial sample on the target voiceprint recognition system, specifically including: Determining the attack success rate according to the number of voice adversarial samples that are incorrectly identified in the recognition result of the target voiceprint recognition system and the total number of voice adversarial samples input into the target voiceprint recognition system; The attack cost is determined according to the cost of generating the speech adversarial sample and the number of queries of the speech adversarial sample in the target voiceprint recognition system.

15. A voiceprint recognition system evaluation device, comprising: An acquisition module, used to acquire a target voiceprint recognition system to be evaluated and an adversarial sample to be optimized, and determine the recognition type of the target voiceprint recognition system; An input module, used for inputting the adversarial sample to be optimized into the target voiceprint recognition system to obtain a recognition result of the adversarial sample to be optimized by the target voiceprint recognition system; An optimization module, configured to adjust the adversarial sample to be optimized in a preset manner corresponding to the recognition type according to the recognition result to obtain a speech adversarial sample; An attack module, used to attack the target voiceprint recognition system using the voice adversarial sample; A determination module is used to determine an evaluation result of the target voiceprint recognition system according to the attack result of the speech adversarial sample on the target voiceprint recognition system.

16. A computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the method according to any one of claims 1 to 14.

17. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method according to any one of claims 1 to 14 when executing the program.

Images