Audio processing agent robustness evaluation method and related device

Abstract

The invention belongs to the technical field of audio processing, and discloses an audio processing agent robustness evaluation method and a related device, and the method comprises the steps: obtaining an alternative agent of a to-be-evaluated audio processing agent and a plurality of original audio samples; according to the plurality of original audio samples and a plurality of pre-constructed attack vectors, obtaining a plurality of confrontation audio samples through an alternative agent of the to-be-evaluated audio processing agent; inputting the plurality of adversarial audio samples into a preset audio processing agent to obtain actual outputs of the plurality of adversarial audio samples; and according to the actual output and the expected output of the plurality of adversarial audio samples, obtaining the attack success rate of the plurality of adversarial audio samples, and taking the attack success rate as an audio processing agent robustness evaluation result. The robustness of the audio processing agent is comprehensively tested in multiple scenes, comprehensive and accurate analysis can be achieved, then safety deployment and safety risk control of the audio processing agent are assisted, and reliability and safety of audio processing in actual application are guaranteed.

Description

Technical Field

[0001] The present invention belongs to the technical field of audio processing, and relates to a robustness evaluation method for an audio processing intelligent agent and a related device. Background Art

[0002] Multimodal large models have demonstrated excellent performance in natural language processing, image generation, and speech processing, and have become a hot topic in current artificial intelligence research. By using large-scale data sets, attention mechanisms, and multimodal encoders, large models can capture the features and contextual relationships in inputs such as text, images, and speech, and complete complex tasks in a variety of scenarios. However, with the popularity of multimodal large models, researchers have also found that the security of multimodal large models faces new threats. Attackers can insert hidden misleading information into normal input sentences, speech, and images to generate adversarial samples, causing the large model to produce the response expected by the attacker. The problem of multimodal adversarial samples has threatened the private assets and safe operation of large models.

[0003] The emergence of large audio models has made audio-based attacks a more covert and effective attack method. First, compared with attacks in other modes, the concealment of audio perturbation attacks is greatly enhanced. Audio perturbations can be designed in the frequency domain where the human ear is not sensitive, making it difficult or even impossible for the human ear to detect audio perturbations. Secondly, audio-based attacks can affect large audio models in a large range while maintaining concealment. The applied audio perturbations can be broadcast through independent audio playback devices, thereby simultaneously affecting all large audio models within the broadcast range without the presence of hardware or network connections. Finally, this attack method can cause the model to fail to work properly, the meaning of the input speech to be distorted, and even the output of the model to be manipulated, which will seriously affect the security and reliability of the large audio model.

[0004] As an advanced application based on a large audio model, the core function of audio processing agents relies on the accurate recognition, processing and generation of audio signals. In complex and changeable practical application environments, audio signals are often accompanied by various interferences and noises. The robustness of audio processing agents is directly related to their stability and reliability in practical applications. Therefore, how to accurately evaluate the robustness of audio processing agents is crucial to the safe deployment and security risk control of audio processing agents. However, due to the difficulty in quantifying the distinction between original audio and adversarial audio, existing evaluation methods often use the total energy of adversarial audio perturbations to characterize the difference between original audio and adversarial attack audio. This evaluation method rarely considers the difference in audio frequency and is difficult to correctly reflect the human ear's perception of audio similarity. Combined with the diversity of audio agent tasks, the existing multimodal large model robustness evaluation methods lack credibility and comprehensiveness when evaluating the robustness of audio processing agents. Summary of the invention

[0005] The purpose of the present invention is to overcome the above-mentioned shortcomings of the prior art and provide an audio processing agent robustness evaluation method and related devices.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] In a first aspect, the present invention provides a method for evaluating the robustness of an audio processing agent, comprising: obtaining a substitute agent for the audio processing agent to be evaluated and a number of original audio samples; obtaining a number of adversarial audio samples through the substitute agent of the audio processing agent to be evaluated based on the number of original audio samples and a number of pre-constructed attack vectors; wherein the attack vector is a triple consisting of a disturbance source, a threat module and an attack target; inputting the number of adversarial audio samples into a preset audio processing agent to obtain actual outputs of the number of adversarial audio samples; and obtaining the attack success rate of the number of adversarial audio samples based on the actual outputs and expected outputs of the number of adversarial audio samples as a result of the robustness evaluation of the audio processing agent.

[0008] Optionally, the disturbance sources include: malicious audio playback process, malicious audio sender and malicious audio broadcaster; the threat module includes: perception module, reasoning module, memory module and cooperation module of the audio processing agent; the attack targets include: confidentiality, integrity and availability of the audio processing agent.

[0009] Optionally, the method of obtaining a substitute agent for the audio processing agent to be evaluated includes: in a white box attack scenario, using the audio processing agent to be evaluated as a substitute agent for the audio processing agent to be evaluated; in a black box attack scenario, obtaining an audio processing agent that has the same type of input and output as the audio processing agent to be evaluated and is trained with the same data set as the data set of the audio processing agent to be evaluated or with a similarity less than a preset similarity threshold, as a substitute agent for the audio processing agent to be evaluated; or, obtaining an audio processing agent that is fine-tuned on the basic agent model of the audio processing agent to be evaluated as a substitute agent for the audio processing agent to be evaluated.

[0010] Optionally, the method of obtaining several adversarial audio samples through a substitute agent of the audio processing agent to be evaluated based on several original audio samples and several pre-constructed attack vectors includes: constructing the expected output of the audio processing agent to be evaluated under the current attack vector; performing a perturbation step on the current original audio sample to a predetermined number of iterations to obtain the current adversarial audio sample; wherein the perturbation step includes: inputting the current original audio sample into the substitute agent to obtain a substitute output of the current original audio sample; and obtaining an output loss value of the current original audio sample based on the substitute output of the current original audio sample and the expected output of the audio processing agent to be evaluated under the current attack vector; and calculating the gradient matrix of the output loss value with respect to the current original audio sample through a back propagation mechanism based on the output loss value of the current original audio sample, and superimposing a perturbation of a preset step size on the current original audio sample with the gradient matrix as the perturbation direction; traversing several original audio samples and several pre-constructed attack vectors, and performing the above steps respectively to obtain several adversarial audio samples.

[0011] Optionally, the disturbance of the preset step size is obtained in the following manner: taking the gradient matrix as the disturbance direction, adopting the adversarial attack algorithm to obtain an unnormalized disturbance signal; inputting the unnormalized disturbance signal into the Mel filter to obtain a disturbance signal of the enhanced key spectrum; calculating the total energy of the disturbance signal of the enhanced key spectrum to obtain the step size of the existing disturbance; obtaining the ratio of the preset disturbance step size to the step size of the existing disturbance as a scaling coefficient, multiplying the unnormalized disturbance signal by the scaling coefficient to obtain the disturbance of the preset step size.

[0012] Optionally, superimposing a disturbance of a preset step length on the current original audio sample includes: directly adding the current original audio sample and the disturbance of the preset step length in the form of vector addition; or, receiving the current original audio sample and the disturbance of the preset step length played simultaneously in a real environment.

[0013] Optionally, obtaining the attack success rate of several adversarial audio samples based on the actual output and expected output of several adversarial audio samples includes: obtaining the semantic similarity between the actual output and the expected output of the current adversarial audio sample, when the semantic similarity is greater than a preset semantic similarity threshold, marking the current adversarial audio sample as a successful attack; and counting the proportion of adversarial audio samples marked as successful attacks to all adversarial audio samples to obtain the attack success rate of several adversarial audio samples.

[0014] According to a second aspect of the present invention, a method for evaluating the robustness of an audio processing agent is provided, comprising: a data acquisition module for acquiring a substitute agent of the audio processing agent to be evaluated and a number of original audio samples; an adversarial sample construction module for obtaining a number of adversarial audio samples through a substitute agent of the audio processing agent to be evaluated based on a number of original audio samples and a number of pre-constructed attack vectors; wherein the attack vector is a triple consisting of a disturbance source, a threat module and an attack target; an evaluation module for inputting a number of adversarial audio samples into a preset audio processing agent to obtain actual outputs of the number of adversarial audio samples; and obtaining the attack success rate of the number of adversarial audio samples based on the actual outputs and expected outputs of the number of adversarial audio samples as a result of the robustness evaluation of the audio processing agent.

[0015] According to a third aspect of the present invention, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned audio processing agent robustness evaluation method when executing the computer program.

[0016] According to a fourth aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned audio processing agent robustness evaluation method are implemented.

[0017] Compared with the prior art, the present invention has the following beneficial effects:

[0018] The robustness evaluation method of the audio processing agent of the present invention realizes the generation of several adversarial audio samples based on several pre-constructed attack vectors, several original audio samples and alternative agents of the audio processing agent to be evaluated, and then inputs the several adversarial audio samples into the preset audio processing agent to obtain the actual output of the several adversarial audio samples, and finally obtains the attack success rate of the several adversarial audio samples according to the actual output and expected output of the several adversarial audio samples as the robustness evaluation result of the audio processing agent. Among them, the attack vector is a triple composed of the disturbance source, the threat module and the attack target, which covers the diverse audio adversarial attack scenarios to the greatest extent, and then by constructing the adversarial audio samples based on the attack vector, the robustness of the audio processing agent facing the adversarial audio samples can be comprehensively tested in various scenarios, and the attack effects of various audio attacks on the audio processing agent can be comprehensively evaluated, thereby realizing a comprehensive and accurate analysis of the robustness of the audio processing agent, and then assisting the safe deployment and security risk control of the audio processing agent, and ensuring the reliability and security of audio processing in actual application. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1This is a flow chart of a method for evaluating the robustness of an audio processing agent according to an embodiment of the present invention.

[0020] Figure 2 Schematic diagram of the adversarial audio sample generation principle according to an embodiment of the present invention.

[0021] Figure 3 This is a structural block diagram of the audio processing agent robustness evaluation system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0022] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0023] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0024] The present invention is further described in detail below in conjunction with the accompanying drawings:

[0025] See also Figure 1 In one embodiment of the present invention, a method for evaluating the robustness of an audio processing agent is provided, which effectively solves the shortcomings of existing robustness evaluation methods in terms of lack of credibility and comprehensiveness in evaluating the robustness of audio processing agents, and achieves accurate and comprehensive evaluation of the robustness of audio processing agents.

[0026] Specifically, the method for evaluating the robustness of an audio processing agent of the present invention comprises the following steps:

[0027] S1: Obtain a substitute agent for the audio processing agent to be evaluated and several original audio samples.

[0028] S2: Based on several original audio samples and several pre-constructed attack vectors, several adversarial audio samples are obtained by replacing the audio processing agent to be evaluated; among which, the attack vector is a triple consisting of the disturbance source, the threat module and the attack target.

[0029] S3: Input several adversarial audio samples into a preset audio processing agent to obtain the actual output of several adversarial audio samples; and obtain the attack success rate of several adversarial audio samples based on the actual output and expected output of several adversarial audio samples as the robustness evaluation result of the audio processing agent.

[0030] The robustness evaluation method of the audio processing agent of the present invention realizes the generation of several adversarial audio samples based on several pre-constructed attack vectors, several original audio samples and alternative agents of the audio processing agent to be evaluated, and then inputs the several adversarial audio samples into the preset audio processing agent to obtain the actual output of the several adversarial audio samples, and finally obtains the attack success rate of the several adversarial audio samples according to the actual output and expected output of the several adversarial audio samples as the robustness evaluation result of the audio processing agent. Among them, the attack vector is a triple composed of the disturbance source, the threat module and the attack target, which covers the diverse audio adversarial attack scenarios to the greatest extent, and then by constructing the adversarial audio samples based on the attack vector, the robustness of the audio processing agent facing the adversarial audio samples can be comprehensively tested in various scenarios, and the attack effects of various audio attacks on the audio processing agent can be comprehensively evaluated, thereby realizing a comprehensive and accurate analysis of the robustness of the audio processing agent, and then assisting the safe deployment and security risk control of the audio processing agent, and ensuring the reliability and security of audio processing in actual application.

[0031] In a possible implementation, the disturbance sources include: a malicious audio playback process, a malicious audio sender, and a malicious audio broadcaster; the threat module includes: a perception module, a reasoning module, a memory module, and a cooperation module of an audio processing agent; and the attack targets include: confidentiality, integrity, and availability of the audio processing agent.

[0032] Explanatory: malicious audio playback process, develop a process dedicated to covertly playing malicious audio, users may mistakenly download such programs and suffer attacks; malicious audio sender, publish malicious audio on the network or certain applications, agents may mistakenly access such audio and suffer attacks; malicious audio broadcaster, broadcast malicious audio in private or public places, agents may mistakenly receive such audio and suffer attacks. Confidentiality, steal audio processing agents or user assets; integrity, cause specific output or behavior of audio processing agents to be abnormal; availability, make audio processing agents unavailable.

[0033] Explanatory, the attack vector can be composed of any instance of a disturbance source, a threat module, and an attack target. For example, the attack vector can be designed as a triple in the form of: (malicious audio playback process development, reasoning module, confidentiality). By combining disturbance sources, threat modules, and attack targets, etc., a comprehensive attack vector list is obtained. On this basis, according to the alternative agent of the audio processing agent to be evaluated, for each attack vector, an adversarial audio sample is iteratively generated for each original audio sample.

[0034] In one possible implementation, the method of obtaining a substitute agent for the audio processing agent to be evaluated includes: in a white box attack scenario, using the audio processing agent to be evaluated as a substitute agent for the audio processing agent to be evaluated; in a black box attack scenario, obtaining an audio processing agent that has the same type of input and output as the audio processing agent to be evaluated and is trained with the same data set as the data set of the audio processing agent to be evaluated or with a similarity less than a preset similarity threshold, as a substitute agent for the audio processing agent to be evaluated; or, obtaining an audio processing agent that is fine-tuned on the basic agent model of the audio processing agent to be evaluated as a substitute agent for the audio processing agent to be evaluated.

[0035] Exemplarily, before applying an alternative agent to the audio processing agent to be evaluated, a data set for training the alternative agent to the audio processing agent to be evaluated is obtained, including but not limited to audio instruction samples and corresponding expected responses; then the alternative agent is trained using the training set of the alternative agent to the audio processing agent to be evaluated until the accuracy of the alternative agent on the test set is no less than a preset accuracy threshold.

[0036] In one possible implementation, see Figure 2 , the method of obtaining several adversarial audio samples through a substitute agent of the audio processing agent to be evaluated based on several original audio samples and several pre-constructed attack vectors includes: constructing the expected output of the audio processing agent to be evaluated under the current attack vector; performing a perturbation step on the current original audio sample to a predetermined number of iterations to obtain the current adversarial audio sample; wherein the perturbation step includes: inputting the current original audio sample into the substitute agent to obtain a substitute output of the current original audio sample; and obtaining an output loss value of the current original audio sample based on the substitute output of the current original audio sample and the expected output of the audio processing agent to be evaluated under the current attack vector; and calculating the gradient matrix of the output loss value with respect to the current original audio sample through a back propagation mechanism based on the output loss value of the current original audio sample, and superimposing a perturbation of a preset step size on the current original audio sample with the gradient matrix as the perturbation direction; traversing several original audio samples and several pre-constructed attack vectors, and performing the above steps respectively to obtain several adversarial audio samples.

[0037] Explanatory, the target output of the audio processing agent is determined by the threat module and the attack target type. The corresponding expected output of the audio processing agent to be evaluated can be obtained through the determined attack vector.

[0038] Exemplarily, taking the confidentiality of the reasoning module as an example, a possible expected output can be set as a system prompt word for the audio processing agent.

[0039] Specifically, when a number of adversarial audio samples are obtained through adversarial attack, any adversarial attack algorithm can be used. In this implementation, the classic adversarial attack algorithm I-FGSM is used as an example to introduce the process:

[0040] S401: First input the original audio sample, obtain the output of the alternative agent, and calculate the output loss value compared with the expected output. S402: Then use the back propagation mechanism to calculate the gradient matrix of the output loss value with respect to the input original audio sample. S403: Use the direction of the calculated gradient matrix as the perturbation direction, superimpose the perturbation on the original audio sample with a fixed step size, and then return to S401, and iterate repeatedly until the predetermined number of iterations is completed.

[0041] In a possible implementation, the disturbance of the preset step size is obtained in the following manner: taking the gradient matrix as the disturbance direction, adopting the adversarial attack algorithm to obtain an unnormalized disturbance signal; inputting the unnormalized disturbance signal into the Mel filter to obtain a disturbance signal of the enhanced key spectrum; calculating the total energy of the disturbance signal of the enhanced key spectrum to obtain the step size of the existing disturbance; obtaining the ratio of the preset disturbance step size to the step size of the existing disturbance as a scaling coefficient, multiplying the unnormalized disturbance signal by the scaling coefficient, and obtaining the disturbance of the preset step size.

[0042] Explanatory, the perturbation step size, that is, the sum of the energies of specific spectral components of the perturbation audio, is designed to be a fixed value, that is, a preset perturbation step size, in this embodiment.

[0043] In a possible implementation, superimposing a disturbance of a preset step size on the current original audio sample includes: directly adding the current original audio sample and the disturbance of the preset step size in the form of vector addition; or, receiving the current original audio sample and the disturbance of the preset step size played simultaneously in a real environment.

[0044] Explanatory, superimposing disturbances on the original audio sample includes directly superimposing disturbances and superimposing disturbances in a real physical environment. Specifically, the direct superimposing disturbance means directly adding the original audio sample and the disturbance in the form of vector addition in a computer device. The superimposing disturbance in a real physical environment means playing the original audio sample and the disturbance separately through an audio playback device (such as a speaker), and after superimposing in a real physical environment, receiving them through an audio receiving device (such as a microphone).

[0045] In a possible implementation, obtaining the attack success rate of several adversarial audio samples based on actual outputs and expected outputs of several adversarial audio samples includes: obtaining the semantic similarity between the actual output and the expected output of the current adversarial audio sample, and when the semantic similarity is greater than a preset semantic similarity threshold, marking the current adversarial audio sample as a successful attack; and calculating the proportion of adversarial audio samples marked as successful attacks to all adversarial audio samples to obtain the attack success rate of several adversarial audio samples.

[0046] The robustness evaluation method of the audio processing agent of the present invention is essentially to comprehensively evaluate the robustness of the audio processing agent against the multimodal large model adversarial attack based on audio from the perspective of designing attack vectors. Specifically, the attack vector is designed and adversarial audio samples are generated according to the designed attack vectors. The robustness of the audio processing agent to be evaluated against the adversarial audio samples is comprehensively tested in a variety of scenarios to assist the safe deployment and security risk control of the audio processing agent to be evaluated.

[0047] In one possible implementation, it has been experimentally verified that even if the model components, model structures, and model types of the audio processing agents to be evaluated are different, the present invention can achieve a good attack success rate on the vast majority of audio processing agents, thereby obtaining a more reliable adversarial robustness evaluation result.

[0048] The following are device embodiments of the present invention, which can be used to implement the method embodiments of the present invention. For details not disclosed in the device embodiments, please refer to the method embodiments of the present invention.

[0049] See also Figure 3 In another embodiment of the present invention, an audio processing agent robustness evaluation system is provided, which can be used to implement the above-mentioned audio processing agent robustness evaluation method. Specifically, the audio processing agent robustness evaluation system includes a data acquisition module, an adversarial sample construction module and an evaluation module.

[0050] Among them, the data acquisition module is used to obtain a substitute agent for the audio processing agent to be evaluated and several original audio samples; the adversarial sample construction module is used to obtain several adversarial audio samples through the substitute agent of the audio processing agent to be evaluated based on several original audio samples and several pre-constructed attack vectors; wherein the attack vector is a triple consisting of a disturbance source, a threat module and an attack target; the evaluation module is used to input several adversarial audio samples into a preset audio processing agent to obtain the actual output of several adversarial audio samples; and according to the actual output and expected output of several adversarial audio samples, the attack success rate of several adversarial audio samples is obtained as the robustness evaluation result of the audio processing agent.

[0051] All relevant contents of each step involved in the embodiment of the aforementioned audio processing agent robustness evaluation method can be referred to the functional description of the functional modules corresponding to the audio processing agent robustness evaluation system in the embodiment of the present invention, and will not be repeated here.

[0052] The division of modules in the embodiments of the present invention is schematic and is only a logical function division. There may be other division methods in actual implementation. In addition, each functional module in each embodiment of the present invention may be integrated into one processor, or may exist physically separately, or two or more modules may be integrated into one module. The above-mentioned integrated modules may be implemented in the form of hardware or in the form of software functional modules.

[0053] In another embodiment of the present invention, a computer device is provided, the computer device including a processor and a memory, the memory is used to store a computer program, the computer program includes program instructions, and the processor is used to execute the program instructions stored in the computer storage medium. The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc., which are the computing core and control core of the terminal, which are suitable for implementing one or more instructions, and are specifically suitable for loading and executing one or more instructions in the computer storage medium to implement the corresponding method flow or corresponding function; the processor described in the embodiment of the present invention can be used for the operation of the robustness evaluation method of the audio processing agent.

[0054] In another embodiment of the present invention, the present invention also provides a storage medium, specifically a computer-readable storage medium (Memory), which is a memory device in a computer device for storing programs and data. It can be understood that the computer-readable storage medium here can include both built-in storage media in the computer device and, of course, extended storage media supported by the computer device. The computer-readable storage medium provides a storage space, which stores the operating system of the terminal. In addition, one or more instructions suitable for being loaded and executed by the processor are also stored in the storage space, and these instructions can be one or more computer programs (including program codes). It should be noted that the computer-readable storage medium here can be a high-speed RAM memory or a non-volatile memory, such as at least one disk memory. The processor can load and execute one or more instructions stored in the computer-readable storage medium to implement the corresponding steps of the method for evaluating the robustness of the audio processing agent in the above embodiment.

[0055] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0056] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0057] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0058] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0059] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A method for evaluating the robustness of an audio processing agent, characterized in that: include: Obtain a substitute agent for the audio processing agent to be evaluated and several original audio samples; According to several original audio samples and several pre-constructed attack vectors, several adversarial audio samples are obtained by replacing the audio processing agent to be evaluated; wherein the attack vector is a triple consisting of a disturbance source, a threat module and an attack target; Several adversarial audio samples are input into a preset audio processing agent to obtain the actual outputs of the several adversarial audio samples; and based on the actual outputs and expected outputs of the several adversarial audio samples, the attack success rates of the several adversarial audio samples are obtained as the robustness evaluation results of the audio processing agent.

2. The method for evaluating the robustness of an audio processing agent according to claim 1, characterized in that: The disturbance sources include: malicious audio playback process, malicious audio sender and malicious audio broadcaster; the threat module includes: perception module, reasoning module, memory module and cooperation module of the audio processing agent; the attack targets include: confidentiality, integrity and availability of the audio processing agent.

3. The method for evaluating the robustness of an audio processing agent according to claim 1, characterized in that: The alternative agents for obtaining the audio processing agent to be evaluated include: In the scenario of white box attack, the audio processing agent to be evaluated is used as a substitute agent for the audio processing agent to be evaluated; in the scenario of black box attack, an audio processing agent with the same type of input and output as the audio processing agent to be evaluated and trained with the same data set as the data set of the audio processing agent to be evaluated or with a similarity less than a preset similarity threshold is obtained as a substitute agent for the audio processing agent to be evaluated; alternatively, an audio processing agent fine-tuned on the basic agent model of the audio processing agent to be evaluated is obtained as a substitute agent for the audio processing agent to be evaluated.

4. The method for evaluating the robustness of an audio processing agent according to claim 1, characterized in that: The method of obtaining several adversarial audio samples based on several original audio samples and several pre-constructed attack vectors by replacing the audio processing agent to be evaluated includes: Construct the expected output of the audio processing agent to be evaluated under the current attack vector; The current original audio sample is subjected to a perturbation step to a predetermined number of iterations to obtain a current adversarial audio sample; wherein the perturbation step includes: inputting the current original audio sample into the replacement agent to obtain a replacement output of the current original audio sample; and obtaining an output loss value of the current original audio sample according to the replacement output of the current original audio sample and the expected output of the audio processing agent to be evaluated under the current attack vector; and according to the output loss value of the current original audio sample, calculating a gradient matrix of the output loss value with respect to the current original audio sample through a back-propagation mechanism, and superimposing a perturbation of a preset step length on the current original audio sample with the gradient matrix as the perturbation direction; Traverse several original audio samples and several pre-constructed attack vectors, and perform the above steps respectively to obtain several adversarial audio samples.

5. The method for evaluating the robustness of an audio processing agent according to claim 4, characterized in that: The disturbance of the preset step size is obtained by: Taking the gradient matrix as the disturbance direction, an adversarial attack algorithm is used to obtain an unnormalized disturbance signal; The unnormalized disturbance signal is input into the Mel filter to obtain the disturbance signal that strengthens the key spectrum; Calculate the total energy of the disturbance signal that strengthens the key spectrum and obtain the step size of the existing disturbance; The ratio of the preset disturbance step length to the existing disturbance step length is obtained as a scaling factor, and the unnormalized disturbance signal is multiplied by the scaling factor to obtain a disturbance with a preset step length.

6. The method for evaluating the robustness of an audio processing agent according to claim 1, characterized in that: The method of superimposing a disturbance of a preset step length on the current original audio sample comprises: Directly add the current original audio sample to the disturbance of the preset step size in the form of vector addition; Alternatively, receive the current original audio sample and the perturbation of a preset step size played simultaneously in the real environment.

7. The method for evaluating the robustness of an audio processing agent according to claim 1, characterized in that: The attack success rates of the plurality of adversarial audio samples are obtained according to the actual outputs and expected outputs of the plurality of adversarial audio samples, including: Obtaining semantic similarity between the actual output and the expected output of the current adversarial audio sample, and when the semantic similarity is greater than a preset semantic similarity threshold, marking the current adversarial audio sample as a successful attack; The proportion of adversarial audio samples marked as successful attacks to all adversarial audio samples is counted to obtain the attack success rate of several adversarial audio samples.

8. A method for evaluating the robustness of an audio processing agent, characterized in that: include: A data acquisition module, used to obtain a substitute agent of the audio processing agent to be evaluated and a number of original audio samples; The adversarial sample construction module is used to obtain several adversarial audio samples through a substitute agent of the audio processing agent to be evaluated based on several original audio samples and several pre-constructed attack vectors; wherein the attack vector is a triple consisting of a disturbance source, a threat module and an attack target; The evaluation module is used to input several adversarial audio samples into a preset audio processing agent to obtain the actual output of several adversarial audio samples; and obtain the attack success rate of several adversarial audio samples based on the actual output and expected output of several adversarial audio samples as the robustness evaluation result of the audio processing agent.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method for evaluating the robustness of an audio processing agent as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method for evaluating the robustness of an audio processing agent as described in any one of claims 1 to 7 are implemented.

Images