Method and apparatus for security defense of an audio language model

Abstract

The application provides a security defense method and device of an audio language model, the security defense method comprising: determining a sensitivity score of each mel-frequency unit based on the mel-frequency unit and model parameters of each audio sample data; determining mel- gradient sparse mask distribution data based on the sensitivity score; iteratively optimizing initial perturbation parameters based on the mel- gradient sparse mask distribution data and the model parameters to determine target perturbation parameters; in response to receiving real-time audio data, applying corresponding perturbation to the real-time audio data based on the target perturbation parameters and the mel- gradient sparse mask distribution data to determine a target mel-spectrogram, so that the target audio language model infers and generates response information based on the target mel-spectrogram. Through the above method, the audio language model effectively defends against jailbreaking attacks while stably retaining the performance of the audio language model on benign tasks, thereby improving the robustness and usability of the audio language model.

Description

Technical Field

[0001] This application relates to the field of audio language model technology, and in particular to a security defense method and device for audio language models. Background Technology

[0002] Currently, when dealing with jailbreak attacks targeting Audio Language Models (ALMs), existing technologies mainly migrate from areas such as jailbreak defense of large text language models and traditional audio adversarial defense.

[0003] Traditional audio anti-defense methods include Gaussian noise, local smoothing, and downsampling. These methods are effective against acoustic signal-based attacks but cannot defend against semantic-based attacks. Methods transferred from the field of large text language model defense perform well against semantic attacks but cannot effectively defend against acoustic attacks. The existing methods have a narrow and unbalanced defense scope, exhibit obvious bias, and cannot defend against adaptive attacks, thus reducing the robustness and performance of audio language models.

[0004] Furthermore, methods migrated from the field of text-based large language model defense can severely interfere with the audio language model's ability to understand normal speech. This means that even if the user inputs benign and harmless speech, the audio language model may not be able to correctly understand its semantics, reducing the usability of the audio language model. Summary of the Invention

[0005] In view of this, the purpose of this application is to provide a security defense method and apparatus for an audio language model. By optimizing the audio language model to generate a target perturbation parameter guided by a Mel gradient sparse mask, a general security path activation perturbation is determined. During the real-time inference stage of the audio language model, the target perturbation parameter and the Mel gradient sparse mask are applied to the Mel spectrogram corresponding to the input audio to activate the potential security path within the audio language model, thereby achieving effective defense of the audio language model against jailbreak attacks. At the same time, the performance of the audio language model on benign tasks is stably preserved, thereby improving the robustness and usability of the audio language model.

[0006] This application provides a security defense method for an audio language model, the security defense method comprising:

[0007] Acquire multiple preset audio sample data, and determine the sensitivity score corresponding to each of the multiple Mel frequency units corresponding to each audio sample data and the model parameters corresponding to the target audio language model.

[0008] determine a mel gradient sparse mask value corresponding to each of the mel frequency units based on the sensitivity score corresponding to each of the mel frequency units, to determine mel gradient sparse mask distribution data corresponding to the audio sample data;

[0009] based on the mel gradient sparse mask distribution data and the model parameters, iteratively optimize a preset initial perturbation parameter using a preset optimization function to determine a target perturbation parameter;

[0010] In response to the target audio language model receiving real-time audio data, based on the target perturbation parameter and the mel gradient sparse mask distribution data, a corresponding perturbation is applied to the real-time audio data to determine a target mel spectrum corresponding to the real-time audio data, so that the target audio language model infers to generate corresponding response information based on the target mel spectrum.

[0011] Further, the audio sample data is pre-set by the following steps:

[0012] Obtain a plurality of preset attack instruction text data, and convert each of the attack instruction text data into initial audio data using a preset text-to-speech conversion tool;

[0013] Each of the initial audio data is maliciously attacked and processed using a preset jailbreak attack method to generate audio sample data corresponding to each of the initial audio data.

[0014] Further, the model parameters at least include a security cross-entropy loss value and a model availability cross-entropy loss value; and the determination of the sensitivity score corresponding to each of the mel frequency units based on the plurality of mel frequency units corresponding to each of the audio sample data and the model parameters corresponding to the target audio language model comprises:

[0015] For each of the plurality of mel frequency units corresponding to each of the audio sample data, determine a security gradient sensitivity score corresponding to each of the mel frequency units based on the security cross-entropy loss value;

[0016] Determine an availability gradient sensitivity score corresponding to each of the mel frequency units based on the model availability cross-entropy loss value;

[0017] Determine a sensitivity score corresponding to each of the mel frequency units based on the security gradient sensitivity score and the availability gradient sensitivity score.

[0018] Further, the determination of the mel gradient sparse mask value corresponding to each of the mel frequency units based on the sensitivity score corresponding to each of the mel frequency units to determine the mel gradient sparse mask distribution data corresponding to the audio sample data comprises:

[0019] arranging the sensitivity scores corresponding to each of the mel frequency units in a numerical descending order to obtain a numerical descending order result;

[0020] based on the numerical descending order result, selecting a preset number of sensitivity scores from the first order in the sensitivity scores to determine the sensitivity scores as target sensitivity scores;

[0021] for the mel frequency unit corresponding to the target sensitivity score, determining the mel frequency unit as a first mel frequency unit, and determining the mel gradient sparse mask value corresponding to the first mel frequency unit as a first mask value;

[0022] determining the mel frequency unit corresponding to the unselected sensitivity score as a second mel frequency unit, and determining the mel gradient sparse mask value corresponding to the second mel frequency unit as a second mask value;

[0023] based on the first mask value corresponding to the first mel frequency unit and the second mask value corresponding to the second mel frequency unit, determining the mel gradient sparse mask distribution data corresponding to the audio sample data.

[0024] Further, based on the mel gradient sparse mask distribution data and the model parameters, the preset optimization function is used to iteratively optimize the preset initial perturbation parameter to determine the target perturbation parameter, including:

[0025] based on the mel gradient sparse mask distribution data, and the safety cross entropy loss value and the target safety recovery data in the model parameters, a target optimization model corresponding to the preset initial perturbation parameter is constructed using a preset optimization function;

[0026] based on the target optimization model, the initial perturbation parameter is iteratively optimized using a preset projection gradient descent method until the number of iterations reaches a preset iteration number to determine the target perturbation parameter.

[0027] Further, in response to the target audio language model receiving real-time audio data, based on the target perturbation parameter and the mel gradient sparse mask distribution data, the corresponding perturbation is applied to the real-time audio data to determine the target mel spectrum corresponding to the real-time audio data, including:

[0028] in response to the target audio language model receiving real-time audio data, converting the real-time audio data to obtain a mel spectrum corresponding to the real-time audio data;

[0029] apply a corresponding perturbation to the mel spectrogram based on the target perturbation parameter and the mel gradient sparse mask distribution data, to determine a target mel spectrogram corresponding to the mel spectrogram.

[0030] Further, the applying a corresponding perturbation to the mel spectrogram based on the target perturbation parameter and the mel gradient sparse mask distribution data, to determine a target mel spectrogram corresponding to the mel spectrogram, comprises:

[0031] performing element-wise multiplication on the target perturbation parameter and the mel gradient sparse mask distribution data to obtain a mask product result;

[0032] performing element-wise addition on the mask product result and the mel spectrogram to determine the target mel spectrogram.

[0033] Embodiments of the present application also provide a security defense device of an audio language model, the security defense device comprising:

[0034] a sensitivity calculation module configured to obtain a plurality of preset audio sample data, and determine a sensitivity score corresponding to each mel frequency unit based on a plurality of mel frequency units corresponding to each of the audio sample data and model parameters corresponding to a target audio language model;

[0035] a mask determination module configured to determine a mel gradient sparse mask value corresponding to each of the mel frequency units based on the sensitivity score corresponding to each of the mel frequency units, to determine mel gradient sparse mask distribution data corresponding to the audio sample data;

[0036] a perturbation optimization module configured to perform iterative optimization on a preset initial perturbation parameter by using a preset optimization function based on the mel gradient sparse mask distribution data and the model parameters, to determine a target perturbation parameter;

[0037] a model defense module configured to, in response to the target audio language model receiving real-time audio data, apply a corresponding perturbation to the real-time audio data based on the target perturbation parameter and the mel gradient sparse mask distribution data, to determine a target mel spectrogram corresponding to the real-time audio data, so that the target audio language model infers to generate corresponding response information based on the target mel spectrogram.

[0038] Further, the sensitivity calculation module is configured to, when the audio sample data is preset:

[0039] obtain a plurality of preset attack instruction text data, and convert each of the attack instruction text data into initial audio data by using a preset text-to-speech conversion tool;

[0040] The preset jailbreak attack method is used to maliciously attack and process each initial audio data to generate audio sample data corresponding to each initial audio data.

[0041] Further, the model parameters at least include a safety cross-entropy loss value and a model availability cross-entropy loss value; when the sensitive calculation module is used to determine a sensitivity score corresponding to each mel frequency unit based on a plurality of mel frequency units corresponding to each audio sample data and model parameters corresponding to a target audio language model, the sensitive calculation module is used to:

[0042] For a plurality of mel frequency units corresponding to each audio sample data, determine a safety gradient sensitivity score corresponding to each mel frequency unit based on the safety cross-entropy loss value;

[0043] Determine an availability gradient sensitivity score corresponding to each mel frequency unit based on the model availability cross-entropy loss value;

[0044] Determine a sensitivity score corresponding to each mel frequency unit based on the safety gradient sensitivity score and the availability gradient sensitivity score.

[0045] Further, when the mask determination module is used to determine a mel gradient sparse mask value corresponding to each mel frequency unit based on the sensitivity score corresponding to each mel frequency unit to determine mel gradient sparse mask distribution data corresponding to the audio sample data, the mask determination module is used to:

[0046] Numerical descending arrangement is performed on the sensitivity score corresponding to each mel frequency unit to obtain a numerical descending arrangement result;

[0047] Based on the numerical descending arrangement result, a preset number of sensitivity scores are selected from the first order in the sensitivity score to determine the sensitivity score as a target sensitivity score;

[0048] For the mel frequency unit corresponding to the target sensitivity score, the mel frequency unit is determined as a first mel frequency unit, and the mel gradient sparse mask value corresponding to the first mel frequency unit is determined as a first mask value;

[0049] The mel frequency unit corresponding to the unselected sensitivity score is determined as a second mel frequency unit, and the mel gradient sparse mask value corresponding to the second mel frequency unit is determined as a second mask value;

[0050] determine, based on the first mask value corresponding to the first mel-frequency unit and the second mask value corresponding to the second mel-frequency unit, mel- gradient sparse mask distribution data corresponding to the audio sample data.

[0051] Further, when the perturbation optimization module is used to iteratively optimize the preset initial perturbation parameter based on the mel-gradient sparse mask distribution data and the model parameter by using a preset optimization function to determine a target perturbation parameter, the perturbation optimization module is configured to:

[0052] construct, based on the mel-gradient sparse mask distribution data and a safety cross-entropy loss value and target safety recovery data in the model parameter, a target optimization model corresponding to the preset initial perturbation parameter by using a preset optimization function;

[0053] iteratively optimize the initial perturbation parameter based on the target optimization model by using a preset projected gradient descent method until the number of iterations reaches a preset iteration number to determine the target perturbation parameter.

[0054] Further, when the model defense module is used to, in response to the target audio language model receiving real-time audio data, apply corresponding perturbation to the real-time audio data based on the target perturbation parameter and the mel-gradient sparse mask distribution data to determine a target mel-spectrogram corresponding to the real-time audio data, the model defense module is configured to:

[0055] convert, in response to the target audio language model receiving real-time audio data, the real-time audio data to obtain a mel-spectrogram corresponding to the real-time audio data;

[0056] apply corresponding perturbation to the mel-spectrogram based on the target perturbation parameter and the mel-gradient sparse mask distribution data to determine a target mel-spectrogram corresponding to the mel-spectrogram.

[0057] Further, when the model defense module is used to, based on the target perturbation parameter and the mel-gradient sparse mask distribution data, apply corresponding perturbation to the mel-spectrogram to determine a target mel-spectrogram corresponding to the mel-spectrogram, the model defense module is configured to:

[0058] perform element-wise multiplication of the target perturbation parameter and the mel-gradient sparse mask distribution data to obtain a mask product result;

[0059] determine the target mel-spectrogram by adding the mask product result and the mel-spectrogram.

[0060] The embodiment of the present application also provides an electronic device, comprising a processor, a memory and a bus, the memory stores machine readable instructions executable by the processor, when the electronic device is running, the processor and the memory communicate through the bus, and the machine readable instructions are executed by the processor to perform the steps of the security defense method of the audio language model.

[0061] The embodiment of the present application also provides a computer readable storage medium, which stores a computer program, and the computer program performs the steps of the security defense method of the audio language model when executed by a processor.

[0062] The security defense method and device of the audio language model provided by the embodiment of the present application, the security defense method comprises: obtaining a plurality of preset audio sample data, and determining a sensitivity score corresponding to each mel frequency unit based on a plurality of mel frequency units corresponding to each of the audio sample data and model parameters corresponding to a target audio language model; determining a mel gradient sparse mask value corresponding to each of the mel frequency units based on the sensitivity score corresponding to each of the mel frequency units, to determine mel gradient sparse mask distribution data corresponding to the audio sample data; iteratively optimizing a preset initial perturbation parameter by using a preset optimization function based on the mel gradient sparse mask distribution data and the model parameters, to determine a target perturbation parameter; in response to the target audio language model receiving real-time audio data, applying a corresponding perturbation to the real-time audio data based on the target perturbation parameter and the mel gradient sparse mask distribution data, to determine a target mel spectrum corresponding to the real-time audio data, so that the target audio language model infers and generates corresponding response information based on the target mel spectrum.

[0063] Compared with the method of migrating from the text large language model escape defense and traditional audio adversarial defense in the prior art, a target perturbation parameter guided by a mel gradient sparse mask is generated by optimizing the audio language model, to determine a general security path activation perturbation, and the target perturbation parameter and the mel gradient sparse mask are applied to a mel spectrum corresponding to input audio in the real-time inference stage of the audio language model, to activate the potential security path inside the audio language model, to effectively defend against escape attacks of the audio language model, while stably retaining the performance of the audio language model on benign tasks, thereby improving the robustness and usability of the audio language model.

[0064] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the following preferred embodiments are described in detail below, and the accompanying drawings are described as follows. BRIEF DESCRIPTION OF DRAWINGS

[0065] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments. It should be understood that the following drawings only show some of the embodiments of the present application, and therefore should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can also be obtained without creative labor.

[0066] Figure 1 A flowchart of a security defense method for an audio language model provided by an embodiment of the present application;

[0067] Figure 2 A structural schematic diagram of a security defense device for an audio language model provided by an embodiment of the present application;

[0068] Figure 3 A structural schematic diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0069] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the following will combine the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. The components of the embodiments of the present application described and shown in the drawings can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed present application, but only represents selected embodiments of the present application. Based on the embodiments of the present application, every other embodiment obtained by those skilled in the art without creative labor belongs to the scope of protection of the present application.

[0070] It is found through research that, at present, when responding to the scene of jailbreaking attack on an audio language model (ALM), the existing technology mainly migrates from the fields of text large language model jailbreaking defense and traditional audio confrontation defense.

[0071] The existing technology also includes repeating the security responsibility of the model in the system prompt to enhance its resistance. The limitation of this method is that it seriously interferes with the understanding of the model to benign speech, thus sacrificing the usability of the model on normal tasks. In addition, there is also a method of guiding the model by providing contextually safe examples. This method can defend text induction to a certain extent, but also significantly damages the basic speech recognition ability of the model, leading to a significant decline in its performance in normal conversations, and is not suitable for scenes with high requirements for communication quality.

[0072] The traditional audio anti-defense method includes Gaussian noise, local smoothing and down-sampling, etc. The method has certain effect on attacks based on acoustic signals, but cannot defend against semantic attacks. The method transferred from the text large language model defense field performs well against semantic attacks, but cannot effectively resist acoustic attacks. The defense range of the above-mentioned existing methods is narrow and uneven, and there is obvious bias.

[0073] Here, in the white box scenario where the attacker understands the defense mechanism and adjusts the attack strategy accordingly (i.e., adaptive attack), the performance of all baseline defense methods will decrease sharply, making the audio language model unable to resist adaptive attacks. In addition, all existing technologies are directly transferred from other fields, and the behavior diversity and complexity specific to the audio and language modalities after fusion are not considered, which reduces the robustness and performance of the audio language model.

[0074] In addition, the method transferred from the text large language model defense field will seriously interfere with the understanding ability of the audio language model to normal speech, which means that even if the user inputs benign and harmless speech, the audio language model may not be able to correctly understand its semantics, reducing the usability of the audio language model.

[0075] Based on this, the embodiments of the present application provide a security defense method for an audio language model. A target perturbation parameter guided by a mel gradient sparse mask is generated for the audio language model to determine a general security path activation perturbation. In the real-time inference stage of the audio language model, the target perturbation parameter and the mel gradient sparse mask are applied to the mel spectrogram corresponding to the input audio to activate the potential security path inside the audio language model, effectively defending the audio language model against jailbreaking attacks, while stably preserving the performance of the audio language model on benign tasks, thereby improving the robustness and usability of the audio language model.

[0076] Here, based on the known concept that there is a "security path" (Safety Shortcuts) inside a well-aligned audio language model, once the security path is activated, it can guide the audio language model to produce safer behavior. Based on this, a light-weight general acoustic perturbation, i.e., a security path activation perturbation, is applied to the mel spectrogram of the input audio during the inference of the audio language model to efficiently activate the security path and improve the security defense capability of the audio language model.

[0077] Please refer to Figure 1 , Figure 1 The flowchart of the security defense method for an audio language model provided by the embodiments of the present application. As shown in Figure 1 The security defense method for an audio language model provided by the embodiments of the present application includes:

[0078] S101, obtain a plurality of preset audio sample data, and determine a sensitivity score corresponding to each mel frequency unit based on a plurality of mel frequency units corresponding to each of the audio sample data and model parameters corresponding to a target audio language model.

[0079] In the embodiments of the present application, the target audio language model is an audio language model expected to be protected. An audio language model (ALM) is a unified deep learning architecture capable of jointly modeling audio (speech, sound) and text language, realizing deep alignment, understanding and generation between speech and language. The goal of the audio language model is to break the "pipeline" separation of traditional automatic speech recognition (ASR) and text-to-speech (TTS) systems, understand speech signals and language semantics through a unified large model, realize semantic alignment of speech to language, cross-modal reasoning, zero-shot speech understanding and generation, and context modeling in multi-turn voice dialogue, etc.

[0080] It should be noted that the target audio language model described in the embodiments of the present application is generally in a scenario with a jailbreak attack. The "jailbreak attack" of the audio language model refers to that an attacker bypasses the security mechanism built into the model by carefully designed audio input (for example, voice instructions, background sound, adversarial perturbation, etc.), and induces the model to output content that should be limited (for example, illegal information, harmful suggestions, privacy leakage, etc.).

[0081] For example, the jailbreak attack can occur in the following aspects, including but not limited to: speech recognition layer, through adversarial audio perturbation, the speech recognition module misrecognizes malicious instructions as legal text; language model layer, even if the speech recognition module outputs normally, the attacker can also use the prompt injection vulnerability of the language model; end-to-end model: directly optimize the original audio, bypass the intermediate text representation, and realize "black box" jailbreak.

[0082] In the embodiments of the present application, the mel frequency unit refers to a plurality of frequency channels (usually 80 or 128) divided on the mel scale. Each mel frequency unit corresponds to a triangular filter covering a frequency range (narrow in the low frequency region and wide in the high frequency region) to simulate the perceptual sensitivity of the human ear to different frequency bands.

[0083] In the embodiments of the present application, the model parameters corresponding to the target audio language model include but are not limited to a safety cross-entropy loss value, a model availability cross-entropy loss value, and target safety response data.

[0084] In one implementation manner of the present application, in specific implementation, the step of pre-setting the audio sample data in step S101 can include:

[0085] S101A, obtain a plurality of preset attack instruction text data, and convert each of the attack instruction text data into initial audio data by using a preset text-to-speech conversion tool.

[0086] In the embodiments of the present application, the attack instruction text data includes text instructions for attacking the audio language model, and the text-to-speech conversion tool can include a Text-to-Speech (TTS) tool, i.e., a text-to-speech synthesis, which automatically converts input text into natural, fluent, and understandable human voice waveform.

[0087] S101B, maliciously attacking and processing each of the initial audio data by using a preset jailbreak attack method to generate audio sample data corresponding to each of the initial audio data.

[0088] The audio sample data is used to simulate the jailbreak attack on the target audio language model, and further, based on the audio sample data, a sample data set corresponding to the target audio language model can be constructed.

[0089] For example, the jailbreak attack method can include an adversarial jailbreak attack algorithm (AdvWave), which can generate an adversarial audio satisfying the constraint conditions and security policies of the audio language model based on the input audio, so as to achieve the jailbreak attack on the audio language model.

[0090] In an implementable manner of the present application, in specific implementation, the step of determining the sensitivity score corresponding to each of the mel frequency units based on the plurality of mel frequency units corresponding to each of the audio sample data and the model parameters corresponding to the target audio language model in step S101 can include:

[0091] S1011, for each of the plurality of mel frequency units corresponding to each of the audio sample data, determining a security gradient sensitivity score corresponding to each of the mel frequency units based on the security cross-entropy loss value.

[0092] Here, the security cross-entropy loss value represents the cross-entropy loss between the output content of the target audio language model and the preset security reply content.

[0093] In the embodiments of the present application, the security gradient sensitivity score corresponding to each of the mel frequency units is calculated by the following formula.

[0094] .

[0095] wherein, represents the security gradient sensitivity score corresponding to each of the mel frequency units ; represents the security cross-entropy loss value; denotes a time frame corresponding to each mel frequency unit; denotes each time frame; denotes each mel frequency unit.

[0096] In this step, based on the safety cross-entropy loss value, the average gradient absolute value of each mel frequency unit on all time frames corresponding to each mel frequency unit is calculated to determine the safety gradient sensitivity score corresponding to each mel frequency unit.

[0097] S1012, based on the model availability cross-entropy loss value, determine the availability gradient sensitivity score corresponding to each mel frequency unit.

[0098] Here, the model availability cross-entropy loss value represents the cross-entropy loss between the output content of the target audio language model and the real text content.

[0099] In the embodiments of the present application, the availability gradient sensitivity score corresponding to each mel frequency unit is calculated by the following formula.

[0100] .

[0101] wherein, denotes each mel frequency unit corresponding availability gradient sensitivity score; denotes the model availability cross-entropy loss value; denotes a time frame corresponding to each mel frequency unit; denotes each time frame; denotes each mel frequency unit.

[0102] In this step, based on the model availability cross-entropy loss value, the average gradient absolute value of each mel frequency unit on all time frames corresponding to each mel frequency unit is calculated to determine the availability gradient sensitivity score corresponding to each mel frequency unit.

[0103] S1013, based on the safety gradient sensitivity score and the availability gradient sensitivity score, determine the sensitivity score corresponding to each mel frequency unit.

[0104] Here, the goal of optimizing the perturbation parameter is to find the mel frequency unit with a higher safety gradient sensitivity score but a lower availability gradient sensitivity score.

[0105] In the embodiments of the present application, the sensitivity score corresponding to each mel frequency unit is calculated by the following formula.

[0106] .

[0107] wherein, denotes each mel-frequency unit corresponding sensitivity score; denotes each mel-frequency unit corresponding safety gradient sensitivity score; denotes each mel-frequency unit corresponding availability gradient sensitivity score; denotes a preset constant to prevent the denominator from being zero.

[0108] S102, determine a mel gradient sparse mask value corresponding to each of the mel-frequency units based on the sensitivity score corresponding to each of the mel-frequency units, to determine mel gradient sparse mask distribution data corresponding to the audio sample data.

[0109] In the embodiments of the present application, to avoid affecting the performance of the target audio model on normal tasks, the target mel-frequency unit sensitive to safety response but less affecting the speech understanding of the target audio model is accurately identified by determining the mel gradient sparse mask (M-GSM) distribution data, so as to guide the perturbation parameter to be added only on the target mel-frequency unit, thereby greatly reducing the success rate of jailbreaking attack while maintaining the high availability of the target audio model.

[0110] In an implementable manner of the present application, in specific implementation, step S102 can include:

[0111] S1021, arrange the sensitivity scores corresponding to each of the mel-frequency units in numerical descending order to obtain a numerical descending order result.

[0112] For example, assuming that the sensitivity scores corresponding to mel-frequency units f1, f2, f3 and f4 are 1.5, 1.6, 0.8 and 1.9 respectively, the numerical descending order result of the mel-frequency units is f4, f2, f1 and f3.

[0113] S1022, based on the numerical descending order result, select a preset number of sensitivity scores from the first order in the sensitivity scores to determine the sensitivity scores as target sensitivity scores.

[0114] In the embodiments of the present application, the preset number is generally calibrated according to the model test results, for example, when there are 128 mel-frequency units corresponding to the mel-frequency spectrum of each audio sample data, the preset number can be set to 48, and the mel gradient sparse mask distribution data obtained at this time is more suitable for the requirements of perturbation optimization.

[0115] For example, assuming that the values of the mel-frequency bins are arranged in descending order as f4, f2, f1, f3, and the preset number is set as 2, the sensitivity scores corresponding to f4 and f2 are determined as the target sensitivity scores.

[0116] In the embodiment of the present application, the first mask value is generally set as 1.

[0117] In the embodiment of the present application, the first mask value is generally set as 1.

[0118] In the embodiment of the present application, the second mask value is generally set as 0.

[0119] In the embodiment of the present application, the second mask value is generally set as 0.

[0120] S1025, determining the mel- gradient sparse mask distribution data corresponding to the audio sample data based on the first mask value corresponding to the first mel-frequency bin and the second mask value corresponding to the second mel-frequency bin.

[0121] For example, the mel- gradient sparse mask distribution data is represented as follows.

[0122] .

[0123] wherein, the mask value corresponding to each mel-frequency bin, the first mask value, the second mask value; the first mel-frequency bin; the second mel-frequency bin.

[0124] S103, based on the mel- gradient sparse mask distribution data and the model parameters, iteratively optimizing the preset initial perturbation parameter by using a preset optimization function to determine a target perturbation parameter.

[0125] In the embodiment of the present application, the preset initial perturbation parameter is optimized to minimize the expected security loss on the audio sample data.

[0126] In one implementation manner of the present application, in the specific implementation, step S103 can include:

[0127] ​S1031, based on the mel gradient sparse mask distribution data, and the safety cross entropy loss value in the model parameter and the target safety reply data, an initial disturbance parameter corresponding to a target optimization model is constructed by using a preset optimization function.

[0128] In the embodiment of the application, the expression of the target optimization model is as follows.

[0129] .

[0130] wherein, represents the attack instruction text data corresponding to the audio sample data; represents the jailbreak attack method corresponding to the audio sample data; represents the preset initial disturbance parameter; represents the target audio language model; represents the safety cross entropy loss value of the target audio language model; represents the mel gradient sparse mask distribution data; represents the target safety reply data of the target audio language model; represents the mel spectrogram conversion; represents the element-wise product; represents the audio sample data.

[0131] S1032, based on the target optimization model, the initial disturbance parameter is iteratively optimized by using a preset projection gradient descent method, until the number of iterations reaches a preset iteration number, and a target disturbance parameter is determined.

[0132] In the embodiment of the application, the iterative optimization expression corresponding to the projection gradient descent method is as follows.

[0133] .

[0134] wherein, represents the target disturbance parameter; represents the learning rate; represents the projection operation; represents the disturbance amplitude; represents the preset initial disturbance parameter; represents the target audio language model; represents the safety cross entropy loss value of the target audio language model; represents the mel gradient sparse mask distribution data; represents the target safety reply data of the target audio language model; represents the mel spectrogram conversion; represents the element-wise product; represents the audio sample data.

[0135] Here, the preset number of iterations can be determined according to empirical data, specific optimization requirements, and specific parameters of the target audio model.

[0136] S104, in response to the target audio language model receiving real-time audio data, based on the target perturbation parameter and the mel gradient sparse mask distribution data, applying a corresponding perturbation to the real-time audio data, determining the target mel spectrogram corresponding to the real-time audio data, so that the target audio language model infers to generate corresponding response information based on the target mel spectrogram.

[0137] In an embodiment of the present application, each audio data corresponds to a mel spectrogram. Mel spectrogram is a very common feature representation method in speech and audio signal processing, which combines spectral analysis and human auditory perception characteristics, and is closer to human perception of sound frequency than original linear spectrum.

[0138] Here, since the target mel spectrogram is subjected to perturbation processing, the target mel spectrogram activates the safe path of the target audio language model, and the target audio language model will refuse to execute harmful instruction information in the real-time audio data, and output a safe and compliant response information.

[0139] In this way, a general security path activation perturbation (target perturbation parameter) guided by mel gradient sparse mask distribution data is generated through offline optimization, and the target perturbation parameter is efficiently applied to the mel spectrogram of real-time audio data in the inference stage of the target audio language model to activate the potential security path inside the audio language model, effectively defend against potential jailbreak attacks in real-time audio data, while maximizing the performance of the audio language model on benign tasks.

[0140] In an embodiment of the present application, in specific implementation, the step S104 of responding to the target audio language model receiving real-time audio data, based on the target perturbation parameter and the mel gradient sparse mask distribution data, applying a corresponding perturbation to the real-time audio data, determining the target mel spectrogram corresponding to the real-time audio data can include:

[0141] S1041, in response to the target audio language model receiving real-time audio data, converting the real-time audio data to obtain a mel spectrogram corresponding to the real-time audio data.

[0142] In this step, in a specific implementation, first, the real-time audio data is first framed according to a fixed frame length and frame shift, and each frame is windowed to reduce boundary effects; then, a short-time Fourier transform is performed on each frame (to obtain a linear frequency spectrum; then, the spectral energy is mapped to a mel frequency conforming to human ear perception through a set of triangular filters distributed according to the mel scale; finally, the energy of each mel band is logarithmically transformed to form a mel spectrum vector of the current frame, which is processed in a time sliding manner, that is, a mel spectrum graph corresponding to the real-time audio data can be obtained.

[0143] S1042, based on the target perturbation parameter and the mel gradient sparse mask distribution data, a corresponding perturbation is applied to the mel spectrum graph to determine a target mel spectrum graph corresponding to the mel spectrum graph.

[0144] In an implementation manner of the present application, in a specific implementation, step S1042 can include:

[0145] S10421, element-wise multiplication of the target perturbation parameter and the mel gradient sparse mask distribution data is performed to obtain a mask product result.

[0146] S10422, the mask product result is added to the mel spectrum graph corresponding to the target mel spectrum graph.

[0147] In the embodiments of the present application, the target mel spectrum graph is determined by the following formula.

[0148]

[0149] wherein, represents real-time audio data corresponding target mel spectrum graph; represents real-time audio data; represents real-time audio data corresponding mel spectrum graph; represents a target perturbation parameter; represents mel gradient sparse mask distribution data; represents a mask product result.

[0150] In this way, the calculation overhead of the above step of determining the target mel spectrum graph is extremely small, only involving one mask multiplication and one addition.

[0151] In summary, based on the target perturbation parameter and the mel gradient sparse mask distribution data, a corresponding perturbation is applied to the real-time audio data input to the target audio language model, a security path activation framework can be formed to activate the security path when the target audio language model is in inference, thereby protecting the target audio language model without modifying or retraining the model.

[0152] Further, by determining the mel gradient sparse mask distribution data, the mel spectrum frequency region sensitive to the "jailbreak defense" target and not sensitive to the "speech recognition" normal task can be accurately identified, so as to strictly limit the acoustic disturbance in the key mel spectrum frequency region, thereby maximizing the defense effect of the target audio language model, while minimizing the impact on the general performance (i.e., model availability) of the target audio language model.

[0153] Further, a single and universal perturbation of the target perturbation parameter can be effectively generalized to unseen attacks and audio samples. The target perturbation parameter can be directly and low-cost applied to the input audio during the actual inference of the target audio language model, without any inference delay, thereby realizing real-time and efficient protection of the target audio language model.

[0154] The security defense method for an audio language model provided by the embodiments of the present application generates a target perturbation parameter guided by a mel gradient sparse mask for an audio language model to determine a universal security path activation perturbation. The target perturbation parameter and the mel gradient sparse mask are applied to the mel spectrum graph corresponding to the input audio in the real-time inference stage of the audio language model to activate the potential security path inside the audio language model, thereby realizing effective defense of the audio language model against jailbreak attacks, while stably preserving the performance of the audio language model on benign tasks, and further improving the robustness and availability of the audio language model.

[0155] Please refer to Figure 2 , Figure 2 The structure diagram of a security defense device for an audio language model provided by the embodiments of the present application is shown in FIG. 1. As shown in FIG. 1, the security defense device 200 includes: Figure 2 A sensitive computing module 210 is configured to obtain a plurality of preset audio sample data, and determine a sensitivity score corresponding to each mel frequency unit based on the model parameters corresponding to the target audio language model and the plurality of mel frequency units corresponding to each audio sample data.

[0156] A mask determining module 220 is configured to determine a mel gradient sparse mask value corresponding to each mel frequency unit based on the sensitivity score corresponding to each mel frequency unit, so as to determine the mel gradient sparse mask distribution data corresponding to the audio sample data.

[0157] A perturbation optimization module 230 is configured to perform iterative optimization on a preset initial perturbation parameter based on the mel gradient sparse mask distribution data and the model parameters, and determine a target perturbation parameter by using a preset optimization function.

[0158] ​

[0159] The model defense module 240 is configured to, in response to the target audio language model receiving real-time audio data, impose a corresponding perturbation on the real-time audio data based on the target perturbation parameter and the mel gradient sparse mask distribution data, determine a target mel spectrogram corresponding to the real-time audio data, so that the target audio language model infers and generates corresponding response information based on the target mel spectrogram.

[0160] Further, the sensitive computing module 210 is configured to, when pre-setting the audio sample data:

[0161] obtain a plurality of preset attack instruction text data, and convert each of the attack instruction text data into initial audio data using a preset text-to-speech conversion tool;

[0162] perform malicious attack processing on each of the initial audio data using a preset jailbreak attack method to generate audio sample data corresponding to each of the initial audio data.

[0163] Further, the model parameters at least include a security cross-entropy loss value and a model availability cross-entropy loss value; and the sensitive computing module 210 is configured to, when determining a sensitivity score corresponding to each of the mel frequency units based on the model parameters corresponding to the target audio language model and the plurality of mel frequency units corresponding to each of the audio sample data:

[0164] determining, for each of the plurality of mel frequency units corresponding to each of the audio sample data, a security gradient sensitivity score corresponding to each of the mel frequency units based on the security cross-entropy loss value;

[0165] determining, based on the model availability cross-entropy loss value, an availability gradient sensitivity score corresponding to each of the mel frequency units;

[0166] determining, based on the security gradient sensitivity score and the availability gradient sensitivity score, a sensitivity score corresponding to each of the mel frequency units.

[0167] Further, the mask determination module 220 is configured to, when determining a mel gradient sparse mask value corresponding to each of the mel frequency units based on the sensitivity score corresponding to each of the mel frequency units to determine the mel gradient sparse mask distribution data corresponding to the audio sample data:

[0168] arranging the sensitivity scores corresponding to each of the mel frequency units in descending numerical order to obtain a descending numerical order result;

[0169] Based on the numerical descending arrangement result, a preset number of sensitivity scores are selected from the first order in the sensitivity scores to determine the target sensitivity score;

[0170] For the mel frequency unit corresponding to the target sensitivity score, the mel frequency unit is determined as a first mel frequency unit, and the mel gradient sparse mask value corresponding to the first mel frequency unit is determined as a first mask value;

[0171] The mel frequency unit corresponding to the unselected sensitivity score is determined as a second mel frequency unit, and the mel gradient sparse mask value corresponding to the second mel frequency unit is determined as a second mask value;

[0172] Based on the first mask value corresponding to the first mel frequency unit and the second mask value corresponding to the second mel frequency unit, the mel gradient sparse mask distribution data corresponding to the audio sample data is determined.

[0173] Further, when the perturbation optimization module 230 is used to determine the target perturbation parameter by iteratively optimizing the preset initial perturbation parameter based on the mel gradient sparse mask distribution data and the model parameter using a preset optimization function, the perturbation optimization module 230 is used to:

[0174] Based on the mel gradient sparse mask distribution data, and the safety cross entropy loss value and the target safety recovery data in the model parameter, a target optimization model corresponding to the preset initial perturbation parameter is constructed using a preset optimization function;

[0175] Based on the target optimization model, the initial perturbation parameter is iteratively optimized using a preset projected gradient descent method until the number of iterations reaches a preset iteration number, and the target perturbation parameter is determined.

[0176] Further, when the model defense module 240 is used to determine the target mel spectrum diagram corresponding to the real-time audio data by applying a corresponding perturbation to the real-time audio data based on the target perturbation parameter and the mel gradient sparse mask distribution data in response to the target audio language model receiving real-time audio data, the model defense module 240 is used to:

[0177] In response to the target audio language model receiving real-time audio data, the real-time audio data is converted to obtain a mel spectrum diagram corresponding to the real-time audio data;

[0178] Based on the target perturbation parameter and the mel gradient sparse mask distribution data, a corresponding perturbation is applied to the mel spectrum diagram to determine a target mel spectrum diagram corresponding to the mel spectrum diagram.

[0179] Further, when the model defense module 240 is used to apply a corresponding perturbation to the mel spectrogram based on the target perturbation parameter and the mel gradient sparse mask distribution data to determine a target mel spectrogram corresponding to the mel spectrogram, the model defense module 240 is configured to:

[0180] perform element-wise multiplication on the target perturbation parameter and the mel gradient sparse mask distribution data to obtain a mask product result;

[0181] perform element-wise addition on the mask product result and the mel spectrogram to determine the target mel spectrogram.

[0182] The security defense device for the audio language model provided by the embodiments of the present application generates a target perturbation parameter guided by a mel gradient sparse mask for the audio language model to optimize, to determine a general security path activation perturbation. In the real-time inference stage of the audio language model, the target perturbation parameter and the mel gradient sparse mask are applied to the mel spectrogram corresponding to the input audio to activate the potential security path inside the audio language model, to effectively defend the audio language model against jailbreaking attacks, while stably retaining the performance of the audio language model on benign tasks, thereby improving the robustness and usability of the audio language model.

[0183] Please refer to Figure 3 , Figure 3 for a structural schematic diagram of an electronic device provided by the embodiments of the present application. As shown in Figure 3 , the electronic device 300 includes a processor 310, a memory 320, and a bus 330.

[0184] The memory 320 stores machine-readable instructions executable by the processor 310. When the electronic device 300 is running, the processor 310 communicates with the memory 320 through the bus 330. The machine-readable instructions, when executed by the processor 310, can perform the steps of the security defense method for the audio language model in the method embodiment as shown above Figure 1 . For specific implementation manners, refer to the method embodiments, which will not be described here.

[0185] The embodiments of the present application also provide a computer-readable storage medium having a computer program stored thereon. When the computer program is run by a processor, the steps of the security defense method for the audio language model in the method embodiment as shown above Figure 1 can be performed. For specific implementation manners, refer to the method embodiments, which will not be described here.

[0186] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system, device and unit described above can refer to the corresponding process in the foregoing method embodiments, which will not be repeated here.

[0187] In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented in other ways. The device embodiments described above are only schematic, for example, the division of the units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some communication interfaces, devices or units, and can be electrical, mechanical or other forms.

[0188] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0189] In addition, the functional units in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit.

[0190] If the functions are realized in the form of software function units and sold or used as independent products, they can be stored in a nonvolatile computer readable storage medium executable by a processor. Based on this understanding, the technical solutions of the present application or the essential part or part of the technical solutions that make contributions to the prior art can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The foregoing storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk and various program code storage media.

[0191] Finally, it should be noted that the above-described embodiments are merely specific embodiments of the present application, which are used to illustrate the technical solutions of the present application, but not to limit the same. The protection scope of the present application is not limited thereto. Although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those skilled in the art that any skilled person in the art can still modify or easily think of changes to the technical solutions recorded in the foregoing embodiments, or make equivalent replacements to some of the technical features, within the technical scope disclosed by the present application. The modifications, changes or replacements do not make the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A security defense method for an audio language model, characterized in that, The security defense methods include: Acquire multiple preset audio sample data, and determine the sensitivity score corresponding to each of the multiple Mel frequency units corresponding to each audio sample data and the model parameters corresponding to the target audio language model. Based on the sensitivity score corresponding to each Mel frequency unit, the Mel gradient sparse mask value corresponding to each Mel frequency unit is determined to determine the Mel gradient sparse mask distribution data corresponding to the audio sample data; Based on the Mel gradient sparse mask distribution data and the model parameters, the preset initial perturbation parameters are iteratively optimized using a preset optimization function to determine the target perturbation parameters; In response to the target audio language model receiving real-time audio data, based on the target perturbation parameters and the Mel gradient sparse mask distribution data, a corresponding perturbation is applied to the real-time audio data to determine the target Mel spectrogram corresponding to the real-time audio data, so that the target audio language model can infer and generate corresponding response information based on the target Mel spectrogram.

2. The method according to claim 1, characterized in that, The audio sample data is pre-set using the following steps: Acquire multiple preset attack command text data, and use a preset text-to-speech conversion tool to convert each attack command text data into initial audio data; The initial audio data is maliciously processed using a preset jailbreak attack method to generate audio sample data corresponding to each initial audio data.

3. The method according to claim 1, characterized in that, The model parameters include at least the security cross-entropy loss value and the model availability cross-entropy loss value; the determination of the sensitivity score corresponding to each Mel frequency unit based on the multiple Mel frequency units corresponding to each audio sample data and the model parameters corresponding to the target audio language model includes: For each of the multiple Mel frequency units corresponding to each audio sample data, a security gradient sensitivity score is determined for each Mel frequency unit based on the security cross-entropy loss value. Based on the model availability cross-entropy loss value, determine the availability gradient sensitivity score corresponding to each Mel frequency unit; Based on the security gradient sensitivity score and the availability gradient sensitivity score, a sensitivity score corresponding to each Mel frequency unit is determined.

4. The method according to claim 1, characterized in that, The step of determining the Mel gradient sparse mask value corresponding to each Mel frequency unit based on the sensitivity score corresponding to each Mel frequency unit, in order to determine the Mel gradient sparse mask distribution data corresponding to the audio sample data, includes: The sensitivity scores corresponding to each Mel frequency unit are sorted in descending order to obtain the numerical descending order result; Based on the descending order of the numerical values, a predetermined number of sensitivity scores are selected from the first position in the sensitivity scores to determine the sensitivity score as the target sensitivity score. For the Mel frequency unit corresponding to the target sensitivity score, the Mel frequency unit is determined as the first Mel frequency unit, and the Mel gradient sparse mask value corresponding to the first Mel frequency unit is determined as the first mask value; The Mel frequency unit corresponding to the unselected sensitivity score is determined as the second Mel frequency unit, and the Mel gradient sparse mask value corresponding to the second Mel frequency unit is determined as the second mask value. Based on the first mask value corresponding to the first Mel frequency unit and the second mask value corresponding to the second Mel frequency unit, the Mel gradient sparse mask distribution data corresponding to the audio sample data is determined.

5. The method according to claim 1, characterized in that, The step of determining the target perturbation parameters by iteratively optimizing the preset initial perturbation parameters using a preset optimization function based on the Mel gradient sparse mask distribution data and the model parameters includes: Based on the Mel gradient sparse mask distribution data, as well as the safe cross-entropy loss value and target safe response data in the model parameters, a target optimization model corresponding to the preset initial perturbation parameters is constructed using a preset optimization function. Based on the target optimization model, the initial perturbation parameters are iteratively optimized using a preset projection gradient descent method until the number of iterations reaches the preset number of iterations, thereby determining the target perturbation parameters.

6. The method according to claim 1, characterized in that, In response to the target audio language model receiving real-time audio data, based on the target perturbation parameters and the Mel gradient sparse mask distribution data, a corresponding perturbation is applied to the real-time audio data to determine the target Mel spectrogram corresponding to the real-time audio data, including: In response to the target audio language model receiving real-time audio data, the real-time audio data is transformed to obtain the Mel spectrogram corresponding to the real-time audio data; Based on the target perturbation parameters and the Mel gradient sparse mask distribution data, a corresponding perturbation is applied to the Mel spectrum map to determine the target Mel spectrum map corresponding to the Mel spectrum map.

7. The method according to claim 6, characterized in that, The step of applying a corresponding perturbation to the Mel spectrum based on the target perturbation parameters and the Mel gradient sparse mask distribution data to determine the target Mel spectrum corresponding to the Mel spectrum includes: The target perturbation parameter is multiplied element-wise with the Mel gradient sparse mask distribution data to obtain the mask product result. The sum of the mask product result and the corresponding Mel spectrogram is determined as the target Mel spectrogram.

8. A security defense device for an audio language model, characterized in that, The security defense device includes: The sensitivity calculation module is used to acquire multiple preset audio sample data, and determine the sensitivity score corresponding to each of the multiple Mel frequency units corresponding to each audio sample data and the model parameters corresponding to the target audio language model. The mask determination module is used to determine the Mel gradient sparse mask value corresponding to each Mel frequency unit based on the sensitivity score corresponding to each Mel frequency unit, so as to determine the Mel gradient sparse mask distribution data corresponding to the audio sample data; The perturbation optimization module is used to iteratively optimize the preset initial perturbation parameters based on the Mel gradient sparse mask distribution data and the model parameters using a preset optimization function to determine the target perturbation parameters. The model defense module is used to respond to the target audio language model receiving real-time audio data, and to apply a corresponding perturbation to the real-time audio data based on the target perturbation parameters and the Mel gradient sparse mask distribution data, thereby determining the target Mel spectrogram corresponding to the real-time audio data, so that the target audio language model can infer and generate corresponding response information based on the target Mel spectrogram.

9. An electronic device, characterized in that, include: The device includes a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus. The machine-readable instructions are executed by the processor to perform the steps of the security defense method for the audio language model as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of the security defense method for the audio language model as described in any one of claims 1 to 7.

Images