Backdoor detection method and system based on attention abnormity of text graph model

Abstract

The invention provides a backdoor detection method and system based on attention anomalies of a text graph model, and the method comprises the steps: obtaining a to-be-detected data set and a pre-trained text graph model, and enabling the text graph model to be obtained through the pre-training of a data set containing a backdoor sample; generating an image based on each cue word text in the to-be-tested data set by adopting a pre-trained text graph model, and extracting a trans-attention graph sequence corresponding to each denoising time step of each cue word text in a preset denoising time step interval; based on the cross-attention map sequence of each cue word text, calculating a first judgment parameter and a second judgment parameter of the cue word text; if any one of the first judgment parameter and the second judgment parameter is smaller than or equal to a preset threshold value, the sample to which the prompt word text belongs is a backdoor sample. The method breaks through the limitation of a conventional static feature detection method, achieves the effective recognition of a novel backdoor attack method with higher concealment, and improves the detection robustness and generalization capability.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence security technology, specifically to the field of backdoor detection using deep learning, and more specifically, to a backdoor detection method and system based on attention anomalies in a text-based graph model. Background Technology

[0002] In recent years, text-based image diffusion models have been widely applied in various fields, such as art and design, healthcare, and general tasks, due to their superior image synthesis capabilities. Furthermore, the successful application of these models has driven the development of multiple open-source platforms, attracting numerous users to upload and download third-party models for secondary development. However, the rapid popularization of text-based image diffusion models has also been accompanied by a sharp increase in backdoor risks. Attackers can subtly implant text-level backdoors during the training or fine-tuning phases through loss manipulation (Rickrolling, TwT, etc.) or model editing (EvilEdit, etc.). These backdoors can maintain the quality of normal prompt word generation while being instantaneously activated by specific trigger text, forcing the model to output malicious images preset by the attacker, achieving targeted, covert, and difficult-to-detect hijacking of the generation behavior.

[0003] Faced with the aforementioned text backdoor threats, the industry currently still mainly uses a backdoor detection approach oriented towards discriminative models. The existing main technical methods are as follows: feature extraction in the frequency or spatial domain is performed on backdoor and benign image test samples; different categories are scanned; and abnormal behavior of backdoor samples or models is observed for detection. However, compared to discriminative models, generative models (such as text-based image diffusion models) do not have fixed category outputs. Their generation results are high-dimensional continuous image distributions, lacking directly comparable classification probabilities. Therefore, traditional category response analysis methods are difficult to apply. Furthermore, among current defenses against text-based image diffusion models, researchers have proposed backdoor sample detection methods based on output diversity. Existing methods mainly identify backdoor samples based on static anomalies in attention structure or output diversity. Although existing methods have some effectiveness in detecting early typical backdoor samples, they are no longer suitable for the more covert backdoor attack algorithms that have emerged in recent years.

[0004] In summary, existing backdoor detection schemes rely on static features and are unable to cope with new and covert attacks. Therefore, there is an urgent need for a backdoor sample detection scheme that is more discriminative and adaptable.

[0005] It should be noted that the background information presented here is only for illustrating relevant information about the present invention to aid in understanding the technical solutions of the present invention, and does not imply that the relevant information is necessarily prior art. In the absence of evidence indicating that the relevant information was disclosed before the filing date of this invention, the relevant information should not be considered prior art. Summary of the Invention

[0006] Therefore, the purpose of this invention is to overcome the shortcomings of the prior art and provide a backdoor detection method and system based on attention anomalies in a text-based graph model.

[0007] The objective of this invention is achieved through the following technical solution:

[0008] According to a first aspect of the present invention, a backdoor detection method based on attention anomalies of a text-based graph model is proposed for identifying backdoor samples in a test dataset. The method includes: S1, acquiring a test dataset and a pre-trained text-based graph model, wherein the test dataset contains multiple samples composed of prompt text and images, and the text-based graph model is pre-trained using a dataset containing backdoor samples; S2, using the pre-trained text-based graph model to generate an image based on each prompt text in the test dataset, and extracting the cross-attention map sequence corresponding to each prompt text at each denoising time step within a preset denoising time step interval, wherein each cross-attention map sequence includes the cross-attention map corresponding to each token of a prompt text in its corresponding denoising time step; S3. Based on all cross-attention map sequences of each prompt word text, obtain the difference between the total change of the cross-attention map corresponding to the end token of the prompt word text within the preset denoising time step interval and the average of the total change of the cross-attention map corresponding to all semantic tokens, and use this difference as the first determination parameter; S4. Based on all cross-attention map sequences of each prompt word text, obtain the difference between the total change of the end token state of the prompt word text within the preset denoising time step interval and the average of the total change of the semantic token state, and use this difference as the second determination parameter; S5. If either the first determination parameter or the second determination parameter of the prompt word text is less than or equal to a preset threshold, then the sample to which the prompt word text belongs is a backdoor sample.

[0009] Optionally, S3 includes: obtaining the change amount of each token of the prompt text across the attention map corresponding to each denoising time step within a preset denoising time step interval, wherein the change amount of a token across the attention map corresponding to a denoising time step is the Frobenius norm of the difference between the token and the cross-attention map corresponding to the previous denoising time step; and determining a first determination parameter of the prompt text based on the change amount of all tokens of the prompt text across the attention map corresponding to each time step within the preset denoising time step interval using a first preset method.

[0010] Optionally, the first preset method is:

[0011]

[0012] in, Indicates the first decision parameter. and These represent the first and last denoising time steps within the preset denoising time step interval, respectively. This indicates the end token of the prompt text within the preset denoising time step interval. The change across the attention map corresponding to each denoising time step. This represents the total change across the attention map of the end token of the cue text within the preset denoising time step interval. This indicates the total number of tokens corresponding to the prompt text. This indicates the first denoising step of the prompt text within the preset denoising time interval. The semantic token in the ... The change across the attention map corresponding to each denoising time step. This represents the average total change across attention maps corresponding to all semantic tokens in the prompt text within a preset denoising time step interval.

[0013] Optionally, S4 includes: constructing a state transition model for the prompt text based on all attention map sequences of the prompt text, the state transition model being used to predict the state of all tokens in the next denoising time step based on the state of all tokens in the previous denoising time step; obtaining the initial state of all tokens corresponding to the prompt text, wherein the initial state of each token is the Frobenius norm of the token across the attention map in the first denoising time step within a preset denoising time step interval; predicting the state of all tokens in each denoising time step within the preset denoising time step interval based on the constructed state transition model, using the initial states of all tokens as input; obtaining the state change amount of each token in the prompt text in each denoising time step within the preset denoising time step interval, wherein the state change amount of a token in a denoising time step is the difference between the state of the token in the current denoising time step and the state in the previous denoising time step; and determining the second determination parameter of the prompt text based on all state changes of all tokens in each time step within the preset denoising time step interval using a second preset method.

[0014] Optionally, the state transition model is:

[0015]

[0016] in, This indicates that all tokens in the prompt text are within the preset denoising time step interval. The state corresponding to each denoising time step. This indicates that all tokens in the prompt text are within the preset denoising time step interval. The state corresponding to each denoising time step. Given a pre-defined L-dimensional diagonal matrix, For coupling strength parameters, The range of values ​​is , Indicates based on the first The graph Laplacian matrix constructed from the cross-attention graph sequences corresponding to each denoising time step.

[0017] Optionally, the second preset method is:

[0018]

[0019] in, This represents the second decision parameter. and These represent the first and last denoising time steps within the preset denoising time step interval, respectively. This indicates the end of the token's journey within the preset denoising time step interval. The state change amount corresponding to each denoising time step. This represents the total change in the token state of the prompt text within the preset denoising time step interval. This indicates the total number of tokens corresponding to the prompt text. Indicates the first The semantic token within the preset denoising time step interval The state change amount corresponding to each denoising time step. This represents the average total change in the state of all semantic tokens within a preset denoising time step interval for the prompt text.

[0020] Optionally, the preset denoising time step interval includes the first denoising time steps of all denoising time steps of the text image model. The noise reduction time step.

[0021] Optionally, the preset threshold is an interval. Any value in the range.

[0022] According to a second aspect of the present invention, a backdoor detection system for implementing the method described in the first aspect of the present invention is proposed. The system includes a text-based image model, a cross-attention map sequence extraction module, a first detection module, a second detection module, and a backdoor sample determination module, wherein: the text-based image model is used to generate an image based on each prompt word text in the dataset to be tested; the cross-attention map sequence extraction module is used to extract the cross-attention map sequence corresponding to each denoising time step within a preset denoising time step interval for each prompt word text in the dataset to be tested from the text-based image model; the first detection module is used to determine the structure of the prompt word text within the preset denoising time step interval based on all the extracted cross-attention map sequences of each prompt word text. The first determination parameter is the difference between the total change in the cross-attention map corresponding to the bundle token and the average of the total change in the cross-attention map corresponding to all semantic tokens. The second detection module is used to determine the difference between the total change in the end token state of the prompt text and the average of the total change in the semantic token state within a preset denoising time step interval based on all cross-attention map sequences of each extracted prompt text, and uses this difference as the second determination parameter for the prompt text. The backdoor sample determination module is used to determine the sample to which the description text belongs if either the first determination parameter or the second determination parameter is less than or equal to a preset threshold as a backdoor sample.

[0023] Compared with the prior art, the advantages of the present invention are as follows:

[0024] This invention breaks through the limitations of traditional static feature detection methods. By modeling the temporal evolution behavior of the attention mechanism during the diffusion process, it achieves effective identification of novel backdoor attacks with stronger concealment, thereby improving the robustness and generalization ability of the detection. Attached Figure Description

[0025] The embodiments of the present invention will be further described below with reference to the accompanying drawings, wherein:

[0026] Figure 1 This is a schematic diagram illustrating the steps of a backdoor detection method based on an attention anomaly using a text-based graph model according to an embodiment of the present invention.

[0027] Figure 2 This is a schematic diagram of a backdoor detection system based on an attention anomaly of a text-based graph model according to an embodiment of the present invention. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of this invention clearer, the invention is further described in detail below through specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0029] As mentioned in the background section, existing backdoor detection schemes rely on static features and are difficult to deal with new and covert attacks (such as loss manipulation and model editing). They also suffer from problems such as significantly reduced detection performance, insufficient generalization ability, and difficulty in applying them to model security detection in complex scenarios.

[0030] Before describing the embodiments of the present invention in detail, some of the terms used therein are explained as follows:

[0031] End Token ( <eos>token): The end token plays a special role in marking the boundary of a sentence. It is a special symbol used to indicate the end position in the text of the prompt word.

[0032] Semantic Token: A semantic token is a token used to represent the actual words or symbols in the prompt text.

[0033] It should be noted that the inventors, through in-depth research on the text-based image model, discovered that normal samples, in the early stages of the text-based image model diffusion process (denoising process), <eos>The token plays a dominant role. As the denoising process progresses, the model's attention will gradually shift from... <eos>Tokens are transferred to other tokens. Therefore, <eos>The dynamic evolution rate of a token is typically higher than the average of other tokens. In contrast, for backdoor samples, because the model is forced to learn specific triggering patterns, the triggers interfere with this process by controlling the overall image structure, thereby... <eos>Competition arises between the token and other tokens, leading to <eos>The token's dynamic evolution rate is abnormal.

[0034] Based on the inventors' research findings, this invention proposes a backdoor detection scheme based on attention anomalies in a text-based graph model. This scheme captures cross-attention maps generated by the text-based graph model in real time during image generation and compares them... <eos>The transformation rate between the cross-attention graph corresponding to the token and the cross-attention graph corresponding to the semantic token is used to determine whether it is a backdoor sample, thereby achieving effective identification of new backdoor attacks with stronger concealment and improving the robustness and generalization ability of backdoor detection.

[0035] According to one embodiment of the present invention, a backdoor detection method based on attention anomalies in a text-based graph model is proposed. (See attached diagram.) Figure 1 In summary, the method includes steps S1, S2, S3, S4, and S5. To better understand the present invention, each step of the present invention will be described in detail below with reference to specific embodiments and accompanying drawings.

[0036] I. Step S1

[0037] According to one embodiment of the present invention, in S1 of the present invention, a test dataset and a pre-trained text-to-image model are obtained, wherein the test dataset contains multiple samples composed of prompt words and images, and the text-to-image model is trained using a dataset containing backdoor samples.

[0038] According to one embodiment of the present invention, a contaminated dataset containing backdoor samples is proactively introduced during the training phase. Abnormal attention responses of these backdoor samples are statistically analyzed and captured. The cross-attention maps generated during the generation process are extracted, and their evolution rates are calculated to infer whether a sample is a backdoor sample. Furthermore, although abnormal evolution rates are a method-independent, universal identification feature, the training set should still cover backdoor samples generated by various backdoor algorithms (such as data poisoning, loss manipulation, and model editing) as much as possible to maximize the generalization and recall of cross-attention evolution rate detection for unknown backdoor types.

[0039] II. Step S2

[0040] According to an embodiment of the present invention, in S2 of the present invention, a pre-trained text-to-image model is used to generate an image based on each prompt word text in the dataset to be tested, and the cross-attention map sequence corresponding to each prompt word text in each denoising time step within a preset denoising time step interval is extracted, wherein each cross-attention map sequence includes a cross-attention map corresponding to each token of a prompt word text in a denoising time step.

[0041] According to one embodiment of the present invention, the pre-trained model injects the semantic information of the prompt text into spatial features through a cross-attention mechanism, thereby forming an explicit activation map with a "semantic-pixel" correspondence. For normal samples, since the end token plays a dominant role in the early stage of the diffusion process, the evolution rate of the cross-attention map corresponding to the end token is usually higher than that of the cross-attention maps of other tokens. However, for backdoor samples, since a strong "trigger word-target image" mapping is deliberately implanted during the training stage, the cross-attention map corresponding to the end token exhibits an abnormal phenomenon of slow dynamic evolution in the early stage of the diffusion process. Therefore, extracting the cross-attention map sequence corresponding to the test sample can provide a quantifiable, interpretable, and highly correlated discriminative basis with the internal mechanism of the model for subsequent detection based on evolution rate, thereby achieving high sensitivity and robustness in the identification of backdoor samples.

[0042] According to one embodiment of the present invention, the preset denoising time step interval needs to include the early stage of denoising the text image model. Preferably, the preset denoising time step interval is selected from the early stages of all denoising time steps of the text image model. Regarding the denoising time step, it should be understood that the range of values ​​for the preset denoising time step interval is only illustrative and is not exhaustive. Implementers can adjust it freely according to actual needs.

[0043] III. Step S3

[0044] According to an embodiment of the present invention, in S3 of the present invention, based on all cross-attention map sequences of each prompt word text, the difference between the total change of the cross-attention map corresponding to the end token of the prompt word text within a preset denoising time step interval and the average of the total change of the cross-attention map corresponding to all semantic tokens is obtained, and the difference is used as the first determination parameter.

[0045] According to an embodiment of the present invention, in S3, each attention map is treated as an independent spatial feature, the dynamic evolution rate is calculated using the Frobenius norm, and the sum of the relative evolution rates under multiple denoising time steps is used as the first detection index of the sample. Specifically, S3 includes: obtaining the change amount of each token of the prompt text across attention maps corresponding to each denoising time step within a preset denoising time step interval, wherein the change amount of a token across attention maps corresponding to a denoising time step is the Frobenius norm of the difference between the token and the cross-attention map corresponding to the previous denoising time step; and determining the first determination parameter of the prompt text based on the change amount of all tokens of the prompt text across attention maps corresponding to each time step within the preset denoising time step interval using a first preset method.

[0046] According to one embodiment of the present invention, the first The token in the 1st The change in the cross-attention map corresponding to each denoising time step is ,in, and The numbers represent the order of the numbers. The tokens are respectively in the... The noise reduction time step and the first The corresponding cross-attention map at each denoising time step. This represents the function for calculating the Frobenius norm.

[0047] According to one embodiment of the present invention, in order to characterize <eos>The relative differences between Tokens and other Tokens in dynamic changes, the first preset method adopted in this invention is: This invention proposes the first... Each noise reduction step <eos>The relative evolution rate between tokens and semantic tokens is ,in, Indicates the first Each noise reduction step <eos>The amount of change across the attention graph corresponding to the token. Indicates the first The noise reduction time step is the first The amount of change across the attention graph corresponding to each token. This represents the total number of tokens corresponding to the sample (i.e., the total number of tokens contained in the token sequence). Furthermore, to improve robustness, this invention accumulates the relative evolution rate within a preset denoising time step interval to obtain a first judgment parameter. Specifically, the calculation of the first judgment parameter can be expressed by the formula:

[0048]

[0049] in, Indicates the first decision parameter. and These represent the first and last denoising time steps within the preset denoising time step interval, respectively. This indicates the end token of the prompt text within the preset denoising time step interval. The change across the attention map corresponding to each denoising time step. This represents the total change across the attention map of the end token of the cue text within the preset denoising time step interval. This indicates the total number of tokens corresponding to the prompt text. This indicates the first denoising step of the prompt text within the preset denoising time interval. The semantic token in the ... The change across the attention map corresponding to each denoising time step. This represents the average total change across attention maps corresponding to all semantic tokens in the prompt text within a preset denoising time step interval.

[0050] IV. Step S4

[0051] According to an embodiment of the present invention, in S4 of the present invention, based on all cross-attention map sequences of each prompt word text, the difference between the total change of the end token state of the prompt word text within a preset denoising time step interval and the average of the total change of all semantic token states is obtained, and the difference is used as a second determination parameter.

[0052] According to one embodiment of the present invention, in S4, the present invention models the spatial dependency relationship of attention graphs between different tokens, constructs a state transition model through a graph structure, and extracts... <eos>The dynamic state change rate of the token is compared with the average state change rate of other tokens as a second judgment parameter. Specifically, S4 includes: S41, constructing a state transition model for the prompt text based on all attention map sequences of the prompt text, which is used to predict the state of all tokens in the next denoising time step based on the state of all tokens in the prompt text in the previous denoising time step; S42, obtaining the initial state of all tokens corresponding to the prompt text, wherein the initial state of each token is the Frobenius norm of the first denoising time step corresponding to the attention map within the preset denoising time step interval; S43, based on the constructed state transition model, using the dynamic state change rate of all tokens... The initial state is to predict the state of each token in the preset denoising time step interval for each token in the input; S44, obtain the state change amount of each token in the prompt text in the preset denoising time step interval for each token, wherein the state change amount of a token in a denoising time step is the difference between the state of the token in the current denoising time step and the state in the previous denoising time step; S45, determine the second judgment parameter of the prompt text based on the state change amount of all tokens in the preset denoising time step interval for each token in the prompt text using the second preset method.

[0053] According to one embodiment of the present invention, in S42, the present invention considers the spatial correlation between different tokens across attention maps by modeling the attention map sequence as a dynamic system to capture global evolution patterns. Specifically, each cross-attention map corresponding to a sample at each denoising time step is first regarded as a graph node, and the weight of the edge between any two nodes is defined as the Frobenius norm of the difference between the two cross-attention maps (i.e., ...). A graph structure is constructed, and a state transition model is established based on it. The state transition model is used to solve the state of each token corresponding to the preset denoising time step, and the global evolution pattern is captured based on the state changes of the tokens.

[0054] According to one embodiment of the present invention, the state transition model is as follows:

[0055]

[0056] in, This indicates that all tokens in the prompt text are within the preset denoising time step interval. The state corresponding to each denoising time step. This indicates that all tokens in the prompt text are within the preset denoising time step interval. The state corresponding to each denoising time step. For a pre-defined L-dimensional diagonal matrix, preferably... The value of the first element to the second-to-last element on the diagonal is -1, and the value of the last element is -10. For coupling strength parameters, The range of values ​​is , Let be the Laplace matrix of the graph. The value of each element is as follows:

[0057]

[0058] in, For the prompt text in the first The difference matrix constructed across the attention map sequence corresponding to the denoising time step, its th OK The value of the element in the column is the first In the first noise reduction time step The cross-attention graph corresponding to the first token and the first token The Frobenius norm of the differences across attention graphs corresponding to each token, and the result after normalizing the Frobenius norm. express The OK Column elements, express The OK The elements of the column.

[0059] V. Step S5

[0060] According to an embodiment of the present invention, in S5 of the present invention, if either the first determination parameter or the second determination parameter of the prompt text is less than or equal to a preset threshold, then the sample to which the prompt text belongs is a backdoor sample.

[0061] According to one embodiment of the present invention, preferably, the preset threshold is an interval. Any value in the above. It should be understood that the range of values ​​for the above preset threshold is merely exemplary and not exhaustive, and implementers can adjust it freely according to their needs.

[0062] According to one embodiment of the present invention, the first determination parameter indicates the sample in the early stage of denoising. <eos>The second decision parameter is the difference between the change in the attention map corresponding to a token and the average change in the attention maps corresponding to other tokens. It indicates the performance of samples in the early stages of denoising. <eos>The difference between the state change of a token and the average state change of other tokens. Both parameters reflect the early stages of the denoising process. <eos>Whether the token plays a dominant role is determined when any difference in the prompt text is less than or equal to a preset threshold. <eos>Competition arises between the token and other tokens, leading to <eos>The token has a low evolution rate in the early stages of the denoising process, i.e. <eos>If the token does not play a dominant role in the early stages of the denoising process, then the sample corresponding to the prompt text is identified as a backdoor sample.

[0063] To facilitate understanding of this invention, the process of determining backdoor samples will be described below with reference to examples.

[0064] For example, a pre-trained text-to-image diffusion model is obtained, whose text encoder transforms the input text into a graph containing... A token sequence of tokens, where the last token is <eos>Token. Assume a backdoor attack scenario where the attacker constructs a backdoor model by embedding specific trigger words (e.g., "cf") into the training data and associating them with specific concepts (e.g., "white teacup"). During inference, any text prompt containing "cf" will cause the model to generate an image containing "white teacup". Extract cross-attention map sequences. Assume there are two test samples: Sample 1, "A beautiful landscape", and Sample 2, "cf A beautiful landscape" (containing the suspected trigger word "cf"). Input the text prompt words into the text-to-image model and execute the image generation process. Obtain the cross-attention map sequences corresponding to the two samples within a preset denoising time step interval and calculate the first and second decision parameters for the normal sample and the backdoor sample. Assume the first and second decision parameters for Sample 1 are 0.05 and 0.06, respectively, and the first and second decision parameters for Sample 2 are 0.005 and 0.05, respectively. Assuming the preset threshold is 0.01, if both the first and second judgment parameters of sample 1 are greater than the preset threshold, then sample 1 is judged as a normal sample. If the first judgment parameter of sample 2 is less than the preset threshold, then sample 2 is judged as a backdoor sample.

[0065] According to an embodiment of the present invention, in order to implement the above-described backdoor detection method, the present invention proposes a backdoor detection system based on attention anomalies in a text-based graph model, see appendix. Figure 2 The system includes a text-based image model, a cross-attention map sequence extraction module, a first detection module, a second detection module, and a backdoor sample determination module. The text-based image model generates images based on each prompt word text in the test dataset. The encoding module in the text-based image model encodes the prompt word text into a token sequence, and the denoising module performs multiple rounds of denoising based on the token sequence, generating a cross-attention map sequence in each round. The cross-attention map sequence extraction module extracts the cross-attention map sequence corresponding to each prompt word text in the test dataset for each denoising time step within a preset denoising time step interval from the text-based image model. The first detection module determines a preset cross-attention map sequence based on all extracted cross-attention map sequences of each prompt word text. The first determination parameter for the prompt text is the difference between the total change in the cross-attention map corresponding to the end token of the prompt text within the denoising time step interval and the average of the total change in the cross-attention map corresponding to all semantic tokens. The second detection module is used to determine the difference between the total change in the end token state of the prompt text and the average of the total change in the semantic token state within the preset denoising time step interval based on all the cross-attention map sequences of each extracted prompt text, and the difference is used as the second determination parameter for the prompt text. The backdoor sample determination module is used to determine the sample to which the description text belongs if either the first determination parameter or the second determination parameter is less than or equal to a preset threshold as a backdoor sample.

[0066] In summary, this invention breaks through the limitations of traditional static feature detection methods. By modeling the temporal evolution behavior of the attention mechanism during the diffusion process, it achieves effective identification of novel backdoor attacks with stronger concealment. It can effectively improve the robustness and generalization ability of backdoor detection, and provide a backdoor detection scheme with good detection efficiency and model compatibility that is applicable to various backdoor implantation algorithms.

[0067] It should be noted that although the steps are described in a specific order above, it does not mean that the steps must be executed in the above specific order. In fact, some of these steps can be executed concurrently, or even in a different order, as long as the required function can be achieved.

[0068] This invention can be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of the invention.

[0069] Computer-readable storage media can be tangible devices that hold and store instructions for use by an instruction execution device. Computer-readable storage media can include, for example, but not limited to, electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination thereof.

[0070] The various embodiments of the present invention have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.< / eos> < / eos> < / eos> < / eos> < / eos> < / eos> < / eos> < / eos> < / eos> < / eos> < / eos> < / eos> < / eos> < / eos> < / eos> < / eos> < / eos> < / eos>

Claims

1. A backdoor detection method based on attention anomalies in a text-based graph model, used to identify backdoor samples in a test dataset, characterized in that, The method includes: S1. Obtain the dataset to be tested and a pre-trained text-to-image model, wherein the dataset to be tested contains multiple samples composed of prompt words and images, and the text-to-image model is pre-trained using a dataset containing backdoor samples. S2. A pre-trained text-to-image model is used to generate an image based on each prompt word text in the dataset to be tested, and the cross-attention map sequence corresponding to each prompt word text in each denoising time step within the preset denoising time step interval is extracted. Each cross-attention map sequence includes the cross-attention map corresponding to each token of a prompt word text in its corresponding denoising time step. S3. Based on all cross-attention map sequences of each prompt word text, obtain the difference between the total change of the cross-attention map corresponding to the end token and the average of the total change of the cross-attention map corresponding to all semantic tokens within the preset denoising time step interval of the prompt word text, and use the difference as the first determination parameter. S4. Based on all cross-attention map sequences of each prompt word text, obtain the difference between the total change of the end token state of the prompt word text within the preset denoising time step interval and the average of the total change of all semantic token states, and use the difference as the second determination parameter. S5. If either the first or second determination parameter of the prompt text is less than or equal to a preset threshold, then the sample to which the prompt text belongs is a backdoor sample.

2. The method according to claim 1, characterized in that, S3 includes: Get the change amount of each token in the prompt text across the attention map corresponding to each denoising time step within the preset denoising time step interval. The change amount of a token across the attention map corresponding to a denoising time step is the Frobenius norm of the difference between the token and the cross attention map corresponding to the previous denoising time step. The first determination parameter of the prompt text is determined by the first preset method based on the change of the cross-attention map of all tokens in the prompt text at each time step within the preset denoising time step interval.

3. The method according to claim 2, characterized in that, The first preset method is: in, Indicates the first decision parameter. and These represent the first and last denoising time steps within the preset denoising time step interval, respectively. This indicates the end token of the prompt text within the preset denoising time step interval. The change across the attention map corresponding to each denoising time step. This represents the total change across the attention map of the end token of the cue text within the preset denoising time step interval. This indicates the total number of tokens corresponding to the prompt text. This indicates the first denoising step of the prompt text within the preset denoising time interval. The semantic token in the ... The change across the attention map corresponding to each denoising time step. This represents the average total change across attention maps corresponding to all semantic tokens in the prompt text within a preset denoising time step interval.

4. The method according to claim 1, characterized in that, S4 includes: A state transition model for the prompt text is constructed based on all attention map sequences of the prompt text. This state transition model is used to predict the state of all tokens in the next denoising time step based on the state of all tokens in the previous denoising time step. Get the initial state of all tokens corresponding to the prompt text, where the initial state of each token is the Frobenius norm of the cross attention map corresponding to the first denoising time step within the preset denoising time step interval of the token. Based on the constructed state transition model, the initial state of all tokens is used as input to predict the state of each token in each denoising time step within the preset denoising time step interval. Get the state change amount of each Token in the prompt text for each denoising time step within the preset denoising time step interval. The state change amount of a Token in a denoising time step is the difference between the state of the Token in the current denoising time step and the state in the previous denoising time step. The second judgment parameter of the prompt text is determined by the second preset method based on all state changes of all tokens in each prompt text at each time step within the preset denoising time step interval.

5. The method according to claim 4, characterized in that, The state transition model is as follows: in, This indicates that all tokens in the prompt text are within the preset denoising time step interval. The state corresponding to each denoising time step. This indicates that all tokens in the prompt text are within the preset denoising time step interval. The state corresponding to each denoising time step. Given a pre-defined L-dimensional diagonal matrix, For coupling strength parameters, The range of values ​​is , Indicates based on the first The graph Laplacian matrix constructed from the cross-attention graph sequences corresponding to each denoising time step.

6. The method according to claim 5, characterized in that, The second preset method is: in, This represents the second decision parameter. and These represent the first and last denoising time steps within the preset denoising time step interval, respectively. This indicates the end of the token's journey within the preset denoising time step interval. The state change amount corresponding to each denoising time step. This represents the total change in the token state of the prompt text within the preset denoising time step interval. This indicates the total number of tokens corresponding to the prompt text. Indicates the first The semantic token within the preset denoising time step interval The state change amount corresponding to each denoising time step. This represents the average total change in the state of all semantic tokens within a preset denoising time step interval for the prompt text.

7. The method according to any one of claims 1-6, characterized in that, The preset denoising time step interval includes the first of all denoising time steps in the Wensheng image model. The noise reduction time step.

8. The method according to claim 1, characterized in that, The preset threshold is an interval. Any value in the range.

9. A backdoor detection system for implementing the method of any one of claims 1-8, characterized in that, The system includes a text graph model, a cross-attention graph sequence extraction module, a first detection module, a second detection module, and a backdoor sample determination module, wherein: The text-based image model is used to generate images based on each prompt word text in the dataset to be tested. The cross-attention map sequence extraction module is used to extract the cross-attention map sequence corresponding to each denoising time step within a preset denoising time step interval from the text graph model for each prompt word text in the dataset to be tested. The first detection module is used to determine the average of the total change of the cross attention map corresponding to the end token of the prompt word text and the total change of the cross attention map corresponding to all semantic tokens within a preset denoising time step interval based on all cross attention map sequences of each extracted prompt word text, and use the difference as the first determination parameter of the prompt word text. The second detection module is used to determine the difference between the total change in the end token state of the prompt word text and the average of the total change in all semantic token states within a preset denoising time step interval, based on all cross-attention map sequences of each extracted prompt word text, and to use the difference as the second determination parameter of the prompt word text. The backdoor sample determination module is used to determine that the sample to which the description text belongs is a backdoor sample if either the first determination parameter or the second determination parameter is less than or equal to a preset threshold.

10. A computer-readable storage medium, characterized in that, It contains a computer program that can be executed by a processor to implement the steps of the method according to any one of claims 1 to 8.

Images