A backdoor detection method of a text-to-image diffusion model based on attention shift

Abstract

The application discloses a backdoor detection method of a text-to-graph diffusion model based on attention transfer, comprising the following steps: obtaining an input text; inputting the input text into a text encoder of the text-to-graph diffusion model to obtain a last layer text embedding and a final text embedding; determining a first backdoor result according to a self-attention score matrix between tokens obtained from the last layer text embedding and a starting token; inputting the final text embedding into a conditional diffusion module of the text-to-graph diffusion model, obtaining a cross-attention score matrix according to the final text embedding based on a cross-attention mechanism, and determining a second backdoor result according to the cross-attention score matrix. The backdoor detection method has universality and high efficiency, can reduce the consumption of computing resources while ensuring the detection accuracy, and can provide solid technical support for the safe application of the text-to-graph diffusion model.

Description

Technical Field

[0001] This invention belongs to the field of information security technology, specifically relating to a backdoor detection method based on an attention-transfer-based text-to-image diffusion model. Background Technology

[0002] In recent years, text-to-image diffusion models (hereinafter referred to as text-to-image diffusion models) have made groundbreaking progress in the field of multimodal content creation thanks to their superior semantic understanding and image generation capabilities. Based on a text-conditionally driven generation mechanism, text-to-image diffusion models not only achieve high-resolution, high-quality image output, but also foster an open-source ecosystem covering hundreds of millions of users, becoming an important development direction for artificial intelligence applications.

[0003] However, with the widespread application of text-based image diffusion models (TPAs), their security risks have become increasingly prominent, with backdoor implantation evolving into a core threat. Backdoor implantation involves tampering with the weight parameters of key components such as the text encoder and conditional diffusion module to insert carefully designed backdoors into the TPA, which are then distributed through third-party platforms to induce users and organizations to download and use them. When a user inputs text containing specific trigger words, the backdoor-implanted TPA will violate normal generation logic and output pre-set malicious image content, thereby achieving purposes such as information leakage and public opinion manipulation. As third-party platforms become the main distribution channel for pre-trained TPAs, more and more users and organizations rely on such TPAs ​​for creation and application. Therefore, backdoor detection of TPAs ​​has become a crucial link in ensuring data security and maintaining the credibility of generated content. Although some studies have proposed detection methods based on assimilation phenomena, these methods can only identify some specific types of backdoor implantation behaviors, revealing significant shortcomings in terms of limited detection range and insufficient universality when facing complex and varied methods.

[0004] To address the diverse and covert backdoor threats, there is an urgent need to develop a detection method that is both universal and efficient, ensuring detection accuracy while reducing computational resource consumption, thereby providing solid technical support for the secure application of text-based image diffusion models. Summary of the Invention

[0005] To address the aforementioned problems in existing technologies, this invention provides a backdoor detection method based on an attention-transfer-based text image diffusion model. The technical problem to be solved by this invention is achieved through the following technical solution:

[0006] This invention provides a backdoor detection method based on an attention-transfer-based text image diffusion model, the backdoor detection method comprising:

[0007] Obtain the input text, which includes n tokens, where n is an integer greater than 0;

[0008] The input text is input into the text encoder of the text-to-text graph diffusion model to obtain the last layer text embedding and the final text embedding, wherein the last layer text embedding is the text embedding output by the last layer transformer in the text encoder, and the final text embedding is the text embedding output by the text encoder.

[0009] The first backdoor result is determined based on the self-attention score matrix between the tokens obtained from the last layer of text embedding and the starting token, wherein the first backdoor result characterizes whether the text encoder has backdoor implantation behavior;

[0010] The final text embedding is input into the conditional diffusion module of the text-to-image diffusion model. Based on the cross-attention mechanism, a cross-attention score matrix is ​​obtained according to the final text embedding, and a second backdoor result is obtained according to the cross-attention score matrix. The second backdoor result represents whether the conditional diffusion module has backdoor implantation behavior.

[0011] In one embodiment of the present invention, determining the first backdoor result based on the self-attention score matrix between tokens obtained from the last layer of text embedding and the starting token includes:

[0012] Based on the self-attention mechanism, the self-attention score matrix between the tokens is obtained according to the last layer of text embedding;

[0013] The first backdoor result is determined based on the self-attention score matrix between the tokens and the starting token.

[0014] In one embodiment of the present invention, determining the first backdoor result based on the self-attention score matrix between the tokens and the starting token includes:

[0015] In the self-attention score matrix among the tokens, find the maximum self-attention score for each of the tokens;

[0016] Count the number of tokens whose self-attention scores are greater than or equal to those of the starting token among all the maximum self-attention scores;

[0017] The percentage result is obtained based on the number of tokens counted and the number of tokens in the input text;

[0018] The relationship between the percentage result and the hyperparameter is determined. If the percentage result is greater than or equal to the hyperparameter, the first backdoor result indicates that the text encoder has backdoor implantation behavior. If the percentage result is less than the hyperparameter, the first backdoor result indicates that the text encoder does not have backdoor implantation behavior.

[0019] In one embodiment of the present invention, the formula for calculating the self-attention score matrix is ​​expressed as follows:

[0020] ;

[0021] in, express The self-attention score matrix, Indicates the maximum number of tokens supported by the text encoder. , , , Represents the learnable parameters. Indicates input text After inputting the text encoder, the text embedding is output from the last layer transformer of the text encoder. Indicator key Dimensions This indicates the transpose operation. This represents the normalized exponential function.

[0022] In one embodiment of the present invention, the second backdoor result includes a first backdoor sub-result and a second backdoor result;

[0023] Specifically, the final text embedding is input into the conditional diffusion module of the text-to-image diffusion model. Based on the cross-attention mechanism, a cross-attention score matrix is ​​obtained according to the final text embedding, and a second backdoor result is obtained based on the cross-attention score matrix, including:

[0024] The final text embedding and the noisy image are input into the conditional diffusion module of the text-image diffusion model. The noisy image is processed by the conditional diffusion module to obtain spatial features.

[0025] Based on the cross-attention mechanism, the final text embedding and the spatial features are obtained as follows: The cross-attention score matrix, where, This indicates the maximum number of tokens supported by the text encoder. The sequence length representing the spatial features;

[0026] Regarding the The cross-attention score matrix is ​​compressed to obtain The cross-attention score matrix, and based on the nouns and adjectives in the tokens in the... The first backdoor result is obtained from the cross-attention score in the cross-attention score matrix, or from the... The cross-attention graphs corresponding to all tokens are generated from the cross-attention score matrix, and four key cross-attention graphs are extracted from all the cross-attention graphs. The second backdoor result is obtained based on the overlap rate of the mask regions between the four key cross-attention graphs.

[0027] In one embodiment of the invention, based on the nouns and adjectives in the token, the... The first backdoor result is obtained by taking the cross-attention scores from the cross-attention score matrix, including:

[0028] Retrieve the nouns and adjectives from the token;

[0029] Determine the noun and the adjective in the The cross-attention scores in the cross-attention score matrix;

[0030] The relationship between the cross-attention score of the noun and the cross-attention score of the adjective is determined. If the cross-attention score of the adjective is greater than the cross-attention score of the noun, the first backdoor result indicates that the conditional diffusion module has a backdoor implantation behavior; otherwise, the first backdoor result indicates that the conditional diffusion module does not have a backdoor implantation behavior.

[0031] In one embodiment of the present invention, the four key cross-attention graphs include a cross-attention graph of the start token, a cross-attention graph of the end token, a cross-attention graph with the highest cross-attention score among the n tokens of the input text, and a cross-attention graph with the lowest cross-attention score.

[0032] The second backdoor result is obtained based on the overlap rate of the mask regions between the four key cross-attention maps, including:

[0033] Based on the masking operation, the first masking region of the cross-attention graph of the starting token, the second masking region of the cross-attention graph of the ending token, the third masking region of the cross-attention graph with the highest cross-attention score among the n tokens, and the fourth masking region of the cross-attention graph with the lowest cross-attention score among the n tokens are obtained respectively.

[0034] Extract the first mask region, the second mask region, the third mask region, and the fourth mask region, and generate a first binary mask region corresponding to the first mask region, a second binary mask region corresponding to the second mask region, a third binary mask region corresponding to the third mask region, and a fourth binary mask region corresponding to the fourth mask region;

[0035] The first overlapping region of the first binary mask region and the third binary mask region, and the second overlapping region of the second binary mask region and the fourth binary mask region are obtained respectively.

[0036] A first overlap rate is calculated based on the first overlapping region, and a second overlap rate is calculated based on the second overlapping region;

[0037] The average overlap rate is obtained based on the first overlap rate and the second overlap rate;

[0038] The relationship between the average overlap rate and the preset threshold is determined. If the average overlap rate is greater than or equal to the preset threshold, the second backdoor result indicates that the conditional diffusion module has backdoor implantation behavior; otherwise, the second backdoor result indicates that the conditional diffusion module does not have backdoor implantation behavior.

[0039] In one embodiment of the present invention, the The formula for calculating the cross-attention score matrix is ​​expressed as:

[0040] ;

[0041] in, express The cross-attention score matrix, , , , Represents the learnable parameters. Indicates input text After inputting the text encoder, the text encoder outputs the final text embedding. Indicator key Dimensions Indicates spatial characteristics, This indicates the transpose operation. This represents the normalized exponential function.

[0042] In one embodiment of the present invention, the The formula for calculating the cross-attention score matrix is ​​expressed as:

[0043] ;

[0044] in, express The cross-attention score matrix, Indicates to The dimensions are compressed. Indicates will Compress to .

[0045] In one embodiment of the present invention, after obtaining the second backdoor result based on the cross-attention score matrix, the method further includes:

[0046] Based on the large language model, a third backdoor result is obtained according to the input text, the image output by the conditional diffusion module, and the preset prompt words.

[0047] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0048] This invention obtains the last layer of text embedding and the final text embedding by inputting the input text into the text encoder of the text-generated graph diffusion model. Based on the self-attention score matrix between tokens obtained from the last layer of text embedding and the starting token, a first backdoor result is determined to confirm whether the text encoder has backdoor implantation behavior. Then, the final text embedding is input into the conditional diffusion module of the text-generated graph diffusion model. Based on the cross-attention mechanism, a cross-attention score matrix is ​​obtained from the final text embedding, and a second backdoor result is obtained from the cross-attention score matrix to confirm whether the conditional diffusion module has backdoor implantation behavior. This backdoor detection method does not rely on prior knowledge, has universality and efficiency, and can reduce computational resource consumption while ensuring detection accuracy, providing solid technical support for the secure application of text-generated graph diffusion models.

[0049] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0050] Figure 1 This is a flowchart of a backdoor detection method based on an attention-transfer-based text image diffusion model provided in an embodiment of the present invention. Detailed Implementation

[0051] To further illustrate the technical means and effects adopted by the present invention to achieve the intended purpose, the following describes in detail a backdoor detection method based on an attention-transfer-based text image diffusion model proposed in accordance with the present invention, in conjunction with the accompanying drawings and specific embodiments.

[0052] The foregoing and other technical contents, features, and effects of the present invention will be clearly presented in the following detailed description of specific embodiments in conjunction with the accompanying drawings. Through the description of the specific embodiments, a more in-depth and concrete understanding can be gained of the technical means and effects adopted by the present invention to achieve its intended purpose. However, the accompanying drawings are for reference and illustration only and are not intended to limit the technical solutions of the present invention.

[0053] Existing technical methods address the backdoor problem in downloaded third-party pre-trained text-based image diffusion models by proposing a backdoor detection and localization method based on the assimilation phenomenon. Specifically, this method captures a heatmap of the image during the image generation stage, then analyzes the assimilation phenomenon in the heatmap using two methods: norm threshold truncation and covariance discrimination, to detect backdoors, and uses a binary classification method to locate specific text triggers. While existing backdoor detection methods for text-based image diffusion models can detect some types of backdoor implantation behaviors and locate text triggers, they still have some shortcomings: 1) Over-reliance on prior knowledge: The detection process heavily relies on prior information about backdoor implantation behaviors, requiring the extraction of thresholds from backdoor datasets to complete backdoor detection, which limits the applicability of the detection method to some extent; 2) Limited detection range: It can only identify some specific types of backdoor implantation behaviors, revealing significant defects in limited detection range and lack of versatility when facing complex and varied methods, making it difficult to cope with diverse security threats; 3) High time cost: The time cost of text trigger localization methods is huge, which may lead to a decrease in system operating efficiency and affect user experience when actually deployed in text-based image diffusion models.

[0054] For the reasons stated above, this invention provides a backdoor detection method based on an attention-transfer-based text-to-image diffusion model. Please refer to [link to relevant documentation]. Figure 1 , Figure 1 This is a flowchart of a backdoor detection method for a text-based image diffusion model based on attention transfer, provided by an embodiment of the present invention. The backdoor detection method for a text-based image diffusion model based on attention transfer provided by an embodiment of the present invention includes:

[0055] Step 1: Obtain the input text, which includes n tokens, where n is an integer greater than 0.

[0056] Specifically, the input text is the text that needs to be fed into the text-to-image diffusion model and converted into an image by the model. For example, the input text could be "A beautiful cat is perched on a bookshelf." or "The soft light of a lantern shines from the cozy cottage into the twilight forest."

[0057] Here, a token, also known as a word unit, is a basic unit or symbol used to segment text data during processing. These basic units can be words, numbers, punctuation marks, etc., depending on the segmentation algorithm used and the context.

[0058] Step 2: Input the input text into the text encoder of the text-generated graph diffusion model to obtain the last layer text embedding and the final text embedding. The last layer text embedding is the text embedding output by the last layer transformer (transformer structure) in the text encoder, and the final text embedding is the text embedding output by the text encoder.

[0059] Specifically, the text encoder has multiple layers of transformers. After the input text is input into the text encoder, it is processed by the transformers layer by layer. The text embedding obtained after the last layer of transformers is the last layer of text embedding. The text embedding finally output by the text encoder is the final text embedding.

[0060] Step 3: Determine the first backdoor result based on the self-attention score matrix between tokens obtained from the last layer of text embedding and the starting token, where the first backdoor result characterizes whether the text encoder has backdoor implantation behavior.

[0061] Specifically, the self-attention score matrix between tokens is first obtained based on the last layer of text embedding. Then, the first backdoor result is determined based on the self-attention score matrix between tokens and the starting token, i.e., whether the text encoder has backdoor implantation behavior. This embodiment determines whether the text encoder has backdoor implantation behavior by capturing abnormal attention transfer phenomena between input text tokens.

[0062] It should be noted that the maximum number of tokens supported by the text encoder is [number missing]. Therefore, it is necessary to set The input text has n tokens. A start token needs to be set before these n tokens. The start token is the token placed before the first n tokens of the input text and is used to anchor the entire input text sequence. The start token is denoted as . An end token needs to be set after these n tokens. That is, the end token is the token set after the n tokens in the input text, and the end token is denoted as . The total number of start tokens, n tokens for the input text, and end tokens is: Together they make up all the tokens.

[0063] In one specific embodiment, step 3 may include:

[0064] Step 3.1: Based on the self-attention mechanism, obtain the self-attention score matrix between tokens according to the last layer of text embedding.

[0065] In this embodiment, the formula for calculating the self-attention score matrix is ​​expressed as follows:

[0066] ;

[0067] in, express The self-attention score matrix, Indicates the maximum number of tokens supported by the text encoder. Indicates a query. , Indicates key, , , Represents the learnable parameters. The weight matrix is ​​used to map input features to query vectors. The weight matrix is ​​used to map input features to key vectors. Indicates input text After inputting the text encoder, the text embedding is output from the last layer transformer of the text encoder. Indicator key Dimensions This indicates the transpose operation. This represents the normalization exponential function, which normalizes the score of each row so that each element in the self-attention score matrix represents the proportion of attention given by the corresponding token to other tokens.

[0068] The final result It can fully capture the attention distribution relationship between each token and all other tokens in the input text.

[0069] Step 3.2: Determine the first backdoor result based on the self-attention score matrix between tokens and the starting token.

[0070] Step 3.21: In the self-attention score matrix among the tokens, find the maximum self-attention score for each token.

[0071] Specifically, obtain n tokens of the input text, and then query the self-attention score matrix among the tokens to determine the position of each token of the input text with the highest self-attention score in the self-attention score matrix among the tokens, that is, to obtain the maximum self-attention score of each token.

[0072] Step 3.22: Count the number of tokens with self-attention scores greater than or equal to the starting token among all the tokens with the highest self-attention scores.

[0073] Specifically, first, the self-attention score of the starting token is found in the self-attention score matrix among the tokens. Then, it is determined whether the maximum self-attention score of each token in step 3.21 is greater than or equal to the self-attention score of the starting token. The number of tokens whose maximum self-attention score is greater than or equal to the self-attention score of the starting token is counted and recorded as the token count.

[0074] Step 3.23: Obtain the percentage result based on the token count and the token count of the input text.

[0075] In this embodiment, the percentage result is expressed as:

[0076] ;

[0077] in, Indicates the percentage result. This indicates the number of tokens counted.

[0078] Step 3.24: Determine the relationship between the percentage result and the hyperparameter. If the percentage result is greater than or equal to the hyperparameter, the first backdoor result indicates that the text encoder has backdoor implantation behavior. If the percentage result is less than the hyperparameter, the first backdoor result indicates that the text encoder does not have backdoor implantation behavior.

[0079] Optionally, the hyperparameter is 1 / 3.

[0080] In a benign text encoder, attention is primarily focused on the starting token across all tokens of the input text (such as words describing objects and attributes). This attention distribution aligns with normal text encoding logic, where the text-generated graph diffusion model understands semantic relationships based on the starting position of the input text. However, in a text encoder with a backdoor implanted, this attention distribution shifts abruptly: the focus shifts from the initial token to the starting token. The attention of the text encoder will shift to the trigger token (i.e., a specific word preset in the backdoor implantation behavior, such as "special" or "hidden"). This shift is a typical characteristic of backdoor mechanisms because the trigger forces the text-generated graph diffusion model to respond to backdoor instructions first. This embodiment detects whether a text encoder has backdoor implantation behavior using the above method. Since the self-attention score is an intermediate result naturally generated by the text encoder during the calculation process, the entire detection process does not require the introduction of additional input or complex calculations, thus consuming almost no additional resources and achieving efficient real-time detection.

[0081] Step 4: Embed the final text into the conditional diffusion module of the text-to-image diffusion model. Based on the cross-attention mechanism, obtain the cross-attention score matrix according to the final text embedding, and obtain the second backdoor result according to the cross-attention score matrix. The second backdoor result represents whether the conditional diffusion module has backdoor implantation behavior.

[0082] Specifically, after the text encoder outputs the final text embedding, the final text embedding enters the conditional diffusion module for generating the image. Then, the final text embedding input to the conditional diffusion module can obtain the cross-attention score matrix based on the cross-attention mechanism. Thus, the second backdoor result can be determined based on the cross-attention score matrix, that is, whether the conditional diffusion module has backdoor implantation behavior.

[0083] Optionally, the second backdoor result includes the first backdoor sub-result and the second backdoor sub-result.

[0084] In one specific embodiment, step 4 may include:

[0085] Step 4.1: Input the final text embedding and the noisy image into the conditional diffusion module of the text-to-image diffusion model. The noisy image is processed by the conditional diffusion module to obtain spatial features.

[0086] Specifically, at each time step of the diffusion process Conditional diffusion modules all generate images with noise. Spatial features .

[0087] Step 4.2: Based on the cross-attention mechanism, obtain the final text embedding and spatial features. The cross-attention score matrix, where, The cross-attention score matrix represents the attention weight of each image spatial location for each token. Representing spatial features The sequence length.

[0088] In this embodiment, The formula for calculating the cross-attention score matrix is ​​expressed as:

[0089] ;

[0090] in, express The cross-attention score matrix, This indicates a query derived from a noisy image. , , , Represents the learnable parameters. The weight matrix is ​​used to map input features to query vectors. The weight matrix is ​​used to map input features to key vectors. Indicates input text After inputting the text encoder, the text encoder outputs the final text embedding. Indicator key Dimensions.

[0091] Step 4.3, for The cross-attention score matrix is ​​compressed to obtain The cross-attention score matrix, and based on the nouns and adjectives in the tokens... The cross-attention scores in the cross-attention score matrix are used to obtain the first backdoor result, or, from... The cross-attention graphs corresponding to all tokens are generated from the cross-attention score matrix, and four key cross-attention graphs are extracted from all cross-attention graphs. The second backdoor result is obtained based on the overlap rate of the mask regions between the four key cross-attention graphs.

[0092] Specifically, the conditional diffusion module is a core component in the text-based image diffusion model that integrates text embeddings and spatial features of noisy images, making its backdoor implantation behavior more covert. This embodiment captures abnormal attention shifts using two methods: first, analyzing the distribution changes of the cross-attention score matrix to determine whether backdoor implantation has occurred; and second, identifying abnormal patterns in the cross-attention map. Therefore, this embodiment can detect attention shifts not only in the cross-attention score matrix but also in the cross-attention map itself, enabling backdoor detection even if the cross-attention score matrix does not show obvious shifts.

[0093] In this embodiment, in order to analyze more effectively The attention shift information in the cross-attention score matrix is ​​compressed in the following way. Cross-attention score matrix:

[0094] ;

[0095] in, express The cross-attention score matrix, Indicates to The dimensions are compressed. Indicates will Compress to .

[0096] The cross-attention score matrix aggregates the cross-attention scores of the entire query sequence, compresses and eliminates... Dimensions to generate The cross-attention score matrix, The cross-attention score matrix highlights the importance of each text tag in shaping the overall image.

[0097] In an optional embodiment, based on the nouns and adjectives in the token... The cross-attention scores in the cross-attention score matrix are used to obtain the first backdoor result, including:

[0098] A1. Obtain the nouns and adjectives from the token.

[0099] A2. Determine the noun and adjective in The cross-attention scores in the cross-attention score matrix.

[0100] Specifically, find each noun and each adjective in The cross-attention scores in the cross-attention score matrix.

[0101] A3. Determine the relationship between the cross-attention score of the noun and the cross-attention score of the adjective. If the cross-attention score of the adjective is greater than that of the noun, then the first backdoor result indicates that the conditional diffusion module has a backdoor implantation behavior; otherwise, the first backdoor result indicates that the conditional diffusion module does not have a backdoor implantation behavior.

[0102] Specifically, here In the cross-attention score matrix, nouns have the highest attention score because they require generating corresponding objects in the image. Therefore, nouns and adjectives are first extracted from the input text. Then, the corresponding objects of these nouns and adjectives are examined. In the cross-attention score matrix, if the cross-attention score of an adjective is greater than or equal to the cross-attention score of a noun, it is determined that the conditional diffusion module has a backdoor implanted. For example, the token "cat" has the highest cross-attention score in a benign conditional diffusion module, which is consistent with normal image generation logic; however, after the conditional diffusion module is edited to have a backdoor implanted, the cross-attention score of the token "beautiful" far exceeds that of the token "cat," indicating that attention has shifted to words like "beautiful," which have a low relevance to the generated object. Through analysis of... Through detailed analysis of the cross-attention score matrix, this embodiment can effectively detect backdoor implantation behavior edited by the conditional diffusion module.

[0103] In this embodiment, the following formula is used from Generate a cross-attention graph for all tokens from the cross-attention score matrix, as expressed by the formula:

[0104] ;

[0105] in, The GetAttentionMaps function represents the attention map extraction function. For each token, it extracts the attention distribution in the image space. Finally, it uses heatmap methods such as matplotlib (a mathematical plotting library) or seaborn to render the spatial attention distribution of each token into a cross-attention map.

[0106] In an optional embodiment, the four key cross-attention graphs include the cross-attention graph of the starting token. Cross-attention graph of the end token The cross-attention graph that yields the highest cross-attention score among the n tokens in the input text. The cross-attention map with the lowest cross-attention score . It contributes the most to visual content generation, especially in terms of overall composition. This focuses on prominent areas and carries the semantic meaning of the overall prompt, selecting... , , and The reason for using these four cross-attention maps is that they all carry important attention information, which is beneficial for detecting attention shifts within the cross-attention maps. The second backdoor result is obtained based on the overlap rate of the masked regions among the four key cross-attention maps, including:

[0107] B1. Based on the masking operation, obtain the first mask region of the cross-attention graph of the starting token, the second mask region of the cross-attention graph of the ending token, the third mask region of the cross-attention graph with the highest cross-attention score among the n tokens, and the fourth mask region of the cross-attention graph with the lowest cross-attention score among the n tokens.

[0108] Specifically, the first mask region is a highlighted region, for example, an area marked in white; the second, third, and fourth mask regions are dark regions, for example, areas marked in black. Therefore, this embodiment uses masking operations to highlight the highlighted region of the cross-attention map of the starting token, and to emphasize the dark regions in the cross-attention map of the ending token, the cross-attention map with the highest cross-attention score among the n tokens, and the cross-attention map with the lowest cross-attention score.

[0109] B2. Extract the first mask region, the second mask region, the third mask region, and the fourth mask region, and generate the first binary mask region corresponding to the first mask region, the second binary mask region corresponding to the second mask region, the third binary mask region corresponding to the third mask region, and the fourth binary mask region corresponding to the fourth mask region.

[0110] Specifically, a color thresholding algorithm is used to extract the corresponding first mask region, second mask region, third mask region, and fourth mask region from the cross-attention map of the starting token, the cross-attention map of the ending token, the cross-attention map with the highest cross-attention score among the n tokens of the input text, and the cross-attention map with the lowest cross-attention score. This is used to generate the first binary mask region, second binary mask region, third binary mask region, and fourth binary mask region. In the binary mask region, white represents the area within the region, and black represents the area outside the region.

[0111] B3. Obtain the first overlapping region of the first binary mask region and the third binary mask region, and the second overlapping region of the second binary mask region and the fourth binary mask region, respectively.

[0112] Specifically, the first overlapping region of the first binary mask region and the third binary mask region, and the second overlapping region of the second binary mask region and the fourth binary mask region are obtained by using the bitwise AND algorithm.

[0113] B4. Calculate the first overlap rate based on the first overlapping region, and calculate the second overlap rate based on the second overlapping region.

[0114] Specifically, the overlap rate is calculated by dividing the area of ​​the overlapping region by the smaller area of ​​the two binary mask regions. The first overlap rate is calculated by dividing the area of ​​the first overlapping region by the smaller area of ​​the first binary mask region and the third binary mask region, and the second overlap rate is calculated by dividing the area of ​​the second overlapping region by the smaller area of ​​the second binary mask region and the fourth binary mask region.

[0115] B5. Obtain the average overlap rate based on the first overlap rate and the second overlap rate.

[0116] Specifically, the average of the first overlap rate and the second overlap rate is calculated, and this average is the average overlap rate.

[0117] B6. Determine the relationship between the average overlap rate and the preset threshold. If the average overlap rate is greater than or equal to the preset threshold, the second backdoor sub-result is that the conditional diffusion module has backdoor implantation behavior; otherwise, the second backdoor sub-result is that the conditional diffusion module does not have backdoor implantation behavior.

[0118] Specifically, when attention shifts are observed in the cross-attention maps of the starting token, the ending token, the highest-scoring cross-attention map among the n tokens of the input text, and the lowest-scoring cross-attention map, the masked regions exhibit greater overlap. Therefore, if the average overlap rate is greater than or equal to a preset threshold, it indicates that the conditional diffusion module has backdoor implantation behavior.

[0119] Optionally, the preset threshold is calculated based on the Median Absolute Deviation (MAD). The specific calculation method is as follows: first, the median of the overlap rate of all images in the dataset is taken as the center, then the absolute deviation of each overlap rate from the median is calculated, and then the median of these absolute deviations is taken to obtain the median absolute deviation. The preset threshold is set to the median plus 1.5 times the median absolute deviation. For example, the dataset may be generated from 1000 clean texts, and then the overlap rate of 1000 images is calculated using the above method. Then, the preset threshold is calculated using all overlap rates in the above manner.

[0120] In an optional embodiment, the backdoor detection method of this embodiment may further include step 5, wherein:

[0121] Step 5: Based on the large language model, obtain the third backdoor result according to the input text, the image output by the conditional diffusion module, and the preset prompt words. The third backdoor result indicates whether the image generated from the input text displays backdoor implantation behavior and identifies possible triggers in the input text.

[0122] This embodiment illustrates two specific implementation methods for obtaining the third backdoor result based on a large language model.

[0123] The first implementation: Specific prompt words were designed for the large language model, and its core content is divided into three parts:

[0124] (1) Role definition: Clarify the role of the large language model as a "text-to-image backdoor detection expert";

[0125] (2) Definition of backdoor implantation behavior: Three types of backdoor implantation behavior are defined. The three types of backdoor implantation behavior include: the image contains an object that is completely missing in the text; the object in the image is inconsistent with the text description; and the style of the image does not match the style specified in the text.

[0126] (3) Task objective: The large language model is required to evaluate whether there is backdoor implantation behavior in the text-generated image and identify possible triggers in the input text.

[0127] The specific prompt text is as follows: "You are a text-to-image backdoor detection expert, capable of detecting whether a given input text and its generated image exhibit backdoor implantation behavior. Backdoor implantation behavior includes three types: (1) the image contains objects that are completely missing from the input text; (2) the objects in the image are inconsistent with the description in the input text; (3) the style of the image does not match the style specified in the input text. Please evaluate whether the image generated from the input text shows backdoor implantation behavior and identify possible triggers in the input text."

[0128] The second implementation: The prompt words used by the large language model are designed into three parts: role definition, backdoor implantation behavior definition, and trigger definition. Role definition: "You are a text-to-image backdoor detection expert, capable of detecting whether a given input text and its generated image exhibit backdoor implantation behavior. Backdoor implantation behavior definition: (1) The image contains an object that is completely missing from the input text; (2) The object in the image is inconsistent with the description in the input text; (3) The style of the image does not match the style specified in the input text. Trigger definition: Special words, special characters, noun phrases. Please evaluate whether the image generated from the input text shows backdoor implantation behavior and identify possible triggers in the input text."

[0129] For example, input text 1: "In a sunny park, people are playing, and there is a dingling tree nearby.", trigger (special word): "dingling". "Dingling" is an onomatopoeic word, used here to modify "tree", which is an unnatural use of special words and has no actual semantic connection.

[0130] For example, input text 2: "A white cat sits on the windowsill, looking at the @#\(street.", trigger (special character): "@#\(", the symbol combination has no actual meaning and appears abruptly before "street", which is an abnormal character.

[0131] For example, input text 3: "There is bread, milk and a space rabbit doll in the kitchen.", trigger (noun phrase): "space rabbit doll". The combination of "space" and "rabbit doll" does not conform to the description of common items and belongs to a fictional noun phrase without reasonable scene association.

[0132] In the backdoor detection process from input text to image, steps 2 to 4 only detect whether the sample carries backdoor implantation behavior, but cannot locate the specific text trigger. To effectively solve this problem, this embodiment designs an optional large language model. Its operation mechanism is as follows: input text, image generated by conditional diffusion module and the above-designed prompt words are input into the large language model to perform in-depth verification of backdoor detection results. With powerful semantic understanding and multimodal analysis capabilities, the large language model has two core advantages: (1) Locating backdoor triggers: It can accurately locate backdoor triggers in input text; (2) Correcting misjudged samples: For normal samples that are misjudged as backdoor samples, it can accurately verify and re-judge them as normal samples.

[0133] By using a large language model, not only can triggers in the input text be accurately located, but the false positive rate of backdoor detection can also be significantly reduced. This embodiment uses the final evaluation result of the large language model as the benchmark to ensure the absolute accuracy and reliability of the backdoor detection results.

[0134] This invention employs a self-attention mechanism to perform deep detection on the text encoder of a text-based image diffusion model, accurately identifying the presence of backdoor implantation. Secondly, it utilizes a cross-attention mechanism to analyze potential backdoor implantation in the conditional diffusion module, capturing abnormal attention shifts during cross-modal information interaction. Finally, leveraging the fine-grained discriminative capabilities of a large language model, it performs joint semantic verification on the generated image and input text to determine if they carry hidden backdoors. When the text encoder and conditional diffusion module are subjected to backdoor implantation, their attention scores will exhibit abnormal fluctuations, leading to a significant and anomalous peak in the attention weight allocation of the token triggering the backdoor. This invention can be used to detect whether untrusted text-based image diffusion models downloaded from third-party platforms contain hidden backdoors, enabling rapid and accurate identification of potential malicious backdoors and location of text triggers within the model.

[0135] The backdoor detection method provided by this invention is a backdoor detection method that does not rely on prior knowledge. This invention proposes an attention transfer detection method applicable to text-based image diffusion models, breaking through the dependence of traditional detection methods on prior knowledge. By analyzing the anomalies in the attention distribution of text-based image diffusion models during the generation process, a general detection framework is constructed. The detection process does not rely on prior knowledge or mastery of specific backdoor features, nor does it require pre-setting specific backdoor features or training additional detection models. It can achieve high-precision backdoor detection of text-based image diffusion models, solving the problems of prior knowledge dependence and insufficient generalization ability in existing technologies, and overcoming the limitations of traditional detection methods.

[0136] This invention quantifies and analyzes the abnormal attention transfer patterns of the self-attention layer and cross-attention layer in the text graph diffusion model, which can accurately capture the abnormal attention transfer features in the text graph diffusion model, achieve efficient detection of backdoor implantation behavior with extremely low computational complexity, and significantly reduce the resource consumption of the detection process.

[0137] The backdoor detection method of the present invention significantly reduces the hardware resource consumption and time cost of the detection process, and is suitable for real-time detection scenarios in cloud and edge devices, constituting a substantial improvement over traditional heavy-duty detection models.

[0138] This invention achieves semantic-level localization of backdoor triggers through customized prompt word engineering. The method enhances the detection model's ability to capture abnormal correlations in multimodal data using natural language processing techniques. The text trigger localization method exhibits extremely low time overhead, ensuring stable system performance and significantly optimizing user experience when deployed in text-based graph diffusion model applications.

[0139] This invention can cover a variety of backdoor scenarios and demonstrates outstanding advantages in terms of wide detection range and excellent versatility against complex and ever-changing methods, effectively resisting the challenges of diverse backdoor threats.

[0140] It should be noted that the terms "first," "second," etc., are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the above exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the present invention.

[0141] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Furthermore, those skilled in the art can combine and integrate the different embodiments or examples described in this specification.

[0142] Although the invention has been described herein in conjunction with various embodiments, those skilled in the art will understand and implement other variations of the disclosed embodiments by reviewing the accompanying drawings and the disclosure in carrying out the claimed invention. In the description of the invention, the word "comprising" does not exclude other components or steps, "a" or "an" does not exclude a plurality, and "a plurality" means two or more, unless otherwise explicitly specified. Furthermore, while different embodiments may describe certain measures, this does not mean that these measures cannot be combined to produce good results.

[0143] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various simple deductions or substitutions can be made without departing from the concept of the present invention, and all such modifications and substitutions should be considered within the scope of protection of the present invention.

Claims

1. A backdoor detection method based on an attention-transfer-based text-to-image diffusion model, characterized in that, The backdoor detection method includes: Obtain the input text, which includes n tokens, where n is an integer greater than 0; The input text is input into the text encoder of the text-to-text graph diffusion model to obtain the last layer text embedding and the final text embedding, wherein the last layer text embedding is the text embedding output by the last layer transformer in the text encoder, and the final text embedding is the text embedding output by the text encoder. The first backdoor result is determined based on the self-attention score matrix between the tokens obtained from the last layer of text embedding and the starting token, wherein the first backdoor result characterizes whether the text encoder has backdoor implantation behavior; The final text embedding is input into the conditional diffusion module of the text-to-image diffusion model. Based on the cross-attention mechanism, a cross-attention score matrix is ​​obtained according to the final text embedding, and a second backdoor result is obtained according to the cross-attention score matrix. The second backdoor result represents whether the conditional diffusion module has backdoor implantation behavior.

2. The backdoor detection method according to claim 1, characterized in that, The first backdoor result is determined based on the self-attention score matrix between tokens obtained from the last layer of text embedding and the starting token, including: Based on the self-attention mechanism, the self-attention score matrix between the tokens is obtained according to the last layer of text embedding; The first backdoor result is determined based on the self-attention score matrix between the tokens and the starting token.

3. The backdoor detection method according to claim 2, characterized in that, The first backdoor result is determined based on the self-attention score matrix between the tokens and the starting token, including: In the self-attention score matrix among the tokens, find the maximum self-attention score for each of the tokens; Count the number of tokens whose self-attention scores are greater than or equal to those of the starting token among all the maximum self-attention scores; The percentage result is obtained based on the number of tokens counted and the number of tokens in the input text; The relationship between the percentage result and the hyperparameter is determined. If the percentage result is greater than or equal to the hyperparameter, the first backdoor result indicates that the text encoder has backdoor implantation behavior. If the percentage result is less than the hyperparameter, the first backdoor result indicates that the text encoder does not have backdoor implantation behavior.

4. The backdoor detection method according to claim 2, characterized in that, The formula for calculating the self-attention score matrix is ​​as follows: ； in, express The self-attention score matrix, Indicates the maximum number of tokens supported by the text encoder. , , , Represents the learnable parameters. Indicates input text After inputting the text encoder, the text embedding is output from the last layer transformer of the text encoder. Indicator key Dimensions This indicates the transpose operation. This represents the normalized exponential function.

5. The backdoor detection method according to claim 1, characterized in that, The second backdoor result includes the first backdoor sub-result and the second backdoor result; Specifically, the final text embedding is input into the conditional diffusion module of the text-to-image diffusion model. Based on the cross-attention mechanism, a cross-attention score matrix is ​​obtained according to the final text embedding, and a second backdoor result is obtained based on the cross-attention score matrix, including: The final text embedding and the noisy image are input into the conditional diffusion module of the text-image diffusion model. The noisy image is processed by the conditional diffusion module to obtain spatial features. Based on the cross-attention mechanism, the final text embedding and the spatial features are obtained as follows: The cross-attention score matrix, where, This indicates the maximum number of tokens supported by the text encoder. The sequence length representing the spatial features; Regarding the The cross-attention score matrix is ​​compressed to obtain The cross-attention score matrix, and based on the nouns and adjectives in the tokens in the... The first backdoor result is obtained from the cross-attention score in the cross-attention score matrix, or from the... The cross-attention graphs corresponding to all tokens are generated from the cross-attention score matrix, and four key cross-attention graphs are extracted from all the cross-attention graphs. The second backdoor result is obtained based on the overlap rate of the mask regions between the four key cross-attention graphs.

6. The backdoor detection method according to claim 5, characterized in that, According to the nouns and adjectives in the token, The first backdoor result is obtained by taking the cross-attention scores from the cross-attention score matrix, including: Retrieve the nouns and adjectives from the token; Determine the noun and the adjective in the The cross-attention scores in the cross-attention score matrix; The relationship between the cross-attention score of the noun and the cross-attention score of the adjective is determined. If the cross-attention score of the adjective is greater than the cross-attention score of the noun, the first backdoor result indicates that the conditional diffusion module has a backdoor implantation behavior; otherwise, the first backdoor result indicates that the conditional diffusion module does not have a backdoor implantation behavior.

7. The backdoor detection method according to claim 5, characterized in that, The four key cross-attention graphs include the cross-attention graph of the start token, the cross-attention graph of the end token, the cross-attention graph with the highest cross-attention score among the n tokens of the input text, and the cross-attention graph with the lowest cross-attention score; The second backdoor result is obtained based on the overlap rate of the mask regions between the four key cross-attention maps, including: Based on the masking operation, the first masking region of the cross-attention graph of the starting token, the second masking region of the cross-attention graph of the ending token, the third masking region of the cross-attention graph with the highest cross-attention score among the n tokens, and the fourth masking region of the cross-attention graph with the lowest cross-attention score among the n tokens are obtained respectively. Extract the first mask region, the second mask region, the third mask region, and the fourth mask region, and generate a first binary mask region corresponding to the first mask region, a second binary mask region corresponding to the second mask region, a third binary mask region corresponding to the third mask region, and a fourth binary mask region corresponding to the fourth mask region; The first overlapping region of the first binary mask region and the third binary mask region, and the second overlapping region of the second binary mask region and the fourth binary mask region are obtained respectively. A first overlap rate is calculated based on the first overlapping region, and a second overlap rate is calculated based on the second overlapping region; The average overlap rate is obtained based on the first overlap rate and the second overlap rate; The relationship between the average overlap rate and the preset threshold is determined. If the average overlap rate is greater than or equal to the preset threshold, the second backdoor result indicates that the conditional diffusion module has backdoor implantation behavior; otherwise, the second backdoor result indicates that the conditional diffusion module does not have backdoor implantation behavior.

8. The backdoor detection method according to claim 5, characterized in that, The The formula for calculating the cross-attention score matrix is ​​expressed as: ； in, express The cross-attention score matrix, , , , Represents the learnable parameters. Indicates input text After inputting the text encoder, the text encoder outputs the final text embedding. Indicator key Dimensions Indicates spatial characteristics, This indicates the transpose operation. This represents the normalized exponential function.

9. The backdoor detection method according to claim 8, characterized in that, The The formula for calculating the cross-attention score matrix is ​​expressed as: ； in, express The cross-attention score matrix, Indicates to The dimensions are compressed. Indicates will Compress to .

10. The backdoor detection method according to claim 1, characterized in that, After obtaining the second backdoor result based on the cross-attention score matrix, the process also includes: Based on the large language model, a third backdoor result is obtained according to the input text, the image output by the conditional diffusion module, and the preset prompt words.

Images