Cross-modal collaborative anti-attack method and device

Abstract

The present invention discloses a cross-modal collaborative adversarial attack method and device, which relates to the field of artificial intelligence security. It is used to solve the current problems of difficulty in modeling cross-modal collaborative attacks and difficulty in building optimization mechanisms in the field of large language model adversarial attacks. The cross-modal collaborative attack provided by this method can circumvent these security alignment mechanisms based on single-modal content feedback, thereby breaking through existing protection measures and increasing the complexity and unpredictability of attacks; furthermore, by optimizing adversarial perturbations in key cross-modal semantic subspaces and combining historical and future gradient information, the efficiency, convergence speed and stability of perturbation optimization can be improved, avoiding falling into local optimal solutions, thereby improving the practicality and operability of attacks.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence security, and more specifically to a cross-modal collaborative counterattack method and device. Background Art

[0002] As an important branch of the field of generative artificial intelligence (AI), large language models (LLM) are increasingly becoming a key force in promoting social progress and technological innovation. However, the LLM backbone architecture is extremely complex, the weight parameters are huge, and the decision-making mechanism operates in a black box. Although the interactive understanding and content generation capabilities are constantly improving, its reasoning illusion affects the credibility of the generated content, and the emergence of intelligence leads to the unpredictability of model decisions. The sensitivity to exogenous attacks and endogenous security vulnerabilities are becoming increasingly prominent, resulting in unprecedented challenges in application security, decision reliability, and capability controllability, inducing security risks in many fields.

[0003] Most existing research focuses on attacks using text input and prompt injection. However, the effectiveness of attacks against text instruction prompts is easily weakened by preference-based reinforcement learning alignment technology. This type of value alignment mechanism uses AI feedback to mark the generated text content, automatically completes LLM adversarial training, directly rejects illegal instructions and harmful queries, and ensures the generation of text content based on harmless, useful, and true principles. In other words, this type of single-modal attack for text instructions is difficult to overcome the security alignment mechanism based on text content feedback, is easily automatically detected and forcibly filtered, and is difficult to form an advanced persistent threat attack situation.

[0004] At present, adversarial attack technology for LLM has developed from single-modal attack to multi-modal attack, which mainly uses the characteristics of multi-modal data such as text, image, audio, and video to independently design adversarial prompts; because it does not consider the representation complementarity of each modal semantics and the adversarial synergy of cross-modal interaction, nor does it effectively integrate the harmful text semantics and the adversarial graph semantics generated by the prompt words, and there is also a lack of cross-modal collaborative adversarial attack modeling research, resulting in the existing multi-modal attacks. Although they can circumvent the security alignment mechanism based on text content feedback, the attack effectiveness is still limited, the attack success rate is low, and the attack cost remains high. It is even more difficult to break through the security alignment limitations based on multimodal content feedback; further, in the design of adversarial attack strategies for LLM, a pure gradient descent search strategy is usually used to optimize adversarial perturbations in a single-modal input semantic space. Since there is no effective cross-modal embedded semantic space adversarial perturbation optimization mechanism, there is a lack of positioning of key graphic semantic subspaces, and there is no fusion of momentum information of historical gradients and massive information of future gradients in the perturbation optimization process to obtain cross-modal generalized gradients, the optimization efficiency is low, the convergence stability is poor, and it is even more difficult to generate adversarial perturbations with the minimum cross-modal cost. Summary of the invention

[0005] The embodiments of the present invention provide a cross-modal collaborative attack method and device to solve the current problems in the field of large language model adversarial attacks, such as difficulty in modeling cross-modal collaborative attacks and difficulty in building optimization mechanisms.

[0006] The embodiment of the present invention provides a cross-modal collaborative anti-attack method, including:

[0007] The harmful text is obtained by modifying the initial text based on the harmful semantics related to the attack topic, determining the initial image related to the harmful text in the initial key cross-modal semantic subspace, generating an adjustment strategy according to the harmful semantics, and obtaining an initial adversarial image according to the adjustment strategy; the first key semantics in the initial text is input into the prediction model to obtain a plurality of first similar words similar to the first key semantics, and the first similar words replace the first key semantics to obtain the initial adversarial text; wherein the initial adversarial image and the initial adversarial sample form an initial adversarial image-text multimodal data sample;

[0008] Determine a first loss function of the initial adversarial image and the initial image, and obtain the gradient of the image modality, the Jacobian matrix of the image modality, and the Hessian matrix of the image modality in sequence based on the first loss function; perform weighted fusion on the gradient of the image modality, the Jacobian matrix of the image modality, the Hessian matrix of the image modality, the gradient of the text modality, the Jacobian matrix of the text modality, and the Hessian matrix of the text modality to obtain a cross-modal generalized gradient; wherein the image modality includes an initial adversarial image and an initial image, and the text modality includes an initial adversarial text and an initial text;

[0009] Based on the cross-modal generalized gradient, the initial adversarial text obtains a plurality of second similar words similar to the second key semantics through the top-k prediction of the top k rankings, and the second key semantics are replaced in sequence according to the score order of the second similar words to obtain a first adversarial text, and the first adversarial text and the initial adversarial text are input into the attack target model, and if the obtained first predicted label is different from the initial predicted label and meets the attack theme, the first adversarial text is determined as the final adversarial text for successful attack;

[0010] A text key semantic subspace is obtained based on the key similar words included in the final adversarial text, and feature alignment is performed on the text key semantic subspace and the image key semantic subspace based on the attention mechanism; the key similar words included in the text key semantic subspace are strengthened to obtain strengthened key semantics, the strengthened key semantics are mapped to the image key semantic subspace, and the second key area included in the image key semantic subspace is modified according to the strengthened key semantics to obtain a strengthened key area; and a strengthened adversarial text and a strengthened adversarial image are obtained according to the strengthened key semantics and the strengthened key area, respectively.

[0011] Preferably, before modifying the initial text based on harmful semantics related to the attack topic to obtain the harmful text, the method further includes:

[0012] According to a set semantic localization method, a first key area of ​​an initial image and a first key semantic of an initial text are determined; according to the first key area and the first key semantic, an initial association point between the initial image and the initial text is determined, and according to the initial association point, an initial key cross-modal semantic subspace is obtained; wherein the initial text and the initial image constitute an initial multimodal data sample.

[0013] Preferably, determining a first key area of ​​an initial image and a first key semantics of an initial text according to a set semantic localization method, determining an initial association point of the initial image and the initial text according to the first key area and the first key semantics, and obtaining an initial key cross-modal semantic subspace according to the initial association point specifically include:

[0014] Determine the key area of ​​the initial image based on the Grad-CAM++ graph semantic localization method, and determine the first keyword of the initial text based on the word mask text semantic localization method;

[0015] Determine an influence region that affects the classification result of the initial image according to the first key region, determine influence words included in the initial text according to the first key semantics, the influence words are used to influence the prediction result of the attack target model, and determine initial association points with semantic correspondence among the plurality of influence regions and the plurality of influence words;

[0016] Converting pixel information of the influence area corresponding to the initial association point in the initial image into a visual feature vector of fixed dimension, and converting the influence words corresponding to the initial association point in the initial text into text vector information;

[0017] The visual feature vector and the text vector information are combined to obtain a plurality of joint feature vectors, and an initial key cross-modal semantic subspace is obtained according to the plurality of joint feature vectors.

[0018] Preferably, the determining of the first loss function of the initial adversarial image and the initial image, and sequentially obtaining the gradient of the image modality, the Jacobian matrix of the image modality, and the Hessian matrix of the image modality based on the first loss function, specifically includes:

[0019] Inputting the initial image and the initial adversarial image into the attack target model, obtaining the true image label and the image prediction category label in turn, obtaining a first loss function according to the true image label and the image prediction category label, and obtaining the gradient of the image modality by back propagation in the first loss function;

[0020] Obtaining the Jacobian matrix of the image modality according to the first loss function and the Jacobian matrix;

[0021] Obtaining a Hessian matrix of the image modality according to the first loss function and the Hessian matrix;

[0022] Among them, the first loss function, the gradient of the image modality, the Jacobian matrix of the image modality and the Hessian matrix of the image modality are as follows:

[0023]

[0024]

[0025]

[0026]

[0027] in, represents the first loss function, represents the true label of the initial image, represents the predicted class probability of the initial adversarial image, The base is The logarithmic function of represents the gradient of the image modality, The Jacobian matrix representing the image modality, represents the Hessian matrix of the image modality, represents the input data in image modality, Represents different categories in the image modality.

[0028] Preferably, the cross-modal generalized gradient is as follows:

[0029]

[0030] in, represents the cross-modal generalized gradient, represents the weighting coefficient of the gradient of the image modality, α∈[0,1] ; represents the weighting coefficients of the Jacobian matrix of the image modality, β∈[0,1] ; represents the weighting coefficients of the Hessian matrix of the image modality, γ∈[0,1] ; represents the weighting coefficient of the gradient of the text modality, δ∈[0,1] ; represents the weight coefficients of the Jacobian matrix of the text modality, ϵ∈[0,1] ; represents the weight coefficients of the Hessian matrix of the text modality, ζ∈[0,1] ; represents the gradient of the image modality, The Jacobian matrix representing the image modality, represents the Hessian matrix of the image modality, A gradient representing the text modality, The Jacobian matrix representing the text modality, represents the Hessian matrix of the text modality, represents the input data in image modality, Represents input data for a text modal.

[0031] Preferably, after inputting the first adversarial text and the initial adversarial text into the attack target model, the method further includes:

[0032] If the obtained first predicted label is similar to the initial predicted label, find the first text predicted label output by the attack target model with the maximum misprediction risk from the first adversarial texts corresponding to the plurality of second similar words with similar second key semantics;

[0033] The first adversarial text having the greatest risk of misprediction is adjusted according to the cross-modal generalized gradient to obtain the final adversarial text.

[0034] The embodiment of the present invention provides a cross-modal collaborative anti-attack device, including:

[0035] A first obtaining unit is used to modify an initial text based on harmful semantics related to an attack topic to obtain harmful text, determine an initial image related to the harmful text in an initial key cross-modal semantic subspace, generate an adjustment strategy according to the harmful semantics, and obtain an initial adversarial image according to the adjustment strategy; input a first key semantic in the initial text into a prediction model to obtain a plurality of first similar words similar to the first key semantic, and replace the first key semantic with the first similar words to obtain an initial adversarial text; wherein the initial adversarial image and the initial adversarial sample form an initial adversarial graphic-text multimodal data sample;

[0036] A second obtaining unit is used to determine a first loss function of the initial adversarial image and the initial image, and based on the first loss function, sequentially obtain the gradient of the image modality, the Jacobian matrix of the image modality, and the Hessian matrix of the image modality; the gradient of the image modality, the Jacobian matrix of the image modality, the Hessian matrix of the image modality, the gradient of the text modality, the Jacobian matrix of the text modality, and the Hessian matrix of the text modality are weightedly fused to obtain a cross-modal generalized gradient; wherein the image modality includes an initial adversarial image and an initial image, and the text modality includes an initial adversarial text and an initial text;

[0037] A determination unit is used to obtain a plurality of second similar words similar to the second key semantics through the top-k predictions of the first k rankings of the initial adversarial text based on the cross-modal generalized gradient, replace the second key semantics in sequence according to the score order of the second similar words, and obtain a first adversarial text, input the first adversarial text and the initial adversarial text into the attack target model, and if the obtained first predicted label is different from the initial predicted label and meets the attack theme, determine the first adversarial text as the final adversarial text for successful attack;

[0038] The third obtaining unit is used to obtain a text key semantic subspace based on the key similar words included in the final adversarial text, and perform feature alignment on the text key semantic subspace and the image key semantic subspace based on the attention mechanism; strengthen the key similar words included in the text key semantic subspace to obtain enhanced key semantics, map the enhanced key semantics to the image key semantic subspace, modify the second key area included in the image key semantic subspace according to the enhanced key semantics, and obtain an enhanced key area; and obtain an enhanced adversarial text and an enhanced adversarial image according to the enhanced key semantics and the enhanced key area, respectively.

[0039] An embodiment of the present invention provides a computer device, which includes a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the above-mentioned cross-modal collaborative anti-attack method.

[0040] An embodiment of the present invention provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the processor executes the above-mentioned cross-modal collaborative anti-attack method.

[0041] The embodiment of the present invention provides a cross-modal collaborative counterattack method and device, which includes: modifying an initial text based on harmful semantics related to an attack topic to obtain harmful text, determining an initial image related to the harmful text in an initial key cross-modal semantic subspace, generating an adjustment strategy according to the harmful semantics, and obtaining an initial adversarial image according to the adjustment strategy; inputting a first key semantic in the initial text into a prediction model to obtain a plurality of first similar words similar to the first key semantics, and replacing the first key semantics with the first similar words to obtain an initial adversarial text; wherein the initial adversarial image and the initial adversarial sample form an initial adversarial graphic-text multimodal data sample; determining a first loss function of the initial adversarial image and the initial image, and obtaining the gradient of the image modality, the Jacobian matrix of the image modality, and the Hessian matrix of the image modality in turn based on the first loss function; performing weighted fusion on the gradient of the image modality, the Jacobian matrix of the image modality, the Hessian matrix of the image modality, the gradient of the text modality, the Jacobian matrix of the text modality, and the Hessian matrix of the text modality to obtain a cross-modal generalized gradient; wherein , the image modality includes an initial adversarial image and an initial image, and the text modality includes an initial adversarial text and an initial text; based on the cross-modal generalized gradient, the initial adversarial text obtains a plurality of second similar words similar to the second key semantics through the top-k rankings, and replaces the second key semantics in sequence according to the score order of the second similar words to obtain a first adversarial text, and inputs the first adversarial text and the initial adversarial text into the attack target model. If the obtained first prediction label is different from the initial prediction label and meets the attack theme, the first adversarial text is determined as the final adversarial text with a successful attack; a text key semantic subspace is obtained based on the key similar words included in the final adversarial text, and feature alignment is performed on the text key semantic subspace and the image key semantic subspace based on the attention mechanism; the key similar words included in the text key semantic subspace are strengthened to obtain strengthened key semantics, and the strengthened key semantics are mapped to the image key semantic subspace, and the second key region included in the image key semantic subspace is modified according to the strengthened key semantics to obtain a strengthened key region; and a strengthened adversarial text and a strengthened adversarial image are obtained according to the strengthened key semantics and the strengthened key region, respectively.Existing multimodal protection mechanisms usually rely on single-mode or simple multimodal feedback. The cross-modal collaborative attack provided by this method can circumvent these security alignment mechanisms based on single-modal content feedback, thereby breaking through existing protection measures and increasing the complexity and unpredictability of attacks; secondly, by optimizing adversarial perturbations in key cross-modal semantic subspaces and combining historical and future gradient information, the efficiency, convergence speed and stability of perturbation optimization can be improved, avoiding falling into local optimal solutions, thereby improving the practicality and operability of attacks; by effectively integrating cross-modal semantic information, the accuracy and effectiveness of adversarial attacks can be significantly improved, and the success rate of attacks can be increased; through cross-modal collaborative optimization, the design of adversarial perturbations can be more accurate, reducing unnecessary calculations and resource consumption, thereby reducing the overall cost of attacks and improving attack efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0043] Figure 1 A schematic diagram of a cross-modal collaborative anti-attack method flow chart provided by an embodiment of the present invention;

[0044] Figure 2 A schematic diagram of the structure of a cross-modal collaborative anti-attack device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0045] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0046] Figure 1 A schematic diagram of a cross-modal collaborative anti-attack method provided by an embodiment of the present invention is shown in FIG. Figure 1 As shown, the method comprises the following steps:

[0047] Step 101, based on harmful semantics related to the attack topic, the initial text is modified to obtain harmful text, an initial image related to the harmful text is determined in the initial key cross-modal semantic subspace, an adjustment strategy is generated according to the harmful semantics, and the initial image is used to obtain an initial adversarial image according to the adjustment strategy; a first key semantic in the initial text is input into a prediction model to obtain a plurality of first similar words similar to the first key semantic, and the first similar words replace the first key semantic to obtain an initial adversarial text; wherein the initial adversarial image and the initial adversarial sample form an initial adversarial image-text multimodal data sample;

[0048] Step 102, determining a first loss function of the initial adversarial image and the initial image, and obtaining the gradient of the image modality, the Jacobian matrix of the image modality, and the Hessian matrix of the image modality in sequence based on the first loss function; performing weighted fusion on the gradient of the image modality, the Jacobian matrix of the image modality, the Hessian matrix of the image modality, the gradient of the text modality, the Jacobian matrix of the text modality, and the Hessian matrix of the text modality to obtain a cross-modal generalized gradient; wherein the image modality includes the initial adversarial image and the initial image, and the text modality includes the initial adversarial text and the initial text;

[0049] Step 103: Based on the cross-modal generalized gradient, the initial adversarial text obtains a plurality of second similar words similar to the second key semantics through the top-k prediction of the top k rankings, and replaces the second key semantics in sequence according to the score order of the second similar words to obtain a first adversarial text. The first adversarial text and the initial adversarial text are input into the attack target model. If the obtained first predicted label is different from the initial predicted label and meets the attack theme, the first adversarial text is determined as the final adversarial text for successful attack.

[0050] Step 104, obtaining a text key semantic subspace based on the key similar words included in the final adversarial text, and aligning the features of the text key semantic subspace and the image key semantic subspace based on the attention mechanism; strengthening the key similar words included in the text key semantic subspace to obtain enhanced key semantics, mapping the enhanced key semantics to the image key semantic subspace, modifying the second key area included in the image key semantic subspace according to the enhanced key semantics to obtain an enhanced key area; and obtaining an enhanced adversarial text and an enhanced adversarial image according to the enhanced key semantics and the enhanced key area, respectively.

[0051] In an embodiment of the present invention, the attack target model is a model based on the Transformer architecture, such as GPT-4Vision (Generative Pretrained Transformer -4 with Vision, GPT-4V) or BERT (Bidirectional Encoder Representations from Transformers, BERT model).

[0052] Before step 101, a graphic-text multimodal data sample is first obtained. The graphic-text multimodal data sample refers to a data set containing multiple modalities. In the embodiment of the present invention, the graphic-text multimodal data sample mainly includes two modalities: image and text. The data of these two modalities exist simultaneously in the same data sample, and there is some intrinsic connection between them.

[0053] Specifically, in the multimodal data sample of images and text, the images are presented in the form of two-dimensional pixel data. Simply put, the digital photos and computer-drawn graphics that we usually see are stored and represented in the computer as two-dimensional pixel matrices. These images can depict a specific visual scene, such as a beautiful landscape, a busy street, or a specific object, such as a cat or a book. Text has multiple functions in this data sample. It can be a detailed description of the image to help more fully understand the content expressed by the image; it can also be an annotation of certain specific elements in the image, such as annotating the names of animals such as "cat" and "dog" in an animal picture.

[0054] In an embodiment of the present invention, the image-text multimodal data sample obtained from the data set is referred to as an initial image-text multimodal data sample. Based on the above description, it can be seen that the initial image and the initial text coexist and are related to each other in the initial image-text multimodal data sample.

[0055] For an initial image in the initial multimodal data sample, it is necessary to determine a first key region of the initial image; and for an initial text in the initial multimodal data sample, it is necessary to determine a first key semantics of the initial text.

[0056] In the embodiment of the present invention, the first key area of ​​the initial image is determined by using a key map semantic positioning method based on Grad-CAM (Gradient-weighted Class Activation Mapping)++. In practical applications, the core idea of ​​Grad-CAM++ is to introduce pixel-level weighting of the output gradient for a specific position by modeling the contribution of each pixel to the final feature map more finely, thereby providing a measure of the importance of each pixel in the feature map. The calculation method and principle of the first key area obtained by the Grad-CAM++ algorithm are described in detail below.

[0057] For the initial image, preprocess the initial image so that the size of the initial image meets the input requirements of the subsequent convolutional neural network. The preprocessed initial image is input into the convolutional neural network for forward propagation. In this process, the image will pass through each convolution layer, pooling layer, etc. in turn, and finally obtain the output result of the network, that is, the prediction score of each category. According to the output of the network, find the prediction score of a target category corresponding to the initial image.

[0058] Furthermore, for the convolutional feature map exist The pixel value at , the score of the target category is calculated by the following formula right The first-order gradient of:

[0059]

[0060] in, represents the score of the target category, Represents the convolutional feature map exist The pixel value at represents the target category, Indicates Layer feature map, and They respectively represent the element positions on a certain layer of feature map.

[0061] Next, Grad-CAM++ introduces the second-order gradient and third-order gradient information and determines the weight according to the following formula :

[0062]

[0063] in, Indicates the target category In Element positions in layer feature maps The weight of the position can also be expressed as: In the The target category in the layer feature map Contribution of Represents the convolutional feature map exist The pixel value at Represents the convolutional feature map exist The pixel value at Represents the score of the target category right The second-order gradient of Represents the score of the target category right The third-order gradient of and Respectively represent The spatial coordinate index of the position in the layer feature map.

[0064] Next, Grad-CAM++ uses the calculated weights To weight each pixel value in the convolution feature map, generate a weighted feature map as shown below :

[0065]

[0066] in, represents the weighted feature map, Indicates the target category In Element positions in layer feature maps The weight of Represents the convolutional feature map exist The pixel value at represents the score of the target category, Represents the activation function.

[0067] In an embodiment of the present invention, Indicates Layer feature map for target category The importance weight of represents the contribution of each pixel in the convolutional layer to the target category. Operation ensures that only regions with a positive impact on the category score are retained, suppressing the reverse impact of negative gradients on the decision.

[0068] Next, Grad-CAM++ generates the final class activation map by summing the weighted feature maps of all convolutional channels, which represents the class of the target in the image. Areas where classification decisions have significant contributions:

[0069]

[0070] in, Indicates the target category The position in the class activation map below The final value of the image indicates the location of the category The final class activation map shows the regions of the initial image that make important contributions to the classification decision of a given image category.

[0071] Finally, Grad-CAM++ converts the final class activation map Mapped to the initial image, a heat map is generated. By setting a threshold or selecting a high contribution area in the heat map, the first key area can be extracted from the heat map, which represents the part of the image that has the greatest impact on the final classification result.

[0072] In an embodiment of the present invention, word mask-based key text semantic positioning is adopted, and certain words in the text are masked (i.e., temporarily removed or replaced) to significantly change the prediction results of the trained neural network model; by observing and comparing the difference values ​​of the trained neural network model output before and after masking, it is inferred which words are crucial to the correct prediction of the trained neural network model.

[0073] For the initial text, choose to use a trained neural network model, such as BERT, GPT (Generative Pretrained Transformer), etc. The neural network outputs the predicted score of the category based on the input initial text. The initial text is represented as a series of words:

[0074]

[0075] in, Represents the initial text, Indicates the first words.

[0076] In the masking operation, a word in the initial text is replaced with the [MASK] token, and a masked text like the following can be generated:

[0077]

[0078] in, Represents mask text, Indicates the first words, Indicates the first words.

[0079] Next, we calculate the logit output based on the original text and the masked text. By comparing the logit outputs before and after masking, we can calculate the logit output of each word. Importance score:

[0080]

[0081] in, represents the logit output of the initial text, represents the logit output in the masked text, The label representing the target category.

[0082] By calculating the difference between the two, we can get the word The degree of influence on the model prediction. If the logit output changes significantly after masking a word, it means that the word is very important to the model's prediction; conversely, if the change is small, it means that the word has little influence on the prediction.

[0083] For example, the trained BERT model is used to classify the sentiment of movie reviews (positive or negative), and the target category is "positive" sentiment. Specifically, the initial text S is "This movie is really great. I highly recommend it." After tokenizing it, we get ; For words , perform mask operation to generate S , w 0 ={[MASK] ， movie ， is ， really ， great ， . ， I ， highly ， recommend ， it ， .} . Input the initial text and mask text into the BERT model respectively to obtain the logit value of the target category "positive" sentiment. , , then the word is calculated by formula (7) The importance score is ; According to the above method, the initial text , such as the logit value of the target category "positive" sentiment obtained by masking the text is , then the word In this embodiment, after determining the importance scores of all words, the sizes of these scores are compared. Since the importance score of the word "great" is relatively large, it shows that it plays an important role in the model's prediction of the comment as "positive" sentiment; while some words with smaller scores, such as "is", may have relatively small impact on the prediction results.

[0084] Furthermore, according to the first key region and the first key semantics determined by the above method, the initial association point of the initial text and the initial image can be determined. For example, there is a picture of a kitchen with a green refrigerator, a white stove and some tableware. The initial image is processed using the Grad-CAM++ method, and the target category is "refrigerator". After calculation, the first key region obtained is the green door part of the refrigerator in the image; the corresponding initial text is "There is a green refrigerator and a white stove in the kitchen". Using the word mask method and the BERT model as the discriminator, the first key semantics obtained after calculation are "green" and "refrigerator". The first key semantics of "green" forms an association point with the first key region of the green door of the refrigerator in the initial image; the first key semantics of "refrigerator" also forms an association point with the first key region where the entire refrigerator is located.

[0085] Furthermore, the initial key transmembrane state semantic subspace is obtained based on the initial association point. Combining the above small example, for the first key area of ​​the green door of the refrigerator, its visual features are extracted through a certain layer of CNN (Convolutional Neural Network) to obtain a 128-dimensional The feature vector contains the color, texture and other information of the area. For the two first key semantics of "green" and "refrigerator", the BERT model is used to obtain their word vector representations, which are recorded as and , the dimension is also 128. For the "green" connection point, and Concatenate to get a 256-dimensional joint feature vector For the “refrigerator” connection point, and Concatenate and get the joint feature vector . Assuming there are multiple such samples, a set of joint feature vectors is obtained. The PCA (Principal Component Analysis) method is used to reduce the dimension of these vectors from 256 to 50 dimensions. This 50-dimensional space is the key cross-modal semantic subspace, in which the semantic information of the initial image and the initial text are interrelated, which can provide more valuable information for subsequent multimodal analysis, model training and other tasks.

[0086] In the embodiment of the present invention, the initial text and the initial image constitute an initial multimodal data sample.

[0087] In step 101, based on the initial key cross-modal semantic subspace and according to the attack target model, harmful semantics related to the attack topic are obtained, and the initial image is guided to generate an initial image perturbation and an initial adversarial image based on the harmful semantics; based on the first similar word similar to the first key semantics included in the initial text, an initial text perturbation and an initial adversarial text are generated.

[0088] Specifically, the first key semantics included in the initial text is modified according to the harmful semantics to obtain the harmful text, the initial image related to the harmful text is found and determined in the initial key cross-modal semantic subspace, and an adjustment strategy for adjusting the initial image is generated according to the harmful semantics. The initial image obtains an initial adversarial image and an initial image perturbation according to the adjustment strategy. Furthermore, the first key semantics in the initial text is input into the prediction model, and a plurality of first similar words similar to the first key semantics can be obtained based on the prediction model, and the first similar words are used to replace the first key semantics to obtain the initial adversarial text. It should be noted that the prediction model here is the BERT model.

[0089] For example, an intelligent security monitoring system can determine whether there are potential safety hazards in the scene by analyzing the initial image and initial text at the same time to ensure the safety of public places. The initial image is a normal picture of people coming and going at the entrance of a shopping mall, and the initial text describes that "customers enter the mall to shop in an orderly manner."

[0090] In this embodiment, the attacker aims to make the system misjudge the scene as a dangerous scene, so the attacker starts to generate the initial adversarial image and initial adversarial text: Specifically, the attacker introduces the harmful semantics "suspicious package", and after the initial text is modified, the harmful text is "someone placed a suspicious package at the entrance of the mall". Based on this harmful semantics, the system locks the ground area near the gate in the initial image as the key disturbance object based on the key cross-modal semantic subspace; further, by using gradient-based optimization algorithms and other means, not only the light and shadow of the "ground area near the gate" becomes dim and blurred, as if something is really placed there, but also a package-like outline is added through image processing technology, and some fuzzy patterns similar to warning signs are simulated around the outline to make it look more suspicious, which closely echoes the "a suspicious package is placed" in the harmful text, and the generated initial adversarial image is extremely confusing. Accordingly, for the first key semantics in the initial text, such as "orderly entry", it is input into the BERT model to obtain multiple first similar words with similar semantics to the first key semantics "orderly entry", such as "step in" and "enter in a line", etc. Replace "enter in an orderly manner" with "enter one after another" and you get the initial confrontational text "Customers enter the mall one after another to shop."

[0091] Based on the above embodiments, it can be seen that the initial adversarial image and the initial adversarial text work together, and on the surface they still maintain a certain scene relevance, both of which are descriptions of the shopping mall entrance scene, but they quietly change the first key area and the first key semantics, guiding the model to judge in the wrong direction, causing it to misjudge the originally safe shopping mall entrance scene as a scene with potential safety hazards, so as to achieve the attacker's purpose of interfering with the normal operation of the security monitoring system.

[0092] It should be noted that in the above embodiment, the initial image obtains the initial adversarial image based on the adjustment strategy, and the initial text can obtain the initial adversarial text based on the first similar word. Therefore, it can be determined that the adjustment strategy is the initial image perturbation, and the first similar word is the initial text perturbation.

[0093] In this embodiment, the initial adversarial image and the initial adversarial text form an initial adversarial image-text multimodal data sample.

[0094] In step 102, the initial image and the initial adversarial image are respectively input into the attack target model, the attack target model classifies the input initial image and outputs the true label of the initial image; the attack target model classifies the input initial adversarial image and outputs the predicted category probability of the initial adversarial image;

[0095] Furthermore, by setting the method, the first probability is obtained according to the true label of the initial image and the predicted category probability of the initial adversarial image, which can be obtained by The distance function or cross entropy function is used to construct the first loss function, specifically:

[0096]

[0097]

[0098] in, represents the true label of the initial image, represents the predicted class probability of the initial adversarial image, represents the first loss function, The base is The logarithmic function of Indicates taking the largest Distance is used as the first loss function, Represents different categories in the image modality, Represents a vector Norm.

[0099] Furthermore, the first loss function calculates the gradient of the image modality through back propagation, and the gradient of the image modality represents the influence of each pixel of the initial adversarial image on the loss:

[0100]

[0101] in, represents the gradient of the image modality, represents the first loss function, Represents input data of image modality.

[0102] In an embodiment of the present invention, by calculating the gradient of the image modality, it is possible to guide which areas of the initial adversarial image need to be modified to guide the attack target model to misclassify.

[0103] Furthermore, the first loss function is input into the Jacobian matrix to obtain the Jacobian matrix of the image modality:

[0104]

[0105] in, Jacobian matrix representing the image modality.

[0106] In an embodiment of the present invention, the Jacobian matrix is ​​the first-order derivative of the first loss function with respect to the image modality. Each element of the Jacobian matrix represents the degree of influence of each pixel in the image modality on the change of the first loss function. According to the Jacobian matrix, the direct influence of each pixel on the first loss function can be known, that is, which pixel needs to be modified to maximize the first loss function, so that the target model produces an incorrect classification.

[0107] Furthermore, the first loss function is input into the Hessian matrix to obtain the Hessian matrix of the image modality:

[0108]

[0109] in, Represents the Hessian matrix of the image modality.

[0110] In an embodiment of the present invention, the Hessian matrix is ​​the second-order derivative of the first loss function with respect to the image modality, and each element of the Hessian matrix represents the accelerating effect of each pixel perturbation in the image modality on the first loss function; the Hessian matrix reveals the accelerating effect of each pixel included in the image modality on the change of the first loss function. According to the Hessian matrix, it can help identify nonlinear regions in the first loss function and avoid premature local minima, and can help to more finely adjust the adversarial perturbation.

[0111] Accordingly, the initial text and the initial adversarial text are respectively input into the attack target model, and the attack target model classifies the input initial text and outputs the true label of the initial text; the attack target model classifies the input initial adversarial text and outputs the predicted category probability of the initial adversarial text;

[0112] Furthermore, by setting the method, the second probability is obtained according to the true label of the initial text and the predicted category probability of the initial adversarial text, which can be obtained by The distance function or cross entropy function is used to construct the second loss function, specifically:

[0113]

[0114]

[0115] in, represents the true label of the initial text, represents the predicted category probability of the initial adversarial text, represents the second loss function, The base is The logarithmic function of Indicates taking the largest Distance is used as the first loss function, Represents different categories in a text modal.

[0116] Furthermore, the second loss function calculates the gradient of the text modality through back propagation. The gradient of the text modality represents the impact of each word of the initial adversarial text on the loss:

[0117]

[0118] in, A gradient representing the text modality, represents the second loss function, Represents input data for a text modal.

[0119] Furthermore, the second loss function is input into the Jacobian matrix and the Hessian matrix in turn to obtain the Jacobian matrix of the text modality and the Hessian matrix of the text modality:

[0120]

[0121]

[0122] in, The Jacobian matrix representing the text modality, Represents the Hessian matrix of the text modality.

[0123] In an embodiment of the present invention, the Jacobian matrix provides information about the sensitivity of each input feature (such as text vocabulary) to the loss by calculating the first-order derivative of the second loss function. The Jacobian matrix is ​​used to reveal the sensitivity of each input feature to the second loss, helping the optimization algorithm to determine which vocabulary most needs adjustment, and guiding the optimization algorithm to accurately adjust the direction of the adversarial disturbance. The Hessian matrix provides acceleration information of the change of the second loss function by calculating the second-order derivative of the second loss function, helping the optimization algorithm avoid over-adjustment or entering an unstable area, thereby improving the stability and robustness of the adversarial disturbance.

[0124] In an embodiment of the present invention, the above-determined gradient of the image modality, the Jacobian matrix of the image modality, the Hessian matrix of the image modality, the gradient of the text modality, the Jacobian matrix of the text modality, and the Hessian matrix of the text modality are weighted fused to generate a cross-modal generalized gradient:

[0125]

[0126] in, represents the cross-modal generalized gradient, represents the weighting coefficient of the gradient of the image modality, α∈ [0,1] ; represents the weighting coefficients of the Jacobian matrix of the image modality, β∈[0,1] ; represents the weighting coefficients of the Hessian matrix of the image modality, γ∈[0,1] ; represents the weighting coefficient of the gradient of the text modality, δ∈[0,1] ; represents the weight coefficients of the Jacobian matrix of the text modality, ϵ∈[0,1] ; represents the weight coefficients of the Hessian matrix of the text modality, ζ∈[0,1] ; represents the gradient of the image modality, The Jacobian matrix representing the image modality, represents the Hessian matrix of the image modality, A gradient representing the text modality, The Jacobian matrix representing the text modality, represents the Hessian matrix of the text modality, represents the input data in image modality, Represents input data for a text modal.

[0127] In this step, by calculating and weighting the gradient, Jacobian matrix and Hessian matrix of the image modality and the text modality, the adversarial attack effect between the image modality and the text modality can be effectively coordinated to improve the accuracy, efficiency and robustness of the attack; this method can not only accurately control the influence between different modalities, but also improve the overall attack performance in complex cross-modal adversarial attack scenarios. In practical applications, the final generated cross-modal generalized gradient combines the gradient of the image modality, the Jacobian matrix of the image modality, the Hessian matrix of the image modality, the gradient of the text modality, the Jacobian matrix of the text modality and the Hessian matrix of the text modality, which provides gradient information for perturbation optimization on the image modality and the text modality, and provides guidance and basis for perturbation generation in subsequent steps.

[0128] In step 103, based on the cross-modal generalized gradient determined in step 102, the initial adversarial text is predicted by Top-k (Top- k ranking) to obtain multiple second similar words with similar second key semantics to the initial adversarial text. In practical applications, the Top-k prediction score usually refers to the top k results with the highest scores when a certain model (possibly a language model) predicts the text. Based on these prediction scores, multiple second similar words with second key semantics in the initial adversarial text are found. For example, for the second key semantics "apple", the second similar words with similar semantics to "apple" are found by the cosine similarity calculation method based on word vectors, such as "banana" and "orange". These second similar words constitute a set of candidate words for subsequent replacement.

[0129] Furthermore, the second key semantics included in the initial adversarial text are replaced in sequence according to the size of the second similar word score to obtain the first adversarial text, and the first adversarial text sentence is input into the attack target model. If the first text prediction label given by the attack target model is different from the initial text prediction label obtained by the initial adversarial text, the first adversarial text corresponding to the first text prediction label at this time is determined as the final adversarial text.

[0130] Furthermore, if the first text prediction label given by the attack target model is similar to the initial text prediction label obtained by the initial adversarial text, it is necessary to first determine multiple second similar words that are similar to the second key semantics, and then determine the first adversarial text corresponding to each second similar word in turn, input the first adversarial text into the attack target model in turn, determine the first text prediction label obtained after each first adversarial text is input into the attack target model, and determine a first adversarial text with the greatest risk of misprediction from multiple first text prediction labels. At this time, the second similar word corresponding to the determined first adversarial text may cause the attack target model to make an incorrect judgment word. Therefore, after fine-tuning or transforming the first adversarial text according to the cross-modal generalized gradient, the final adversarial text can be obtained.

[0131] For example, the original sentence of the initial adversarial text is a sentence describing healthy food, and the initial text prediction label obtained by the attack target model is "healthy". When the initial adversarial text is replaced by the second similar word to obtain the first adversarial text, the first adversarial text is input into the attack target model to obtain the first text prediction label of "unhealthy". The first adversarial text corresponding to "unhealthy" can be determined as the attack sentence to be found by the method, and the first adversarial text determined here can be determined as the final adversarial text.

[0132] If the first text prediction label "dog" output by the attack target model for the input first adversarial sample is opposite to the initial text prediction label "cat" obtained by the attack target model for the initial adversarial sample, and the first text prediction label "dog" at this time meets the attack theme. Then the first adversarial text corresponding to "dog" can be determined as the final adversarial text; if the previous attempt failed to successfully find the first adversarial text that misclassifies the attack target model, then select one from the remaining first adversarial texts obtained from the second similar words, which can make the first text prediction label output by the attack target model closest to the "dog" category (i.e., the first adversarial text with the greatest risk of misprediction), further fine-tune or transform the selected first adversarial text (such as adjusting the color tone, changing the shape of a certain area, etc.), and then generate a fine-tuned adversarial text again and input it to the attack target model, until a fine-tuned adversarial text that can successfully mislead the attack target model is found, and the fine-tuned adversarial text is determined as the final adversarial text.

[0133] In an embodiment of the present invention, the above steps focus on fine-tuning the initial adversarial text, guided by the actual feedback of the attack target model. On the basis of the generated initial adversarial text, the second similar words similar to the second key semantics are mined using Top-k prediction, and the second key semantics are replaced in sequence according to the score order of the second similar words to obtain the first adversarial text. The second adversarial text (such as the first adversarial text after "fighting" is replaced with "wrestling") is input into the attack target model together with the initial adversarial text to observe the changes in the model's predicted labels. Once it is found that the first adversarial text can make the attack target model give a first predicted label that is opposite to the initial predicted label of the initial adversarial text and conforms to the attack theme, the first adversarial text is locked as the final adversarial text for a successful attack. This step screens out the most misleading first adversarial text by continuously trying to replace subtle changes in the second key semantics, thereby strengthening the ability of the first adversarial text to deceive the model alone.

[0134] In the embodiment of the present invention, by optimizing the adversarial text, the adversarial attack can be hidden in natural language, making the attack more covert and difficult to detect, thereby improving the stability and effectiveness of the attack. In addition, the small tampering ensures that the effectiveness of the adversarial sample does not rely on large-scale modification of the original text, reducing the risk of being identified by manual and automated detection systems.

[0135] In step 104, based on the final adversarial text determined in step 103, the lexical analysis tool NLTK (Natural Language Toolkit) in natural language processing is used to perform part-of-speech tagging and word form restoration, and a certain verb in the final adversarial text is identified as a key similar word. The identified verb here is closely related to the harmful semantics expected by the method and is the core vocabulary of the harmful semantics. Based on the extracted key similar words, the pre-trained word vector model Word to Vector is used to obtain its word vector representation. For example, if the word vector dimension of the key similar word is 300 dimensions, this vector contains information such as the position of the key similar word in the semantic space and the semantic relationship with other words. At the same time, words with similar semantics to the key similar words are collected and obtained by calculating the cosine similarity of the word vectors. The word vectors of these similar words are also included in the construction of the text key semantic subspace, forming a vector set around the semantics of the key similar words, that is, the text key semantic subspace is obtained.

[0136] Furthermore, for the initial adversarial image, a pre-trained convolutional neural network (such as VGG19 (Visual Geometry Group 19 - layer network)) is used to extract image features, and the feature maps output by the intermediate layer of the network contain the semantic information of different regions of the initial adversarial image. The feature maps are processed using models related to the attention mechanism to highlight the areas related to the key semantics of the text. For example, the spatial attention module weights the feature maps by calculating the attention weight of each position, making the features of the action areas related to the semantics of the key similar words more prominent.

[0137] The attention weights are obtained by calculating the similarity (such as cosine similarity) between the word vector and the feature vector at each position of the feature map. Based on these attention weights, the feature maps of the key semantic subspace of the image are weighted and summed to obtain the image feature representation aligned with the key similar words included in the key semantic subspace of the text. This ensures that the semantics of the key similar words in the final adversarial text closely correspond to the action area in the initial adversarial image, achieving cross-modal feature alignment.

[0138] Further, the key similar words included in the text key semantic subspace are strengthened, and the strengthening can be performed by increasing the semantic strength of the key similar words. For example, using the semantic expansion tool, find words with stronger semantics than the key similar words. At the same time, in the text key semantic subspace, the word vectors related to the key similar words are adjusted, and the uniqueness of the key similar words in the semantic space and the relevance to the harmful semantics can be enhanced by increasing the values ​​on certain dimensions of the word vectors. After the above operations, the strengthened key semantics are obtained. Further, the strengthened key semantics are mapped to the image key semantic subspace, and the second key area included in the image key semantic subspace is modified according to the strengthened key semantics to obtain a strengthened key area that echoes the strengthened key semantics. Finally, the key similar words in the final adversarial text are replaced with the strengthened vocabulary to obtain the strengthened adversarial text, and the strengthened adversarial image is obtained after modifying the second key area included in the image key semantic subspace. Further, the strengthened adversarial text and the strengthened adversarial image are referred to as the final adversarial image-text multimodal data samples.

[0139] In an embodiment of the present invention, by strengthening and aligning the key semantics in the text key semantic subspace and the image key semantic subspace, the semantic and feature consistency of the final adversarial text and the initial adversarial image is ensured, thereby achieving a stronger cross-modal adversarial attack effect. By strengthening the key areas of the text and image, the misleading ability of the adversarial sample is maximized, while maintaining the naturalness and consistency of the text and image, reducing the risk of detection. This optimization process not only enhances the success rate of the adversarial attack, but also ensures the concealment and robustness of the attack, so that the final generated enhanced adversarial text and enhanced adversarial image can effectively confuse the attack target model and avoid excessive disturbance and unnatural performance.

[0140] For example, assuming that the target model is a social media content review model, which is used to determine whether a picture or text post contains harmful information, in this embodiment, it is necessary to generate a final adversarial image and a final adversarial text that can bypass the review, that is, a final adversarial picture and text multimodal data sample.

[0141] Specifically, a photo of children playing football on the playground is selected as the initial image, and the corresponding initial text is "On a sunny day, the children are happily playing football on the playground." The initial image and initial text are called the initial image-text multimodal data samples. A pre-trained multimodal analysis model is used to determine the children's action area and the football's position area in the initial image as the first key area; "playing football" and "happy" in the initial text are the first key semantics. These first key areas and first key semantics jointly determine the initial association points, and then obtain the initial key cross-modal semantic subspace containing semantics such as sports and joyful atmosphere.

[0142] Step 201, in this embodiment, the attack target is to make the audit model misjudge it as harmful content. Therefore, the harmful semantics generated according to the attack target model is "fierce conflict"; the initial text is modified according to the harmful semantics "fierce conflict" to obtain the harmful text "On a sunny day, the children had a fierce conflict on the playground". In the initial key cross-modal semantic subspace, the initial image part related to "fierce conflict" is locked, such as the moment when the children's physical contact is slightly intense when fighting for football; further, an adjustment strategy is formulated for the initial image determined to correspond to the harmful text, and the initial image is modified according to the adjustment strategy, such as blurring the picture of the surrounding teacher's supervision, partially enlarging the children's slightly excited expressions, and adjusting the initial image according to the adjustment strategy to obtain the initial adversarial image. At the same time, "playing football" is input into the prediction model to obtain the first similar words of "fighting" and "fighting", and the initial adversarial text is obtained after replacement, which is "On a sunny day, the children are fighting on the playground".

[0143] Step 202, for the initial adversarial image and the initial image, the loss function based on the L2 distance is used to measure the difference, and the gradient of the image modality, the Jacobian matrix of the image modality, and the Hessian matrix of the image modality are calculated in sequence through back propagation. Similarly, similar operations are performed on the initial adversarial text and the initial text. Assuming that the weight of the image modality is set to 0.6 and the weight of the text modality is set to 0.4, these matrices are weighted and fused to obtain the cross-modal generalized gradient, which is used to guide the subsequent adversarial sample optimization direction, such as adjusting the initial adversarial image and the initial adversarial text in the direction that makes the attack target model more likely to misjudge.

[0144] Step 203, for the initial adversarial text "On a sunny day, the children were fighting on the playground", through Top-k prediction, assuming that similar words to "fight" include "wrestle" and "mutual fight", "fight" is replaced in order of score. First, replace it with "wrestle" to obtain the first adversarial text "On a sunny day, the children were fighting on the playground", and input the first adversarial text and the initial adversarial text into the attack target model. If the attack target model gives a harmful label to the "wrestle" text but not to the initial adversarial text, then the first adversarial text including "wrestle" can be determined as the final adversarial text with a successful attack.

[0145] Step 204, for the final adversarial text "On a sunny day, the children were fighting on the playground", the lexical analysis tool in natural language processing is used to perform part-of-speech tagging and word form restoration, and the verb "fight" is identified as a key similar word, because it is a core word closely related to the harmful semantics "fierce conflict" expected in this embodiment. Based on the extracted key similar word "fight", its word vector representation is obtained using a pre-trained word vector model. At the same time, words with similar semantics to "fight", such as "fight" and "struggle", are collected, and the word vectors of these similar words are also included in the construction of the text key semantic subspace, forming a vector set around the semantics of "fight", i.e., the text key semantic subspace.

[0146] For the initial adversarial image whose key semantic subspace has been determined, a pre-trained convolutional neural network is used to extract image features, and the feature map is processed using a model related to the attention mechanism to highlight the areas related to the key semantic subspace of the text. For example, the spatial attention module weights the feature map by calculating the attention weight of each position, so that the "wrestling" in the final adversarial text closely corresponds to the children's body movements in the initial adversarial image.

[0147] The word vector of "fight" in the key semantic subspace of the text is associated with the feature map of the key semantic subspace of the image. The feature map of the key semantic subspace of the image is weighted and summed to obtain the image feature representation aligned with the key semantic "fight" in the text. This ensures that the semantics of "fight" in the final adversarial text closely corresponds to the children's body movements in the initial adversarial image, achieving cross-modal feature alignment.

[0148] The key similar word "scuffle" is strengthened. At the text semantic level, it can be strengthened by increasing its semantic strength. For example, using the semantic expansion tool, find words with stronger semantics than "scuffle", such as "evil fight". At the same time, in the text key semantic subspace, adjust the word vectors related to "scuffle". By increasing the value of certain dimensions of the word vector, its uniqueness in the semantic space and its relevance to harmful semantics can be enhanced. After the above operation, the strengthened key semantics are obtained. The strengthened key semantics here not only contain the stronger "fierce conflict" semantics, but also have a more prominent and unique representation in the semantic space.

[0149] Furthermore, the enhanced key semantics are mapped to the key semantic subspace of the image. Based on the enhanced "scuffle" semantics (such as the more intense conflict semantics represented by "evil fight"), the second key area in the image that needs to be further modified is determined. In the image, the children's movements and facial expressions are adjusted to make their body movements more exaggerated and show a more intense confrontation; at the same time, the details of the children's angry expressions are enhanced, such as enlarging the pupils and deepening the tension of the facial muscles.

[0150] The enhanced word "fierce fight" is used to replace the word "fight" in the initial adversarial text, and the enhanced adversarial text "On a sunny day, the children fought fiercely on the playground" is obtained. This text is more semantically consistent with the harmful content characteristics expected by this method. After modifying and adjusting the key areas of the image, the enhanced adversarial image is generated. The image at this time can better reflect the harmful semantics of "fierce conflict" visually, and has a higher consistency with the enhanced adversarial text in terms of semantics and features.

[0151] This method generates more deceptive fortified adversarial text and fortified adversarial images, enhancing the ability of cross-modal adversarial attacks to mislead social media content review models, while maintaining the naturalness and consistency of text and images, reducing the risk of detection.

[0152] In summary, an embodiment of the present invention provides a cross-modal collaborative adversarial attack method, including: modifying an initial text based on harmful semantics related to an attack topic to obtain harmful text, determining an initial image related to the harmful text in an initial key cross-modal semantic subspace, generating an adjustment strategy according to the harmful semantics, and obtaining an initial adversarial image according to the adjustment strategy; inputting a first key semantic in the initial text into a prediction model to obtain a plurality of first similar words similar to the first key semantics, and replacing the first key semantics with the first similar words to obtain an initial adversarial text; wherein the initial adversarial image and the initial adversarial sample form an initial adversarial image-text multimodal data sample; determining a first loss function of the initial adversarial image and the initial image, and obtaining the gradient of the image modality, the Jacobian matrix of the image modality, and the Hessian matrix of the image modality in turn based on the first loss function; performing weighted fusion on the gradient of the image modality, the Jacobian matrix of the image modality, the Hessian matrix of the image modality, the gradient of the text modality, the Jacobian matrix of the text modality, and the Hessian matrix of the text modality to obtain a cross-modal generalized gradient; The image modality includes an initial adversarial image and an initial image, and the text modality includes an initial adversarial text and an initial text; based on the cross-modal generalized gradient, the initial adversarial text obtains a plurality of second similar words similar to the second key semantics through Top-k prediction, and the second key semantics are replaced in sequence according to the score order of the second similar words to obtain a first adversarial text; the first adversarial text and the initial adversarial text are input into the attack target model; if the obtained first prediction label is different from the initial prediction label and meets the attack theme, the first adversarial text is determined as the final adversarial text with successful attack; a text key semantic subspace is obtained based on the key similar words included in the final adversarial text, and feature alignment is performed on the text key semantic subspace and the image key semantic subspace based on the attention mechanism; the key similar words included in the text key semantic subspace are strengthened to obtain strengthened key semantics, the strengthened key semantics are mapped to the image key semantic subspace, and the second key area included in the image key semantic subspace is modified according to the strengthened key semantics to obtain a strengthened key area; and a strengthened adversarial text and a strengthened adversarial image are obtained according to the strengthened key semantics and the strengthened key area, respectively.Existing multimodal protection mechanisms usually rely on single-mode or simple multimodal feedback. The cross-modal collaborative attack provided by this method can circumvent these security alignment mechanisms based on single-modal content feedback, thereby breaking through existing protection measures and increasing the complexity and unpredictability of attacks; secondly, by optimizing adversarial perturbations in key cross-modal semantic subspaces and combining historical and future gradient information, the efficiency, convergence speed and stability of perturbation optimization can be improved, avoiding falling into local optimal solutions, thereby improving the practicality and operability of attacks; by effectively integrating cross-modal semantic information, the accuracy and effectiveness of adversarial attacks can be significantly improved, and the success rate of attacks can be increased; through cross-modal collaborative optimization, the design of adversarial perturbations can be more accurate, reducing unnecessary calculations and resource consumption, thereby reducing the overall cost of attacks and improving attack efficiency.

[0153] Based on the same inventive concept, an embodiment of the present invention provides a cross-modal collaborative anti-attack device. Since the principle of solving the technical problem of the device is similar to the cross-modal collaborative anti-attack method, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0154] like Figure 2 As shown, the device includes a first obtaining unit 201 , a second obtaining unit 202 , a determining unit 203 and a third obtaining unit 204 .

[0155] The first obtaining unit 201 is used to modify the initial text based on the harmful semantics related to the attack topic to obtain the harmful text, determine the initial image related to the harmful text in the initial key cross-modal semantic subspace, generate an adjustment strategy according to the harmful semantics, and obtain the initial adversarial image according to the adjustment strategy; input the first key semantics in the initial text into the prediction model to obtain a plurality of first similar words similar to the first key semantics, and replace the first key semantics with the first similar words to obtain the initial adversarial text; wherein the initial adversarial image and the initial adversarial sample form an initial adversarial image-text multimodal data sample;

[0156] The second obtaining unit 202 is used to determine the first loss function of the initial adversarial image and the initial image, and obtain the gradient of the image modality, the Jacobian matrix of the image modality, and the Hessian matrix of the image modality in sequence based on the first loss function; the gradient of the image modality, the Jacobian matrix of the image modality, the Hessian matrix of the image modality, the gradient of the text modality, the Jacobian matrix of the text modality, and the Hessian matrix of the text modality are weightedly fused to obtain a cross-modal generalized gradient; wherein the image modality includes the initial adversarial image and the initial image, and the text modality includes the initial adversarial text and the initial text;

[0157] A determination unit 203 is used to obtain a plurality of second similar words similar to the second key semantics through the top-k prediction of the first k rankings of the initial adversarial text based on the cross-modal generalized gradient, replace the second key semantics in sequence according to the score order of the second similar words, and obtain a first adversarial text, input the first adversarial text and the initial adversarial text into the attack target model, and if the obtained first predicted label is different from the initial predicted label and meets the attack theme, determine the first adversarial text as the final adversarial text for successful attack;

[0158] The third obtaining unit 204 is used to obtain a text key semantic subspace based on the key similar words included in the final adversarial text, and perform feature alignment on the text key semantic subspace and the image key semantic subspace based on the attention mechanism; strengthen the key similar words included in the text key semantic subspace to obtain enhanced key semantics, map the enhanced key semantics to the image key semantic subspace, modify the second key area included in the image key semantic subspace according to the enhanced key semantics, and obtain an enhanced key area; and obtain an enhanced adversarial text and an enhanced adversarial image according to the enhanced key semantics and the enhanced key area, respectively.

[0159] It should be understood that the units included in the above cross-modal collaborative anti-attack device are only logical divisions based on the functions implemented by the device. In practical applications, the above units can be superimposed or split. And the functions implemented by the cross-modal collaborative anti-attack device provided in this embodiment correspond one-to-one to the cross-modal collaborative anti-attack method provided in the above embodiment. The more detailed processing flow implemented by the device has been described in detail in the above method embodiment 1, and will not be described in detail here.

[0160] Another embodiment of the present invention also provides a computer device, which includes: a processor and a memory; the memory is used to store computer program code, and the computer program code includes computer instructions; when the processor executes the computer instructions, the electronic device executes each step of the cross-modal collaborative anti-attack method in the method flow described in the above method embodiment.

[0161] Another embodiment of the present invention further provides a computer-readable storage medium, which stores computer instructions. When the computer instructions are executed on a computer device, the computer device executes each step of the cross-modal collaborative anti-attack method in the method flow described in the above method embodiment.

[0162] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0163] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A cross-modal collaborative anti-attack method, characterized in that: include: Modifying an initial text based on harmful semantics related to the attack topic to obtain harmful text, determining an initial image related to the harmful text in an initial key cross-modal semantic subspace, generating an adjustment strategy according to the harmful semantics, and obtaining an initial adversarial image according to the adjustment strategy; Inputting the first key semantics in the initial text into the prediction model to obtain a plurality of first similar words similar to the first key semantics, and replacing the first key semantics with the first similar words to obtain an initial adversarial text; wherein the initial adversarial image and the initial adversarial sample form an initial adversarial image-text multimodal data sample; Determine a first loss function of the initial adversarial image and the initial image, and obtain the gradient of the image modality, the Jacobian matrix of the image modality, and the Hessian matrix of the image modality in sequence based on the first loss function; perform weighted fusion on the gradient of the image modality, the Jacobian matrix of the image modality, the Hessian matrix of the image modality, the gradient of the text modality, the Jacobian matrix of the text modality, and the Hessian matrix of the text modality to obtain a cross-modal generalized gradient; wherein the image modality includes the initial adversarial image and the initial image, and the text modality includes the initial adversarial text and the initial text; Based on the cross-modal generalized gradient, the initial adversarial text obtains a plurality of second similar words similar to the second key semantics through the top-k prediction of the top k rankings, and the second key semantics are replaced in sequence according to the score order of the second similar words to obtain a first adversarial text, and the first adversarial text and the initial adversarial text are input into the attack target model, and if the obtained first predicted label is different from the initial predicted label and meets the attack theme, the first adversarial text is determined as the final adversarial text for successful attack; A text key semantic subspace is obtained based on the key similar words included in the final adversarial text, and feature alignment is performed on the text key semantic subspace and the image key semantic subspace based on the attention mechanism; the key similar words included in the text key semantic subspace are strengthened to obtain strengthened key semantics, the strengthened key semantics are mapped to the image key semantic subspace, and the second key area included in the image key semantic subspace is modified according to the strengthened key semantics to obtain a strengthened key area; and a strengthened adversarial text and a strengthened adversarial image are obtained according to the strengthened key semantics and the strengthened key area, respectively.

2. The method according to claim 1, characterized in that Before modifying the initial text based on the harmful semantics related to the attack topic to obtain the harmful text, the method further includes: Determine the first key area of ​​the initial image and the first key semantics of the initial text according to a set semantic positioning method; Determine initial association points between the initial image and the initial text according to the first key area and the first key semantics, and obtain an initial key cross-modal semantic subspace according to the initial association points; wherein the initial text and the initial image constitute an initial multimodal data sample.

3. The method according to claim 2, characterized in that The determining of a first key region of an initial image and a first key semantics of an initial text according to a set semantic localization method, determining an initial association point of the initial image and the initial text according to the first key region and the first key semantics, and obtaining an initial key cross-modal semantic subspace according to the initial association point specifically includes: Determine the key area of ​​the initial image based on the Grad-CAM++ graph semantic localization method, and determine the first keyword of the initial text based on the word mask text semantic localization method; Determine an influence region that affects the classification result of the initial image according to the first key region, determine influence words included in the initial text according to the first key semantics, the influence words are used to influence the prediction result of the attack target model, and determine initial association points with semantic correspondence among the plurality of influence regions and the plurality of influence words; Converting pixel information of the influence area corresponding to the initial association point in the initial image into a visual feature vector of fixed dimension, and converting the influence words corresponding to the initial association point in the initial text into text vector information; The visual feature vector and the text vector information are combined to obtain a plurality of joint feature vectors, and an initial key cross-modal semantic subspace is obtained according to the plurality of joint feature vectors.

4. The method according to claim 1, characterized in that The determining of the first loss function of the initial adversarial image and the initial image, and sequentially obtaining the gradient of the image modality, the Jacobian matrix of the image modality, and the Hessian matrix of the image modality based on the first loss function, specifically includes: Inputting the initial image and the initial adversarial image into the attack target model, obtaining the true image label and the image prediction category label in turn, obtaining a first loss function according to the true image label and the image prediction category label, and obtaining the gradient of the image modality by back propagation in the first loss function; Obtaining the Jacobian matrix of the image modality according to the first loss function and the Jacobian matrix; Obtaining a Hessian matrix of the image modality according to the first loss function and the Hessian matrix; The first loss function, the gradient of the image modality, the Jacobian matrix of the image modality, and the Hessian matrix of the image modality are as follows: in, represents the first loss function, represents the true label of the initial image, represents the predicted class probability of the initial adversarial image, The base is The logarithmic function of represents the gradient of the image modality, The Jacobian matrix representing the image modality, represents the Hessian matrix of the image modality, represents the input data in image modality, Represents different categories in the image modality.

5. The method according to claim 1, characterized in that The cross-modal generalized gradient is as follows: in, represents the cross-modal generalized gradient, represents the weighting coefficient of the gradient of the image modality, ; represents the weighting coefficients of the Jacobian matrix of the image modality, ; represents the weighting coefficients of the Hessian matrix of the image modality, ; represents the weighting coefficient of the gradient of the text modality, ; represents the weight coefficients of the Jacobian matrix of the text modality, ; represents the weight coefficients of the Hessian matrix of the text modality, ; represents the gradient of the image modality, The Jacobian matrix representing the image modality, represents the Hessian matrix of the image modality, A gradient representing the text modality, The Jacobian matrix representing the text modality, represents the Hessian matrix of the text modality, represents the input data in image modality, Represents input data for a text modal.

6. The method according to claim 1, characterized in that After inputting the first adversarial text and the initial adversarial text into the attack target model, the method further includes: If the obtained first predicted label is similar to the initial predicted label, find the first text predicted label output by the attack target model with the maximum misprediction risk from the first adversarial texts corresponding to the plurality of second similar words with similar second key semantics; The first adversarial text having the greatest risk of misprediction is adjusted according to the cross-modal generalized gradient to obtain the final adversarial text.

7. A cross-modal collaborative anti-attack device, characterized in that: include: A first obtaining unit is used to modify an initial text based on harmful semantics related to an attack topic to obtain harmful text, determine an initial image related to the harmful text in an initial key cross-modal semantic subspace, generate an adjustment strategy according to the harmful semantics, and obtain an initial adversarial image according to the adjustment strategy; Inputting the first key semantics in the initial text into the prediction model to obtain a plurality of first similar words similar to the first key semantics, and replacing the first key semantics with the first similar words to obtain an initial adversarial text; wherein the initial adversarial image and the initial adversarial sample form an initial adversarial image-text multimodal data sample; A second obtaining unit is used to determine a first loss function of the initial adversarial image and the initial image, and based on the first loss function, sequentially obtain the gradient of the image modality, the Jacobian matrix of the image modality, and the Hessian matrix of the image modality; the gradient of the image modality, the Jacobian matrix of the image modality, the Hessian matrix of the image modality, the gradient of the text modality, the Jacobian matrix of the text modality, and the Hessian matrix of the text modality are weightedly fused to obtain a cross-modal generalized gradient; wherein the image modality includes an initial adversarial image and an initial image, and the text modality includes an initial adversarial text and an initial text; A determination unit is used to obtain a plurality of second similar words similar to the second key semantics through the top-k predictions of the first k rankings of the initial adversarial text based on the cross-modal generalized gradient, replace the second key semantics in sequence according to the score order of the second similar words, and obtain a first adversarial text, input the first adversarial text and the initial adversarial text into the attack target model, and if the obtained first predicted label is different from the initial predicted label and meets the attack theme, determine the first adversarial text as the final adversarial text for successful attack; The third obtaining unit is used to obtain a text key semantic subspace based on the key similar words included in the final adversarial text, and perform feature alignment on the text key semantic subspace and the image key semantic subspace based on the attention mechanism; strengthen the key similar words included in the text key semantic subspace to obtain enhanced key semantics, map the enhanced key semantics to the image key semantic subspace, modify the second key area included in the image key semantic subspace according to the enhanced key semantics, and obtain an enhanced key area; and obtain an enhanced adversarial text and an enhanced adversarial image according to the enhanced key semantics and the enhanced key area, respectively.

8. A computer device, characterized in that: The computer device includes a memory and a processor, the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the cross-modal collaborative anti-attack method as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that: A computer program is stored, and when the computer program is executed by a processor, the processor executes the cross-modal collaborative anti-attack method as described in any one of claims 1-6.

Images