An image interference method, device, equipment, storage medium and program product

Abstract

This invention provides an image interference method, apparatus, device, storage medium, and program product, relating to the field of data processing technology. The method includes: acquiring an attribute graph of a first image, the attribute graph including multiple nodes, multiple edges, and a node feature matrix, the node feature matrix being used to characterize the features of different nodes among the multiple nodes; generating a first perturbation adjacency matrix based on the node feature matrix, the first perturbation adjacency matrix being used to characterize the structural perturbation of different nodes; generating a first node feature perturbation parameter corresponding to each of the multiple nodes based on the multiple edges; and perturbing each node based on the first node feature perturbation parameter corresponding to each node and the first perturbation adjacency matrix, respectively, to obtain an interference image, the interference image being used to train an image recognition model. This invention can improve the effect of image interference.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and specifically to an image interference method, apparatus, device, storage medium, and program product. Background Technology

[0002] Graph Neural Networks (GNNs) are models used for image recognition, capable of accurately identifying image features. In related technologies, to improve the recognition accuracy of GNNs, perturbations are added to the sample images during training to enhance the GNN's resistance to interference. However, these technologies often use a single structural perturbation to the sample images, ignoring the unique features of different nodes within the image. This makes the perturbed images easily recognizable by the GNN, resulting in poor image perturbation effectiveness.

[0003] It is evident that the related technologies suffer from poor image interference performance. Summary of the Invention

[0004] This invention provides an image interference method, apparatus, device, storage medium, and program product to solve the problem of poor image interference effect in related technologies.

[0005] To solve the above problems, the present invention is implemented as follows: In a first aspect, embodiments of the present invention provide an image interference method, comprising: Obtain the attribute graph of the first image, the attribute graph including multiple nodes, multiple edges, and a node feature matrix, the node feature matrix being used to characterize the features of different nodes among the multiple nodes; A first perturbation adjacency matrix is ​​generated based on the node feature matrix. The first perturbation adjacency matrix is ​​used to characterize the structural perturbation of different nodes. Generate first node feature perturbation parameters for each of the multiple nodes based on the multiple edges; Based on the first node feature perturbation parameters and the first perturbation adjacency matrix corresponding to each node, the nodes are perturbed to obtain perturbation images, which are used to train the image recognition model.

[0006] Secondly, embodiments of the present invention provide an image interference device, comprising: The acquisition module is used to acquire the attribute graph of the first image, the attribute graph including multiple nodes, multiple edges, and a node feature matrix, the node feature matrix being used to characterize the features of different nodes among the multiple nodes; The first generation module is used to generate a first perturbation adjacency matrix based on the node feature matrix. The first perturbation adjacency matrix is ​​used to characterize the structural perturbation of different nodes. The second generation module is used to generate first node feature perturbation parameters for each of the multiple nodes based on the multiple edges; The interference module is used to interfere with each node based on the first node feature perturbation parameters and the first perturbation adjacency matrix corresponding to each node, so as to obtain an interference image, which is used to train the image recognition model.

[0007] Thirdly, embodiments of the present invention also provide an electronic device, including a transceiver and a processor. The transceiver is used to acquire an attribute graph of a first image. The attribute graph includes multiple nodes, multiple edges, and a node feature matrix. The node feature matrix is ​​used to characterize the features of different nodes among the multiple nodes. The processor is configured to generate a first perturbation adjacency matrix based on the node feature matrix, wherein the first perturbation adjacency matrix is ​​used to characterize the structural perturbation of different nodes. The processor is further configured to generate first node feature perturbation parameters corresponding to each of the plurality of nodes based on the plurality of edges; The processor is further configured to perturb each node based on the first node feature perturbation parameter and the first perturbation adjacency matrix corresponding to each node, thereby obtaining a perturbation image, which is used to train the image recognition model.

[0008] Fourthly, embodiments of the present invention provide an electronic device, including: a processor, a memory, and a program stored in the memory and executable on the processor, wherein the program, when executed by the processor, implements the steps of the image interference method described in the first aspect.

[0009] Fifthly, embodiments of the present invention provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the image interference method described in the first aspect.

[0010] In a sixth aspect, the present invention also provides a computer program product, including computer instructions, which, when executed by a processor, implement the steps of the image interference method described in the first aspect.

[0011] In this embodiment of the invention, an attribute graph of a first image is obtained. The attribute graph includes multiple nodes, multiple edges, and a node feature matrix, which characterizes the features of different nodes among the multiple nodes. A first perturbation adjacency matrix is ​​generated based on the node feature matrix, which characterizes the structural perturbation of different nodes. A first node feature perturbation parameter is generated for each node among the multiple nodes based on the multiple edges. Each node is perturbed based on the first node feature perturbation parameter and the first perturbation adjacency matrix to obtain a perturbed image, which is used to train an image recognition model. Thus, by generating a first perturbation adjacency matrix based on the node feature matrix, generating first node feature perturbation parameters for each node among the multiple nodes based on multiple edges, and perturbing each node based on the first node feature perturbation parameter and the first perturbation adjacency matrix to obtain a perturbed image, a good concealment effect can be achieved while perturbing the image, making it difficult for graph neural networks to directly recognize, thereby effectively improving the image perturbation effect. Attached Figure Description

[0012] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 This is a flowchart of an image interference method provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the relationship between nodes provided in an embodiment of the present invention; Figure 3 This is a flowchart of generating an adversarial graph provided in an embodiment of the present invention; Figure 4 This is a structural diagram of an image interference device provided in an embodiment of the present invention; Figure 5 This is a structural diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0014] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0015] Please see Figure 1 , Figure 1 This is a flowchart of an image interference method provided in an embodiment of the present invention, such as... Figure 1 As shown, it includes the following steps: Step 101: Obtain the attribute graph of the first image. The attribute graph includes multiple nodes, multiple edges, and a node feature matrix. The node feature matrix is ​​used to characterize the features of different nodes among the multiple nodes.

[0016] The first image mentioned above is the image to be interfered with, and the attribute map mentioned above is the attribute map corresponding to the first image. The attribute map can represent the content of the first image. Specifically, the attribute map includes multiple nodes, multiple edges, and a node feature matrix. The multiple nodes are nodes in the first image, and each node is used to represent the situation at different positions in the first image; each edge is used to represent whether there is a connection relationship between two nodes; the node feature matrix is ​​used to represent the characteristics of different nodes. In this way, by obtaining the attribute map of the first image, the situation of the first image can be determined through the multiple nodes, multiple edges, and node feature matrix of the attribute map.

[0017] The attribute graph can be represented as G={V,E,X}, where V={v1, v2, ..., v...}. N} represents the set of nodes, including N nodes (i.e., multiple nodes); E represents the set of edges, including multiple edges; This represents the node feature matrix.

[0018] Furthermore, let N(v) i ) is node v i The set of first-order neighbor nodes, which also contains a labeled subset of nodes. The node label set is [C] = {1, 2, ..., C}, V L and V U These represent the labels of labeled and unlabeled nodes, respectively. The labels determine whether different nodes are labeled (i.e., whether they are recognized). For labeled nodes, it is assumed that the node has been recognized and does not need to be classified by the image recognition model. For unlabeled nodes, it is assumed that the node has not been recognized and the node still needs to be classified by the image recognition model.

[0019] It should be noted that the goal of node classification is to train a [system / mechanism / database]... GNN model with parameters The model learns a function to map nodes to a label set to predict the labels of unlabeled nodes (i.e., classifying nodes using an image recognition model). Adversarial attacks aim to generate node feature perturbations and edge perturbations more effectively. By adding or deleting edges and perturbing node features, the accuracy and confidence of node classification are reduced, while maintaining the stealth of the attack.

[0020] The objective of node classification can be represented by the following formula: ; in the formula Let A be the classification result for the i-th node, and let X be the node to be classified. i Let c be the node feature of the i-th node, and c be the node label.

[0021] Furthermore, the GNN model It consists of K layers and is optimized by minimizing a specific classification loss, such as cross-entropy loss. GNN uses a message-passing mechanism in neural networks to recursively update nodes, specifically as follows: ; W in the formula k Represents node v i The learnable parameters generated after aggregation at the k-th (k>0) layer Represents node v i The hidden representation generated after aggregation at the k-th (k>0) layer. Represents node v i The hidden representation generated after aggregation at the (k-1)th (k>0) layer, Represents node v j The hidden representation is generated after aggregation at the (k-1)th (k>0) layer. Initialize to X i . This refers to an activation function, such as the ReLU function. This represents the aggregation function of neighboring nodes. In the output layer, which is the Kth layer of the model, W is... k Set to C, Set to Softmax.

[0022] Step 102: Generate a first perturbation adjacency matrix based on the node feature matrix. The first perturbation adjacency matrix is ​​used to characterize the structural perturbation of different nodes.

[0023] The first perturbation adjacency matrix is ​​used to characterize the structural perturbation of different nodes. The first perturbation adjacency matrix can be used to add structural perturbation to different nodes.

[0024] In some implementations, the first perturbation adjacency matrix can be based on the perturbation adjacency matrix corresponding to a pre-configured single perturbation graph structure; or, it can be the perturbation adjacency matrix corresponding to a perturbation graph in a pre-configured set of perturbation graphs, wherein the set of perturbation graphs includes multiple perturbation graphs.

[0025] Step 103: Generate the first node feature perturbation parameters corresponding to each of the multiple nodes based on the multiple edges.

[0026] The aforementioned first node feature perturbation parameters are perturbation parameters for each node. The first node feature perturbation parameters corresponding to each node are generated based on multiple edges, so that the first node feature perturbation corresponding to each node can be the same or different.

[0027] In some implementations, generating the first node feature perturbation parameter corresponding to each node based on multiple edges can be achieved by determining the neighboring nodes corresponding to each node based on multiple edges, and then generating the corresponding first node perturbation parameter based on the node features of the neighboring nodes.

[0028] Step 104: Based on the first node feature perturbation parameter and the first perturbation adjacency matrix corresponding to each node, the first node feature perturbation parameter and the first perturbation adjacency matrix are used to perturb each node to obtain a perturbation image. The perturbation image is used to train the image recognition model.

[0029] It should be understood that since the first node feature perturbation parameter is generated for each node and the first perturbation adjacency matrix is ​​for the perturbation of the graph structure, by perturbing each node based on the first node feature perturbation parameter and the first perturbation adjacency matrix corresponding to each node, the perturbed nodes in the obtained perturbed image can achieve both good perturbation effect and concealment, making them difficult for the image recognition model to identify. Furthermore, by training the image recognition model through the perturbation model, the trained recognition model can more accurately identify attacks on the image, effectively improving the accuracy of image recognition.

[0030] In this embodiment of the invention, an attribute graph of a first image is obtained. The attribute graph includes multiple nodes, multiple edges, and a node feature matrix, which characterizes the features of different nodes among the multiple nodes. A first perturbation adjacency matrix is ​​generated based on the node feature matrix, which characterizes the structural perturbation of different nodes. A first node feature perturbation parameter is generated for each node among the multiple nodes based on the multiple edges. Each node is perturbed based on the first node feature perturbation parameter and the first perturbation adjacency matrix to obtain a perturbed image, which is used to train an image recognition model. Thus, by generating a first perturbation adjacency matrix based on the node feature matrix, generating first node feature perturbation parameters for each node among the multiple nodes based on multiple edges, and perturbing each node based on the first node feature perturbation parameter and the first perturbation adjacency matrix to obtain a perturbed image, a good concealment effect can be achieved while perturbing the image, making it difficult for graph neural networks to directly recognize, thereby effectively improving the image perturbation effect.

[0031] In one embodiment, generating the first perturbation adjacency matrix based on the node feature matrix includes: Obtain the initial adjacency matrix corresponding to the attribute graph; A matrix set is generated based on the first preset perturbation range and the initial adjacency matrix, the matrix set including multiple perturbation adjacency matrices; Calculate the first loss value and the second loss value corresponding to each of the plurality of perturbation adjacency matrices. The first loss value is used to characterize the training loss of the corresponding perturbation adjacency matrix for the image recognition model, and the second loss value is used to characterize the perturbation loss of the corresponding perturbation adjacency matrix after perturbing the node. The first perturbation adjacency matrix is ​​determined based on the first loss value and the second loss value, wherein the first perturbation adjacency matrix is ​​the perturbation adjacency matrix with the lowest first loss value and the lowest second loss value among the plurality of perturbation adjacency matrices.

[0032] The initial adjacency matrix described above is the adjacency matrix of the attribute graph. Wherein, let... Let A be the adjacency matrix of the graph. ij Node v is defined i and node v j Does a connection exist between them (i.e., node v)? i and node v j (Whether there is an edge between them), if there is a connection between the nodes, then A ij Equals 1, otherwise A ij The value is 0. This means that the connection relationships between different nodes can be determined through the initial adjacency matrix.

[0033] The aforementioned first preset disturbance range is a pre-configured disturbance budget range, which can be understood as the structural interference needing to be within the disturbance budget range when interfering with the first image, so as to achieve effective interference with the image while avoiding excessive interference that would result in poor concealment.

[0034] The first loss value described above characterizes the training loss of the image recognition model by the corresponding perturbation adjacency matrix, and can be understood as the loss value calculated through the attack loss function. The second loss value described above characterizes the perturbation loss after the corresponding perturbation adjacency matrix perturbs the nodes, and can be understood as the loss value of the image recognition model after training based on the perturbation adjacency matrix. It should be noted that the first perturbation adjacency matrix is ​​the perturbation adjacency matrix among multiple perturbation adjacency matrices where both the first and second loss values ​​are the lowest.

[0035] The first perturbation adjacency matrix can be calculated using the following formula: ; st ; in the formula Let A represent the set of matrices, where A is the initial adjacency matrix. The first loss value, This is the second loss value. Let X be the perturbed adjacency matrix, and X be the node characteristics. These are the optimal parameters obtained by the image recognition model on the perturbation graph.

[0036] In this embodiment of the invention, an initial adjacency matrix corresponding to the attribute graph is obtained; a matrix set is generated based on a first preset perturbation range and the initial adjacency matrix, the matrix set including multiple perturbed adjacency matrices; a first loss value and a second loss value are calculated for each of the multiple perturbed adjacency matrices, the first loss value characterizing the training loss of the corresponding perturbed adjacency matrix on the image recognition model, and the second loss value characterizing the perturbation loss after the corresponding perturbed adjacency matrix interferes with the nodes; a first perturbed adjacency matrix is ​​determined based on the first loss value and the second loss value, the first perturbed adjacency matrix being the perturbed adjacency matrix with the lowest first loss value and second loss value among the multiple perturbed adjacency matrices. Thus, the first perturbed adjacency matrix is ​​obtained through the first preset perturbation range and the initial adjacency matrix.

[0037] In one embodiment, generating the first node feature perturbation parameter corresponding to each of the plurality of nodes based on the plurality of edges includes: A first parameter and a second parameter are calculated based on the multiple edges. The first parameter is used to characterize the similarity between two nodes with an edge, and the second parameter is used to characterize the similarity between two nodes without an edge. The initial node feature perturbation parameters for each node are generated based on the second preset perturbation range; A third loss value is calculated based on the initial node feature perturbation parameters, the first parameter, and the second parameter corresponding to each node. The third loss value is used to characterize the similarity between the edges after the nodes are perturbed by the initial node feature perturbation parameters and the edges before the perturbation. If the third loss value is less than or equal to the first threshold, the initial node feature perturbation parameter corresponding to each node is set as the first node feature perturbation parameter.

[0038] The first parameter is used to characterize the similarity between two nodes with an edge, and the second parameter is used to characterize the similarity between two nodes without an edge. The first and second parameters can be used to determine the similarity when there is a connection between different nodes. Then, node feature interference parameters can be determined based on the first and second parameters, so that the relationship between the node after interference by the determined node feature interference parameters and other edges can be similar to the actual existing edges, thereby effectively improving the concealment of the interference.

[0039] The aforementioned second preset disturbance range is a pre-configured range of node feature disturbance parameters. This second preset disturbance range ensures that the node feature disturbance for each node is within a reasonable range, avoiding the problem of poor concealment caused by excessively large node feature disturbance parameters. The second preset disturbance range can be expressed as [- ].

[0040] For example, the relationship between two nodes is as follows: Figure 2 As shown, node feature X i and X j Determine the similarity S of node pairs ij This determines node v i With v j The probability A of the existence of an edge between them ij This leads to the weight values ​​D of the edges in the GNN model (i.e., the image recognition model). ij Thus, by calculating the first and second parameters based on multiple edges, the following can be obtained: Figure 2 The process is shown below.

[0041] Specifically, two nodes with an edge are considered positive samples, and the first parameter is calculated; two nodes with an edge are considered negative samples, and the second parameter is calculated. The first and second parameters are the mean and variance of the similarity, respectively. The first parameter, μ, is obtained from the similarity of the positive samples.pos and variance σ pos , to represent the similarity range of normal edges; the second parameter, the mean μ, is obtained by calculating the similarity of negative samples. neg and variance σ neg , representing the similarity range of normal non-edges.

[0042] The third loss value is calculated based on the initial node feature perturbation parameters, the first parameter, and the second parameter corresponding to each node, and can be specifically expressed by the following formula: ; In the formula, L causal E is the third loss value. perturbed Let X be the set of perturbation edges (a perturbation edge is an edge connecting two nodes, one of which is the node to be classified), sim() represents the similarity, and X is ... i Let i represent the node characteristics of the i-th node. X represents the node characteristic perturbation parameter of the i-th node. j This represents the node characteristics of the j-th node. Let represent the node characteristic perturbation parameter of the j-th node. For target similarity. Where, in A... ij When =1, ; in A ij When =0, .

[0043] It should be noted that the set of nodes to be disturbed is denoted as V. pert The set of perturbed edges can be obtained from the set of nodes to be perturbed, and at least one of the two nodes connected by each perturbed edge is in the set of nodes to be perturbed.

[0044] In this embodiment of the invention, a first parameter and a second parameter are calculated based on the plurality of edges. The first parameter characterizes the similarity between two nodes with an edge, and the second parameter characterizes the similarity between two nodes without an edge. An initial node feature perturbation parameter is generated for each node based on a second preset perturbation range. A third loss value is calculated based on the initial node feature perturbation parameter, the first parameter, and the second parameter. The third loss value characterizes the similarity between the edge after perturbation by the initial node feature perturbation parameter and the edge before perturbation. If the third loss value is less than or equal to a first threshold, the initial node feature perturbation parameter for each node is set as the first node feature perturbation parameter. Thus, by calculating the third loss value using the first and second parameters, the first node feature perturbation parameter for each node can be determined using the third loss value.

[0045] In one embodiment, the method further includes: If the third loss value is greater than the first threshold, the initial node feature perturbation parameters corresponding to each node are adjusted to obtain the intermediate node feature perturbation parameters. The fourth loss value is calculated based on the intermediate node feature perturbation parameters corresponding to each node, the first parameter, and the second parameter; If the fourth loss value is less than or equal to the first threshold, the intermediate node feature perturbation parameter corresponding to each node is set as the first node feature perturbation parameter.

[0046] In this embodiment of the invention, when the third loss value is greater than the first threshold, the initial node feature perturbation parameters corresponding to each node are adjusted to obtain intermediate node feature perturbation parameters; a fourth loss value is calculated based on the intermediate node feature perturbation parameters corresponding to each node, the first parameter, and the second parameter; when the fourth loss value is less than or equal to the first threshold, the intermediate node feature perturbation parameters corresponding to each node are set as the first node feature perturbation parameters. Thus, by adjusting the node feature perturbation parameters, the fourth loss value calculated after adjusting the node feature perturbation parameters can be less than the first threshold, thereby obtaining the first node feature perturbation parameters for each node.

[0047] The adjustment of the initial node feature perturbation parameters corresponding to each node is performed within a second preset perturbation range.

[0048] In one embodiment, generating the first perturbation adjacency matrix based on the node feature matrix includes: The node feature matrix is ​​processed based on the first set model to predict the first perturbation adjacency matrix. The first set model is used to predict the perturbation adjacency matrix corresponding to different node feature matrices. The step of generating the first node feature perturbation parameters corresponding to each of the plurality of nodes based on the plurality of edges includes: The multiple edges are processed based on the second set model to predict the first node feature perturbation parameter corresponding to each node. The second set model is used to predict the node feature perturbation parameter corresponding to different nodes.

[0049] The first model described above is trained to predict the perturbation adjacency matrix, and the second model described above is trained to predict the node feature perturbation parameters corresponding to different nodes. In some implementations, such as Figure 3As shown, the functions of the first and second model settings can be implemented through an attribute perturbation generator. By using an optimizer, the optimal node feature perturbation is obtained through gradient descent, thus ensuring the concealment of the perturbation.

[0050] In this embodiment of the invention, the node feature matrix is ​​processed based on a first predetermined model to predict the first perturbation adjacency matrix. The first predetermined model is used to predict the perturbation adjacency matrix corresponding to different node feature matrices. The plurality of edges are processed based on a second predetermined model to predict the first node feature perturbation parameter corresponding to each node. The second predetermined model is used to predict the node feature perturbation parameter corresponding to different nodes. Thus, the first perturbation adjacency matrix and the first node feature perturbation parameter corresponding to each node are predicted using both the first and second predetermined models.

[0051] In one embodiment, the first and second specified models are obtained as follows: The first initial model is trained based on the sample data to obtain the first intermediate model. The first initial model is used to predict the perturbation adjacency matrix corresponding to the feature matrix of different nodes. The second initial model is trained based on the sample data to obtain the second intermediate model. The second initial model is used to predict the node feature perturbation parameters corresponding to different nodes. Calculate the first training loss value corresponding to the first intermediate model and the second training loss value corresponding to the second intermediate model. The first training loss value is used to characterize the perturbation loss of the node after the perturbation adjacency matrix predicted by the first intermediate model is perturbed. The second training loss value is used to characterize the similarity between the edge after the node is perturbed by the node feature perturbation parameters predicted by the second intermediate model and the edge before the perturbation. Calculate a third training loss value, which is a weighted sum of the first training loss value and the second training loss value; If the third training loss value is less than the second threshold, the first intermediate model is set as the first set model, and the second intermediate model is set as the second set model.

[0052] It should be noted that for both the first and second model settings, the objective is to predict the first perturbation adjacency matrix and the first node feature perturbation parameters that interfere with the nodes, which can be expressed as: ; In the formula Let X be the node after the perturbation, and let X be the node before the perturbation. The first node's characteristic perturbation parameters, This is the first perturbation adjacency matrix.

[0053] The aforementioned first model is obtained by training a first initial model. A first intermediate model is obtained during the training of the first initial model. The training effect of the first intermediate model is determined by a first training loss value, which can be specifically expressed as follows: ; In the formula, L attack This is the first training loss value. For image recognition models, Let X be the first perturbation adjacency matrix in the sample data. i Let be the node features of the i-th node in the sample data. Y is the node feature perturbation parameter of the i-th node predicted by the first intermediate model. i is the label of the node, and c is the label classified by the image recognition model. is a coefficient.

[0054] The second model is obtained by training the second initial model. A second intermediate model is obtained during the training of the second initial model. The training effect of the second intermediate model is determined by the second training loss value, which can be specifically expressed as follows: ; In the formula, L causal E is the second training loss value. perturbed Let X be the set of perturbation edges in the sample data, sim() represents the similarity, and X i This represents the node feature of the i-th node in the sample data. X represents the node feature perturbation parameter of the i-th node predicted by the second intermediate model. j This represents the node feature of the j-th node in the sample data. This represents the node feature perturbation parameter of the j-th node predicted by the second intermediate model. For target similarity. Where, in A... ij When =1, ; in A ij When =0, .

[0055] Furthermore, the third training loss value is calculated, which can be specifically expressed as: ; In the formula, L total This is the third training loss value. These are the weighting coefficients.

[0056] In this way, by training the first and second predefined models as described above, the first perturbation adjacency matrix and the first node feature perturbation parameters corresponding to each node can be predicted using the first and second predefined models.

[0057] In some implementations, the same functionality as a first and second preset model can be achieved using an attribute perturbation generator. For example... Figure 3 As shown, the adjacency matrix and attribute matrix (i.e., node feature matrix) of the original image (first image) are extracted and input into the attribute perturbation generator through node similarity extraction. Simultaneously, an adversarial graph A is generated through structural perturbation (i.e., the first perturbation adjacency matrix). The adjacency matrix and attribute matrix of adversarial graph A are extracted and input into the attribute perturbation generator. The attribute perturbation generator then calculates L. attack and L causal This is done to obtain the adversarial image B, which is the image after interference.

[0058] In this embodiment of the invention, a first initial model is trained based on sample data to obtain a first intermediate model. The first initial model is used to predict the perturbation adjacency matrix corresponding to the feature matrices of different nodes. A second initial model is trained based on the sample data to obtain a second intermediate model. The second initial model is used to predict the node feature perturbation parameters corresponding to different nodes. A first training loss value corresponding to the first intermediate model and a second training loss value corresponding to the second intermediate model are calculated. The first training loss value is used to characterize the perturbation loss of nodes after the perturbation adjacency matrix predicted by the first intermediate model is perturbed. The second training loss value is used to characterize the similarity between the edges of nodes after perturbation and the edges before perturbation by the node feature perturbation parameters predicted by the second intermediate model. A third training loss value is calculated, which is the weighted sum of the first training loss value and the second training loss value. If the third training loss value is less than a second threshold, the first intermediate model is set as the first preset model, and the second intermediate model is set as the second preset model. In this way, by training the first preset model and the second preset model, the first perturbation adjacency matrix and the first node feature perturbation parameters corresponding to each node can be predicted using the first preset model and the second preset model.

[0059] It should be noted that the methods of this invention can be applied to different fields.

[0060] For example, node classification using GNNs can be applied to transaction networks, where nodes represent transaction accounts and edges represent transaction relationships. The goal is to classify whether a node is a "fraudulent account." It should be understood that fraudsters add temporary transaction edges between themselves and reputable accounts, embedding the "fraudulent account" into the normal transaction community. Therefore, by fine-tuning the characteristics of fraudulent accounts (such as the embedding representation of transaction time and amount) to match the characteristic patterns of the normal community, the model can identify that the account is in a normal context, thereby reducing its fraud risk score and allowing fraudulent transactions to succeed. Subsequent model training using perturbed images allows the model to accurately identify "fraudulent accounts" with these characteristics, improving the recognition accuracy.

[0061] For example, the method of the present invention can be applied to link prediction applications. In such applications, the system typically uses a Generative Neural Network (GNN) to construct link predictions to predict the relationship between users and items, thereby recommending relevant products to users. By inserting certain spurious edges or deleting certain real edges into the recommendation system, the information about users or products is fine-tuned to make attacks difficult to detect, thus introducing errors into the link prediction results.

[0062] For example, the method of this invention can be applied to the field of graph classification. Graph classification typically relies on global structure, and modifying key edges through structural perturbation can affect the global representation of the graph. For instance, in biomedicine, graph classification techniques are used to screen compounds with specific pharmacological properties. Modifying key edges can cause significant changes in the representation of compounds, leading to misclassification. Furthermore, because node feature perturbations hide the perturbation edges, their presence is difficult to detect. By training the model based on the perturbated image, the trained model can accurately identify compounds with specific pharmacological properties, thus improving the recognition accuracy.

[0063] Please see Figure 4 , Figure 4 This is a structural diagram of an image interference device provided in an embodiment of the present invention, as shown below. Figure 4 As shown, the image interference device 400 includes: The acquisition module 401 is used to acquire the attribute map of the first image, the attribute map including multiple nodes, multiple edges, and a node feature matrix, the node feature matrix being used to characterize the features of different nodes among the multiple nodes; The first generation module 402 is used to generate a first perturbation adjacency matrix based on the node feature matrix. The first perturbation adjacency matrix is ​​used to characterize the structural perturbation of different nodes. The second generation module 403 is used to generate first node feature perturbation parameters for each of the plurality of nodes based on the plurality of edges; The interference module 404 is used to interfere with each node based on the first node feature perturbation parameter and the first perturbation adjacency matrix corresponding to each node, so as to obtain an interference image, which is used to train the image recognition model.

[0064] In one embodiment, the first generation module 402 includes: The acquisition unit is used to acquire the initial adjacency matrix corresponding to the attribute graph; The first generation unit is configured to generate a matrix set based on a first preset perturbation range and the initial adjacency matrix, the matrix set including multiple perturbation adjacency matrices; The first calculation unit is used to calculate the first loss value and the second loss value corresponding to each of the plurality of perturbation adjacency matrices. The first loss value is used to characterize the training loss of the corresponding perturbation adjacency matrix on the image recognition model, and the second loss value is used to characterize the perturbation loss after the corresponding perturbation adjacency matrix perturbs the node. The determining unit is configured to determine the first perturbation adjacency matrix based on the first loss value and the second loss value, wherein the first perturbation adjacency matrix is ​​the perturbation adjacency matrix with the lowest first loss value and the lowest second loss value among the plurality of perturbation adjacency matrices.

[0065] In one embodiment, the second generation module 403 includes: The second calculation unit is used to calculate a first parameter and a second parameter based on the plurality of edges. The first parameter is used to characterize the similarity between two nodes with edges, and the second parameter is used to characterize the similarity between two nodes without edges. The second generation unit is used to generate initial node feature perturbation parameters for each node based on the second preset perturbation range; The third calculation unit is used to calculate a third loss value based on the initial node feature perturbation parameters, the first parameter, and the second parameter corresponding to each node. The third loss value is used to characterize the similarity between the edges after the nodes are perturbed by the initial node feature perturbation parameters and the edges before the perturbation. The setting unit is used to set the initial node feature perturbation parameter corresponding to each node as the first node feature perturbation parameter when the third loss value is less than or equal to the first threshold.

[0066] In one embodiment, the image interference device 400 further includes: The adjustment module is used to adjust the initial node feature perturbation parameters corresponding to each node when the third loss value is greater than the first threshold, so as to obtain the intermediate node feature perturbation parameters. The calculation module is used to calculate a fourth loss value based on the intermediate node feature perturbation parameters corresponding to each node, the first parameter, and the second parameter; The setting module is used to set the intermediate node feature perturbation parameter corresponding to each node as the first node feature perturbation parameter when the fourth loss value is less than or equal to the first threshold.

[0067] In one embodiment, the first generation module 402 includes: The first prediction unit is used to process the node feature matrix based on the first set model to predict the first perturbation adjacency matrix. The first set model is used to predict the perturbation adjacency matrix corresponding to different node feature matrices. The second generation module 403 includes: The second prediction unit is used to process the multiple edges based on the second set model to predict the first node feature perturbation parameter corresponding to each node. The second set model is used to predict the node feature perturbation parameter corresponding to different nodes.

[0068] In one embodiment, the first and second specified models are obtained as follows: The first initial model is trained based on the sample data to obtain the first intermediate model. The first initial model is used to predict the perturbation adjacency matrix corresponding to the feature matrix of different nodes. The second initial model is trained based on the sample data to obtain the second intermediate model. The second initial model is used to predict the node feature perturbation parameters corresponding to different nodes. Calculate the first training loss value corresponding to the first intermediate model and the second training loss value corresponding to the second intermediate model. The first training loss value is used to characterize the perturbation loss of the node after the perturbation adjacency matrix predicted by the first intermediate model is perturbed. The second training loss value is used to characterize the similarity between the edge after the node is perturbed by the node feature perturbation parameters predicted by the second intermediate model and the edge before the perturbation. Calculate a third training loss value, which is a weighted sum of the first training loss value and the second training loss value; If the third training loss value is less than the second threshold, the first intermediate model is set as the first set model, and the second intermediate model is set as the second set model.

[0069] The image interference device provided in this embodiment of the invention can realize the various processes of the above-described image interference method, with one-to-one correspondence of technical features and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0070] It should be noted that the image interference device in the embodiments of the present invention can be a device, or it can be a component, integrated circuit, or chip in an electronic device.

[0071] This invention also provides an electronic device, including: a processor, a memory, and a program stored in the memory and executable on the processor, wherein the program, when executed by the processor, implements the above-described functionality. Figure 1 The various processes of the image interference method embodiments shown are all capable of achieving the same technical effect, and will not be described again here to avoid repetition.

[0072] For details, see Figure 5 As shown, this embodiment of the invention also provides an electronic device, including a bus 501, a transceiver 502, an antenna 503, a bus interface 504, a processor 505, and a memory 506.

[0073] The transceiver 502 is used to acquire an attribute map of the first image. The attribute map includes multiple nodes, multiple edges, and a node feature matrix. The node feature matrix is ​​used to characterize the features of different nodes among the multiple nodes. The processor 505 is used to generate a first perturbation adjacency matrix based on the node feature matrix, wherein the first perturbation adjacency matrix is ​​used to characterize the structural perturbation of different nodes. The processor 505 is further configured to generate first node feature perturbation parameters corresponding to each of the plurality of nodes based on the plurality of edges; The processor 505 is further configured to perturb each node based on the first node feature perturbation parameter and the first perturbation adjacency matrix corresponding to each node, thereby obtaining an interference image, which is used to train the image recognition model.

[0074] In one embodiment, generating the first perturbation adjacency matrix based on the node feature matrix includes: Obtain the initial adjacency matrix corresponding to the attribute graph; A matrix set is generated based on the first preset perturbation range and the initial adjacency matrix, the matrix set including multiple perturbation adjacency matrices; Calculate the first loss value and the second loss value corresponding to each of the plurality of perturbation adjacency matrices. The first loss value is used to characterize the training loss of the corresponding perturbation adjacency matrix for the image recognition model, and the second loss value is used to characterize the perturbation loss of the corresponding perturbation adjacency matrix after perturbing the node. The first perturbation adjacency matrix is ​​determined based on the first loss value and the second loss value, wherein the first perturbation adjacency matrix is ​​the perturbation adjacency matrix with the lowest first loss value and the lowest second loss value among the plurality of perturbation adjacency matrices.

[0075] In one embodiment, generating the first node feature perturbation parameter corresponding to each of the plurality of nodes based on the plurality of edges includes: A first parameter and a second parameter are calculated based on the multiple edges. The first parameter is used to characterize the similarity between two nodes with an edge, and the second parameter is used to characterize the similarity between two nodes without an edge. The initial node feature perturbation parameters for each node are generated based on the second preset perturbation range; A third loss value is calculated based on the initial node feature perturbation parameters, the first parameter, and the second parameter corresponding to each node. The third loss value is used to characterize the similarity between the edges after the nodes are perturbed by the initial node feature perturbation parameters and the edges before the perturbation. If the third loss value is less than or equal to the first threshold, the initial node feature perturbation parameter corresponding to each node is set as the first node feature perturbation parameter.

[0076] In one embodiment, the processor 505 is further configured to adjust the initial node feature perturbation parameters corresponding to each node to obtain intermediate node feature perturbation parameters when the third loss value is greater than the first threshold. The processor 505 is further configured to calculate a fourth loss value based on the intermediate node feature perturbation parameters corresponding to each node, the first parameter, and the second parameter; The processor 505 is further configured to, when the fourth loss value is less than or equal to the first threshold, set the intermediate node feature perturbation parameter corresponding to each node as the first node feature perturbation parameter.

[0077] In one embodiment, generating the first perturbation adjacency matrix based on the node feature matrix includes: The node feature matrix is ​​processed based on the first set model to predict the first perturbation adjacency matrix. The first set model is used to predict the perturbation adjacency matrix corresponding to different node feature matrices. The step of generating the first node feature perturbation parameters corresponding to each of the plurality of nodes based on the plurality of edges includes: The multiple edges are processed based on the second set model to predict the first node feature perturbation parameter corresponding to each node. The second set model is used to predict the node feature perturbation parameter corresponding to different nodes.

[0078] In one embodiment, the first and second specified models are obtained as follows: The first initial model is trained based on the sample data to obtain the first intermediate model. The first initial model is used to predict the perturbation adjacency matrix corresponding to the feature matrix of different nodes. The second initial model is trained based on the sample data to obtain the second intermediate model. The second initial model is used to predict the node feature perturbation parameters corresponding to different nodes. Calculate the first training loss value corresponding to the first intermediate model and the second training loss value corresponding to the second intermediate model. The first training loss value is used to characterize the perturbation loss of the node after the perturbation adjacency matrix predicted by the first intermediate model is perturbed. The second training loss value is used to characterize the similarity between the edge after the node is perturbed by the node feature perturbation parameters predicted by the second intermediate model and the edge before the perturbation. Calculate a third training loss value, which is a weighted sum of the first training loss value and the second training loss value; If the third training loss value is less than the second threshold, the first intermediate model is set as the first set model, and the second intermediate model is set as the second set model.

[0079] exist Figure 5 In this document, a bus architecture (represented by bus 501) is used. Bus 501 can include any number of interconnected buses and bridges, linking various circuits including one or more processors represented by processor 505 and memory represented by memory 506. Bus 501 can also link various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and therefore will not be described further herein. Bus interface 504 provides an interface between bus 501 and transceiver 502. Transceiver 502 can be a single element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices over a transmission medium. Data processed by processor 505 is transmitted over a wireless medium via antenna 503, which further receives data and transmits it to processor 505.

[0080] Processor 505 manages bus 501 and general processing, and also provides various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory 506 can be used to store data used by processor 505 during operation.

[0081] Optionally, the processor 505 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or a graphics processing unit (GPU).

[0082] This invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, performs the above-described functions. Figure 1 The various processes corresponding to the image interference method embodiments, and which achieve the same technical effect, will not be described again here to avoid repetition. The computer-readable storage medium mentioned includes, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0083] The present invention also provides a computer program product, including computer instructions that, when executed by a processor, implement the above-described... Figure 1 The various processes of the corresponding image interference recognition method embodiments can achieve the same technical effect, and will not be described again here to avoid repetition.

[0084] In the embodiments of this invention, the terms "first," "second," etc., are used to distinguish similar object parameters and are not necessarily used to describe a specific order or sequence. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products, or devices. Additionally, the use of "and / or" in this application indicates at least one of the connected object parameters, such as A and / or B and / or C, representing seven possibilities: A alone, B alone, C alone, both A and B present, both B and C present, both A and C present, and A, B, and C present.

[0085] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0086] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or second terminal device, etc.) to execute the methods of the various embodiments of this application.

[0087] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. An image interference method, characterized in that, include: Obtain the attribute graph of the first image, the attribute graph including multiple nodes, multiple edges, and a node feature matrix, the node feature matrix being used to characterize the features of different nodes among the multiple nodes; A first perturbation adjacency matrix is ​​generated based on the node feature matrix. The first perturbation adjacency matrix is ​​used to characterize the structural perturbation of different nodes. Generate first node feature perturbation parameters for each of the multiple nodes based on the multiple edges; Based on the first node feature perturbation parameters and the first perturbation adjacency matrix corresponding to each node, the nodes are perturbed to obtain perturbation images, which are used to train the image recognition model.

2. The method as described in claim 1, characterized in that, The step of generating the first perturbation adjacency matrix based on the node feature matrix includes: Obtain the initial adjacency matrix corresponding to the attribute graph; A matrix set is generated based on the first preset perturbation range and the initial adjacency matrix, the matrix set including multiple perturbation adjacency matrices; Calculate the first loss value and the second loss value corresponding to each of the plurality of perturbation adjacency matrices. The first loss value is used to characterize the training loss of the corresponding perturbation adjacency matrix for the image recognition model, and the second loss value is used to characterize the perturbation loss of the corresponding perturbation adjacency matrix after perturbing the node. The first perturbation adjacency matrix is ​​determined based on the first loss value and the second loss value, wherein the first perturbation adjacency matrix is ​​the perturbation adjacency matrix with the lowest first loss value and the lowest second loss value among the plurality of perturbation adjacency matrices.

3. The method as described in claim 1, characterized in that, The step of generating the first node feature perturbation parameters corresponding to each of the plurality of nodes based on the plurality of edges includes: A first parameter and a second parameter are calculated based on the multiple edges. The first parameter is used to characterize the similarity between two nodes with an edge, and the second parameter is used to characterize the similarity between two nodes without an edge. The initial node feature perturbation parameters for each node are generated based on the second preset perturbation range; A third loss value is calculated based on the initial node feature perturbation parameters, the first parameter, and the second parameter corresponding to each node. The third loss value is used to characterize the similarity between the edges after the nodes are perturbed by the initial node feature perturbation parameters and the edges before the perturbation. If the third loss value is less than or equal to the first threshold, the initial node feature perturbation parameter corresponding to each node is set as the first node feature perturbation parameter.

4. The method as described in claim 3, characterized in that, The method further includes: If the third loss value is greater than the first threshold, the initial node feature perturbation parameters corresponding to each node are adjusted to obtain the intermediate node feature perturbation parameters. The fourth loss value is calculated based on the intermediate node feature perturbation parameters corresponding to each node, the first parameter, and the second parameter; If the fourth loss value is less than or equal to the first threshold, the intermediate node feature perturbation parameter corresponding to each node is set as the first node feature perturbation parameter.

5. The method according to any one of claims 1 to 4, characterized in that, The step of generating the first perturbation adjacency matrix based on the node feature matrix includes: The node feature matrix is ​​processed based on the first set model to predict the first perturbation adjacency matrix. The first set model is used to predict the perturbation adjacency matrix corresponding to different node feature matrices. The step of generating the first node feature perturbation parameters corresponding to each of the plurality of nodes based on the plurality of edges includes: The multiple edges are processed based on the second set model to predict the first node feature perturbation parameter corresponding to each node. The second set model is used to predict the node feature perturbation parameter corresponding to different nodes.

6. The method as described in claim 5, characterized in that, The first and second model settings are obtained as follows: The first initial model is trained based on the sample data to obtain the first intermediate model. The first initial model is used to predict the perturbation adjacency matrix corresponding to the feature matrix of different nodes. The second initial model is trained based on the sample data to obtain the second intermediate model. The second initial model is used to predict the node feature perturbation parameters corresponding to different nodes. Calculate the first training loss value corresponding to the first intermediate model and the second training loss value corresponding to the second intermediate model. The first training loss value is used to characterize the perturbation loss of the node after the perturbation adjacency matrix predicted by the first intermediate model is perturbed. The second training loss value is used to characterize the similarity between the edge after the node is perturbed by the node feature perturbation parameters predicted by the second intermediate model and the edge before the perturbation. Calculate a third training loss value, which is a weighted sum of the first training loss value and the second training loss value; If the third training loss value is less than the second threshold, the first intermediate model is set as the first set model, and the second intermediate model is set as the second set model.

7. An image interference device, characterized in that, include: The acquisition module is used to acquire the attribute graph of the first image, the attribute graph including multiple nodes, multiple edges, and a node feature matrix, the node feature matrix being used to characterize the features of different nodes among the multiple nodes; The first generation module is used to generate a first perturbation adjacency matrix based on the node feature matrix. The first perturbation adjacency matrix is ​​used to characterize the structural perturbation of different nodes. The second generation module is used to generate first node feature perturbation parameters for each of the multiple nodes based on the multiple edges; The interference module is used to interfere with each node based on the first node feature perturbation parameters and the first perturbation adjacency matrix corresponding to each node, so as to obtain an interference image, which is used to train the image recognition model.

8. An electronic device, characterized in that, Including transceivers and processors, The transceiver is used to acquire an attribute graph of a first image. The attribute graph includes multiple nodes, multiple edges, and a node feature matrix. The node feature matrix is ​​used to characterize the features of different nodes among the multiple nodes. The processor is configured to generate a first perturbation adjacency matrix based on the node feature matrix, wherein the first perturbation adjacency matrix is ​​used to characterize the structural perturbation of different nodes. The processor is further configured to generate first node feature perturbation parameters corresponding to each of the plurality of nodes based on the plurality of edges; The processor is further configured to perturb each node based on the first node feature perturbation parameter and the first perturbation adjacency matrix corresponding to each node, thereby obtaining a perturbation image, which is used to train the image recognition model.

9. An electronic device, characterized in that, include: A processor, a memory, and a program stored in the memory and executable on the processor, wherein the program, when executed by the processor, implements the steps of the image interference method as described in any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the image interference method as described in any one of claims 1 to 6.

11. A computer program product, characterized in that, It includes computer instructions that, when executed by a processor, implement the steps of the image interference method as described in any one of claims 1 to 6.

Images