Parkinson's disease auxiliary judgment model based on image attention

Abstract

The invention discloses a Parkinson's disease diagnosis method based on graph attention, and the method comprises the steps: building a brain function connection graph based on a resting state functional magnetic resonance image, expanding the information exchange between graph nodes to the global graph through the self-attention mechanism of a Transform model, and achieving the diagnosis of Parkinson's disease. Relative and absolute position codes of the graph are constructed on the basis of a graph Laplacian matrix and characteristic decomposition thereof, node characteristic information and graph topological structure information of the brain function connection graph are fully utilized, and rich information of attention layers of different graphs is further fused through jump connection; the problems of over-smoothing and insufficient aggregation of partial node information are solved, and finally the brain function connection diagram is divided into two types of Parkinson's disease patients and normal people, so that accurate Parkinson's disease computer-aided diagnosis is realized. The highly-discriminated brain region for Parkinson's disease diagnosis has high correlation with the research result of the brain function of the Parkinson's disease in the past, so that the invention provides a new strategy for searching imaging markers for diagnosing Parkinson's disease.

Description

Technical Field

[0001] The present invention relates to the technical field of brain magnetic functional resonance imaging, and in particular to a Parkinson's auxiliary judgment model based on image attention. Background Art

[0002] Parkinson's disease (PD) is a common neurodegenerative disease that mainly affects middle-aged and elderly people. The pathogenesis of the disease is complex. It is currently believed that it is caused by the gradual loss of dopamine neurons in the substantia nigra of the brain, which leads to a decrease in dopamine levels and causes motor dysfunction. Typical symptoms of Parkinson's disease include resting tremor, muscle rigidity, bradykinesia, and postural balance disorders.

[0003] In recent years, with the intensification of global aging, the incidence of Parkinson's disease has been on the rise, and it has become one of the diseases that seriously affect human health and quality of life. Although Parkinson's disease cannot be completely cured at present, comprehensive treatment methods such as drug therapy, surgical treatment, rehabilitation therapy and psychological intervention can significantly improve patients' symptoms and quality of life; for the treatment of Parkinson's disease, researchers have been looking for new drugs and treatments. In recent years, with the development of biotechnology, new treatment strategies such as gene therapy and cell replacement therapy have gradually become research hotspots. In addition, the field of medical devices is also constantly developing, and various auxiliary devices and rehabilitation equipment provide better life support for Parkinson's patients.

[0004] In my country, significant progress has been made in the research and treatment of Parkinson's disease. National policies have given strong support to the research and development of neurological diseases, promoted the launch of a number of innovative drugs and medical devices, and made a series of important breakthroughs in the pathogenesis, diagnostic techniques and treatment methods of Parkinson's disease. However, there are no very clear standards for the auxiliary diagnosis of Parkinson's disease. However, by performing magnetic resonance imaging of the patient's brain, there are about three methods for computer-aided diagnosis and judgment of images based on the type of data used, namely, methods based on speech signals, methods based on functional magnetic resonance imaging, and methods based on positron emission tomography. However, most of this type of research is currently based on original images and relatively traditional single classification algorithms, ignoring the abnormal brain function revealed by the internal correlation of the data.

[0005] For the processing of brain functional connectivity maps, the currently more commonly used graph convolutional neural network updates node information by aggregating local neighborhoods, but fails to fully utilize global node features. In addition, as the number of network layers increases, node features will gradually converge and become difficult to distinguish, resulting in over-smoothing. Summary of the invention

[0006] In view of the shortcomings of the above-mentioned technology that cannot solve the over-smoothing of node features of brain functional connection diagrams and insufficient aggregation of some node information, the present invention provides a Parkinson's auxiliary judgment model based on image attention.

[0007] To achieve the above object, the present invention provides a Parkinson's auxiliary judgment model based on image attention, comprising the following steps:

[0008] S1-1, firstly, resting-state functional magnetic resonance imaging of the individuals was performed, and the raw image data were preprocessed;

[0009] S1-2, aligning the individual's original resting-state functional magnetic resonance image data from the original space to the standard template space;

[0010] S1-3, according to the automatic anatomical labeling (AAL) brain atlas partition template, the resting-state functional magnetic resonance images were divided into 90 brain regions, and the blood oxygen level-dependent signals of the voxels contained in each brain region were averaged to obtain the average time series of each brain region;

[0011] S1-4, calculate the Pearson correlation coefficient matrix P between each brain region, with a size of 90×90, so as to obtain the brain functional connection strength between different brain regions;

[0012] S1-5, taking the brain regions as graph nodes, assigning the top 20% of the brain function connection strength matrix P as 1, and the rest as 0, quantizing it into an adjacency matrix A, and using the connection strength between the brain regions and other brain regions as node features to establish a brain function connection map, and obtaining two results for Parkinson's patients and healthy people through the brain function connection map.

[0013] Wherein, a graph node position code and a graph kernel are generated, the Laplacian matrix of the brain function connection graph is eigen-decomposed and the Laplacian position code is constructed.

[0014] Among them, the adjacency matrix graph kernel is calculated, and the adjacency matrix graph kernel describes the direct first-order connection relationship. The relationship with mutual connection is assigned a value, and the relationship without mutual connection is 0.

[0015] Among them, the brain functional connection map is classified through the graph attention network, and the graph node combination embedding containing the graph node position encoding and graph node features is constructed as the network input; the graph node combination embedding is input into the main part of the graph attention network, the graph attention stack, which contains 3 layers in total, and each layer has the same structure, including two parts: the graph kernel-based attention sublayer and the forward propagation sublayer.

[0016] Among them, the output of each layer of the graph attention stack is input into the skip knowledge readout graph pooling module. The graph attention module stack has multiple layers. After splicing the representation information learned in each layer, the full graph node representation is obtained.

[0017] Among them, through the fully connected layer, the obtained graph representation vector is converted into a two-dimensional vector output. The two dimensions respectively represent the two categories of Parkinson's disease and normal control. The true label of the sample is a one-hot two-dimensional vector, where the dimension corresponding to its category is 1 and the other dimensions are 0. Cross entropy is used as the loss function to obtain the loss function of the entire graph attention network.

[0018] Among them, P is the Pearson correlation coefficient matrix calculated by S3. Each row represents the functional connection strength between a graph node and other graph nodes. The initial node feature Pearson correlation coefficient matrix P is mapped into the graph node embedding space.

[0019] Among them, the formula after the eigenvalue decomposition of the Laplacian matrix L is:

[0020]

[0021] , in the above formula, A is the adjacency matrix, D is the degree matrix, L is the Laplacian matrix of the brain functional connection graph, U represents the eigenvector matrix of L, and λ represents the eigenvalue diagonal matrix;

[0022] Construct the Laplacian positional encoding PE Lap The formula is:

[0023] PE Lap = ψ1(U[:k])

[0024] , in the above formula, ψ1 represents the learnable encoder, and k represents the eigenvector U corresponding to the first k (k < N) smallest eigenvalues.

[0025] Among them, the formula for calculating the adjacency matrix graph kernel K A is:

[0026]

[0027] , in the above formula, A is the adjacency matrix and D is the degree matrix.

[0028] Among them, the formula for calculating the adjacency matrix graph kernel K A is:

[0029]

[0030] Among them, the formula of the attention sublayer MultiHead(X, X, K A ) is:

[0031]

[0032] In the above formula, n is the number of nodes of the graph, X is the input of this layer, which is the node embedding in the first layer, K Ais the adjacency matrix graph kernel, || represents concatenation, H is the number of attention heads, and W O is the learnable projection matrix, head h The calculation formula is:

[0033]

[0034] In the above formula, is the learnable projection matrix, and the calculation formula of PosAttention(·) is:

[0035]

[0036] In the above formula, Q is the query matrix and V is the value matrix.

[0037] Among them, the specific formula for the full-graph node representation H is:

[0038]

[0039] Then the Readout readout method is used to obtain a one-dimensional graph representation through Readout aggregation pooling. The specific formula is:

[0040]

[0041] Where n is the number of graph nodes, h i is a row in the matrix H, || represents concatenation, and max is the maximum value of each dimension in the n row vectors.

[0042] The beneficial effects of the present invention are as follows: compared with the prior art, the Parkinson's auxiliary judgment model based on image attention provided by the present invention uses the self-attention mechanism to expand the information exchange between graph nodes to the global graph, fully learns the node features of the brain functional connection graph of Parkinson's patients, and fully learns the topological features of the graph through the position encoding based on the graph representation and the attention mechanism based on the graph kernel. In addition, the graph pooling based on the jump knowledge connection is used to fuse the multi-scale node representations of different Transformer Encoder layers, solving the problems of over-smoothing and insufficient aggregation of some node information. Compared with the existing computer-aided diagnosis method for Parkinson's disease, the method of the present invention has a higher diagnostic accuracy, and the high-identification brain area for Parkinson's disease diagnosis discovered by the method of the present invention is highly correlated with the previous research results of Parkinson's disease brain function. Therefore, the framework proposed by the present invention provides a new strategy for finding imaging markers for diagnosing Parkinson's disease. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 A flow chart of the steps of the Parkinson's auxiliary judgment model method based on the jump knowledge graph kernel Transformer provided by the present invention;

[0044] Figure 2 This is a detailed schematic diagram of the overall framework of the Parkinson's auxiliary judgment model based on the jump knowledge graph kernel Transformer;

[0045] Figure 3 Schematic diagram of the process of constructing a brain functional connectivity map;

[0046] Figure 4 Schematic diagram of Laplace position coding; (a) represents the matrix composed of the eigenvectors of the Laplace matrix, and (b) represents the distribution of the eigenvectors on the frequency domain axis.

[0047] Figure 5 This is a schematic diagram of the Transformer Encoder layer structure;

[0048] Figure 6 Schematic diagram of the graph pooling module based on skip knowledge reading. DETAILED DESCRIPTION

[0049] In order to more clearly illustrate the present invention, the present invention is further described below in conjunction with the accompanying drawings.

[0050] See also Figure 1-Figure 6 The present invention provides a Parkinson's auxiliary judgment model based on image attention, which constructs a brain function connection map based on resting-state functional magnetic resonance images, uses the self-attention mechanism of the Transformer model to expand the information exchange between graph nodes to the global graph, and constructs the relative and absolute position encoding of the graph based on the graph Laplacian matrix and its eigendecomposition, fully utilizes the node feature information and graph topological structure information of the brain function connection map, further integrates the rich information of different graph attention layers through jump connections, solves the problems of over-smoothing and insufficient aggregation of some node information, and finally divides the brain function connection map into two categories: Parkinson's patients and normal people, thereby realizing accurate computer-aided diagnosis of Parkinson's disease.

[0051] The specific method flow of the present invention is as follows Figure 1 The detailed framework is as shown in Figure 2 As shown, the specific method includes the following steps:

[0052] S1-1, firstly, resting-state functional magnetic resonance imaging of the individuals was performed, and the raw image data were preprocessed;

[0053] S1-2, aligning the individual original resting-state functional magnetic resonance image (rsfMRI original image) data from the original space to the standard template space;

[0054] S1-3, according to the automatic anatomical labeling brain atlas partitioning template, the resting state functional magnetic resonance imaging (rsfMRI) is divided into 90 brain regions, and the blood oxygen level dependent signal (BOLD signal) of the voxels contained in each brain region is averaged to obtain the average time series of each brain region, using the AAL brain partitioning template (Anatomical Automatic Lcabeling);

[0055] S1-4, calculate the Pearson correlation coefficient matrix P between each brain region, with a size of 90 × 90, so as to obtain the brain functional connectivity strength between different brain regions;

[0056] S1-5, see Figure 2 , taking the brain regions as graph nodes, assigning the top 20% of the brain function connection strength matrix P to 1, and the rest to 0, quantizing it into an adjacency matrix A, and using the connection strength between the brain regions and other brain regions as node features to establish a brain function connection map, and obtaining two results for Parkinson's patients and healthy people through the brain function connection map.

[0057] The resting-state functional magnetic resonance imaging of the present invention is a brain imaging technology used to study the functional connectivity and activity patterns of the brain when there is no specific task or stimulation; the DPABI platform, as a processing software for pre-processing images, can support a variety of brain imaging data types such as task-state fMRI, structural MRI, diffusion tensor imaging, etc. The present invention visualizes fMRI images through DPABI, and performs time series correction, head motion correction, spatial standardization, smoothing, denoising and other processing to ensure normal data quality; the AAL brain partition template can automatically annotate brain regions on functional magnetic resonance imaging images based on voxels.

[0058] See also Figure 4 In order to generate graph node position encoding and graph core more accurately, the specific steps include:

[0059] S2-1, perform eigendecomposition on the Laplacian matrix L of the brain function connection map. The specific formula of the Laplacian matrix L is:

[0060]

[0061] In the above formula, A is the adjacency matrix, D is the degree matrix, and L is Figure 4 The Laplace matrix of L, U represents the eigenvector matrix of L, and λ represents the eigenvalue diagonal matrix;

[0062] S2-2, construct Laplace position encoding PE Lap ,like Figure 4 As shown, Laplace position encoding PE Lap The specific formula is:

[0063] PELap = ψ1(U[:k])

[0064] In the above formula, ψ1 represents a learnable encoder that maps the positional encoding to the node embedding space where Figure 4 (a) represents the matrix composed of the eigenvectors U of the Laplacian matrix. Each column represents an eigenvector U, and each row represents the values corresponding to a node in all eigenvectors U. Figure 4 (b) represents distributing the eigenvectors onto the frequency domain axis. The N values of each eigenvector are the topological information of the graph nodes at the corresponding frequencies. To ensure consistent dimensions and avoid the influence of high-frequency noise, only the eigenvectors U corresponding to the first k (k < N) smallest eigenvalues are retained. The eigenvector U describes the topological information of the graph nodes at different frequency bands and can represent the topological environment where the nodes are located from a global perspective. The Laplacian positional encoding PE Lap describes the absolute positional relationship between nodes;

[0065] S2-3. Calculate the adjacency matrix graph kernel K A , the adjacency matrix graph kernel K A describes the first-order connection relationship directly connected to the current node. If there is a mutual connection relationship, it is assigned a value, and if there is no mutual connection relationship, it is 0. The adjacency matrix graph kernel K A The specific formula is:

[0066] After generating the positional encoding and graph kernel for the graph nodes, the brain functional connectivity graph is classified through a graph attention network, which specifically includes the following steps:

[0067] S3-1. Construct a node combined embedding containing the node positional encoding PE Lap and the node feature X emb as the network input. The specific formula is:

[0068] X = X emb + PE Lap

[0069] In the above formula, PE Lap is the Laplacian positional encoding calculated in S2-2. The calculation formula for the node embedding X emb is:

[0070] X enb = ψ2(P)

[0071] In the above formula, P is the Pearson correlation coefficient matrix calculated in S1-3. Each row represents the functional connection strength between a node and other nodes. ψ2 is a learnable encoder that maps the initial node feature P to the node embedding space where;

[0072] S3-2, such as Figure 2 As shown in Figure 1, the node combination calculated by S3-1 is embedded into the main part of the network, the Transformer Encoder stack. The stack consists of 3 layers, using the Transformer Encoder layer structure. Each layer has the same structure, including a graph kernel-based attention sublayer and a forward propagation sublayer. The Transformer Encoder layer structure is shown in Figure 1. Figure 5 As shown, the attention sub-layer formula MultiHead(X,X,K A )for:

[0073]

[0074] In the above formula, X is the input of this layer, which is the node embedding in the first layer, and K A is the adjacency matrix kernel calculated by S2-3, || represents concatenation, H is the number of attention heads, and W O is the learnable projection matrix, head h The calculation formula is:

[0075]

[0076] In the above formula, is a learnable projection matrix, and the calculation formula of PosAttention(·) is:

[0077]

[0078] In the above formula, Q is the query matrix and V is the value matrix. Different from the traditional graph convolution that updates node information by aggregating local neighborhoods, the present invention uses the attention mechanism to enable global nodes to exchange information with each other. The graph kernel contains rich topological structure information, which represents the relative position relationship between nodes. By multiplying the graph kernel with the attention matrix, the attention is more focused on the graph structure, thereby fully learning the global node features and graph structure features of the graph.

[0079] The attention sublayer is followed by the Add&Norm operation, and the specific formula is:

[0080]

[0081] Where D is the degree matrix of the graph. To eliminate the influence of the densely connected parts of the graph, D is used before the Add instruction operation. -1 / 2 Normalize the output of the attention sublayer. After the Add instruction and normalization operation, it becomes the forward propagation sublayer. The specific formula is:

[0082] X″=ReLu(X′W1)W2

[0083] Among them, W1 and W2 are learnable projection matrices, ReLu(·) is a nonlinear activation function, and after the forward propagation sublayer, it is an Add instruction and a normalization operation. The specific formula is:

[0084] X″′=LayerNorm(X″+X′)

[0085] In the above formula, X'' is the input of the next Transformer Encoder layer. The network contains 3 Transformer Encoder layers with the same structure.

[0086] S3-3, input the output of each layer of the Transformer Encoder stack in S3-2 to the skip knowledge readout graph pooling module. The "skip" in skip knowledge refers to the graph node representation X learned by each layer of Transformer Encoder i It is not only input to the next layer of the network, but also directly jumps to the concatenation layer in the network while it is forward propagating. When the Transformer Encoder stack has L layers, the representation information learned in each layer is concatenated to obtain the full-graph node representation H. The specific formula is:

[0087]

[0088] Then the Readout readout method is used to obtain a one-dimensional graph representation through Readout aggregation pooling. The specific formula is:

[0089]

[0090] Where n is the number of graph nodes, h i is a row in the matrix H, || represents concatenation, and max is the maximum value of each dimension in the n row vectors;

[0091] When the brain functional connectivity map is performing attention calculations, due to the structural differences between different brain regions, some brain regions may aggregate too much information from other brain regions, which will cause some useful information to be lost after multiple layers of aggregation; however, brain regions in some positions have less information aggregated in the previous layers, so deeper layers are needed to aggregate more information. The present invention adopts a skip knowledge structure, fully utilizing the attention information between different brain regions captured by each layer of TransformerEncoder, and fusing node representations of different layers to obtain multi-scale information.

[0092] S3-4, through the fully connected layer as a classifier, the graph representation vector obtained in S3-3 is converted into a two-dimensional vector output. The two dimensions represent the two categories of major depressive disorder MDD (Major Depressive Disorder) and health control HC (Health Control). The true label of the sample is a one-hot two-dimensional vector, the dimension corresponding to the category is 1, and the remaining dimensions are 0. The cross entropy is used as the loss function. Finally, the loss function Loss of the entire jump knowledge graph core Transformer network is:

[0093]

[0094] Among them, N is the number of samples, label i is the label vector of sample i, Predict i is the prediction result vector of sample i, α controls the regularization strength, ω is all the parameters of the network, and the calculation formula of the cross entropy loss function H is:

[0095]

[0096] Where p represents the true distribution, q represents the predicted distribution, and the cross entropy loss function H is used to measure the similarity between p and q.

[0097] The label confusion matrix is ​​shown in Table 1. The method of the present invention uses three indicators, accuracy, sensitivity, and specificity, to comprehensively measure the effectiveness of the method. The calculation formulas of the three indicators are:

[0098] Accuracy indicates the proportion of correct predictions among all samples predicted to be Parkinson's disease:

[0099]

[0100] Sensitivity represents the proportion of people with Parkinson's disease who are correctly diagnosed:

[0101]

[0102] Specificity represents the proportion of healthy people who are not misdiagnosed as having Parkinson's disease:

[0103]

[0104]

[0105] Table 1

[0106] The method of the present invention was compared with experiments on its own data set. The compared methods included classic basic methods SVM (support vector machine), GCN (graph convolutional network), GIN (graph isomorphism network), GraphSAGE (graph sampling and aggregation network), FGDN (functional graph discriminant network), BrainGNN (brain graph neural network), and a pure Transformer model that directly uses the features of brain regions as input.

[0107] The advantages of the present invention are:

[0108] 1) The information exchange between graph nodes is expanded to the whole graph using the self-attention mechanism, and the node features of the brain functional connection graph of Parkinson's disease patients are fully learned. The topological features of the graph are fully learned through the position encoding based on graph representation and the attention mechanism based on graph kernel, so as to improve the diagnostic accuracy.

[0109] 2) Graph pooling based on skip knowledge connection is used to fuse multi-scale node representations of different Transformer Encoder layers, solving the problems of over-smoothing and insufficient aggregation of some node information.

[0110] 3) The highly discriminatory brain regions for diagnosing Parkinson's disease discovered in the present invention are highly correlated with previous research results on brain function of Parkinson's disease, providing a new strategy for finding imaging markers for diagnosing Parkinson's disease.

[0111] The above disclosures are only several specific embodiments of the present invention, but the present invention is not limited thereto, and any changes that can be conceived by those skilled in the art should fall within the protection scope of the present invention.

Claims

1. A Parkinson's auxiliary judgment model based on image attention, characterized in that: It includes the following steps: S1-1. First, perform resting-state functional magnetic resonance imaging on an individual and preprocess the original image data; S1-2. Register the original resting-state functional magnetic resonance image data of the individual from the original space to the standard template space; S1-3. According to the automated anatomical labeling (AAL) brain atlas partition template, divide the resting-state functional magnetic resonance image into 90 brain regions, average the blood oxygenation level-dependent signals of the voxels contained in each brain partition, and obtain the average time series of each brain region; S1-4. Calculate the Pearson correlation coefficient matrix P between brain regions, with a size of 90×90, so as to obtain the brain functional connection strength between different brain regions; S1-5. Take the brain regions as graph nodes, assign the top 20% of the large values of the brain functional connection strength matrix P as 1 and the rest as 0, quantify it into the adjacency matrix A, use the connection strength between the brain region and other brain regions as the node feature, establish the brain functional connection graph, and obtain two results of Parkinson's patients and healthy people through the brain functional connection graph.

2. The Parkinson's auxiliary judgment model based on image attention according to claim 1 is characterized in that: Generate graph node position codes and graph kernels, perform eigendecomposition on the Laplacian matrix L of the brain function connection graph and construct the Laplacian position code PE Lap .

3. The Parkinson's auxiliary judgment model based on image attention according to claim 2 is characterized in that: Calculate the adjacency matrix graph kernel K A , adjacency matrix graph kernel K A Describes direct first-order connection relationships. Values ​​are assigned to relationships that are mutually connected, and values ​​are 0 for relationships that are not mutually connected.

4. The Parkinson's auxiliary judgment model based on image attention according to claim 3 is characterized in that: Through the graph attention network, classify the brain functional connection graph, and construct the graph node combined embedding including the graph node position encoding and the graph node feature as the network input; input the graph node combined embedding into the main part of the graph attention network, the graph attention stack, which has multiple layers, and each layer has the same structure, and both include an attention sublayer based on the graph kernel and a forward propagation sublayer.

5. The Parkinson's auxiliary judgment model based on image attention according to claim 4 is characterized in that: Input the outputs of each layer of the graph attention stack into the jump knowledge readout graph pooling module. The graph attention module stack has multiple layers. After splicing the representation information learned by each layer, the full graph node representation H is obtained.

6. The Parkinson's auxiliary judgment model based on image attention according to claim 5 is characterized in that: Convert the obtained graph representation vector into a two-dimensional vector for output. The two dimensions respectively represent the two categories of Parkinson's disease and normal control. The true label of the sample is a one-hot two-dimensional vector, with the dimension corresponding to its category being 1 and the other dimensions being 0. Use cross-entropy as the loss function to obtain the loss function LOSS of the entire graph attention network.

7. The Parkinson's auxiliary judgment model based on image attention according to claim 6 is characterized in that: The formula after the eigenvalue decomposition of the Laplacian matrix L is: In the above formula, A is the adjacency matrix, D is the degree matrix, L is the Laplacian matrix of the brain functional connection graph, U represents the eigenvector matrix of L, and λ represents the eigenvalue diagonal matrix; Construct Laplace position encoding PE Lap The formula is: INSTEAD Lap =ψ1(U[:k]) In the above formula, ψ1 represents the learnable encoder, and k represents the eigenvector U corresponding to the first k (k < N) smallest eigenvalues.

8. The Parkinson's auxiliary judgment model based on image attention according to claim 7 is characterized in that: Calculate the adjacency matrix graph kernel K A The formula is:

9. The Parkinson's auxiliary judgment model based on image attention according to claim 8 is characterized in that: Among them, Attention sub-layer formula MultiHead(X,X,K A )for: In the above formula, n is the number of nodes in the graph, X is the input of this layer, and the first layer is the node embedding, K A is the adjacency matrix graph kernel, || represents concatenation, H is the number of attention heads, and W O is the learnable projection matrix, head h The calculation formula is: In the above formula, is a learnable projection matrix, and the calculation formula of PosAttention(·) is: In the above formula, Q is the query matrix and V is the value matrix.

10. The Parkinson's auxiliary judgment model based on image attention according to claim 9 is characterized in that: The specific formula for the full graph node representation H is: Subsequently, adopt the Readout readout method to obtain a one-dimensional graph representation through Readout aggregation pooling. The specific formula is: Where n is the number of graph nodes, h i is a row in the matrix H, || represents concatenation, and max is the maximum value of each dimension in the n row vectors.

Images