A brain image classification method, a classification device, an apparatus and a storage medium

Abstract

The application relates to the technical field of image processing, in particular to a brain image classification method, a brain image classification device, equipment and a storage medium. A self-attention network is applied to a plurality of features of a detected brain image to calculate the correlation degree between any two features; the plurality of features are corrected according to the correlation degree to obtain a plurality of features after correction; a graph convolution network is applied to the plurality of features after correction to obtain a classification result of the detected brain image output by the graph convolution network. The application inputs the plurality of features into the self-attention network, the self-attention network outputs the correlation degree between the plurality of features, then the plurality of features are corrected according to the correlation degree between the plurality of features, finally the plurality of features after correction are input into the graph convolution network, and the graph convolution network outputs the classification result. The application fully considers the correlation degree between the features, so that the classification result of the application for the brain image is relatively accurate.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, specifically to a brain image classification method, classification device, equipment, and storage medium. Background Technology

[0002] Brain diseases, such as Alzheimer's disease (AD) and autism spectrum disorder (ASD), are often considered a public health challenge, affecting populations from children to the elderly. Accurate identification of brain diseases is often key to implementing early intervention. Intelligent identification methods based on neuroimaging and deep learning are commonly used approaches. The identification process typically involves two parts: constructing discriminative neuroimaging biomarkers and researching the identifier to complete the disease identification. However, complex neuroimaging features and insufficient sample size are the main reasons limiting diagnostic performance.

[0003] Resting-state functional magnetic resonance imaging (rs-fMRI) is currently one of the main imaging tools. Based on the measured oxygen-dependent (BOLD) signals of subjects, functional connectivity networks (FCNs) are constructed to describe the functional interactions between brain regions. FCN features have been extensively studied as neuroimaging biomarkers for the diagnosis of brain diseases. Most FCN studies rely on Pearson correlation coefficient (PC) and sparse representation (SR). PC describes the pairwise relationship between two regions of interest (ROIs), while SR describes the relationship between multiple ROIs. The good performance of the SR series methods indicates that there are synergistic relationships between the activities of multiple brain regions. However, complex synergistic mechanisms make it difficult to construct sufficiently effective FCNs, so the obtained FCN features are often insignificant, which is considered one of the main factors limiting diagnostic performance.

[0004] Graph Convolutional Networks (GCNs) significantly improve the performance of classifiers (used for image classification) by establishing connections between individuals and groups. GCNs and their variants have been widely applied in both supervised and unsupervised learning. A graph consists of nodes and edges, where nodes represent acquired features and edges represent relationships between nodes. Edge weights, as convolution coefficients of filters, play a crucial role in refining the subject's features. Early GCN methods relied solely on imaging features to calculate edge weights. However, existing graph convolutional networks ignore the collaborative relationships (degree of association) between features, classifying brain images based only on their individual features, thus reducing the accuracy of classification results.

[0005] In summary, existing technologies have poor accuracy in classifying brain images.

[0006] Therefore, existing technologies still need to be improved and enhanced. Summary of the Invention

[0007] To address the aforementioned technical problems, this invention provides a brain image classification method, classification device, equipment, and storage medium, which solves the problem of poor accuracy in brain image classification results in existing technologies.

[0008] To achieve the above objectives, the present invention adopts the following technical solution:

[0009] In a first aspect, the present invention provides a brain image classification method, comprising:

[0010] A self-attention network is applied to several features of the brain image being detected to calculate the degree of correlation between any two of the features;

[0011] Based on the degree of correlation, several features are modified to obtain the modified features.

[0012] A graph convolutional network is applied to the modified features to obtain the classification result of the detected brain image output by the graph convolutional network.

[0013] In one implementation, the plurality of features are preprocessed features, and the preprocessing methods for the plurality of features include:

[0014] Extract several original features from the brain image being detected;

[0015] Based on the non-image information of the detected brain image, the original features are divided into original sensitive features and original insensitive features. Each original sensitive feature is a feature that is sensitive to the non-image information, and the original insensitive features are features other than the original sensitive features among the original features.

[0016] Based on the group sensitive features corresponding to each of the original sensitive features, the original sensitive features are adjusted to obtain the preprocessed sensitive features;

[0017] Based on the group insensitive features corresponding to the original insensitive features, the original insensitive features are adjusted to obtain the preprocessed insensitive features.

[0018] In one implementation, based on the non-image information of the detected brain image, the plurality of original features are divided into original sensitive features and original insensitive features. Each original sensitive feature is a feature sensitive to the non-image information, and each original insensitive feature is a feature other than the original sensitive features among the plurality of original features, including:

[0019] Calculate the Euclidean distance between several original features of the detected brain images corresponding to any two of the non-image information;

[0020] Based on the Euclidean distance, the original features are divided into original sensitive features and original insensitive features.

[0021] In one implementation, the sensitive features include site-sensitive features, device-sensitive features, and gender-sensitive features. The site-sensitive features are features associated with a site, where the site is the database containing the detected brain image. The gender-sensitive features are features associated with the gender of the user corresponding to the detected brain image. The device-sensitive features are features associated with the device that acquired the detected brain image.

[0022] In one implementation, both the self-attention network and the graph convolutional network are trained networks, and the training methods for the networks include:

[0023] Based on the similarity between the features of each sample image, the non-image information of each sample image, and the sample classification label corresponding to each sample image, the weight of any two sample images is calculated. The sample classification label is used to characterize the sample classification result corresponding to the sample image.

[0024] The self-attention network and the graph convolutional network are trained based on the weights of any two sample images and the features of each sample image.

[0025] In one implementation, training the self-attention network and the graph convolutional network based on the weights of any two sample images and the features of each sample image includes:

[0026] Based on the weights of any two sample images and each of the sample features of each sample image, each of the sample features of each sample image is corrected to obtain the sample features of each sample image after the first correction.

[0027] The self-attention network is applied to each of the sample features after the first correction of each of the sample images to obtain the degree of correlation between any two sample features output by the self-attention network.

[0028] The sample features after the first correction are corrected according to the degree of correlation between any two features in each sample feature, so as to obtain the sample features after the second correction.

[0029] Apply a graph convolutional network to the sample features after the second correction to obtain the training classification result output by the graph convolutional network;

[0030] Based on the training classification results and the sample classification labels, the parameters of the self-attention network and the graph convolutional network are adjusted to complete the training of the self-attention network and the graph convolutional network.

[0031] In one implementation, adjusting the parameters of the self-attention network and the graph convolutional network based on the training classification results and the sample classification labels to complete the training of the self-attention network and the graph convolutional network includes:

[0032] Obtain the site distance metric between site domains, the device distance metric between device domains, and the gender distance metric between gender domains corresponding to each of the sample images. The site domains correspond to the database where each of the sample images is located, the device domains correspond to the devices that acquired each of the sample images, and the gender domains correspond to the gender of the source user of each of the sample images.

[0033] Based on the training classification results and the sample classification labels, calculate the local loss function;

[0034] Calculate the overall loss function based on the site distance metric, the device distance metric, the gender distance metric, and the local loss function;

[0035] Based on the overall loss function, the parameters of the self-attention network and the graph convolutional network are adjusted to complete the training of the self-attention network and the graph convolutional network.

[0036] Secondly, embodiments of the present invention also provide a brain image classification device, wherein the device comprises the following components:

[0037] The correlation calculation module is used to apply a self-attention network to several features of the detected brain image and calculate the correlation between any two features.

[0038] The feature modification module is used to modify several features based on the correlation degree to obtain the modified features.

[0039] The image classification module is used to apply a graph convolutional network to the corrected features to obtain the classification result of the detected brain image output by the graph convolutional network.

[0040] Thirdly, embodiments of the present invention also provide a terminal device, wherein the terminal device includes a memory, a processor, and a brain image classification program stored in the memory and executable on the processor, wherein when the processor executes the brain image classification program, it implements the steps of the brain image classification method described above.

[0041] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing a brain image classification program, wherein when the brain image classification program is executed by a processor, it implements the steps of the brain image classification method described above.

[0042] Beneficial Effects: This invention first extracts several features from the brain image being detected, then inputs these features into a self-attention network. The self-attention network outputs the correlation between these features. Next, the features are corrected based on their correlation, and finally, the corrected features are input into a graph convolutional network. The graph convolutional network then outputs the classification result corresponding to the detected image containing these features. Because this invention fully considers the correlation (cooperative relationship) between features, the classification results for brain images are highly accurate, thus facilitating doctors to implement appropriate measures based on the brain image classification results. Attached Figure Description

[0043] Figure 1 This is an overall flowchart of the present invention;

[0044] Figure 2 This is a flowchart illustrating the classification of brain diseases in an embodiment of the present invention;

[0045] Figure 3 This is a diagram of the Transformer structure in an embodiment of the present invention;

[0046] Figure 4 This is a schematic diagram comparing the sample image and the feature image in an embodiment of the present invention;

[0047] Figure 5 This is a schematic diagram illustrating the construction of the GCN adjacency matrix in an embodiment of the present invention;

[0048] Figure 6 This is a schematic diagram illustrating how the domain adaptation module alleviates data heterogeneity in an embodiment of the present invention;

[0049] Figure 7 This is a block diagram illustrating the internal structure of a terminal device provided in an embodiment of the present invention. Detailed Implementation

[0050] The technical solutions of the present invention will be clearly and completely described below with reference to the embodiments and accompanying drawings. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0051] Studies have shown that brain diseases (such as Alzheimer's disease (AD) and autism spectrum disorder (ASD)) are often considered a public health challenge, affecting populations from children to the elderly. Accurate identification of brain diseases is often key to implementing early intervention. Intelligent identification methods based on neuroimaging and deep learning are commonly used. The identification process typically involves two parts: constructing discriminative neuroimaging biomarkers and studying the identifier to complete disease identification. However, complex neuroimaging features and insufficient sample size are the main reasons limiting diagnostic performance. Resting-state functional magnetic resonance imaging (rs-fMRI) is currently one of the main imaging tools. Based on measured oxygen-dependent (BOLD) signals from subjects, functional connectivity networks (FCNs) are constructed to describe functional interactions between brain regions. FCN features are widely studied as neuroimaging biomarkers for the diagnosis of brain diseases. Most FCN studies rely on Pearson correlation coefficient (PC) and sparse representation (SR). PC describes pairwise relationships between two regions of interest (ROIs), while SR describes relationships between multiple ROIs. The better performance of the SR series of methods indicates synergistic relationships between the activities of multiple brain regions. However, complex collaborative mechanisms make it difficult to establish sufficiently effective FCNs, resulting in FCN features that are often insignificant, which is considered one of the main factors limiting diagnostic performance. Graph Convolutional Networks (GCNs) significantly improve recognizer performance by establishing connections between individuals and groups. GCNs and their variants have been widely applied in supervised and unsupervised learning. A graph consists of nodes and edges, where nodes represent acquired features and edges represent relationships between nodes. Edge weights, as convolution coefficients of filters, play a crucial role in correcting subject features. Early GCN methods relied solely on imaging features to calculate edge weights. However, existing graph convolutional networks ignore the collaborative relationships (degree of association) between features, classifying brain images based only on individual features, thus reducing the accuracy of classification results.

[0052] To address the aforementioned technical problems, this invention provides a brain image classification method, device, equipment, and storage medium, resolving the issue of poor accuracy in existing brain image classification methods. Specifically, firstly, a self-attention network is applied to several features of the brain image to be detected, calculating the correlation between any two features; then, based on the correlation, several features are corrected to obtain corrected features; finally, a graph convolutional network is applied to the corrected features to obtain the output of the graph convolutional network for the brain image being detected. This embodiment improves the accuracy of the classification results.

[0053] For example, brain images A (for subject A) and B (for subject B) are collected. Features a, b, and c (e.g., the mean square error of grayscale values, brightness values, etc.) are extracted from brain images A and B respectively. Features a, b, and c of brain image A are input into a self-attention network. The network calculates the correlation between each pair of features a, b, and c in brain image A. For instance, if the correlation between features a and b is high, and the correlation between features b and c is low, as is the correlation between features a and c; or if this correlation implies that if feature b is high, then feature a should also be high. However, if feature b in brain image A is high, but feature a is low, it indicates an error in the value of feature a extracted from brain image A. Therefore, feature a needs to be adjusted based on the correlation between the two features to increase its value. Finally, features b, c, and the modified feature a are input into the graph convolutional network, which then outputs the classification result for the detected brain image A. The same operation is performed on the detected brain image B.

[0054] Exemplary methods

[0055] The brain image classification method of this embodiment can be applied to terminal devices, which can be terminal products with image acquisition capabilities, such as image classification devices. In this embodiment, for example... Figure 1 As shown, the brain image classification method specifically includes the following steps:

[0056] S100 trains a graph converter consisting of a self-attention network and a graph convolutional network.

[0057] In this embodiment, the output of the self-attention network is used as the output of the graph convolutional network (GCN), and both the self-attention network and the GCN are trained simultaneously. Step S100 includes the following steps S101 to S109:

[0058] S101, based on the similarity between the various sample features of each sample image, the non-image information of each sample image, and the sample classification label corresponding to each sample image, calculate the weight of any two sample images, whereby the sample classification label is used to characterize the sample classification result corresponding to the sample image.

[0059] In this embodiment, the sample images include brain images of patients with early-stage Alzheimer's disease (AD) and patients with autism spectrum disorder (ASD). As shown in Table 1, rs-fMRI imaging data and non-imaging information (multi-site data source, device type, gender) were collected from 415 subjects (mild cognitive impairment (MCI) and normal control (NC)). This dataset includes 163 NCs, 86 early-stage MCIs (EMCI), and 166 late-stage MCIs (LMCI).

[0060] As shown in Table 2, brain images of 356 patients with ASD and NC were collected, including rs-fMRI imaging data and non-imaging information (i.e., multi-site data sources and gender). In Table 2, NYU, UCLA_1, UM_1, and Yale are databases (sites).

[0061] Table 1

[0062]

[0063] Table 2

[0064] Center NYU UM_1 UCLA_1 Yale Male / female 136 / 35 59 / 23 49 / 6 34 / 14 Age 15.35±6.59 13.89±2.88 13.57±2.34 12.87±2.93 ASD / NC 73 / 98 36 / 46 28 / 27 22 / 26 Devices (model) SIEMENS GE SIEMENS SIEMENS <![CDATA[Voxelsize(mm 3 )]]> 3.0×3.0×4.0 3.438×3.438×3.0 3.0×3.0×4.0 3.4×3.4×4.0 Flipangle(deg) 90 90 90 60 TR(ms) 2000 2000 3000 2000 TE(ms) 15 30 28 25 Bandwidth (Hz / Px) 3906 NA 2442 2520

[0065] In this embodiment, the sample features are as follows: Figure 2 The features shown include eight types: site-sensitive features, gender-sensitive features, device-sensitive features, site-gender-sensitive features, site-device-sensitive features (features affected by both site and device), gender-device-sensitive features, site-gender-device-sensitive features (features affected by site, gender, and device, where gender is the gender of the subject corresponding to the sample image, and device is the device that acquired the sample image), and insensitive features. Among these, site, device, and gender are all non-image information.

[0066] Calculate the adjacency matrix (weights) of site sensitivity features.

[0067] A s (i,j)=sim(x i ,x j )×(1+r s (s i ,s j )+r l (l i ,l j ))

[0068] Calculate the adjacency matrix of gender-sensitive features

[0069] A g(i,j)=sim(x i ,x j )×(1+r g (g i ,g j )+r l (l i ,l j ))

[0070] Adjacency matrix of computing device-sensitive features

[0071] A e (i,j)=sim(x i ,x j )×(1+r e (e i ,e j )+rl(l i ,l j ))

[0072] Calculate the adjacency matrix of gender-sensitive features of the site

[0073] A s_g (i,j)=sim(x i ,x j )×(1+r s (s i ,s j )+r g (g i ,g j )+r l (l i ,l j ))

[0074] Adjacency matrix of site device sensitive features

[0075] A s_e (i,j)=sim(x i ,x j )×(1+r s (s i ,s j )+r e (e i ,e j )+r l (l i ,l j ))

[0076] Calculate the adjacency matrix of gender-sensitive device features

[0077] A g_e(i, j) = sim(x) i x j )×(1+r g (g i g j )+r e (e i e j )+r l (l i , l j ))

[0078] Adjacency matrix of gender-sensitive device features at computing sites

[0079] A s_g_e (i, j) = sim(x) i x j )×(1+r s (s i s j )+r g (g i g j )+r e (e i e j )+r l (l i , l j ))

[0080] Calculate the adjacency matrix of insensitive features

[0081] A in (i, j) = sim(x) i x j )×(1+r l (l i , l j ))

[0082] In the above formulas, A(i,j) represents the edge weight between subject i (sample image i) and j (sample image j), sim(·) is the feature similarity function (the similarity function between sample image i and sample image j), and r s (s i s j ) indicates site s i With site s j The distance between them, r g Represents gender distance, r e Let r1 represent the device type distance and r2 represent the label distance (the labels of these training samples are known). For topic i, define x... i s i g i ei and l i Let x represent the feature vector of sample image i, the site source, gender, device type, and label (presence or absence of disease), respectively. j s j g j e j and l j Let r represent the feature vector of sample image j, site origin, gender, device type, and label (presence or absence of disease), respectively. ρ(·) is the correlation distance function, and σ is the kernel width. s r g r e r l The definition is as follows:

[0083]

[0084]

[0085]

[0086]

[0087] g i =g j This indicates that the gender of the subject in sample image i is the same as the gender of the subject in sample image j.

[0088] For example, the adjacency matrix of the site sensitive features calculated in this embodiment. This is used to represent the degree of correlation between site-sensitive features in sample image i and site-sensitive features in sample image j, that is, to calculate the degree of correlation between the site-sensitive features of group I corresponding to sample image i and the site-sensitive features of group J corresponding to sample image j.

[0089] Adjacency matrix A of gender-sensitive features g This is used to represent the degree of correlation between gender-sensitive features in sample image i and gender-sensitive features in sample image j, that is, to calculate the degree of correlation between the gender-sensitive features of group I corresponding to sample image i and the gender-sensitive features of group J corresponding to sample image j.

[0090] S102, based on the weights of any two sample images and each of the sample features of each sample image, correct each of the sample features of each sample image to obtain each of the sample features after the first correction.

[0091] For example, the sensitive feature of the site after the first correction is A. s X s The gender-sensitive characteristic after the first revision is A.g X g The sensitive feature of the device after the first correction is A. e X e Site gender sensitivity feature A after the first revision s_g X s_g Sensitive features of site equipment after the first correction A s_e x s_e The first revised gender-sensitive feature of the device A g_e x g_e The first revised site gender-sensitive device feature A s_g_e X s_g_e Insensitive feature A after the first correction in X in X s X is a matrix composed of site-sensitive features (uncorrected site-sensitive features) from each sample image. g Let x be a matrix composed of gender-sensitive features (uncorrected gender-sensitive features) in each sample image. e X is a matrix composed of device-sensitive features (uncorrected device-sensitive features) in each sample image. s_g Let x be a matrix consisting of the site gender-sensitive features (uncorrected site gender-sensitive features) in each sample image. g_e X is a matrix composed of gender-sensitive device features (uncorrected gender-sensitive device features) in each sample image. s_g_e Let x be a matrix consisting of the site gender device-sensitive features (uncorrected site gender device-sensitive features) in each sample image. in It is a matrix composed of insensitive features (uncorrected insensitive features) in each sample image.

[0092] A s X s A g X g A e X e A s_g X s_g A s_e X s_e A g_e X g_e A s_g_e x s_g_e A in X in Arranging them into a matrix yields a new matrix composed of the features of each sample after the first correction.

[0093]

[0094] S103, apply the self-attention network to each of the sample features after the first correction of each of the sample images to obtain the degree of correlation between any two sample features output by the self-attention network.

[0095] S104, based on the degree of correlation between any two features in each of the sample features, the sample features after the first correction are corrected to obtain the sample features after the second correction.

[0096] The above eight adjacency matrices constitute Figure 2 The eight subgraphs in the graph capture the interactions between individuals and the population (e.g., the relationship between feature a in image i and feature a in image j), and feature refinement is performed using convolutional filters. Since there are collaborative relationships between brain regions during brain activity, this embodiment designs a suitable Transformer structure to capture the relationships between features. The proposed graph transformer structure will fuse the... Figure 2 The subgraph in the graph is used as input, and a self-attention mechanism is used to capture the relationships between features. For example... Figure 3 As shown, in reorganizing a new feature matrix Then, use the transpose function to... Mapping to a new space (called the feature space) and forming a new matrix. Then, a self-attention module is used to capture the cooperative relationships between features and to refine them. Finally, the transpose function is used to remap the refined feature matrix back to the original space (called the sample space) to form the feature matrix. (The sample features after the second correction). The self-attention mechanism is partly as follows:

[0097]

[0098] It is the output of three feedforward networks, each network using As input. It is a scaling factor. The self-attention mechanism has an architecture similar to GCN, where learnable and trainable weights... It can also be viewed as an adjacency matrix. The multiplication of the two matrices (i.e., Q and KT) captures the synergistic relationships between features (the degree of association between features of two samples). For example... Figure 4 As shown, the sample map comparing the interaction relationships between subjects with the feature map capturing the collaborative relationships between features.

[0099] S105, Apply a graph convolutional network to the sample features after the second correction to obtain the training classification result output by the graph convolutional network.

[0100] The feature matrix is ​​composed of the sample features after the second correction of each sample image. The input is fed into a graph convolutional network, and the training classification results output by the graph convolutional network are used to indicate whether the subject corresponding to each sample image is diseased.

[0101] S106, Obtain the site distance metric between the site domains corresponding to each of the sample images. Device distance metric between device domains Gender distance measurement between gender domains The site domain corresponds to the database where each of the sample images is located, the device domain corresponds to the device that acquired each of the sample images, and the gender domain corresponds to the gender of the user from whom each of the sample images originated.

[0102] This indicates the magnitude of the differences in feature values ​​between different site domain samples. This indicates the magnitude of the difference in feature values ​​between samples from different device domains. This represents the magnitude of the difference in feature values ​​between samples from different gender domains. All three loss terms are obtained by calculating and summing the differences between all features.

[0103] Adopting such Figure 6 The calculation method shown calculates by Let me illustrate the calculation process with an example:

[0104] For each site, the average of all features is calculated. If there are S sites, there are S feature domains. The difference between these S feature domains is evaluated (the difference between features in every two domains is calculated and summed), resulting in...

[0105] S107, Calculate the local loss function based on the training classification results and the sample classification labels.

[0106] S108, based on the station distance metric The device distance measurement The gender distance metric and the local loss function Calculate the overall loss function

[0107]

[0108] α, β, and γ are hyperparameters used to control the contributions of the three distance terms.

[0109] In the overall loss function Add site distance metric Device distance measurement The gender distance metric and local loss function It can alleviate the heterogeneity of multi-site data (including heterogeneity caused by device type and gender), improve the generalization ability of the entire diagnostic system, and proposes a domain-adaptive GCN layer to output the final classification result.

[0110] S109, Based on the overall loss function, adjust the parameters of the self-attention network and the graph convolutional network to complete the training of the self-attention network and the graph convolutional network.

[0111] In this embodiment, the self-attention network has a GCN with a structure similar to the graph convolutional network (GCN). Training the graph converter, which consists of the self-attention network and the graph convolutional network, involves adjusting the GCN parameters in the self-attention network and the GCN parameters in the graph convolutional network. The parameters of both networks are a0 and a0. s a g a e The purpose of training is to continuously adjust these five parameters, a1 (the five weight coefficients obtained from network training), until the overall loss function is less than the threshold, thus completing the training of these two networks.

[0112] This embodiment processes the above five parameters based on the following principles:

[0113] Assume r s r represents the distance between the source and the station. g Represents the gender distance (value is 1 or 0), r e Indicates the distance by device type, r l This represents the label distance (the labels of the training samples are known). For subject i, x i g i e i s i l i Let x represent the feature vector, gender, device type, site source, and tag, respectively; for subject j, x j g j e j s j l j These represent the feature vector, gender, device type, site origin, and tag, respectively. The weights of the corresponding edges in GCN are calculated as follows:

[0114] A(i,j)=sim(x) i x j )×(a0+a s ×r s (s i s j)+a g ×r g (g i g j )+a e ×

[0115] r e (e i e j )+a l ×r l (l i , l j ))

[0116] sta0+a s +a g +a e +a l =1, a0, a s a g a e a l ∈(0,1).

[0117] a0、a s a g a e a l These are trainable attention parameters. The specific implementation mechanism is as follows: First, construct the similarity matrix Sim, the identity matrix I, and the site distance matrix A. s Gender distance matrix A g Device distance matrix A e Label distance matrix A l . Figure 5 The matrix A(i,j) in the matrix can also be represented as: A=Sim⊙(a0×I+a s ×A s +a g ×A g +a e ×A e +a l ×A l ). ⊙Hadamard product. The above matrix (I, A) s A g A e , and A l Using Conv2d(5, 1, 1, 1) for weighted combination, these five coefficients (a0, a...) s a g a e , and a l These correspond to 5 trainable network coefficients (network parameters). The softmax(·) function is then used to process these adaptive coefficients.

[0118] S200, apply a self-attention network to several features of the detected brain image to calculate the degree of correlation between any two features.

[0119] After step S100, the training of the self-attention network and the graph convolutional network is complete. These two trained networks can then be directly used for image classification. In this embodiment, several features are preprocessed features. Preprocessing involves comparing several features extracted from a brain image to a standard range value for each feature. If one or more features in the brain image deviate from the standard range value, that feature is adjusted. Preprocessing includes the following steps:

[0120] S201, Extract several original features from the detected brain image.

[0121] S202, calculate the Euclidean distance between several original features of the detected brain images corresponding to any two of the non-image information.

[0122] S203, based on the Euclidean distance, divide the original features into original sensitive features and original insensitive features.

[0123] In one embodiment, the sensitive feature is a site-sensitive feature (such as a brightness value in a brain image; this feature may deviate from the values ​​in brain images from different sites, thus making it a site-sensitive feature). The following formula is used to determine whether an original feature is a site-sensitive feature:

[0124]

[0125] This represents the feature vector of the detected brain image i from site s1. Let represent the feature vector of the detected brain image j from site s2. Euclidean distance is a very simple and effective distance metric.

[0126] For data from multiple sites, this embodiment assesses the differences between each pair of sites and then uses the average value to represent the final impact. s The larger the value, the greater the difference in the corresponding features of the information across different sites.

[0127] S204, Based on the group sensitive features corresponding to each of the original sensitive features, adjust the original sensitive features to obtain the preprocessed sensitive features.

[0128] For example, if the value of sensitive feature A in the detected image is a, while the value of sensitive feature A (population sensitive feature) in most brain images is b, and b is much greater than a, it indicates that the value of sensitive feature A is abnormal, and the value of sensitive feature A in the detected image a needs to be adjusted to be close to b.

[0129] S205. Adjust the original non-sensitive feature according to the population non-sensitive feature corresponding to the original non-sensitive feature, and obtain the preprocessed non-sensitive feature.

[0130] For example, there are two detected images A and B from site s1, and two detected images C and D from site s2. Each detected image has three original features A, B, and C. The three original features of detected image A form a matrix [A 甲 , A 乙 , A 丙 , the three original features of detected image B form a matrix [B 甲 , B 乙 , B 丙 , the three original features of detected image C form a matrix [C 甲 , C 乙 , C 丙 , and the three original features of detected image D form a matrix [D 甲 , D 乙 , D 丙 . The

[0131] If the second element in d s is greater than the threshold, then the original feature B is a sensitive feature.

[0132] After preprocessing the features through steps S201 to S205, the preprocessed features are then input into the self-attention network, and the correlation degree S300 between several features in the same detected image is calculated. According to the correlation degree, several of the features are corrected to obtain several corrected features.

[0133] S400. Apply the graph convolutional network to several of the corrected features to obtain the classification result of the detected brain image output by the graph convolutional network.

[0134] The graph convolutional network outputs "1" indicating that the user corresponding to the detected brain image has the disease, otherwise the user does not have the disease.

[0135] In summary, this invention first extracts several features from the brain image being detected, then inputs these features into a self-attention network. The self-attention network outputs the degree of correlation between these features. Next, the features are corrected based on this correlation, and finally, the corrected features are input into a graph convolutional network. The graph convolutional network then outputs the classification result corresponding to the detected image containing these features. Because this invention fully considers the degree of correlation (cooperative relationship) between features, the classification results for brain images are highly accurate, thus facilitating doctors to implement appropriate measures based on the brain image classification results.

[0136] Furthermore, the graph transformer of this invention captures the relationships between FCN features, which can alleviate the heterogeneity of multi-site data (including heterogeneity caused by device type and gender), improve the generalization ability of the entire recognition system, and proposes a domain-adaptive GCN layer to output the final classification result. Experimental results show that the knowledge-aware multi-site graph transformer can achieve significant performance in the diagnosis of mild cognitive impairment (MCI) and autism spectrum disorder (ASD).

[0137] First, the sensitivity of features to each non-image information is evaluated. Then, feature-sensitive and feature-insensitive subgraphs are constructed based on the different feature sensitivities. Second, after fusing these subgraphs, a Transformer module is integrated to capture the collaborative relationships between features. Finally, a domain-adaptive GCN is designed to output the final classification result. Its domain adaptation module utilizes multiple loss function terms to mitigate data heterogeneity and improve generalization ability. Finally, the effectiveness of the proposed framework is validated on two diagnostic tasks using a private dataset and two public datasets (obtained from ABIDE and ADNI).

[0138] Exemplary device

[0139] This embodiment also provides a brain image classification device, which includes the following components:

[0140] The correlation calculation module is used to apply a self-attention network to several features of the detected brain image and calculate the correlation between any two features.

[0141] The feature modification module is used to modify several features based on the correlation degree to obtain the modified features.

[0142] The image classification module is used to apply a graph convolutional network to the corrected features to obtain the classification result of the detected brain image output by the graph convolutional network.

[0143] Based on the above embodiments, the present invention also provides a terminal device, the principle block diagram of which can be as follows: Figure 7 As shown, the terminal device includes a processor, memory, network interface, display screen, and temperature sensor connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements a brain image classification method. The display screen can be a liquid crystal display (LCD) or an e-ink display. The temperature sensor is pre-installed inside the terminal device to detect the operating temperature of the internal components.

[0144] Those skilled in the art will understand that Figure 7 The block diagram shown is merely a partial structural diagram related to the present invention and does not constitute a limitation on the terminal device to which the present invention is applied. The specific terminal device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0145] In one embodiment, a terminal device is provided, comprising a memory, a processor, and a brain image classification program stored in the memory and executable on the processor. When the processor executes the brain image classification program, it implements the following operation instructions:

[0146] A self-attention network is applied to several features of the brain image being detected to calculate the degree of correlation between any two of the features;

[0147] Based on the degree of correlation, several features are modified to obtain the modified features.

[0148] A graph convolutional network is applied to the modified features to obtain the classification result of the detected brain image output by the graph convolutional network.

[0149] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided by this invention can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0150] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A brain image classification method, characterized in that, include: A self-attention network is applied to several features of the brain image being detected to calculate the degree of correlation between any two of the features; Based on the degree of correlation, several features are modified to obtain the modified features. A graph convolutional network is applied to the modified features to obtain the classification result of the detected brain image output by the graph convolutional network. Both the self-attention network and the graph convolutional network are trained networks, and the training methods for the networks include: Based on the similarity between the features of each sample image, the non-image information of each sample image, and the sample classification label corresponding to each sample image, the weight of any two sample images is calculated. The sample classification label is used to characterize the sample classification result corresponding to the sample image. The self-attention network and the graph convolutional network are trained based on the weights of any two sample images and the features of each sample image. The step of training the self-attention network and the graph convolutional network based on the weights of any two sample images and the features of each sample image includes: Based on the weights of any two sample images and each of the sample features of each sample image, each of the sample features of each sample image is corrected to obtain the sample features of each sample image after the first correction. The self-attention network is applied to each of the sample features after the first correction of each of the sample images to obtain the degree of correlation between any two sample features output by the self-attention network. The sample features after the first correction are corrected according to the degree of correlation between any two features in each sample feature, so as to obtain the sample features after the second correction. Apply a graph convolutional network to the sample features after the second correction to obtain the training classification result output by the graph convolutional network; Based on the training classification results and the sample classification labels, the parameters of the self-attention network and the graph convolutional network are adjusted to complete the training of the self-attention network and the graph convolutional network.

2. The brain image classification method as described in claim 1, characterized in that, The aforementioned features are preprocessed features, and the preprocessing methods for the aforementioned features include: Extract several original features from the brain image being detected; Based on the non-image information of the detected brain image, the original features are divided into original sensitive features and original insensitive features. Each original sensitive feature is a feature that is sensitive to the non-image information, and the original insensitive features are features other than the original sensitive features among the original features. Based on the group sensitive features corresponding to each of the original sensitive features, the original sensitive features are adjusted to obtain the preprocessed sensitive features; Based on the group insensitive features corresponding to the original insensitive features, the original insensitive features are adjusted to obtain the preprocessed insensitive features.

3. The brain image classification method as described in claim 2, characterized in that, Based on the non-image information of the detected brain image, several original features are divided into original sensitive features and original insensitive features. Each original sensitive feature is a feature sensitive to the non-image information, and each original insensitive feature is a feature other than the original sensitive features among the several original features, including: Calculate the Euclidean distance between several original features of the detected brain images corresponding to any two of the non-image information; Based on the Euclidean distance, the original features are divided into original sensitive features and original insensitive features.

4. The brain image classification method as described in claim 3, characterized in that, The sensitive features include site-sensitive features, device-sensitive features, and gender-sensitive features. The site-sensitive features are features associated with a site, where the site is the database where the detected brain image is located. The gender-sensitive features are features associated with the gender of the user corresponding to the detected brain image. The device-sensitive features are features associated with the device that acquired the detected brain image.

5. The brain image classification method as described in claim 1, characterized in that, The step of adjusting the parameters of the self-attention network and the graph convolutional network based on the training classification results and the sample classification labels to complete the training of the self-attention network and the graph convolutional network includes: Obtain the site distance metric between site domains, the device distance metric between device domains, and the gender distance metric between gender domains corresponding to each of the sample images. The site domains correspond to the database where each of the sample images is located, the device domains correspond to the devices that acquired each of the sample images, and the gender domains correspond to the gender of the source user of each of the sample images. Based on the training classification results and the sample classification labels, calculate the local loss function; Calculate the overall loss function based on the site distance metric, the device distance metric, the gender distance metric, and the local loss function; Based on the overall loss function, the parameters of the self-attention network and the graph convolutional network are adjusted to complete the training of the self-attention network and the graph convolutional network.

6. A brain image classification device, characterized in that, The device comprises the following components: The correlation calculation module is used to apply a self-attention network to several features of the detected brain image and calculate the correlation between any two features. The feature modification module is used to modify several features based on the correlation degree to obtain the modified features. An image classification module is used to apply a graph convolutional network to several modified features to obtain the classification result of the detected brain image output by the graph convolutional network. Both the self-attention network and the graph convolutional network are trained networks, and the training methods for the networks include: Based on the similarity between the features of each sample image, the non-image information of each sample image, and the sample classification label corresponding to each sample image, the weight of any two sample images is calculated. The sample classification label is used to characterize the sample classification result corresponding to the sample image. The self-attention network and the graph convolutional network are trained based on the weights of any two sample images and the features of each sample image. The step of training the self-attention network and the graph convolutional network based on the weights of any two sample images and the features of each sample image includes: Based on the weights of any two sample images and each of the sample features of each sample image, each of the sample features of each sample image is corrected to obtain the sample features of each sample image after the first correction. The self-attention network is applied to each of the sample features after the first correction of each of the sample images to obtain the degree of correlation between any two sample features output by the self-attention network. The sample features after the first correction are corrected according to the degree of correlation between any two features in each sample feature, so as to obtain the sample features after the second correction. Apply a graph convolutional network to the sample features after the second correction to obtain the training classification result output by the graph convolutional network; Based on the training classification results and the sample classification labels, the parameters of the self-attention network and the graph convolutional network are adjusted to complete the training of the self-attention network and the graph convolutional network.

7. A terminal device, characterized in that, The terminal device includes a memory, a processor, and a brain image classification program stored in the memory and executable on the processor. When the processor executes the brain image classification program, it implements the steps of the brain image classification method as described in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a brain image classification program, which, when executed by a processor, implements the steps of the brain image classification method as described in any one of claims 1-5.

Images