A brain disease classification model training method, device, equipment and storage medium

Abstract

The present application relates to the technical field of disease classification model training, in particular to a brain disease classification model training method, device, equipment and storage medium. The brain function connection network and the brain structure connection network complement each other, and the combination of the two can provide more human physiological information. Therefore, the brain connection network formed by the fusion of the brain function connection network and the brain structure connection network has more human physiological information. The brain disease classification model is learned and trained by the brain connection network, so that the brain disease classification model can learn more human physiological information. Therefore, the brain disease classification model after training can provide a classification result with higher accuracy for brain diseases.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of disease classification model training, in particular to a brain disease classification model training method, device and equipment and storage medium. BACKGROUND

[0002] With the unprecedented development of neuroimaging and pattern recognition field, disease diagnosis based on graph convolution learning model (brain disease classification model) is possible. Through the processing of image data by the graph convolution learning model, the type of disease can be obtained. For example, Alzheimer's disease (AD) is a common neurodegenerative disease, and the prevalence rate increases with age. AD usually has chronic or progressive nature, and has multiple high-order cortical function disorders, showing symptoms such as cognitive function decline, judgment ability decline and memory loss, and eventually leading to loss of independent living ability of patients. As an early stage of Alzheimer's disease, the annual conversion rate of Mild Cognitive Impairment (MCI) is 10%-15%, and the conversion rate within 5 years is more than 50%. And Significant Memory Concerns (SMC) is the early stage of MCI. MCI and SMC are different stages of AD, corresponding to the classification results of AD.

[0003] The existing technology only learns and trains the brain disease classification model for one kind of brain image, resulting in low accuracy of the brain disease classification model after training.

[0004] In summary, the brain disease classification model trained by the existing technology has low accuracy.

[0005] Therefore, the existing technology needs to be improved and improved. SUMMARY

[0006] To solve the above technical problems, the present application provides a brain disease classification model training method, device and equipment and storage medium, which solves the problem of low accuracy of the brain disease classification model trained by the existing technology.

[0007] To achieve the above purpose, the present application adopts the following technical scheme:

[0008] In a first aspect, the present application provides a brain disease classification model training method, which comprises:

[0009] Generating a brain function connection network according to a brain function image;

[0010] Generating a brain structure connection network according to a brain structure image;

[0011] fuse the brain functional connection network and the brain structural connection network to obtain a brain connection network;

[0012] train a brain disease classification model according to the brain connection network.

[0013] In an implementation manner, the generating the brain functional connection network according to the brain functional images comprises:

[0014] acquiring time points at which each of the brain functional images is collected;

[0015] obtaining intermediate time points and non-intermediate time points according to an order corresponding to each of the time points;

[0016] matching and correcting the brain functional image of each of the non-intermediate time points with the brain functional image of the intermediate time points to obtain the brain functional image after matching and correction;

[0017] removing shadows and noises in the brain functional image after matching and correction to obtain the brain functional image after preprocessing;

[0018] dividing the brain functional image after preprocessing into functional regions;

[0019] taking each of the functional regions as a node, connecting each node according to a connection relationship of each of the functional regions in the human brain to generate the brain functional connection network.

[0020] In an implementation manner, the generating the brain structural connection network according to the brain structural images comprises:

[0021] dividing the brain structural images into structural regions;

[0022] calculating average anisotropy fractions between each of the structural regions;

[0023] calculating surface areas between gray matters and white matters of each of the structural regions;

[0024] calculating average values of the surface areas of each of the structural regions;

[0025] dividing the average anisotropy fractions by the average values to obtain operation results;

[0026] obtaining connection strengths between each of the structural regions according to the operation results;

[0027] taking each of the structural regions as a node and combining the connection strengths to generate the brain structural connection network.

[0028] In an implementation manner, the fusing the brain functional connection network and the brain structural connection network to obtain the brain connection network comprises:

[0029] extracting parameter information of each functional area in the brain functional connectivity network;

[0030] generating a brain functional data matrix according to the parameter information of each functional area and the position of each functional area in the brain functional connectivity network;

[0031] obtaining a brain functional weighted matrix corresponding to the brain functional data matrix;

[0032] generating a strength penalty matrix according to the connection strength covered by the brain structural connectivity network;

[0033] generating a brain connectivity network according to the brain functional data matrix, the brain functional weighted matrix and the strength penalty matrix.

[0034] In an implementation manner, the generating the strength penalty matrix according to the connection strength covered by the brain structural connectivity network comprises:

[0035] calculating a square value of the connection strength;

[0036] dividing the square value by a preset penalty value to obtain a division result;

[0037] calculating a power operation result with a natural number as a base and the division result as an index;

[0038] generating the strength penalty matrix according to the position of the connection strength in the brain structural connectivity network and the power operation result.

[0039] In an implementation manner, the generating the brain connectivity network according to the brain functional data matrix, the brain functional weighted matrix and the strength penalty matrix comprises:

[0040] establishing a to-be-solved matrix corresponding to the size of the brain functional data matrix, the to-be-solved matrix being a matrix in which each element value in the matrix is unknown;

[0041] multiplying the brain functional data matrix by the to-be-solved matrix to obtain a first matrix;

[0042] subtracting the first matrix from the brain functional data matrix to obtain a second matrix;

[0043] calculating a two-norm of the second matrix;

[0044] calculating a third matrix by multiplying the strength penalty matrix by the to-be-solved matrix;

[0045] calculating a one-norm of the third matrix;

[0046] calculating a fourth matrix by multiplying the brain functional weighted matrix by the to-be-solved matrix;

[0047] calculating a two-norm of the fourth matrix;

[0048] constructing a sparse regularization term according to the two-norm, the one-norm and the two-one-norm;

[0049] calculating each element value in the to-be-solved matrix when the sparse regularization term takes a minimum value, to obtain the to-be-solved matrix after solving;

[0050] generating a brain connection network according to the to-be-solved matrix after solving.

[0051] In an implementation manner, the training of the brain disease classification model according to the brain connection network comprises:

[0052] obtaining individual connection network features of individuals constituting the brain connection network according to the brain connection network, the connection network of each individual comprising a brain functional connection network and a brain structural connection network of the individual;

[0053] calculating an adjacency matrix between each of the individuals, an element in the adjacency matrix being used to represent a difference between each of the individuals;

[0054] fusing the connection network features of the individuals and the adjacency matrix to generate a sparse graph;

[0055] applying a K-nearest neighbor algorithm to the connection network features of the individuals to generate a feature map corresponding to the brain connection network;

[0056] generating a topology map corresponding to the brain connection network according to the brain connection network;

[0057] extracting common features covered by the feature map and the topology map;

[0058] inputting the feature map, the topology map and the common features into the sparse graph respectively to obtain a first embedding matrix corresponding to the feature map, a second embedding matrix corresponding to the topology map and a third embedding matrix corresponding to the common features output by the sparse graph;

[0059] calculating a first attention coefficient of the first embedding matrix, a second attention coefficient of the second embedding matrix and a third attention coefficient of the third embedding matrix;

[0060] multiplying the first embedding matrix by the first attention coefficient, multiplying the second embedding matrix by the second attention coefficient and multiplying the third embedding matrix by the third attention coefficient to obtain a final embedding matrix;

[0061] training a brain disease classification model according to the final embedding matrix.

[0062] In a second aspect, the embodiments of the present application further provide a brain disease classification model training device, wherein the device comprises the following components:

[0063] a brain functional connectivity network generation module configured to generate a brain functional connectivity network according to a brain functional image;

[0064] a brain structural connectivity network generation module configured to generate a brain structural connectivity network according to a brain structural image;

[0065] a fusion module configured to fuse the brain functional connectivity network and the brain structural connectivity network to obtain a brain connectivity network;

[0066] a training module configured to train a brain disease classification model according to the brain connectivity network.

[0067] In a third aspect, the embodiments of the present application further provide a terminal device, wherein the terminal device comprises a memory, a processor, and a brain disease classification model training program stored in the memory and executable on the processor, and the processor implements the steps of the brain disease classification model training method described above when executing the brain disease classification model training program.

[0068] In a fourth aspect, the embodiments of the present application further provide a computer readable storage medium, wherein the computer readable storage medium stores a brain disease classification model training program, and the brain disease classification model training program implements the steps of the brain disease classification model training method described above when executed by a processor.

[0069] Beneficial effects: the brain functional connectivity network and the brain structural connectivity network complement each other, and the combination of the two can provide more human physiological information, so the brain connectivity network formed by the fusion of the brain functional connectivity network and the brain structural connectivity network has more human physiological information, the brain disease classification model is trained from the brain connectivity network, so that the brain disease classification model can learn more human physiological information, and therefore the brain disease classification model after training can provide a classification result with higher accuracy for brain diseases. BRIEF DESCRIPTION OF DRAWINGS

[0070] Figure 1 is a whole flowchart of the present application;

[0071] Figure 2 is a brain disease classification diagram in the embodiments of the present application;

[0072] Figure 3 is a brain network construction, graph attention network, and disease classification collaboration diagram in the embodiments of the present application;

[0073] Figure 4 is a graph attention network diagram in the embodiments of the present application;

[0074] Figure 5 The internal structure principle block diagram of the terminal device provided by the embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0075] The technical solutions in the present application are described clearly and completely below in combination with the embodiments and the accompanying drawings. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0076] It is found through research that, with the unprecedented development of the field of neuroimaging and pattern recognition, disease diagnosis based on a graph convolution learning model (a brain disease classification model) is possible. Through processing of image data by the graph convolution learning model, the type of disease can be obtained. Alzheimer's disease (AD) is a common neurodegenerative disease, and the prevalence rate increases continuously with age. AD usually has a chronic or progressive nature, and a variety of high-order cortical function disorders occur, which are manifested as symptoms such as cognitive function decline, judgment ability reduction, and memory loss, and ultimately lead to loss of independent living ability of the patient. As an early stage of Alzheimer's disease, the annual conversion rate of Mild Cognitive Impairment (MCI) is 10%-15%, and the conversion rate within 5 years is more than 50%. Significant Memory Concerns (SMC) is a pre-stage of MCI. MCI and SMC are different stages of AD, and correspond to the classification results of AD. The existing technology only performs learning and training of the brain disease classification model for one kind of brain image, which leads to low accuracy of the brain disease classification model after training.

[0077] To solve the above technical problems, the present application provides a brain disease classification model training method, device, equipment and storage medium, which solves the problem of low accuracy of the brain disease classification model trained by the prior art. In specific implementation, first, a brain function connection network is generated according to a brain function image; at the same time, a brain structure connection network is generated according to a brain structure image; then the brain function connection network and the brain structure connection network are fused to obtain a brain connection network; finally, a brain disease classification model is trained according to the brain connection network. The brain disease classification model trained in this embodiment has high accuracy.

[0078] For example, brain function images and brain structure images of three people are collected, three brain function connection networks are generated respectively (information on each functional area on the brain function image is extracted, and information on related functional areas is connected together to form a brain function image), three brain structure connection networks are generated respectively (the brain function image includes each structure area, and there is a connection relationship between the structure areas, each structure area and the structure areas with a connection relationship have a certain connection strength (fiber information), and the brain structure connection network is generated by combining each structure area and the connection strength), the three brain function connection networks and the three brain structure connection networks are fused together to form a brain connection network, and a brain disease classification model is learned and trained through the brain connection network. The trained brain disease classification model can identify the type of brain disease.

[0079] Exemplary method

[0080] The brain disease classification model training method of the embodiment can be applied to a terminal device, which can be a terminal product with computing function, such as a computer.

[0081] In this embodiment, the brain function images and the brain structure images of each individual are collected from the ADNI database by a data collection device Magnetom Verio 3.0T machine.

[0082] In this embodiment, as shown in Figure 1 The brain disease classification model training method specifically includes the following steps:

[0083] S100, generating a brain function connection network according to a brain function image.

[0084] For the preprocessing of rs-fMRI data (brain function image), the GRETNA toolbox is used. The data processing process is as follows:

[0085] 1) In order to ensure the effectiveness of the data, the first 10 time points of each subject (individual) are discarded, and then the remaining subjects are corrected by matching all time points with the middle time point.

[0086] 2) Head motion correction is applied to remove head motion artifacts in the rs-fMRI time series.

[0087] 3) Spatial normalization is performed using the DARTEL toolbox.

[0088] 4) Smoothing filtering is performed by a Gaussian kernel function to remove low-frequency drift and high-frequency noise.

[0089] 5) Regress regional mean time series to eliminate the influence of white matter signals, cerebrospinal fluid signals and head motion signals.

[0090] 6) Using Automatic Anatomical Labeling (AAL) to divide the brain space into 90 Regions of Interest (ROI), i.e. functional regions. Finally, a 90x90 functional connectivity (FC) network of each subject is obtained.

[0091] The step S100 comprises the following steps S101-S106:

[0092] S101, obtaining time points at which each of the brain function images is collected.

[0093] S102, obtaining intermediate time points and each non-intermediate time point according to the order of the time points.

[0094] S103, matching and correcting the brain function image of each non-intermediate time point with the brain function image of the intermediate time point to obtain the brain function image after matching and correction.

[0095] Suppose 20 brain function images of 10 individuals are collected, and the time points of the 20 brain function images are different in order. One of the brain function images, denoted as image A, is collected at an intermediate time point. Therefore, the image A can be used to correct the remaining 19 brain function images to obtain the images after matching and correction of the remaining 19 brain function images.

[0096] S104, removing shadows and noises in the brain function image after matching and correction to obtain the brain function image after preprocessing.

[0097] When collecting the brain function image, shadows are formed on the human brain due to external objects, which interfere with the information covered by the brain function image. Therefore, the shadows need to be removed.

[0098] S105, dividing the brain function image after preprocessing into each functional region.

[0099] In one embodiment, each brain function image is divided into 90x90 functional regions by using an AAL template.

[0100] S106, connecting each node according to the connection relationship of each functional region in the human brain to generate a brain function connection network.

[0101] The functional regions with correlation in the 90x90 functional regions are connected to form a brain function connection network.

[0102] S200, generating a brain structure connection network according to the brain structure image.

[0103] PANDA toolbox was used to obtain global brain deterministic fiber tracts from brain structural images. The brain space was divided into 90 ROIs using AAL template on the basis of Fractional Anisotropy (FA) as eigenvector. For the construction of structure connectivity (SC) network from DTI data, the average FA of the link between network nodes was defined as the connection weight in the DTI network, and a 90x90 SC network (brain functional connectivity network) was obtained for each patient.

[0104] Step S200 includes steps S201 to S207 as follows:

[0105] S201, dividing the brain structural image into each structural region ROI i .

[0106] The brain is segmented into R regions of interest (ROIs) i.e. structural regions using AAL (Anatomical Automatic Labeling) template, and each ROI information of all structural images of each subject is constructed into a vector as the information of the structural region.

[0107] S202, calculating the average anisotropy fraction FA between each of the structural regions.

[0108] Each structural region has a certain number of fiber tracts in the human brain, and all fiber tracts n(f) contained in the two structural regions with connection relationship are the average anisotropy fraction FA.

[0109] S203, calculating the surface area a i and a j between gray matter and white matter of each of the structural regions.

[0110] S204, calculating the average value of the surface area of each of the structural regions.

[0111] Structural region ROI i and structural region ROI j have connection relationship, then the average value of the surface area of ROI i and ROI j is used to correct the bias in the estimation of structural connectivity strength caused by different ROI sizes.

[0112] S205, dividing the average anisotropy fraction by the average value to obtain an operation result.

[0113] S206, obtaining the connection strength between each of the structural regions according to the operation result

[0114]

[0115] wherein a i is the surface area between the gray matter and the white matter of the structural region ROI i , a j is the surface area between the gray matter and the white matter of the structural region ROI j , N is 90, i, j ∈ N = {1, 2, …, 90} and i≠j. i j

[0116] S207, combining each of the structural regions with the connection strength to generate a brain structural connection network.

[0117] Each of the structural regions is taken as a point in the network, and the connection strength is taken as the weight between the structural region ROI i and the structural region ROI j , thus forming the brain structural connection network.

[0118] S300, fusing the brain functional connection network and the brain structural connection network to obtain a brain connection network.

[0119] In one embodiment, the brain functional connection network and the brain structural connection network of an individual are fused together to form a connection network unit of the individual, and all the connection network units of the individuals together form the brain connection network as shown in Figure 2 , i.e., the brain network construction and fusion in Figure 2 .

[0120] The step S300 specifically comprises the following steps S301 to S3018:

[0121] S301, extracting parameter information of each of the functional regions in the brain functional connection network.

[0122] The parameter information can be used to represent the gray value of each functional region, the mean square error of the gray values between the pixels, etc.

[0123] S302, generating a brain functional data matrix Y according to the parameter information of each of the functional regions and the position of each of the functional regions in the brain functional connection network.

[0124] S303, obtaining a brain functional weighting matrix B corresponding to the brain functional data matrix; ​​​

[0125] S304, Calculate the connection strength The square value

[0126] S305, the square value Divide by the preset penalty value σ to obtain the division result.

[0127] S306, using the natural number exp as the base and the division result as the exponent, calculate the exponentiation result C. i,j :

[0128]

[0129] C i,j That is, the ROI of the structural region i and structural region ROI j The DTI intensity penalty term between them.

[0130] S307, using the connection strength, position in the brain structure connectivity network, and the result of the exponentiation operation, generate a strength penalty matrix.

[0131] All C i,j According to connection strength Arranging these positions within the brain's structural connectivity network forms the intensity penalty matrix C. For example, C... 3,2 C is the intensity penalty term corresponding to ROI3 and ROI2. 3,2 Its position in the intensity penalty matrix C is the second row and first column.

[0132] S308, establish a matrix W to be solved corresponding to the size of the brain function data matrix, wherein the matrix to be solved is a matrix in which the values ​​of each element are unknown.

[0133] The matrix W to be solved is an empty matrix, which contains no elements.

[0134] S309, Multiply the brain function data matrix by the matrix to be solved to obtain the first matrix.

[0135] S3010, Subtract the first matrix from the brain function data matrix to obtain the second matrix.

[0136] S3011, Calculate the L2 norm of the second matrix.

[0137] S3012, calculate the intensity penalty matrix multiplied by the matrix to be solved to obtain the third matrix.

[0138] S3013, Calculate the first norm of the third matrix.

[0139] S3014, calculate the brain function weighted matrix multiplied by the to-be-solved matrix to obtain a fourth matrix;

[0140] S3015, calculate the two-norm of the fourth matrix.

[0141] S3016, construct a sparse regularization term according to the two-norm, the one-norm, and the two-one-norm of the fourth matrix.

[0142]

[0143] wherein YW is a first matrix, Y-YW is a second matrix, is a two-norm of the second matrix, C⊙W is a third matrix, ||C⊙W||1 is a one-norm of the third matrix, B⊙W is a fourth matrix, and ||B⊙W||1 is a one-norm of the fourth matrix. 2，1 is a two-one-norm of the fourth matrix, and λ1 and λ2 are regularization parameters.

[0144] S3017, calculate the element values in the to-be-solved matrix when the sparse regularization term takes a minimum value to obtain the to-be-solved matrix W after solving.

[0145] The specific element values in the W after solving.

[0146] S3018, generate a brain connection network as shown in Figure 2 according to the to-be-solved matrix after solving.

[0147] The reason why the brain structure connection network and the brain function connection network are combined to generate the brain connection network is that the brain structure is the basis of the brain function, the brain structure information is added to the sparse constraint term to constrain the information of the brain function network of the corresponding subject and control the construction of the brain function network. The Pearson correlation coefficient between the BOLD signals of each brain region of the subject is used to constrain the correlation between the brain function networks.

[0148] S400, train a brain disease classification model according to the brain connection network.

[0149] Step S400 introduces an adjacency matrix on the basis of the brain connection network to generate a sparse graph, and trains a brain disease classification model (graph convolution model) through a graph attention network. The reason why the adjacency matrix is introduced is that the adjacency matrix is used to represent the difference between individuals covered in the brain connection network to improve the training accuracy of the brain disease classification model. The adjacency matrix A is generated based on the following principle:

[0150] Suppose that N subjects (individuals) have a feature matrix Z representing their characteristics. The adjacency matrix A includes all edge weights, A(i, j) represents the edge weight between subjects i and j, sim(.) represents a feature similarity function, and r gr e r c r s r i i i i i represent the feature vector, gender, age, data center and disease status, respectively. The corresponding edge weights on the multi-center sparse graph are calculated as follows:

[0151] A(i, j) = sim(x i , x j ) x (1 + r g (g i , g j ) + r g (a i , a j ) + r e (e i , e j ) + r c (c i , c j ) + r s (s i , s j ))

[0152] where the similarity function is calculated as follows:

[0153]

[0154] where p(·) is a function to calculate the correlation distance, r g , r a , r e , r c and r s are defined as follows:

[0155]

[0156]

[0157]

[0158]

[0159]

[0160] Step S400 introduces the above adjacency matrix to train the model, including the following steps S401 to S4010: ​​​​

[0161] S401, Based on the brain connectivity network, obtain the connectivity network features X of the individual constituting the brain connectivity network, wherein the individual's connectivity network includes the individual's brain functional connectivity network and the individual's brain structural connectivity network.

[0162] S402, calculate the adjacency matrix A between each of the individuals, where the elements of the adjacency matrix are used to characterize the differences between the individuals.

[0163] S403, Integrate the individual connection network features and the adjacency matrix to construct a sparse graph G as input to the graph attention network:

[0164] G = (A, X)

[0165] S404, Based on the brain connectivity network, generate the corresponding data such as... Figure 3 The topology diagram shown is used to obtain specific information encoded in the topology space. S405, based on the brain connectivity network, generate the corresponding information such as... Figure 3 The topology diagram is shown below. Based on the node structure in the feature space, we generate the corresponding brain connectivity network using the K-nearest neighbor algorithm and the node feature matrix X (the individual's connectivity network features), as shown below. Figure 3 The feature diagram shown.

[0166] S406, considering the specificity of node features in the feature space and the information correlation, extract the common features covered by the feature map and the topology map.

[0167] In one embodiment, filtering is performed on the feature map, topology map, and common features, and the filtering is based on the following principle:

[0168] Kipf et al. first proposed the GCN network model, which introduced spectral graph convolution theory and applied Fourier transform and Taylor expansion to improve the filter. Assume there are N nodes forming a graph G = (V, E, A), where V is a set of nodes, E is a set of edges, and A is an adjacency matrix composed of edge weights. The basic principle of GCN is as follows:

[0169]

[0170]

[0171]

[0172]

[0173] Among them, H l W is the feature matrix of all nodes in the l-th layer. lLet be the trainable weight matrix of the Lth layer, and σ be the activation function. Graph convolution theory is applied to... The construction of. In this process, First of all The data is converted to the Fourier domain, then truncated using Chebyshev polynomials, and finally converted back to the original time domain. Using the above propagation rules, the commonly used two-layer GCN structure for classification tasks is represented as follows:

[0174]

[0175] Compared to traditional two-layer neural networks, the pre-multiplication adjacency matrix in the pre-propagation GCN layer... (i.e., Z) This acts as a filter, and the effectiveness of the filter is crucial to the performance of GCN. (x) i The feature vector of object i is represented by its filtering form. It can be described as:

[0176]

[0177] Existing GCN methods apply to all features (x) i Applying the same convolution coefficients (A) i1 A i2 A iN ) is filtered, which ignores the differences between features.

[0178] S407, the feature map, the topology map, and the common features are respectively input into such a system as... Figure 4 The graph attention network shown obtains the first embedding matrix Z corresponding to the feature map output by the sparse graph. F The second embedding matrix Z corresponding to the topology graph T The third embedding matrix Z corresponding to the common features C .

[0179] S408, Calculate the first embedding matrix Z F First attention coefficient α F The second embedding matrix Z T The second attention coefficient α T The third embedding matrix Z C The third attention coefficient α C .

[0180] (α T α C α F ) = att(Z T Z C Z F )

[0181] S409, the first embedding matrix multiplied by the first attention coefficient, plus the second embedding matrix multiplied by the second attention coefficient, plus the third embedding matrix multiplied by the third attention coefficient, yields the final embedding matrix Z:

[0182] Z = α T .Z T +α C .Z C +α F .Z F

[0183] In step S407 above, the first embedding matrix Z is generated. F The second embedding matrix Z T The third embedding matrix Z C It is based on the following principle:

[0184] Using graph G = (A, X) as the working basis, where A ∈ R n×n It is a symmetric adjacency matrix with n nodes, X∈R n×d A is the node feature matrix, where d is the dimension of the node features. Specifically, A... i,j =1 indicates that there is a certain relationship between nodes i and j (e.g., the same type of disease, similar age, same gender, etc.); otherwise, A i,j =0. Node features X are learned through two specific convolutional modules on the feature map and topology map to obtain two embedding information Z. F and Z T To obtain more common features, a common embedding module Z is learned using a common convolutional module with shared parameters. CF and Z CT Considering the impact of different parameters on network performance, a consistency constraint L is used. c To further regulate Z CF and Z CT The "shared" property of L, and using the distributional difference constraint L d Used to guarantee Z F and Z CF and Z T and Z CT The independence between them is determined. Finally, an attention mechanism is used to adaptively fuse these embeddings with the learned weights in order to extract the most relevant embedding information Z for disease classification.

[0185] For the feature space, in order to obtain the node structure in the potential feature space, we construct a graph G based on the K-nearest neighbor algorithm and the node feature matrix X. f =(A f (X). The first layer output in the feature space. It can be represented as:

[0186]

[0187] in This is the weight matrix of the l-th layer in GCN, initially... Let X be the characteristic matrix. yes The angle matrix is ​​then calculated. Finally, specific information Z is captured in the feature space. f The node embeddings. For the topological space, we have the original input graph G. t =(A t (X). Learning output embedding Z based on topological graph. T It can be computed in the same way as the feature space. Therefore, we can obtain specific information encoded in the topological space.

[0188] In node classification tasks, information in the feature space or topology space exhibits certain correlations. It is necessary not only to extract the embeddings of specific nodes in these two spaces but also to extract shared common information. Therefore, this embodiment designs a commonality graph with a parameter-sharing strategy to obtain the shared embedding information in the two spaces.

[0189] First, we utilize the commonality graph from the topological graph G t =(A t Extracting node embeddings from (X) As shown below:

[0190]

[0191] It is the weight matrix of the l-th layer of the commonality graph. It is the node embedding layer of layer l-1. Using commonality graphs from feature graphs (A f When learning node embeddings, we share the same weight matrix in each layer of the common graph. As follows:

[0192]

[0193] in, For the output embedding of layer l, Shared features can be learned using a shared weight matrix. Finally, based on different input maps, two independent output embeddings and one common embedding, Z, can be obtained. CT Z CF Z C Public Embedding Z C for:

[0194] Z C =(ZCT +Z CF ) / 2

[0195] S4010, Train the brain disease classification model based on the final embedding matrix.

[0196] The final embedding matrix is ​​input into the brain disease classification model. If the classification result output by the brain disease classification model is different from the set result, the parameters of the brain disease classification model are adjusted to complete the training of the brain disease classification model.

[0197] In summary, functional connectivity networks and structural connectivity networks complement each other, and their combination provides more physiological information about the human body. Therefore, the brain connectivity network formed by the fusion of functional connectivity networks and structural connectivity networks has more physiological information about the human body. By using the brain connectivity network to learn and train the brain disease classification model, the brain disease classification model can learn more physiological information about the human body. As a result, the trained brain disease classification model can provide more accurate classification results for brain diseases.

[0198] Exemplary apparatus

[0199] This embodiment also provides a brain disease classification model training device, the device comprising the following components:

[0200] The brain functional connectivity network generation module is used to generate brain functional connectivity networks based on brain functional images.

[0201] The brain structure connectivity network generation module is used to generate a brain structure connectivity network based on brain structure images.

[0202] A fusion module is used to fuse the brain functional connectivity network and the brain structural connectivity network to obtain a brain connectivity network;

[0203] The training module is used to train a brain disease classification model based on the brain connectivity network.

[0204] Based on the above embodiments, the present invention also provides a terminal device, the principle block diagram of which can be as follows: Figure 5As shown, the terminal device includes a processor, memory, network interface, display screen, and temperature sensor connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements a method for training a brain disease classification model. The display screen can be an LCD screen or an e-ink screen. The temperature sensor is pre-installed inside the terminal device to detect the operating temperature of the internal components.

[0205] Those skilled in the art will understand that Figure 5 The block diagram shown is merely a partial structural diagram related to the present invention and does not constitute a limitation on the terminal device to which the present invention is applied. The specific terminal device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0206] In one embodiment, a terminal device is provided, comprising a memory, a processor, and a brain disease classification model training program stored in the memory and executable on the processor. When the processor executes the brain disease classification model training program, it implements the following operation instructions:

[0207] Based on brain functional images, generate brain functional connectivity networks;

[0208] Generate a brain structure connectivity network based on brain structure images;

[0209] By fusing the brain functional connectivity network and the brain structural connectivity network, a brain connectivity network is obtained;

[0210] Based on the brain connectivity network, a brain disease classification model is trained.

[0211] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided by this invention can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0212] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for training a brain disease classification model, characterized in that, include: Based on brain functional images, generate brain functional connectivity networks; Generate a brain structure connectivity network based on brain structure images; By fusing the brain functional connectivity network and the brain structural connectivity network, a brain connectivity network is obtained; Based on the aforementioned brain connectivity network, a brain disease classification model is trained; The process of generating a brain functional connectivity network based on brain functional images includes: Acquire the time points at which each of the aforementioned brain function images was acquired; Based on the chronological order of the various time points, the intermediate time points and the various non-intermediate time points are obtained; The brain function images at each of the non-intermediate time points are matched and corrected with the brain function images at the intermediate time points to obtain the matched and corrected brain function images. Shadows and noise are removed from the brain functional image after matching correction to obtain the preprocessed brain functional image; The preprocessed brain functional images are divided into various functional regions; Using each of the aforementioned functional areas as nodes, and connecting the nodes according to the connection relationships of each of the aforementioned functional areas in the human brain, a brain functional connectivity network is generated. The process of generating a brain structure connectivity network based on brain structure images includes: The brain structure image is divided into various structural regions; Calculate the average anisotropy fraction among the various structural regions; Calculate the surface area between the gray and white matter in each of the structural regions; Calculate the average surface area of ​​each of the structural regions; The average anisotropy fraction is divided by the average value to obtain the calculation result; Based on the calculation results, the connection strength between each of the structural regions is obtained; Each of the aforementioned structural regions acts as a node, and the connection strength is combined to generate a brain structural connectivity network. The fusion of the brain functional connectivity network and the brain structural connectivity network to obtain a brain connectivity network includes: Extract parameter information of each functional region in the brain functional connectivity network; A brain function data matrix is ​​generated based on the parameter information of each functional area and the position of each functional area in the brain function connectivity network; Obtain the brain function weighted matrix corresponding to the brain function data matrix; A strength penalty matrix is ​​generated based on the connection strength covered by the brain structure connectivity network. A brain connectivity network is generated based on the brain function data matrix, the brain function weighting matrix, and the intensity penalty matrix. The step of training a brain disease classification model based on the brain connectivity network includes: Based on the brain connectivity network, the connectivity network characteristics of the individual constituting the brain connectivity network are obtained, and the individual's connectivity network includes the individual's brain functional connectivity network and the individual's brain structural connectivity network. Calculate the adjacency matrix between each of the individuals, where the elements of the adjacency matrix are used to characterize the differences between the individuals; By fusing the individual connection network features with the adjacency matrix, a sparse graph is generated; The K-nearest neighbor algorithm is applied to the connectivity network features of the individual to generate feature maps corresponding to the brain connectivity network; Based on the brain connectivity network, a topology map corresponding to the brain connectivity network is generated; Extract the common features covered by the feature map and the topology map; The feature map, the topology map, and the common feature are respectively input into the sparse graph to obtain the first embedding matrix corresponding to the feature map, the second embedding matrix corresponding to the topology map, and the third embedding matrix corresponding to the common feature output by the sparse graph. Calculate the first attention coefficient of the first embedding matrix, the second attention coefficient of the second embedding matrix, and the third attention coefficient of the third embedding matrix; The first embedding matrix is ​​multiplied by the first attention coefficient, plus the second embedding matrix multiplied by the second attention coefficient, plus the third embedding matrix multiplied by the third attention coefficient, to obtain the final embedding matrix; Based on the final embedding matrix, a brain disease classification model is trained.

2. The brain disease classification model training method as described in claim 1, characterized in that, The step of generating a strength penalty matrix based on the connection strength covered by the brain structure connectivity network includes: Calculate the square of the connection strength; Divide the squared value by the preset penalty value to obtain the division result; Using natural numbers as the base and the division result as the exponent, calculate the result of the exponentiation operation; A strength penalty matrix is ​​generated based on the position of the connection strength in the brain structure connectivity network and the result of the exponentiation operation.

3. The brain disease classification model training method as described in claim 1, characterized in that, The step of generating a brain connectivity network based on the brain function data matrix, the brain function weighting matrix, and the intensity penalty matrix includes: Establish a solution matrix corresponding to the size of the brain function data matrix, wherein the solution matrix is ​​a matrix in which the values ​​of each element are unknown; Multiply the brain function data matrix by the matrix to be solved to obtain the first matrix; Subtract the first matrix from the brain function data matrix to obtain the second matrix; Calculate the L2 norm of the second matrix; The intensity penalty matrix is ​​multiplied by the matrix to be solved to obtain the third matrix; Calculate the norm of the third matrix; The fourth matrix is ​​obtained by multiplying the brain function weighted matrix by the matrix to be solved. Calculate the 2-1 norm of the fourth matrix; Based on the L2 norm, L1 norm, and L2-L1 norm, a sparse regularization term is constructed; The values ​​of each element in the matrix to be solved are calculated when the sparse regularization term reaches its minimum value, thus obtaining the solved matrix to be solved. Based on the solved matrix, a brain connectivity network is generated.

4. A brain disease classification model training device, characterized in that, The device comprises the following components: The brain functional connectivity network generation module is used to generate brain functional connectivity networks based on brain functional images. The brain structure connectivity network generation module is used to generate a brain structure connectivity network based on brain structure images. A fusion module is used to fuse the brain functional connectivity network and the brain structural connectivity network to obtain a brain connectivity network; The training module is used to train a brain disease classification model based on the brain connectivity network. The process of generating a brain functional connectivity network based on brain functional images includes: Acquire the time points at which each of the aforementioned brain function images was acquired; Based on the chronological order of the various time points, the intermediate time points and the various non-intermediate time points are obtained; The brain function images at each of the non-intermediate time points are matched and corrected with the brain function images at the intermediate time points to obtain the matched and corrected brain function images. Shadows and noise are removed from the brain functional image after matching correction to obtain the preprocessed brain functional image; The preprocessed brain functional images are divided into various functional regions; Using each of the aforementioned functional areas as nodes, and connecting the nodes according to the connection relationships of each of the aforementioned functional areas in the human brain, a brain functional connectivity network is generated. The process of generating a brain structure connectivity network based on brain structure images includes: The brain structure image is divided into various structural regions; Calculate the average anisotropy fraction among the various structural regions; Calculate the surface area between the gray and white matter in each of the structural regions; Calculate the average surface area of ​​each of the structural regions; The average anisotropy fraction is divided by the average value to obtain the calculation result; Based on the calculation results, the connection strength between each of the structural regions is obtained; Each of the aforementioned structural regions acts as a node, and the connection strength is combined to generate a brain structural connectivity network. The fusion of the brain functional connectivity network and the brain structural connectivity network to obtain a brain connectivity network includes: Extract parameter information of each functional region in the brain functional connectivity network; A brain function data matrix is ​​generated based on the parameter information of each functional area and the position of each functional area in the brain function connectivity network; Obtain the brain function weighted matrix corresponding to the brain function data matrix; A strength penalty matrix is ​​generated based on the connection strength covered by the brain structure connectivity network. A brain connectivity network is generated based on the brain function data matrix, the brain function weighting matrix, and the intensity penalty matrix. The step of training a brain disease classification model based on the brain connectivity network includes: Based on the brain connectivity network, the connectivity network characteristics of the individual constituting the brain connectivity network are obtained, and the individual's connectivity network includes the individual's brain functional connectivity network and the individual's brain structural connectivity network. Calculate the adjacency matrix between each of the individuals, where the elements of the adjacency matrix are used to characterize the differences between the individuals; By fusing the individual connection network features with the adjacency matrix, a sparse graph is generated; The K-nearest neighbor algorithm is applied to the connectivity network features of the individual to generate feature maps corresponding to the brain connectivity network; Based on the brain connectivity network, a topology map corresponding to the brain connectivity network is generated; Extract the common features covered by the feature map and the topology map; The feature map, the topology map, and the common feature are respectively input into the sparse graph to obtain the first embedding matrix corresponding to the feature map, the second embedding matrix corresponding to the topology map, and the third embedding matrix corresponding to the common feature output by the sparse graph. Calculate the first attention coefficient of the first embedding matrix, the second attention coefficient of the second embedding matrix, and the third attention coefficient of the third embedding matrix; The first embedding matrix is ​​multiplied by the first attention coefficient, plus the second embedding matrix multiplied by the second attention coefficient, plus the third embedding matrix multiplied by the third attention coefficient, to obtain the final embedding matrix; Based on the final embedding matrix, a brain disease classification model is trained.

5. A terminal device, characterized in that, The terminal device includes a memory, a processor, and a brain disease classification model training program stored in the memory and executable on the processor. When the processor executes the brain disease classification model training program, it implements the steps of the brain disease classification model training method as described in any one of claims 1-3.

6. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a brain disease classification model training program, which, when executed by a processor, implements the steps of the brain disease classification model training method as described in any one of claims 1-3.

Images