Brain disease diagnosis method, storage medium, and electronic device

Abstract

Embodiments of the present disclosure disclose a brain disease diagnosis method, a storage medium, and an electronic device. The brain disease diagnosis method comprises: acquiring medical imaging data of a brain; preprocessing the medical imaging data, to obtain a brain tissue image and a brain map; separately inputting the brain tissue image and the brain map into a pre-trained brain disease diagnosis model, the brain disease model comprising at least one convolution module, at least one map convolution module, at least one fusion module, and an output module; extracting a geometric texture feature of the brain tissue image by means of the convolution module, and, by means of the map convolution module, extracting from the brain map a topological relationship feature between brain regions; fusing the geometric texture feature and the topological relationship feature to a node feature by means of the fusion module, to obtain a new node feature; and, by means of the output module and on the basis of the new node feature, determining a brain disease corresponding to the medical imaging data, and outputting the brain disease.

Description

Brain disease diagnosis method, storage medium and electronic device Technical Field

[0001] The embodiments of the present disclosure relate to, but are not limited to, the field of data processing technology, and specifically to a brain disease diagnosis method, storage medium, and electronic device. Background Art

[0002] Brain diseases are threatening human health. Common brain diseases include epilepsy, Parkinson's disease, Alzheimer's disease, depression, and schizophrenia. According to a survey report, approximately 10 million new cases of brain diseases occur in my country each year, with approximately 75% resulting in mortality or disability. Early detection and treatment are crucial to preventing brain diseases, so timely and effective diagnosis is essential.

[0003] Structural MRI (Magnetic resonance imaging, multimodal magnetic resonance imaging, abbreviated as MRI) has multiple modal imaging information, providing very valuable information for the diagnosis of brain diseases. Taking Alzheimer's disease as an example, it is one of the most common neurodegenerative diseases. Structural MRI examinations can not only clarify the degree of brain atrophy, but also perform exclusionary diagnoses of diseases such as cerebrovascular disease and stroke. It is a non-invasive medical imaging technology and has a wide range of applications in the study of biological differences in brain diseases. However, the human eye cannot perceive the smallest degree of atrophy, and manual diagnosis is time-consuming and inefficient. With the development of computer vision, the application of deep learning methods to structural MRI diagnosis has become a research hotspot.

[0004] Summary of the Invention

[0005] The following is a summary of the subject matter described in detail herein. This summary is not intended to limit the scope of the claims.

[0006] In one aspect, the present disclosure provides a method for diagnosing a brain disease, comprising:

[0007] Acquiring medical imaging data of the brain;

[0008] Preprocessing the medical imaging data to obtain a brain tissue image and a brain map; wherein the brain map includes a plurality of nodes, each node having a node feature; and the nodes correspond one to one with brain regions;

[0009] The brain tissue image and brain map are respectively input into a pre-trained brain disease diagnosis model, wherein the brain disease model includes at least one convolution module, at least one graph convolution module, at least one fusion module, and an output module; the geometric texture features of the brain tissue image are extracted by the convolution module, and the topological relationship features between brain regions are extracted from the brain map by the graph convolution module; the geometric texture features and the topological relationship features are fused with the node features to obtain new node features by the fusion module; and the brain disease corresponding to the medical imaging data is determined by the output module based on the new node features and the brain disease is output.

[0010] In an exemplary embodiment, fusing the geometric texture feature and the topological relationship feature with the original node feature to obtain a new node feature through the fusion module includes:

[0011] A first processing is performed on the brain tissue image containing the geometric texture features to obtain geometric texture features corresponding to each brain region; and the geometric texture features and topological relationship features corresponding to each brain region are fused to the node features corresponding to the brain region to obtain new node features.

[0012] In an exemplary embodiment, the first processing of the brain tissue image containing geometric texture features includes:

[0013] Using a preset brain atlas template, the brain tissue image containing geometric texture features is divided into multiple blocks, each block corresponding to a brain region;

[0014] The geometric texture features of each brain region are flattened and integrated.

[0015] In an exemplary embodiment, fusing the geometric texture features and topological relationship features corresponding to each brain region to the node features corresponding to the brain region to obtain new node features includes:

[0016] The integrated geometric texture features are spliced ​​onto the corresponding node features, and the geometric texture features and topological relationship features of the node are fused with the node features of the node through a second shared multi-layer perceptron to obtain new node features.

[0017] In an exemplary embodiment, determining the brain disease corresponding to the medical image data according to the new node feature includes:

[0018] Transforming the new node features by a preset aggregation function;

[0019] Inputting the transformed node features into the fully connected hierarchy;

[0020] The brain disease corresponding to the medical image data is determined according to the output value of the fully connected layer.

[0021] In an exemplary embodiment, the preprocessing of the medical imaging data to obtain a brain tissue image and a brain map includes:

[0022] Obtaining a brain tissue image by performing a second processing on the medical image data;

[0023] Segmenting the brain tissue image using a preset method to obtain a tissue probability image; obtaining a gray matter volume image based on the tissue probability image; dividing the gray matter volume image into a plurality of brain regions using a preset brain atlas template, determining the KL divergence between any two brain regions based on the values ​​of a plurality of voxels in the gray matter volume images of the two brain regions, and constructing a connectivity matrix based on the KL divergence between all any two brain regions;

[0024] Each brain region is regarded as a node, and a brain map is constructed according to the connectivity matrix.

[0025] In an exemplary embodiment, the second processing sequentially includes image rectification, reconfiguration of the scanning axis, cropping of a portion of the image, removal of the skull, and first spatial registration;

[0026] The first spatial registration includes registering the brain tissue image after the skull is removed to the MNI standard space using a preset registration method.

[0027] In an exemplary embodiment, obtaining a gray matter volume image according to the gray matter probability image includes:

[0028] performing a second spatial registration on the gray matter probability image, and obtaining a gray matter volume image based on the deformation field parameters obtained in the second spatial registration process and the gray matter probability image;

[0029] The performing a second spatial registration on the gray matter probability image includes:

[0030] The gray matter probability image is registered to the MNI standard space using a preset registration method.

[0031] In an exemplary embodiment, obtaining a gray matter volume image based on the deformation field parameters obtained in the second spatial registration process and the tissue probability image includes:

[0032] multiplying the tissue probability image by the nonlinear part of the deformation field parameter to obtain a relative gray matter volume image;

[0033] The relative gray matter volume image is smoothed to obtain a gray matter volume image.

[0034] In an exemplary embodiment, determining the KL divergence between any two brain regions based on the values ​​of n voxels in each of the two brain regions includes:

[0035] The probability density function of each brain region is calculated based on the values ​​of n voxels in the region, and the KL divergence between any two brain regions is calculated based on the probability density functions of the two brain regions.

[0036] In an exemplary embodiment, the connectivity matrix is ​​transformed into a binary matrix according to a preset threshold.

[0037] In an exemplary embodiment, constructing a brain map according to the connectivity matrix includes:

[0038] Calculate the network characteristics of each node according to the binary matrix;

[0039] Calculate the KL divergence between each node and other nodes;

[0040] Determine the node characteristics of each node according to the KL divergence between each node and other nodes and the network characteristics of each node;

[0041] The Euclidean distance between the center point of each brain region and the center points of other brain regions is used as an edge; the adjacency matrix of the corresponding nodes of each brain region is constructed based on the edge of each brain region;

[0042] A brain map is constructed according to the node features of each node and the adjacency matrix.

[0043] In an exemplary embodiment, the acquiring of medical imaging data of the brain includes: receiving medical imaging data input by a user;

[0044] Before obtaining a brain disease by using any one of the above-mentioned brain disease diagnosis methods on the medical image data, the method further comprises: receiving a brain disease diagnosis instruction input by a user on an operation interface;

[0045] The outputting of the brain disease includes: obtaining the name of the brain disease diagnosed from the medical imaging data.

[0046] On the other hand, an embodiment of the present disclosure provides a non-transitory computer-readable storage medium storing program instructions, which can implement any of the above methods when executed.

[0047] On the other hand, an embodiment of the present disclosure provides an electronic device, including a processor and a memory storing a computer program that can be run on the processor, wherein the processor implements the steps of any one of the methods described above when executing the program.

[0048] Still other aspects will become apparent upon reading and understanding the accompanying drawings and detailed description.

[0049] Summary of the Figures

[0050] The accompanying drawings are intended to provide a further understanding of the technical solutions of the present disclosure and constitute a part of the specification. Together with the embodiments of the present disclosure, they are used to explain the technical solutions of the present disclosure and do not constitute a limitation of the technical solutions of the present disclosure. The shapes and sizes of the components in the drawings do not reflect the actual scale and are intended only to illustrate the contents of the present disclosure.

[0051] FIG1 is a schematic diagram of a method for diagnosing brain diseases according to an embodiment of the present disclosure;

[0052] FIG2 is a flow chart of a preprocessing method according to an embodiment of the present disclosure;

[0053] FIG3 is a flowchart of generating a mind map according to an embodiment of the present disclosure;

[0054] FIG4 is a flow chart of a training method according to an embodiment of the present disclosure;

[0055] FIG5 is a schematic diagram of diagnosing a brain disease using a brain disease diagnosis model according to an embodiment of the present disclosure;

[0056] FIG6 is a schematic structural diagram of an electronic device in an embodiment of the present disclosure.

[0057] Details

[0058] The present disclosure describes a plurality of embodiments, but the description is exemplary rather than restrictive, and it is apparent to those skilled in the art that there may be more embodiments and implementations within the scope of the embodiments described in the present disclosure. Although many possible feature combinations are shown in the drawings and discussed in the embodiments, many other combinations of the disclosed features are also possible. Unless specifically limited, any feature or element of any embodiment may be used in combination with any other feature or element in any other embodiment, or may replace any other feature or element in any other embodiment.

[0059] The present disclosure includes and contemplates combinations of features and elements known to those of ordinary skill in the art. The disclosed embodiments, features, and elements of the present disclosure may also be combined with any conventional features or elements to form a unique inventive solution defined by the claims. Any features or elements of any embodiment may also be combined with features or elements from other inventive solutions to form another unique inventive solution defined by the claims. Therefore, it should be understood that any feature shown and / or discussed in this disclosure may be implemented individually or in any appropriate combination. Therefore, the embodiments are not subject to other limitations except for the limitations set forth in the appended claims and their equivalents. In addition, various modifications and changes may be made within the scope of protection of the appended claims.

[0060] In addition, when describing representative embodiments, the specification may have presented the method and / or process as a specific sequence of steps. However, to the extent that the method or process does not rely on the specific order of the steps described herein, the method or process should not be limited to the steps in the specific order described. As will be understood by those skilled in the art, other orders of steps are also possible. Therefore, the specific order of the steps set forth in the specification should not be interpreted as a limitation on the claims. In addition, the claims to the method and / or process should not be limited to performing their steps in the order written, and those skilled in the art can readily understand that these orders can be changed and still remain within the spirit and scope of the disclosed embodiments.

[0061] Unless otherwise defined, the technical or scientific terms used in this disclosure have the usual meanings understood by persons of ordinary skill in the art to which this disclosure belongs. The words "first", "second" and similar terms used in this disclosure do not indicate any order, quantity or importance, but are merely used to distinguish different components. In this disclosure, "plurality" may refer to two or more numbers. Words such as "include" or "comprising" mean that the elements or objects preceding the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects.

[0062] In order to keep the following description of the embodiments of the present disclosure clear and concise, the present disclosure omits detailed descriptions of some known functions and components. The drawings of the embodiments of the present disclosure only relate to the structures related to the embodiments of the present disclosure, and other structures can refer to the general design.

[0063] The inventors of the present disclosure have found that patients with mild cognitive impairment do not undergo significant structural changes in the brain area compared to normal people, that is, for MRI images, patients with mild cognitive impairment have very small changes compared to normal people, are easily interfered by noise, and are difficult to identify. The inventors of the present disclosure have also found that in some embodiments, a block-based approach is adopted to cut the image into pieces, and then each piece of image is input into the model training, so that subtle feature changes will become obvious relative to each piece of image, but this method will ignore the relationship between the image blocks, that is, the positional topological relationship, and each piece of image that is cropped will carry background and other interference information. Therefore, the inventors considered pre-processing the medical imaging data, that is, extracting the brain tissue image, and constructing a brain map based on the brain tissue image, and then extracting geometric texture features from the brain tissue image, and extracting topological relationship features between brain regions from the brain map, thereby solving the technical problems of image background and other interference information redundancy and missing topological position relationship between image blocks when using deep learning technology to diagnose diseases with magnetic resonance imaging (MRI) image data. To this end, the present disclosure provides a method for diagnosing brain diseases, as shown in Figure 1, comprising the following steps 11 to 13:

[0064] Step 11, obtaining medical imaging data of the brain;

[0065] Step 12, preprocessing the medical imaging data to obtain brain tissue images and brain maps;

[0066] Step 13: Input the brain tissue image and brain map into a pre-trained brain disease diagnosis model respectively, wherein the brain disease model includes at least one convolution module, at least one graph convolution module, at least one fusion module, and an output module; extract the geometric texture features of the brain tissue image through the convolution module, and extract the topological relationship features between brain regions from the brain map through the graph convolution module; fuse the geometric texture features and the topological relationship features to the node features to obtain new node features through the fusion module; determine the brain disease corresponding to the medical imaging data according to the new node features through the output module and output the brain disease.

[0067] In the disclosed embodiments, brain medical imaging data is preprocessed to obtain brain tissue images and brain maps, geometric texture features are extracted from the brain tissue images, topological relationship features between brain regions are extracted from the brain maps, and the geometric texture features and topological relationship features are fused with the node features of the brain map nodes, so that the node features of the brain map nodes have both local geometric texture features and global topological features, thereby solving the technical problems of redundant image background and other interference information and missing topological position relationships between image blocks when using deep learning technology to diagnose diseases with MRI imaging data.

[0068] Exemplarily, the convolution module may include t convolution submodules and pooling submodules, where t is a natural number, and each convolution submodule includes a convolution Conv layer, a BN layer, and a ReLU layer. The pooling submodule includes a pooling layer.

[0069] Exemplarily, t may be 3.

[0070] Exemplarily, the graph convolution module may include a graph convolutional neural network (GCN). A graph convolutional neural network (GCN) is a method for extracting features from graph data, thereby using these features to perform node classification and graph classification on the graph data.

[0071] Exemplarily, there may be L convolution modules, where L is a natural number.

[0072] Exemplarily, there may be L graph convolution modules;

[0073] Exemplarily, there may be L fusion modules.

[0074] Exemplarily, L may be 2 or other values.

[0075] In an exemplary embodiment, fusing the geometric texture feature and the topological relationship feature with the original node feature to obtain a new node feature through the fusion module includes:

[0076] A first processing is performed on the brain tissue image containing the geometric texture features to obtain geometric texture features corresponding to each brain region; and the geometric texture features and topological relationship features corresponding to each brain region are fused to the node features corresponding to the brain region to obtain new node features.

[0077] In an exemplary embodiment, the first processing of the brain tissue image containing geometric texture features includes:

[0078] Using a preset brain atlas template, the brain tissue image containing geometric texture features is divided into multiple blocks, each block corresponding to a brain region;

[0079] The geometric texture features of each brain region are flattened and integrated.

[0080] Exemplarily, the preset brain map template may be an ALL90 brain region template.

[0081] In an exemplary embodiment, fusing the geometric texture features and topological relationship features corresponding to each brain region to the node features corresponding to the brain region to obtain new node features includes:

[0082] The integrated geometric texture features are spliced ​​onto the corresponding node features, and the geometric texture features and topological relationship features of the node are fused with the node features of the node through a second shared multi-layer perceptron to obtain new node features.

[0083] In an exemplary embodiment, determining the brain disease corresponding to the medical image data according to the new node feature includes:

[0084] Transforming the new node features by a preset aggregation function;

[0085] Inputting the transformed node features into the fully connected hierarchy;

[0086] The brain disease corresponding to the medical image data is determined according to the output value of the fully connected layer.

[0087] Exemplarily, the preset aggregation function may be aggregation functions such as sum, average, and maximum.

[0088] Exemplarily, the number of fully connected layers may be 3.

[0089] Exemplarily, the normalization may be softmax.

[0090] The present disclosure provides a preprocessing method, as shown in FIG2 , including the following steps 21 to 26:

[0091] Step 21, obtaining a brain tissue image by performing a second processing on the medical image data;

[0092] Step 22, segmenting the brain tissue image using a preset method to obtain a gray matter probability image;

[0093] Step 23, obtaining a gray matter volume image based on the gray matter probability image;

[0094] Step 24, dividing the gray matter volume image into m brain regions using a preset brain atlas template, and determining the KL divergence between any two brain regions based on the values ​​of n voxels in the gray matter volume images of the two brain regions;

[0095] Step 25, constructing an m×m morphological connectivity matrix based on the KL divergence between any two brain regions;

[0096] Step 26: construct a brain map using each brain region as a node according to the morphological connectivity matrix.

[0097] In the disclosed embodiments, by preprocessing medical imaging data of the brain to obtain standardized brain tissue images, a morphological connectivity matrix is ​​constructed based on the standardized brain tissue images, and a brain map is constructed based on the morphological connectivity matrix, thereby solving the technical problem of redundant image background and other interfering information when using deep learning technology to diagnose diseases with magnetic resonance imaging (MRI) image data.

[0098] In an exemplary embodiment, the medical imaging data may be magnetic resonance imaging (MRI).

[0099] In an exemplary embodiment, the second processing sequentially includes image rectification, reconfiguration of the scanning axis, cropping of a portion of the image, removal of the skull, and first spatial registration;

[0100] Exemplarily, image correction may include correcting medical image data by a front-joint-back-joint method and applying an N4 bias field to correct intensity non-uniformity.

[0101] Exemplarily, the purpose of reconfiguring the scan axis is to match the standard orientation of the MNI-152 (Montreal Neurological Institute-152) space.

[0102] For example, the automatic threshold technology can be used to crop out the image of unnecessary parts such as the neck.

[0103] For example, by removing the skull, an image containing only brain tissue can be obtained.

[0104] In an exemplary embodiment, the first spatial registration includes registering the brain tissue image after skull removal to the MNI standard space using a preset registration method.

[0105] In an exemplary embodiment, the first spatial registration may be volume spatial registration.

[0106] The preset registration method may be a preliminary linear affine registration using an MNI-152 template and a nonlinear high-order registration, wherein the nonlinear high-order registration may use DARTEL technology.

[0107] In step 22, the brain tissue image may be segmented into three components, namely, gray matter, white matter, and cerebrospinal fluid, using an adaptive maximum a posteriori technique, thereby obtaining a tissue probability image of each component.

[0108] In step 23, obtaining a gray matter volume image according to the gray matter probability image may include:

[0109] A second spatial registration is performed on the gray matter probability image, and a gray matter volume image is obtained based on the deformation field parameters obtained in the second spatial registration process and the gray matter probability image.

[0110] Exemplarily, the second spatial registration may include linear affine registration and nonlinear high-order registration.

[0111] In an exemplary embodiment, performing a second spatial registration on the gray matter probability image includes:

[0112] The gray matter probability image is registered to the MNI standard space using a preset registration method.

[0113] That is, the gray matter probability image is registered to the MNI standard space through the second space registration.

[0114] In an exemplary embodiment, obtaining a gray matter volume image based on the deformation field parameters obtained in the second spatial registration process and the gray matter probability image includes:

[0115] multiplying the gray matter probability image by the nonlinear part of the deformation field parameter to obtain a relative gray matter volume image;

[0116] The relative gray matter volume image is smoothed to obtain a gray matter volume image.

[0117] Gray matter volume images are three-dimensional, with each point being a voxel, which contains over 1,000,000 neurons. The value of each voxel represents the volume of gray matter at that location, so the volume of gray matter at different locations, i.e., the volume of different voxels, can be determined based on the gray matter volume image.

[0118] The deformation length parameters may be arranged in the form of a Jacobian determinant.

[0119] The relative gray matter volume image is obtained by multiplying the gray matter probability image with the nonlinear change part of the Jacobian determinant to correct the individual brain volume.

[0120] Exemplarily, the smoothing process may be a Gaussian spatial smoothing process to reduce registration errors and improve the signal-to-noise ratio.

[0121] In step 24, n and m are natural numbers. n can be selected based on experience, balancing the stability of the probability density function and the computational complexity. n can be 128 or 1. The preset brain map template can be the AAL90 brain map template. m can be determined based on the number of brain regions in the preset brain map template. For example, m for the ALL90 brain region template is 90.

[0122] In an exemplary embodiment, determining the KL divergence between any two brain regions based on the values ​​of n voxels in each of the two brain regions may include: calculating the probability density function of each brain region based on the values ​​of n voxels in the brain region, and calculating the KL divergence between the two brain regions based on the probability density functions of any two brain regions.

[0123] For example, assume that the KL divergence D between the probability density function of brain region Q and the probability density function of brain region P is KL (P||Q) is defined as:

[0124] Where i represents the number of pixels. P(i) represents the probability density function of brain region P calculated based on the volume values ​​of the i-th pixel in the brain region; Q(i) represents the probability density function of brain region Q calculated based on the volume values ​​of the i-th pixel in the brain region.

[0125] However, D KL (P||Q) and D KL (Q||P) are not equal, so for symmetric measurement, we need to KL (P||Q) and D KL (Q||P) is summed, that is, the KL divergence D between the probability density function of brain area Q and the probability density function of brain area P KL (P,Q) is defined as:

[0126] In step 25, in an exemplary embodiment, before constructing the m×m morphological connectivity matrix based on the KL divergence between all arbitrary two brain regions, the method includes: transforming the KL divergence between all arbitrary two brain regions into the range of 0 to 1;

[0127] That is, the KL divergence between the P and Q brain areas is transformed to the range of 0 to 1 by the following formula:

[0128] Among them, KLS(P,Q)=1 means that the distribution of brain area P and brain area Q is the same.

[0129] The matrix element S in the m×m morphological connectivity matrix ij represents the KL divergence between brain regions i and j calculated according to formula (2) or formulas (2) and (3). The values ​​of i and j are less than or equal to n and greater than or equal to 1, respectively. The KL divergence of self-connection is set to 0.

[0130] In an exemplary embodiment, a preset threshold is set to convert the m×m morphological connectivity matrix into a binary matrix, that is, the value of the matrix is ​​1 or 0, for obtaining network features.

[0131] In step 26, in an exemplary embodiment, constructing a brain map using each brain region as a node according to the morphological connectivity matrix may be as shown in FIG3 , which includes the following steps 261-265:

[0132] Step 261, calculating the network characteristics of each node according to the binary matrix;

[0133] Step 262, calculating the KL divergence between each node and other nodes;

[0134] Step 263, determining a feature vector of each node based on the KL divergence between each node and other nodes and the network characteristics of each node;

[0135] Step 264 , using the Euclidean distance between the center point of each brain region and the center points of other brain regions as an edge; constructing an adjacency matrix of the nodes corresponding to each brain region based on the edge of each brain region;

[0136] Step 265: construct a brain map based on the feature vector of each node and the adjacency matrix.

[0137] In step 261 , the network characteristics may include degree, clustering coefficient, degree centrality, eigenvector centrality, etc.

[0138] Degree represents the number of nodes directly connected to a node. The size of the degree reflects the importance of the node in the entire graph.

[0139] The clustering coefficient refers to the ratio of the actual number of edges between a node's neighboring nodes to the maximum possible number of edges. The higher the clustering coefficient, the closer the connection between the nodes.

[0140] Degree Centrality means that the larger the degree of a node, the higher the degree centrality of the node, and the more important the node is in the graph.

[0141] Eigenvector centrality means that the importance of a node depends on the number of its neighboring nodes, that is, the degree of the node, and also on the importance of its neighboring nodes. In other words, eigenvector centrality expands degree centrality by incorporating the importance of neighboring nodes.

[0142] In step 262 , calculating the KL divergence between each node and other nodes is the same as calculating the KL divergence between brain regions.

[0143] That is, the KL divergence between the brain regions corresponding to the nodes can be used as the KL divergence between the nodes.

[0144] In step 263, the KL divergence between each node and every other node does not include the KL divergence of self-connections. The m×m morphological connectivity matrix becomes (m-1)×(m-1). The KL divergence between each m-1-dimensional node and every other node is concatenated with the node's degree, clustering coefficient, degree centrality, and eigenvector centrality to obtain an m+q-1-dimensional feature, which is then used as the node's eigenvector.

[0145] The feature vector of a node is a representation of the node feature of the node.

[0146] For example, the adjacency matrix can be expressed as

[0147] Among them, d i,j Represents the Euclidean distance between nodes i and j.

[0148] Network characteristics may also include betweenness centrality, closeness centrality, etc.

[0149] The present disclosure also provides a method for training a brain disease diagnosis model, as shown in FIG4 , including the following steps 41 to 43:

[0150] Step 41, obtaining a plurality of labeled brain medical image data;

[0151] Step 42, preprocessing each of the medical image data according to the aforementioned preprocessing method to obtain a brain tissue image and a brain map corresponding to each of the medical image data;

[0152] Step 43 , training the brain disease diagnosis model to be trained according to the brain tissue images, brain maps and annotations corresponding to each of the medical imaging data and the initial parameters of the brain disease diagnosis model to be trained until a preset stopping condition is reached.

[0153] In the disclosed embodiment, by preprocessing the medical imaging data of the brain according to the aforementioned preprocessing method, the brain disease diagnosis model to be trained is trained based on the standardized brain tissue images and the corresponding brain maps and annotations, so that the fully trained brain disease diagnosis model can solve the technical problem of redundant image background and other interference information when using deep learning technology to diagnose diseases with magnetic resonance imaging (MRI) image data.

[0154] In an exemplary embodiment, the training of the brain disease diagnosis model to be trained includes:

[0155] Repeat the following operations L times: extracting geometric texture features of the brain tissue image from each brain tissue image through a first preset operation; extracting topological relationship features between brain regions from a brain map corresponding to the brain tissue image through a second preset operation; updating feature vectors of nodes according to the geometric texture features and the topological relationship features; updating the brain tissue image with a brain tissue image containing the geometric texture features; and updating the brain map with a brain map containing the updated feature vectors of the nodes; wherein L is a natural number;

[0156] The diagnosis result is obtained based on the feature vector obtained after L operations;

[0157] The initial parameters of the brain disease diagnosis model are updated according to the diagnosis result and the preset loss function, and the brain disease diagnosis model with the updated initial parameters is retrained for the next time.

[0158] In the disclosed embodiment, during the training of the brain disease diagnosis model, the medical imaging data of the brain are preprocessed to obtain brain tissue images and brain maps, geometric texture features are extracted from the brain tissue images, topological relationship features between brain regions are extracted from the brain maps, and the geometric texture features and topological relationship features are fused with the feature vectors of the brain map nodes, so that the features of the brain map nodes have both local geometric texture features and global topological features, thereby improving the diagnostic performance of the brain disease diagnosis model for brain diseases and solving the technical problem of missing topological position relationships between image blocks when using deep learning technology to diagnose diseases with MRI imaging data.

[0159] In an exemplary embodiment, L may be 2 or other values.

[0160] In an exemplary embodiment, the first preset operation includes: t convolution combination operations and pooling operations; t is a natural number; and the convolution combination includes Conv, BN, and ReLU.

[0161] In an exemplary embodiment, t may be 3.

[0162] In an exemplary embodiment, the second preset operation includes: a graph convolution operation.

[0163] Graph convolution operations can be implemented through graph convolutional neural networks.

[0164] For the x-th graph convolution operation, that is, the feature matrix H output by the x-th layer of the graph convolution layer x+1 Can be: H x+1 =ReLU(A x H x W x ) (5);

[0165] Among them, Wx ∈R (m+q-1)×(m+q-1) Is the coefficient matrix of graph convolution. A x is the adjacency matrix corresponding to the xth layer determined according to formula (4). x is the feature matrix output by the (x-1)th layer of the graph convolutional layer, where x≤L.

[0166] In an exemplary embodiment, updating the feature vector of the node according to the geometric texture feature and the topological relationship feature includes:

[0167] The brain tissue image containing geometric texture features is processed; the processed geometric texture features and topological relationship features corresponding to each brain region are fused to the feature vector of the node corresponding to the brain region to update the feature vector of the node.

[0168] In an exemplary embodiment, processing the brain tissue image containing geometric texture features includes:

[0169] Using a preset brain atlas template, the brain tissue image containing geometric texture features is divided into m blocks, each block corresponding to a brain region;

[0170] Obtain geometric texture features of each brain region;

[0171] The geometric texture features of each brain region are flattened and then integrated through a shared multi-layer perceptron.

[0172] In an exemplary embodiment, fusing the processed geometric texture features and topological relationship features corresponding to each brain region into the feature vector of the node corresponding to the brain region to update the feature vector of the node includes:

[0173] The integrated features are spliced ​​onto the feature vector of the corresponding node and the geometric texture features and topological relationship features of the node are fused with the feature vector of the node through a shared multi-layer perceptron to update the feature vector of the node.

[0174] In an exemplary embodiment, obtaining a diagnosis result based on a feature vector obtained after L operations includes:

[0175] The feature vector obtained after L operations is transformed by a preset aggregation function to obtain a transformed feature vector;

[0176] Inputting the transformed feature vector into a fully connected layer;

[0177] The output value of the fully connected layer is normalized to obtain the diagnosis result.

[0178] Exemplarily, the preset aggregation function may be aggregation functions such as sum, average, and maximum.

[0179] Exemplarily, the number of fully connected layers may be 3.

[0180] Exemplarily, the normalization may be softmax.

[0181] FIG5 shows a schematic diagram of a brain disease diagnosis model according to an embodiment of the present disclosure. BN and ReLU in the convolution combination are omitted in FIG5 . In FIG5 , geometric texture features are extracted from the brain tissue image through the convolution module, and topological relationship features are extracted through the graph convolution module. The extracted geometric texture features are processed and fused together with the topological relationship features to the node features of the original node to update the node features. Then, geometric texture features are extracted again from the brain tissue image containing geometric texture features, and topological relationship features are extracted again from the brain map containing topological relationship features. The extracted geometric texture features are processed and fused together with the topological relationship features to the node whose node features were updated last time, and the node features are updated again. Finally, the overall feature vector of the node features is obtained through the graph readout operation (i.e., the readout operation), and after the fully connected hierarchy (FC), the diagnosis result is obtained by softmax (i.e., the above-mentioned normalization) and output.

[0182] In an embodiment of the present disclosure, an iterative training method is used to train the brain disease diagnosis model to be trained. By iteratively using training data (i.e., a plurality of labeled brain medical imaging data) for training, each iteration updates the parameters of the model to gradually improve performance. An iterative cycle includes the process of calculating the diagnosis result, calculating the loss function (Loss Function) to measure the difference between the diagnosis result and the true value (i.e., the annotation), and updating the network model parameters. Iterative training gradually optimizes the model through multiple iterations until a preset stopping condition is reached, such as convergence or the maximum number of iterations is reached. In this example, the loss function can be a cross-entropy loss function, which converges faster in the early stages of training.

[0183] In an exemplary embodiment, methods such as self-attention mechanism and skip connection can be added to extract the geometric texture features of the brain tissue image.

[0184] In an exemplary embodiment, the size of the preset brain atlas template is adjusted using the nearest neighbor interpolation method and scaled to the same size as the brain tissue image containing geometric texture features to facilitate brain region correspondence.

[0185] In an exemplary embodiment, an application scenario is provided in which the brain disease diagnosis model can be pre-installed in a computer device, which can be located in a hospital or research institute. To facilitate user use, an operation interface can be designed to receive medical imaging data input by the user and a brain disease diagnosis instruction input by the user on the operation interface. The medical imaging data input by the user is pre-processed and then input into the brain disease diagnosis model. The diagnosis result is output from the brain disease diagnosis model for doctors in the hospital or researchers in the research institute to perform the next step. For example, the diagnosis result can be used by doctors to diagnose and treat diseases. The diagnosis result obtained using the brain disease diagnosis method of this embodiment is more accurate and more convenient for doctors or researchers to perform the next step.

[0186] The embodiments of the present disclosure also provide a non-volatile computer-readable storage medium storing program instructions, which, when executed, can implement any of the above-mentioned preprocessing methods or the training method of the brain disease diagnosis model or the brain disease diagnosis method.

[0187] An embodiment of the present disclosure also provides an electronic device, comprising a processor and a memory storing a computer program that can be run on the processor, wherein when the processor executes the program, it implements the steps of any of the above-mentioned preprocessing methods or the training method of the brain disease diagnosis model or the brain disease diagnosis method.

[0188] In an exemplary embodiment, Figure 6 is a schematic diagram of the structure of an electronic device in an embodiment of the present disclosure. As shown in Figure 6, the device 60 includes: at least one processor 601; at least one memory 602 and a bus 603 connected to the processor 601; wherein the processor 601 and the memory 602 communicate with each other via the bus 603; the processor 601 is configured to call program instructions in the memory 602 to execute the steps of the classification model generation method or classification method in any of the above embodiments.

[0189] The processor can be a central processing unit (CPU), a microprocessor (MPU), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), a transistor logic device, etc., and this disclosure does not limit this.

[0190] Memory can include read-only memory (ROM) and random access memory (RAM), and provides instructions and data to the processor. Some memory may also include non-volatile random access memory. For example, the memory may also store device type information.

[0191] In addition to the data bus, the bus may also include a power bus, a control bus, a status signal bus, etc. However, for the sake of clarity, various buses are all labeled as buses in FIG6 .

[0192] During implementation, the processing performed by the processing device can be completed by hardware integrated logic circuits in the processor or by instructions in the form of software. That is, the method steps of the embodiments of the present disclosure can be embodied as being executed by a hardware processor, or by a combination of hardware and software modules in the processor. The software module can be located in a storage medium such as random access memory, flash memory, read-only memory, programmable read-only memory, or electrically erasable programmable memory, registers, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware. To avoid repetition, it will not be described in detail here.

[0193] It will be appreciated by those skilled in the art that all or some of the steps, systems, and functional modules / units in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In hardware implementations, the functional modules / units mentioned in the above description are not divided equally; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all components may be implemented as software executed by a processor, such as a digital signal processor or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or temporary medium). As is well known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable, and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, it is well known to those skilled in the art that communication media generally embodies computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0194] Although the embodiments disclosed in this disclosure are as described above, the contents described are merely embodiments adopted to facilitate understanding of the disclosure and are not intended to limit the disclosure. Any person skilled in the art to which the disclosure belongs may make any modifications and changes in the form and details of the implementation without departing from the spirit and scope of the disclosure. However, the scope of patent protection of the disclosure shall still be based on the scope defined by the attached claims.

Claims

1. A method for diagnosing a brain disease, comprising: Acquiring medical imaging data of the brain; Preprocessing the medical imaging data to obtain a brain tissue image and a brain map; wherein the brain map includes a plurality of nodes, each node having a node feature; and the nodes correspond one to one with brain regions; The brain tissue image and brain map are respectively input into a pre-trained brain disease diagnosis model, wherein the brain disease model includes at least one convolution module, at least one graph convolution module, at least one fusion module, and an output module; the geometric texture features of the brain tissue image are extracted by the convolution module, and the topological relationship features between brain regions are extracted from the brain map by the graph convolution module; the geometric texture features and the topological relationship features are fused with the node features to obtain new node features by the fusion module; and the brain disease corresponding to the medical imaging data is determined by the output module based on the new node features and the brain disease is output.

2. The method for diagnosing a brain disease according to claim 1, wherein: The step of fusing the geometric texture feature and the topological relationship feature to the original node feature to obtain a new node feature by the fusion module includes: A first processing is performed on the brain tissue image containing the geometric texture features to obtain geometric texture features corresponding to each brain region; and the geometric texture features and topological relationship features corresponding to each brain region are fused to the node features corresponding to the brain region to obtain new node features.

3. The method for diagnosing a brain disease according to claim 2, wherein: The first processing of the brain tissue image containing geometric texture features includes: Using a preset brain atlas template, the brain tissue image containing geometric texture features is divided into multiple blocks, each block corresponding to a brain region; The geometric texture features of each brain region are flattened and integrated.

4. The method for diagnosing a brain disease according to claim 3, wherein: The step of fusing the geometric texture features and topological relationship features corresponding to each brain region to the node features corresponding to the brain region to obtain new node features includes: The integrated geometric texture features are spliced onto the corresponding node features, and the geometric texture features and topological relationship features of the node are fused with the node features of the node through a second shared multi-layer perceptron to obtain new node features.

5. The method for diagnosing a brain disease according to claim 1, wherein: Determining the brain disease corresponding to the medical image data according to the new node feature includes: Transforming the new node features by a preset aggregation function; Inputting the transformed node features into the fully connected hierarchy; The brain disease corresponding to the medical image data is determined according to the output value of the fully connected layer.

6. The method for diagnosing a brain disease according to claim 1, wherein: The preprocessing of the medical imaging data to obtain a brain tissue image and a brain map includes: Obtaining a brain tissue image by performing a second processing on the medical image data; Segmenting the brain tissue image using a preset method to obtain a tissue probability image; obtaining a gray matter volume image based on the tissue probability image; dividing the gray matter volume image into a plurality of brain regions using a preset brain atlas template, determining the KL divergence between any two brain regions based on the values of a plurality of voxels in the gray matter volume images of the two brain regions, and constructing a connectivity matrix based on the KL divergence between all any two brain regions; Each brain region is regarded as a node, and a brain map is constructed according to the connectivity matrix.

7. The method for diagnosing a brain disease according to claim 6, wherein: The second processing sequentially includes image rectification, reconfiguration of scanning axis, cropping of part of the image, removal of skull and first spatial registration; The first spatial registration includes registering the brain tissue image after the skull is removed to the MNI standard space using a preset registration method.

8. The method for diagnosing a brain disease according to claim 7, wherein: The step of obtaining a gray matter volume image according to the gray matter probability image includes: performing a second spatial registration on the gray matter probability image, and obtaining a gray matter volume image based on the deformation field parameters obtained in the second spatial registration process and the gray matter probability image; The performing a second spatial registration on the gray matter probability image includes: The gray matter probability image is registered to the MNI standard space using a preset registration method.

9. The method for diagnosing a brain disease according to claim 8, wherein: The step of obtaining a gray matter volume image based on the deformation field parameters obtained in the second spatial registration process and the tissue probability image includes: multiplying the tissue probability image by the nonlinear part of the deformation field parameter to obtain a relative gray matter volume image; The relative gray matter volume image is smoothed to obtain a gray matter volume image.

10. The brain disease diagnosis method according to claim 6, wherein: Determining the KL divergence between any two brain regions based on the values of n voxels in each of the two brain regions includes: The probability density function of each brain region is calculated based on the values of n voxels in the region, and the KL divergence between any two brain regions is calculated based on the probability density functions of the two brain regions.

11. The method for diagnosing a brain disease according to claim 6, wherein: The connectivity matrix is transformed into a binary matrix according to a preset threshold.

12. The brain disease diagnosis method according to claim 11, wherein: The constructing of a brain map according to the connectivity matrix comprises: Calculate the network characteristics of each node according to the binary matrix; Calculate the KL divergence between each node and other nodes; Determine the node characteristics of each node according to the KL divergence between each node and other nodes and the network characteristics of each node; The Euclidean distance between the center point of each brain region and the center points of other brain regions is used as an edge; the adjacency matrix of the corresponding nodes of each brain region is constructed based on the edge of each brain region; A brain map is constructed according to the node features of each node and the adjacency matrix.

13. The method for diagnosing a brain disease according to claim 1, wherein: The acquiring of medical imaging data of the brain includes: receiving medical imaging data input by a user; Before obtaining a brain disease by using the brain disease diagnosis method according to any one of claims 1 to 12 on the medical image data, the method further comprises: receiving a brain disease diagnosis instruction input by a user on an operation interface; The outputting of the brain disease includes: obtaining the name of the brain disease diagnosed from the medical imaging data.

14. A non-transitory computer-readable storage medium storing program instructions, which can implement the method according to any one of claims 1 to 13 when the program instructions are executed.

15. An electronic device comprising a processor and a memory storing a computer program executable on the processor, wherein: When the processor executes the program, the steps of the method according to any one of claims 1 to 13 are implemented.

Images