Knowledge and data-driven brain network computing method, device, electronic device and storage medium

Abstract

The embodiment of the present invention discloses a knowledge and data-driven brain network calculation method and device, the method comprising: obtaining a sample brain image, constructing a brain region mask based on prior knowledge, locating each brain region in the brain image, and obtaining an image of each brain region in the brain image; extracting the topological features of each brain region image through multiple feature extraction modules based on spatial attention; learning the connection relationship between the topological features of each brain region image, and constructing a brain network based on the topological features and the connection relationship, wherein the brain network is composed of nodes and edges, wherein the nodes represent each brain region, and the edges represent the brain connection strength between each brain region; predicting the disease category of the sample based on the brain network, obtaining disease-related information to constrain the distribution of the brain network, so that the brain network contains more disease features and connections, and optimizing the brain network. The present invention solves the problems of existing brain network calculation methods, such as small scope of application, poor compatibility, strong subjectivity, high computational cost, low efficiency and accuracy.

Description

Technical Field

[0001] The present invention relates to the field of medical image processing technology, and in particular to a knowledge and data-driven brain network computing method, device, electronic device and storage medium. Background Art

[0002] Brain networks that contain disease-related characteristics and connections not only help improve the diagnosis of neurodegenerative diseases, but also provide the prerequisites for analyzing abnormal brain network connections and changing trends at different stages of the disease. For example, patients with early Alzheimer's disease will show changes in brain connections, which are reflected by structural magnetic resonance imaging technology. Clinical brain imaging examinations are now mainly used to find visible abnormalities in diseases other than Alzheimer's disease, excluding interference from other diseases in diagnosis, and cannot achieve accurate diagnosis. In contrast, using brain networks can provide more accurate results because brain networks represent a comprehensive map of neural activity.

[0003] However, existing related technologies can only construct a single brain network, with a small scope of use, which is not conducive to popularization and use. Existing brain network construction methods rely on software templates and manually set different preprocessing parameters to construct the connection relationship between brain regions. They contain a large amount of redundant connection information and do not highlight disease-related abnormal brain connections and characteristics. They are highly subjective, resulting in large calculation errors of brain connections and low accuracy of disease prediction.

[0004] In addition, existing brain network construction methods require complex brain image preprocessing and brain connection calculation processes, which have high time and computational costs and low efficiency.

[0005] Therefore, there is an urgent need for a knowledge- and data-driven brain network calculation method with wide applicability, strong compatibility, strong objectivity, low computational cost, high efficiency and accuracy. Summary of the Invention

[0006] The embodiments of the present invention provide a knowledge and data-driven brain network computing method, device, electronic device and storage medium to solve the problems existing in related technologies such as small scope of application, poor compatibility, strong subjectivity, high computing cost, and low efficiency and accuracy.

[0007] Wherein, the technical solution adopted by the present invention is:

[0008] According to one aspect of the present invention, a knowledge and data-driven brain network calculation method includes: obtaining brain images of samples, constructing brain region masks based on prior knowledge, locating each brain region in the brain image, and obtaining images of each brain region in the brain image; extracting topological features of each brain region image through multiple spatial attention-based feature extraction modules; learning the connection relationship between the topological features of each brain region image, and constructing a brain network based on the topological features and connection relationship, wherein the brain network is composed of nodes and edges, the nodes represent each brain region, and the edges represent the brain connection strength between each brain region; predicting the sample disease category based on the brain network to obtain disease-related information, and constraining the distribution of the brain network based on the disease-related information so that the brain network contains more disease features and connections, and optimizing the brain network.

[0009] According to one aspect of the present invention, a knowledge and data-driven brain network computing device comprises: a brain region perception module based on anatomical knowledge, for constructing a brain region mask according to prior knowledge, locating each brain region in a brain image, and obtaining an image of each brain region in the brain image; a brain disease topological feature extraction module, for extracting the topological features of each brain region image through a plurality of feature extraction modules based on spatial attention; a brain connection learning module, for learning the connection relationship between the topological features of each brain region image, and constructing a brain network based on the topological features and the connection relationship; a brain disease knowledge learning module, for predicting the sample disease category according to the brain network, obtaining disease-related information, constraining the distribution of the brain network, making the brain network contain more disease features and connections, and optimizing the brain network.

[0010] According to one aspect of the present invention, an electronic device includes a processor and a memory, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the knowledge- and data-driven brain network computing method as described above is implemented.

[0011] According to one aspect of the present invention, a storage medium stores a computer program thereon, which implements the knowledge and data driven brain network computing method as described above when executed by a processor.

[0012] According to one aspect of the present invention, a computer program product includes a computer program, the computer program is stored in a storage medium, a processor of a computer device reads the computer program from the storage medium, and the processor executes the computer program, so that the computer device implements the knowledge and data-driven brain network computing method as described above when executing the computer program.

[0013] In the above technical solution, the present invention realizes a knowledge and data-driven brain network calculation method with wide applicability, high efficiency and accuracy.

[0014] Specifically, the present invention first uses anatomical knowledge to construct a learnable brain region mask and divides brain images into brain regions through a brain region perception module based on anatomical knowledge, thereby overcoming the difficulty in locating brain regions caused by different degrees of brain region atrophy during the pathological process of neurodegenerative diseases; then, the topological features of each brain region are obtained through a brain disease topological feature extraction module, and a spatial attention mechanism is used to perform secondary positioning of the brain region in the process of feature extraction, thereby refining the boundaries of each brain region and overcoming the problem of mismatch between brain regions and topological features; secondly, a brain connection learning module is used to learn the connection relationship between brain region topological features, and a complete brain network is constructed based on the extracted topological features and the learned connection relationship, effectively fitting the true distribution of the brain network and overcoming the problem of low efficiency and accuracy in the existing technology; finally, the learned brain network is used as the input of the brain disease knowledge learning module to optimize the brain network. The optimized brain network is input into the brain disease prediction module to obtain the disease prediction result, thereby realizing end-to-end generation of various types of brain networks from brain images of multiple modalities, with a wide range of applications, strong compatibility, strong objectivity and low computational cost.

[0015] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0017] Figure 1 is a flowchart of a knowledge and data driven brain network computing method according to an exemplary embodiment;

[0018] Figure 2 yes Figure 1 A schematic diagram of step 110 in one embodiment of the corresponding embodiment;

[0019] Figure 3 yes Figure 1 A schematic diagram of step 130 in one embodiment of the corresponding embodiment;

[0020] Figure 4 yes Figure 3 Schematic diagram of the structural flow of the feature extraction module in the corresponding embodiment;

[0021] Figure 5 yes Figure 1 A schematic diagram of step 150 in one embodiment of the corresponding embodiment;

[0022] Figure 6 yes Figure 1 A schematic diagram of step 170 in one embodiment of the corresponding embodiment;

[0023] Figure 7 is a flowchart illustrating a knowledge- and data-driven brain network computing method for training a disease prediction model according to an exemplary embodiment;

[0024] Figure 8 yes Figure 7 A structural flow chart of a disease prediction model trained based on a knowledge- and data-driven brain network computing method according to an embodiment;

[0025] Figure 9 is a block diagram of a knowledge and data driven brain network computing device according to an exemplary embodiment;

[0026] Figure 10 is a hardware structure diagram of an electronic device according to an exemplary embodiment;

[0027] Figure 11 It is a block diagram of an electronic device according to an exemplary embodiment.

[0028] The above-mentioned drawings have shown clear embodiments of the present invention, which will be described in more detail below. These drawings and textual descriptions are not intended to limit the scope of the present invention in any way, but rather to illustrate the concept of the present invention for those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0029] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. In the following description, when referring to the drawings, like numbers in different figures represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all possible embodiments consistent with the present invention. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present invention, as detailed in the appended claims.

[0030] Existing related technologies rely on various software toolkits, and one toolkit can only construct a brain network from brain images of a specific modality. It has a small scope of application, poor flexibility and is not conducive to popularization and use.

[0031] At the same time, existing related technologies not only need to rely on experience to set a large number of parameters, which is highly subjective, but the reconstructed brain network contains a large amount of redundant information and does not highlight abnormal brain connections, resulting in low disease prediction accuracy. In addition, complex preprocessing pipelines and calculation processes are required to reconstruct brain connections, which consumes a lot of time and computing resources.

[0032] From the above, we can see that the relevant technologies still have defects such as small scope of application, poor compatibility, strong subjectivity, high calculation cost, and low efficiency and accuracy.

[0033] To this end, the knowledge and data-driven brain network calculation method provided in this application uses prior knowledge to construct a learnable brain region mask in the brain region perception module based on anatomical knowledge to fuzzy locate each brain region in the brain image, thereby overcoming the difficulty in locating the brain region caused by different degrees of brain region atrophy during the disease pathological process; in the brain disease topological feature extraction module, a lightweight convolutional neural network is used to extract the topological features of the brain region, and in the process of feature extraction, a spatial attention mechanism is used to perform secondary positioning of the brain region, refine the boundaries of each brain region, and ensure that the extracted topological features come from specific brain regions; in the brain connection learning module, a generative learning strategy is used to learn the connections between the topological features of the brain regions. The invention improves the accuracy of brain network construction from multiple aspects, thereby improving the accuracy of disease prediction. The knowledge and data-driven brain network calculation method is applicable to disease prediction models and knowledge and data-driven brain network computing devices. The disease prediction models and knowledge and data-driven brain network computing devices can be deployed on electronic devices configured with von Neumann architecture. For example, the electronic device can be a desktop computer, a laptop computer, a server, etc.

[0034] See also Figure 1 , an embodiment of the present application provides a knowledge and data-driven brain network computing method, which is applicable to electronic devices, for example, the electronic device can be a desktop computer, a laptop computer, a server, etc.

[0035] In the following method embodiments, for ease of description, the execution subject of each step of the method is taken as an electronic device as an example for illustration, but this does not constitute a specific limitation.

[0036] like Figure 1 As shown, the method may include the following steps:

[0037] Step 110 , obtaining a sample brain image, constructing a brain region mask based on prior knowledge, locating each brain region in the brain image, and obtaining an image of each brain region in the brain image.

[0038] Specifically, the center point coordinates and relative volume sizes of the images of each brain region in the brain image are obtained based on prior knowledge, the masks of the images of each brain region are constructed based on the center point coordinates and relative volume sizes of the images of each brain region, and the images of each brain region in the brain image are obtained based on the brain image and the masks of the images of each brain region.

[0039] In a possible implementation, the prior knowledge represents anatomical knowledge, physiological knowledge related to the human brain, etc., which is not limited here.

[0040] In one possible implementation, see Figure 2 ,According to anatomical knowledge, the center point coordinates and relative volume sizes of ,N brain regions are input into the anatomical knowledge based ,brain region perception module, a mask of each brain region is constructed, and ,the brain image and the mask are element-wise multiplied to obtain the image of ,each brain region.

[0041] The formula for constructing the brain region mask is as follows:

[0042]

[0043] Among them, ROI i Represents the mask of the i-th brain region, X, Y, Z represent the coordinates of the three dimensions respectively, (x i ,y i ,z i ,v i ) represents the anatomical knowledge of the i-th brain region, θ i Indicates the construction of ROI i The learnable parameters.

[0044] The formula for obtaining brain region images is as follows:

[0045] dti i =DTI×ROI i

[0046] Among them, dti i Represents the image data of the i-th brain region.

[0047] After the above process, a learnable brain region mask is used to fuzzy locate each brain region in the brain image, overcoming the difficulty of brain region positioning caused by different degrees of brain region atrophy during the disease process, ensuring that the subsequently extracted topological features correspond one-to-one with the brain region, and thereby improving the accuracy of brain network construction.

[0048] Step 130 : extracting topological features of each brain region image through multiple spatial attention-based feature extraction modules.

[0049] Specifically, convolutional layers are used to extract simple primary topological features, the number of channels is expanded and the size of each brain region image is reduced, and multiple spatial attention-based feature extraction modules are used to extract the topological features of each brain region image step by step.

[0050] In one possible implementation, see Figure 3Since the brain region image data after preliminary brain region positioning is a 3D single-channel image, we first use two convolutional layers to extract simple primary topological features, expand the number of channels and reduce the size of the brain region image, and use M spatial attention-based feature extraction modules (SAFE-block) as the core modules to extract the topological features of the white matter fiber bundles of each brain region step by step.

[0051] Among them, the structural design process of the feature extraction module (SAFE-block) is as follows Figure 4 As shown, F in ∈R C×H×W×D For the input feature map, H×W×D is the size of the feature map, and C is the number of channels of the feature map. First, a convolutional layer Point-Wise Conv layer is used to reduce the number of channels, and then a convolutional layer Depth-Wise Conv layer is used to extract the features of each channel to obtain the feature map F1∈R C / 2×H×W×D , then introduce the spatial attention layer to accurately locate the brain area, perform global average pooling (AvgPool) and maximum pooling (MaxPool) on the feature map F1 in the channel dimension to obtain two 1×H×W×D channel representations, and splice these two channel representations together, and then pass through a convolution layer Conv layer with a Sigmoid activation function to obtain the attention map a, and multiply the attention map a and F1 to obtain the feature map F2∈R C / 2×H×2×D , then use the Point-WiseConv layer to restore the number of channels to C, and obtain the feature map F3∈R through the jump connection C×H×W×D , speed up the efficiency of forward propagation of information and alleviate the gradient vanishing problem in error back propagation, and use the Conv layer to adjust the number of channels and the scale of the feature map to obtain the output feature map F out ∈R C′×H′×W′×D′ , use a Point-Wise Conv layer to fuse and summarize the information of all channels to obtain a single-channel feature map F i , F i Flattened as the topological feature v of the i-th brain region i ∈R 1×d , d represents the feature dimension, and the above process is repeated to obtain the topological features P∈R of all brain regions N×d .

[0052] After the above process, a lightweight convolutional neural network is used to extract the topological features of the brain regions step by step. In the process of feature extraction, the spatial attention mechanism is used to perform secondary positioning of the brain regions, refine the boundaries of each brain region, and ensure that the extracted topological features come from specific brain regions, thereby improving the accuracy of brain network construction.

[0053] Step 150 : learning the connection relationship between the topological features of the images of various brain regions, and constructing a brain network based on the topological features and the connection relationship.

[0054] Specifically, if Figure 5 As shown in the figure, the brain network is composed of nodes and edges, where nodes represent independent brain regions and edges represent the strength of brain connections between brain regions. The brain network can be represented by a graph G = (A, P), which includes two elements: brain connection A and topological feature P.

[0055] In one possible implementation, a learnable parameter matrix M∈R is used d×d and the nonlinear mapping function e -|·| From the topological features P∈R N×d Learning brain connections A∈R N×N .

[0056] The specific formula for learning brain connections is as follows:

[0057]

[0058] Among them, the learnable parameter matrix M can be implemented using a fully connected layer without a bias term. The number of neurons in the input layer and the output layer is d. Through this calculation method, it can be guaranteed that the learned brain connections must be non-negative and meet the symmetry conditions, which is consistent with the physical meaning of brain connections, thereby improving the accuracy of brain network construction.

[0059] Through the above process, a generative learning strategy is used to learn the connection relationship between the topological features of brain regions, and a complete brain network is constructed based on the extracted topological features and the learned connection relationship. When generating the brain network, not only the topological features of each brain region are extracted to form a feature matrix, but also the brain connection matrix is ​​learned based on the topological features. This provides a basis for the subsequent analysis of brain network abnormalities in the disease process from the two perspectives of topological features and brain connections, thereby improving the accuracy of disease prediction.

[0060] Step 170 , predicting the disease category of the sample based on the brain network, obtaining disease-related information, and constraining the distribution of the brain network based on the disease-related information so that the brain network contains more disease features and connections, thereby optimizing the brain network.

[0061] Specifically, high-dimensional topological features in the brain network are obtained, and multiple high-dimensional topological features are spliced ​​to obtain classification features. Disease categories of samples are predicted based on the classification features to obtain rich disease-related information, guiding knowledge and data-driven brain network computational models to generate brain networks that are more in line with the real distribution, so that the brain network contains more disease features and connections, and the brain network is optimized.

[0062] In one possible implementation, such as Figure 6As shown in the figure, the generated brain network is optimized by three sub-modules: parallel multi-graph convolutional layer (Multi-GCN), average pooling layer and fully connected layer.

[0063] Among them, a multi-graph convolutional layer (Multi-GCN) was designed based on the idea that convolutional neural networks (CNNs) use multiple filters to capture multiple feature spaces. K parallel double-layer GCNs are used to simultaneously capture the potential complex interaction features in the brain network.

[0064] The specific calculation formula is as follows:

[0065]

[0066] in, represents the interaction feature matrix captured by the i-th double-layer GCN, and They represent the parameter matrices to be trained in the two-layer GCN, and ReLU is the activation function of each layer of GCN.

[0067] Among them, since the topological features of the brain network after Multi-GCN filtering have a high dimension, directly flattening the topological features as the input of the fully connected layer will result in a large number of parameters to be trained in the classifier, increasing the training difficulty and computational cost. Therefore, the average pooling layer is used to reduce the dimension of the features. The calculation process is as follows:

[0068]

[0069] in, Indicates P i The high-order topological features obtained after average pooling, j represents the jth brain region.

[0070] Furthermore, the classification features are obtained by concatenating K high-order topological features Input a fully connected layer for classification. The number of neurons in the input layer is 3d2, and the number of neurons in the output layer is the number of disease types C (C=5).

[0071] After obtaining the truly distributed brain network, the generated brain network can be used to analyze the mechanism of the disease and develop biomarkers, which can highlight abnormal brain connections and characteristics related to the disease. This is conducive to analyzing the abnormal brain network connections and change patterns in the pathological process of neurodegenerative diseases, and lays a solid foundation for the intelligent detection and tracing of neurodegenerative diseases.

[0072] Furthermore, a disease prediction module is used to predict the disease category of the sample based on the optimized brain network and disease-related information to obtain a disease prediction result for the sample. The disease prediction result can be used for clinical diagnosis. The disease prediction module uses a graph pooling strategy to summarize the high-order topological features of the brain network, which greatly improves the classification accuracy of neurodegenerative diseases and lays the foundation for early screening of the disease and individual precision intervention. The topological structure loss between the brain connection generated by the joint brain connection learning module and the empirical brain connection obtained by the software template jointly guides the training of the model, so that disease-related knowledge runs through the entire brain network construction process, which provides convenience for analyzing the pathogenic mechanism of neurodegenerative diseases from the brain network level. The present invention simultaneously realizes brain network generation analysis and disease prediction functions.

[0073] Through the above process, the embodiment of the present invention provides a knowledge and data-driven brain network calculation method, which accurately locates brain regions in brain images and extracts topological features of brain diseases based on anatomical knowledge and spatial attention mechanisms. It then combines generative learning strategies to construct brain networks and mine abnormal brain connections and features related to brain diseases, laying the foundation for early screening and precise personalized intervention of neurodegenerative diseases. It is applied to the generation of structural brain networks, abnormal brain connection analysis, and disease prediction of neurodegenerative diseases. Therefore, the embodiment of the present invention can achieve brain network generation analysis and disease prediction with a wide range of applications, high efficiency, and high accuracy.

[0074] See also Figure 7 , an embodiment of the present application provides a training method for a disease prediction model based on a knowledge and data-driven brain computing method, which is applicable to electronic devices, for example, the electronic device can be a desktop computer, a laptop computer, a server, etc.

[0075] In the following method embodiments, for ease of description, the execution subject of each step of the method is taken as an electronic device as an example for illustration, but this does not constitute a specific limitation.

[0076] Each brain image and brain region anatomical knowledge (x, y, z, v) is input into the model, and a brain network is generated after passing through the anatomical knowledge-based brain region perception module, the brain disease topological feature extraction module and the brain connection learning module. The brain network is then input into the brain disease knowledge learning module to obtain the disease classification results and optimize the brain network. The training includes two loss functions and a training strategy.

[0077] Specifically, since neurodegenerative diseases can cause brain regions to atrophy to varying degrees, focusing only on the global topological structure of the brain network and ignoring the local topological structure will underestimate the role of specific brain regions in brain network generation. Therefore, the global and local topological structure differences between the brain connections generated by the model and the empirical brain connections are used as topological structure losses.

[0078] The formula for topology loss is as follows:

[0079] L topo =L g_topo +L p_topo .

[0080] Among them, L g_topo Represents the global topological structure loss, through the brain connection A and the experience brain connection The mean absolute error (MAE) between them is calculated.

[0081] The formula for mean absolute error is as follows:

[0082]

[0083] Among them, L p_topo Represents the local topological structure loss, through the brain connection A and the experience brain connection The closeness centrality (CCD) difference calculation L between nodes p_topo .

[0084] The specific calculation formula is as follows:

[0085]

[0086] Furthermore, the disease classification label l output by the brain disease knowledge learning module is compared with the real label The cross entropy loss between them is used as the classification loss to obtain disease-related information.

[0087] The specific formula for classification loss is as follows:

[0088]

[0089] Where n is the number of input samples, s i represents the brain image of the i-th sample, represents the actual distribution of sample labels, q(l|s i ) represents the distribution of disease labels predicted by the brain disease knowledge learning module.

[0090] In the present invention, all samples are divided into training set, validation set and test set in the ratio of 8:1:1. The model is trained on the training set, and then the optimal hyperparameters are selected through the validation set. Finally, the model is tested on the test set. The training process is divided into two steps. First, the anatomical knowledge-based brain region perception module, brain disease topology feature extraction module and brain connection learning module are fixed. The classification loss L is used to calculate the model. claThe parameters of the brain disease knowledge learning module are trained to ensure that the brain disease knowledge learning module can accurately classify the generated brain network and extract rich disease-related information. Then, the brain disease knowledge learning module is fixed and the topological structure loss L is reduced by the hyperparameter α. topo and classification loss L cla Weighted sum training is performed on the parameters of the anatomical knowledge-based brain region perception module, brain disease topological feature extraction module and brain connection learning module. The topological structure loss makes the brain connection distribution generated by the model gradually close to the empirical brain connection distribution, and the classification loss introduces disease-related knowledge to correct the topological characteristics and connection distribution of the brain network, so that the generated brain network contains more disease-related information and is closer to the real distribution.

[0091] The formula for weighted summation is as follows:

[0092] L G =α×L topo +(1-α)×L p_topo .

[0093] Among them, the hyperparameter α controls the degree of influence of disease-related knowledge on the brain network. The larger α is, the greater the influence of disease-related knowledge on the generation of brain network.

[0094] Furthermore, each module in the model reaches its optimal state during training. The brain region perception module based on anatomical knowledge can accurately locate each brain region, the brain disease topological feature extraction module can extract disease-related topological features, the brain connection learning module can generate brain connections that are more consistent with the actual distribution, and the brain disease knowledge learning module can predict diseases more accurately.

[0095] During the model testing phase, the brain imaging data to be tested is input into the model to obtain a brain network that is highly correlated with the disease, and a more accurate prediction is made of the disease stage of the test sample. By analyzing the brain network, abnormal characteristic indicators of the disease at different stages are calculated, such as changes in the number of increased and decreased brain connections, changes in brain connection strength, etc., to provide detection markers for early diagnosis of the disease.

[0096] At the same time, if Figure 8 As shown, the learned brain area masks, brain disease topological feature extraction module, brain connection learning module, and brain disease knowledge learning module are extracted and migrated to form an end-to-end disease prediction model based on knowledge and data-driven brain network computing methods.

[0097] See also Figure 9 In an exemplary embodiment, a knowledge and data driven brain network computing device 900 is provided.

[0098] The device 900 includes but is not limited to: a brain region perception module 910 based on anatomical knowledge, a brain disease topological feature extraction module 930, a brain connection learning module 950 and a brain disease knowledge learning module 970.

[0099] The brain region perception module 910 based on anatomical knowledge is used to construct a brain region mask according to prior knowledge, locate each brain region in the brain image, and obtain an image of each brain region in the brain image.

[0100] The brain disease topological feature extraction module 930 is used to extract the topological features of each brain region image through multiple spatial attention-based feature extraction modules.

[0101] The brain connection learning module 950 is used to learn the connection relationship between the topological features of each brain area image and construct a brain network based on the topological features and connection relationship.

[0102] The brain disease knowledge learning module 970 is used to predict the sample disease category based on the brain network, obtain disease-related information, constrain the distribution of the brain network, make the brain network contain more disease features and connections, and optimize the brain network.

[0103] It should be noted that the knowledge and data-driven brain network computing device provided in the above embodiments only uses the division of the above-mentioned functional modules as an example when performing brain network computing. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the knowledge and data-driven brain network computing device will be divided into different functional modules to complete all or part of the functions described above.

[0104] In addition, the knowledge and data-driven brain network computing device provided in the above embodiments belongs to the same concept as the embodiments of the knowledge and data-driven brain network computing method, in which the specific manner in which each module performs operations has been described in detail in the method embodiments and will not be repeated here.

[0105] Figure 10 The following is a schematic diagram of the structure of an electronic device according to an exemplary embodiment.

[0106] It should be noted that the electronic device is only an example adapted for this application and cannot be considered to provide any limitation on the scope of use of this application. The electronic device cannot be interpreted as needing to rely on or must have Figure 10 One or more components of exemplary electronic device 2000 are shown.

[0107] The hardware structure of the electronic device 2000 may vary greatly due to different configurations or performances, such as Figure 10As shown, the electronic device 2000 includes a power supply 210 , an interface 230 , at least one memory 250 , and at least one central processing unit (CPU) 270 .

[0108] Specifically, the power supply 210 is used to provide operating voltage for various hardware devices on the electronic device 2000 .

[0109] The interface 230 includes at least one wired or wireless network interface 231 for interacting with external devices.

[0110] Of course, in other examples adapted by this application, the interface 230 may further include at least one serial-to-parallel conversion interface 233, at least one input-output interface 235, and at least one USB interface 237, etc. Figure 10 As shown, this does not constitute a specific limitation.

[0111] The memory 250 serves as a carrier for resource storage and can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored thereon include an operating system 251, application 253 and data 255, etc. The storage method can be temporary storage or permanent storage.

[0112] Among them, the operating system 251 is used to manage and control the various hardware devices and application programs 253 on the electronic device 2000, so as to enable the central processing unit 270 to calculate and process the massive data 255 in the memory 250. It can be WindowsServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.

[0113] The application program 253 is a computer program that performs at least one specific task based on the operating system 251 and may include at least one module ( Figure 10 (not shown), each module can respectively include a computer program for the electronic device 2000. For example, the knowledge and data driven brain network computing device can be regarded as an application 253 deployed on the electronic device 2000.

[0114] The data 255 may be photos, pictures, etc. stored in a disk, or may be brain imaging data, etc., stored in the memory 250 .

[0115] The central processing unit 270 may include one or more processors and is configured to communicate with the memory 250 via at least one communication bus to read the computer programs stored in the memory 250, thereby performing operations and processing on the massive amount of data 255 in the memory 250. For example, the knowledge and data-driven brain network computing method can be implemented by the central processing unit 270 reading a series of computer programs stored in the memory 250.

[0116] In addition, the present application can also be implemented through hardware circuits or hardware circuits combined with software. Therefore, the implementation of the present application is not limited to any specific hardware circuits, software, or a combination of the two.

[0117] See also Figure 11 In an embodiment of the present application, an electronic device 4000 is provided. The electronic device 400 may include: a desktop computer, a laptop computer, a server, etc.

[0118] exist Figure 11 In the embodiment, the electronic device 4000 includes at least one processor 4001, at least one communication bus 4002 and at least one memory 4003.

[0119] The processor 4001 and the memory 4003 are connected, for example, via a communication bus 4002. Optionally, the electronic device 4000 may further include a transceiver 4004, which may be used for data exchange between the electronic device and other electronic devices, such as data transmission and / or data reception. It should be noted that in actual applications, the number of transceivers 4004 is not limited to one, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of the present application.

[0120] Processor 4001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It may implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 4001 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and the like.

[0121] The communication bus 4002 may include a path for transmitting information between the above components. The communication bus 4002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The communication bus 4002 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 11Only one thick line is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0122] The memory 4003 may be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited to these.

[0123] The memory 4003 stores a computer program, and the processor 4001 reads the computer program stored in the memory 4003 through the communication bus 4002 .

[0124] When the computer program is executed by the processor 4001 , the knowledge- and data-driven brain network computing method in the above-mentioned embodiments is implemented.

[0125] In addition, an embodiment of the present application provides a storage medium on which a computer program is stored. When the computer program is executed by a processor, the knowledge and data-driven brain network computing method in the above-mentioned embodiments is implemented.

[0126] In an embodiment of the present application, a computer program product is provided, comprising a computer program stored in a storage medium. A processor of a computer device reads the computer program from the storage medium and executes the computer program, causing the computer device to perform the knowledge- and data-driven brain network computing method described in each of the above embodiments.

[0127] Compared with the related art, the present invention has the following beneficial effects:

[0128] 1. This paper proposes a new knowledge- and data-driven brain network calculation method. Compared with existing technologies, this paper integrates anatomical knowledge with generative learning strategies to construct brain networks from brain images end-to-end, eliminating the tedious data preprocessing steps in existing methods, achieving a high degree of automation and improving the efficiency of brain network calculation.

[0129] 2. The present invention proposes a brain region perception module based on anatomical knowledge, which can realize the rapid positioning of brain regions, overcoming the positioning difficulties caused by different degrees of brain region atrophy during the pathological process of neurodegenerative diseases and the feature extraction difficulties caused by the small size of the brain region.

[0130] 3. This paper proposes a new method for extracting topological features of brain regions. It adds a spatial attention mechanism to a lightweight convolutional neural network to perform secondary positioning of brain regions, guiding the model to extract topological features in specific brain regions. Compared with existing brain region feature extraction methods, it establishes a stronger correspondence between topological features and brain regions.

[0131] 4. This invention proposes a new brain connectivity calculation method, which constructs the connection relationship between brain regions through a learnable network. Compared with existing statistical methods, it can directly and objectively reflect the actual connection between brain regions.

[0132] 5. This paper proposes a new disease knowledge learning module, which extracts rich disease-related features through a parallel multi-layer graph convolutional neural network and summarizes the features in combination with a graph pooling strategy, thereby improving the prediction accuracy of neurodegenerative diseases.

[0133] 6. The present invention uses topological structure loss to guide the model to generate a brain network that approximates the empirical distribution, and introduces disease-related knowledge through classification loss to correct the distribution of the brain network, making it closer to the actual distribution.

[0134] 7. The brain network generated by this invention contains more disease-related features and connections, and is more suitable for in-depth and personalized disease intelligent auxiliary diagnosis and treatment tasks such as pathological analysis and disease tracing.

[0135] 8. The present invention uses brain images as input, adopts a data-driven approach to learn the distribution of brain networks, and uses brain network analysis methods to mine abnormal brain connections and characteristics related to neurodegenerative diseases, which is beneficial for early intervention and treatment of patients. At the same time, the present invention is applicable to medical diagnosis and treatment scenarios of various brain diseases. By modifying the output parameters of the model and adding more modal data, various types of brain networks can be generated and applied to the predictive treatment of various neurodegenerative diseases.

[0136] It should be understood that although the steps in the flowcharts of the accompanying drawings are shown in sequence as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some of the steps in the flowcharts of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0137] The above content is only a preferred exemplary embodiment of the present invention and is not intended to limit the implementation scheme of the present invention. Ordinary technicians in this field can easily make corresponding changes or modifications based on the main concept and spirit of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection required by the claims.

Claims

1. A knowledge and data driven brain network computing method, characterized in that The method comprises: Acquire a brain image of a sample, construct a brain region mask based on prior knowledge, locate each brain region in the brain image, and obtain an image of each brain region in the brain image; Extracting topological features of each brain region image step by step through multiple spatial attention-based feature extraction modules; Learning the connection relationship between the topological features of the brain region images, and constructing a brain network based on the topological features and the connection relationship, wherein the brain network is composed of nodes and edges, wherein the nodes represent the brain regions and the edges represent the brain connection strength between the brain regions; The disease category of the sample is predicted according to the brain network to obtain disease-related information, the distribution of the brain network is constrained based on the disease-related information so that the brain network contains more disease features and connections, and the brain network is optimized, which includes obtaining high-dimensional topological features in the brain network; multiple high-dimensional topological features are spliced ​​to obtain classification features, the disease category of the sample is predicted according to the classification features, and disease-related information is obtained. The distribution of the brain network is constrained based on the disease-related information so that the brain network contains more disease features and connections, and the brain network is optimized.

2. The method according to claim 1, wherein The method predicts the disease category of the sample based on the brain network to obtain disease-related information, constrains the distribution of the brain network based on the disease-related information so that the brain network contains more disease features and connections, and optimizes the brain network, including: The disease category of the sample is predicted based on the optimized brain network and disease-related information to obtain a disease prediction result of the sample.

3. The method according to claim 1, wherein The step of constructing a brain region mask based on prior knowledge, locating each brain region in the brain image, and obtaining an image of each brain region in the brain image includes: According to prior knowledge, the center coordinates and relative volume size of each brain region in the brain image are obtained; Construct the mask of each brain region image according to the center point coordinates and relative volume size of each brain region image; The images of the various brain regions in the brain image are obtained according to the masks of the brain image and the images of the various brain regions.

4. The method according to claim 1, wherein The topological features of each brain region image are extracted by multiple feature extraction modules based on spatial attention, including: Convolutional layers are used to extract simple primary topological features, expand the number of channels, and reduce the size of images of various brain regions.

5. The method according to claim 1, wherein The spatial attention-based feature extraction module includes multiple depth convolution blocks and point convolution blocks.

6. The method according to any one of claims 1 to 5, characterized in that The method is used to train a disease prediction model, which includes a brain region perception module based on anatomical knowledge, a brain disease topological feature extraction module, a brain connection learning module and a brain disease prediction module. Each module is a trained machine learning model with corresponding prediction capabilities.

7. A knowledge and data driven brain network computing device, characterized in that: The device comprises: The brain region perception module based on anatomical knowledge is used to construct brain region masks based on prior knowledge, locate each brain region in the brain image, and obtain the image of each brain region in the brain image; A brain disease topological feature extraction module is used to extract the topological features of each brain region image step by step through multiple spatial attention-based feature extraction modules; The brain connectivity learning module is used to learn the connectivity between the topological features of each brain region image and construct a brain network based on the topological features and connectivity; The brain disease knowledge learning module is used to predict the disease category of the sample based on the brain network, obtain disease-related information, constrain the distribution of the brain network so that the brain network contains more disease features and connections, and optimize the brain network, including obtaining high-dimensional topological features in the brain network; splicing multiple high-dimensional topological features to obtain classification features, predicting the disease category of the sample based on the classification features, obtaining disease-related information, and constraining the distribution of the brain network based on the disease-related information so that the brain network contains more disease features and connections, and optimizing the brain network.

8. An electronic device, characterized in that: include: at least one processor, at least one memory, and at least one communication bus, wherein: The memory stores a computer program, and the processor reads the computer program in the memory through the communication bus; When the computer program is executed by the processor, the knowledge and data-driven brain network computing method according to any one of claims 1 to 6 is implemented.

9. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the knowledge and data-driven brain network computing method according to any one of claims 1 to 6 is implemented.

Images