Brain network unified representation calculation method, device, electronic device and storage medium

Abstract

The embodiment of the present invention discloses a method and device for calculating the unified representation of a brain network. The method includes: obtaining a sample brain image, and guiding the brain image to align with the standard brain space based on the standard brain space provided by the reference image to obtain a brain region structural deformation field of the brain image; learning the local deformation of the brain image through deformable convolution, adjusting the local deformation difference of the brain region structural deformation field, and obtaining an optimized brain region structural deformation field; applying the optimized brain region structural deformation field to the brain image to obtain a moving image, and obtaining regional information of multiple brain regions related to the disease by constraining the similarity between the moving image and the reference image; constructing a brain network based on the regional information of each brain region, and optimizing the brain network by graph comparison learning between the subgraphs of the brain network and their diffusion graphs to obtain a reconstructed brain network. The present invention solves the problems of strong subjectivity, low efficiency, low accuracy and low degree of automation in the existing technology.

Description

Technical Field

[0001] The present invention relates to the field of medical image processing technology, and in particular to a method, device, electronic device and storage medium for unified brain network representation calculation. Background Art

[0002] Brain imaging data of different modalities are usually used to construct three types of brain networks: morphological networks, structural networks, and functional networks. In 2007, the cortical thickness of each brain region was used as a morphological feature to construct the brain morphological network for the first time, and the small-world property of the brain network was discovered. Since then, a large number of complex network analysis methods based on graph theory have begun to study the connection patterns and graph property analysis between brain regions, and are used to explore the diagnosis of neurodegenerative diseases from structure to function. Graph neural networks are one of the current research hotspots in deep learning technology and have become the most widely used method for extracting information from graphs to perform neurodegenerative disease diagnosis.

[0003] However, existing brain network construction methods mainly rely on medical image processing software packages and rely on empirical knowledge to manually set a large number of parameters. The degree of automation is not high and the subjectivity is strong, resulting in inaccurate brain network construction and thus inaccurate disease prediction results.

[0004] In addition, existing technologies construct brain networks based on statistical analysis at the population level. Reconstructing brain connections between brain regions is complex and requires continuous iterative optimization to find the globally optimal model. The number of parameters is large and the processing is very time-consuming. It is also impossible to build personalized brain networks for individuals, making it difficult to explore the potential pathogenesis of the disease and achieve accurate diagnosis.

[0005] Therefore, there is an urgent need for a unified brain network representation and calculation method with strong objectivity, high efficiency, high degree of automation and high accuracy. Summary of the Invention

[0006] The embodiments of the present invention provide a method, device, electronic device and storage medium for unified brain network representation calculation to solve the problems of strong subjectivity, low efficiency, low accuracy and low degree of automation existing in related technologies.

[0007] Wherein, the technical solution adopted by the present invention is:

[0008] According to one aspect of the present invention, a method for calculating unified representation of a brain network includes: obtaining a brain image of a sample, and based on a standard brain space provided by a reference image, guiding the brain image to align with the standard brain space to obtain a brain region structural deformation field of the brain image; learning the local deformation of the brain image through deformable convolution, adjusting the local deformation difference of the brain region structural deformation field to obtain an optimized brain region structural deformation field; applying the optimized brain region structural deformation field to the brain image to obtain a moving image for indicating the distribution of brain regions, and obtaining regional information of multiple brain regions related to the disease by constraining the similarity between the moving image and the reference image; constructing an initial brain network based on the regional information of each brain region, and optimizing the initial brain network by performing graph comparison learning between the subgraphs of the brain network and their diffusion maps to obtain a reconstructed brain network, wherein the reconstructed brain network is used to indicate the unified representation of the brain network of the sample. According to one aspect of the present invention, a brain network unified representation calculation device includes: a global positioning module for acquiring a brain image of a sample, and based on a standard brain space provided by a reference image, guiding the brain image to align with the standard brain space to obtain a brain region structural deformation field of the brain image; a local perception module for learning the local deformation of the brain image through deformable convolution, adjusting the local deformation difference of the brain region structural deformation field, and obtaining an optimized brain region structural deformation field; a weighted coding module for applying the optimized brain region structural deformation field to the brain image to obtain a moving image for indicating the distribution of brain regions, and obtaining regional information of multiple brain regions related to the disease by constraining the similarity between the moving image and the reference image; a brain network reconstruction module for constructing an initial brain network based on the regional information of each brain region, and optimizing the initial brain network by performing graph comparison learning between the subgraphs of the brain network and their diffusion graphs to obtain a reconstructed brain network.

[0009] According to one aspect of the present invention, an electronic device includes a processor and a memory, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the unified brain network representation calculation method as described above is implemented.

[0010] According to one aspect of the present invention, a storage medium stores a computer program thereon, which, when executed by a processor, implements the above-mentioned unified brain network representation calculation method.

[0011] According to one aspect of the present invention, a computer program product includes a computer program, the computer program is stored in a storage medium, a processor of a computer device reads the computer program from the storage medium, and the processor executes the computer program, so that when the computer device executes the computer program, the unified brain network representation calculation method as described above is implemented.

[0012] In the above technical solution, the present invention realizes the unified representation calculation of brain networks with strong subjectivity, high efficiency, high degree of automation and high accuracy.

[0013] Specifically, the present invention obtains regional information of multiple brain regions related to the disease through a global positioning module guided by the spatial information of the brain structure provided by the reference image and a weighted coding module based on the learning of brain abnormality representation, and constructs a relatively accurate initial brain network based on the regional information of each brain region. The initial brain network is optimized by using the idea of ​​graph comparative learning through a brain network reconstruction module based on the embedding of brain disease topological features, and then based on the brain network representation indicated by the reconstructed brain network, the disease category of the sample is predicted to obtain the disease prediction result of the sample, further accurately locates the disease-related brain regions and abnormal connection features, explores the potential pathogenesis of the disease, improves the accuracy of the unified representation calculation of the brain network, and thus improves the accuracy of disease prediction. In addition, each module in the disease prediction model used for disease prediction uses deep learning technology to have corresponding prediction capabilities, improves the degree of automation and efficiency, enhances subjectivity, and thus solves the problems of strong objectivity, low efficiency, low accuracy and automation in the existing technology.

[0014] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0016] Figure 1 is a flowchart of a method for calculating a unified representation of a brain network according to an exemplary embodiment;

[0017] Figure 2 yes Figure 1 A schematic diagram of step 110 in one embodiment of the corresponding embodiment;

[0018] Figure 3 yes Figure 2 Schematic diagram of the flow of convolution and downsampling operations in the corresponding embodiment;

[0019] Figure 4 yes Figure 2 Schematic diagram of optimizing the brain region structural deformation field in the corresponding embodiment;

[0020] Figure 5 yes Figure 1 A schematic diagram of step 130 in one embodiment of the corresponding embodiment;

[0021] Figure 6yes Figure 1 A schematic diagram of step 150 in one embodiment of the corresponding embodiment;

[0022] Figure 7 is a schematic diagram illustrating an overall framework of a disease prediction model based on a unified brain network representation calculation method according to an exemplary embodiment;

[0023] Figure 8 is a schematic diagram illustrating a training process of a disease prediction model based on a unified brain network representation calculation method according to an exemplary embodiment;

[0024] Figure 9 is a block diagram of a brain network unified representation computing device according to an exemplary embodiment;

[0025] Figure 10 This is the experimental result of the simulation experiment of the embodiment of the present invention;

[0026] Figure 11 is a hardware structure diagram of an electronic device according to an exemplary embodiment;

[0027] Figure 12 It is a block diagram of an electronic device according to an exemplary embodiment.

[0028] The above-mentioned drawings have shown clear embodiments of the present invention, which will be described in more detail below. These drawings and textual descriptions are not intended to limit the scope of the present invention in any way, but rather to illustrate the concept of the present invention for those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0029] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. In the following description, when referring to the drawings, like numbers in different figures represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all possible embodiments consistent with the present invention. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present invention, as detailed in the appended claims.

[0030] Existing related technologies use software to construct brain networks, which requires manual setting of a large number of parameters. The parameter selection is subjective and relies on rich clinical experience references. The brain network construction results are not unique, and the final experimental results are difficult to reproduce and fairly compare under the same conditions, resulting in inaccurate brain network construction.

[0031] At the same time, existing related technologies are aimed at tasks such as brain segmentation or classification. It is difficult to accurately locate abnormal changes in brain areas, effectively locate clinically disease-related brain areas, identify abnormal connections in brain networks, and provide a reference for disease diagnosis.

[0032] From the above, we can see that the relevant technologies still have defects such as strong objectivity, low efficiency, low accuracy and low degree of automation.

[0033] To this end, the brain network unified representation calculation method provided in this application takes raw brain image data as input, combines the two tasks of brain network construction and graph representation learning, and utilizes the ability of deep learning automatic feature extraction and the idea of ​​brain network-based graph neural network optimization to reconstruct the brain network corresponding to each sample. It is not limited to the registration or alignment of images of different modalities, but further learns the knowledge of brain region boundaries, accurately locates the spatial position of the brain region divided by the template, uses texture feature similarity and gradient smoothness constraints to improve the accuracy of regional positioning, proposes a multi-classification cross entropy loss function, maps multiple input samples to corresponding disease categories, and utilizes the powerful nonlinear fitting ability of the multilayer perceptron to improve the high-precision prediction results on disease categories, improves the accuracy of brain network construction from multiple aspects, and thus improves the accuracy of disease prediction. At the same time, the brain network unified representation calculation method is applicable to a unified representation calculation device and a disease prediction device based on the unified representation calculation method. The unified representation calculation device and the disease prediction device based on the unified representation calculation method can be deployed in an electronic device. For example, the electronic device can be a computer device configured with a von Neumann architecture, and the computer device includes but is not limited to a desktop computer, a laptop computer, a server, etc.

[0034] See also Figure 1 , an embodiment of the present application provides a unified brain network representation calculation method, which is applicable to electronic devices, for example, the electronic device can be a desktop computer, a laptop computer, a server, etc.

[0035] In the following method embodiments, for ease of description, the execution subject of each step of the method is taken as an electronic device as an example for illustration, but this does not constitute a specific limitation.

[0036] like Figure 1 As shown, the method may include the following steps:

[0037] Step 110 , obtaining a brain image of a sample, and guiding the alignment of the brain image with the standard brain space based on the standard brain space provided by the reference image, to obtain a brain region structural deformation field of the brain image.

[0038] The brain image can be any of structural T1-weighted images (T1WI), diffusion-weighted images (DTI), and resting-state functional MRI (rs-fMRI). The reference image is a manually aligned MNI brain template that provides a standard brain space.

[0039] In one possible implementation, see Figure 2In the encoding stage, the brain image and the reference image are downsampled respectively to obtain the low-dimensional representation of the brain image and the low-dimensional representation of the reference image; in the decoding stage, the low-dimensional representation of the reference image guides the alignment of the low-dimensional representation of the brain image to obtain the brain area structural deformation field of the brain image.

[0040] In one possible implementation, the alignment process in the decoding stage may include the following steps: in the decoding stage, the low-dimensional features of the brain image and the reference image are upsampled respectively to obtain the high-dimensional features of the brain image and the high-dimensional features of the reference image; through the jump connection between the encoding and decoding stages, the low-dimensional features of the brain image and the reference image are spliced ​​and implicitly aligned with the corresponding high-dimensional features in the channel direction; the high-dimensional features of the brain image are fused with the high-dimensional features of the reference image, and the fused high-dimensional features of the brain image are transformed and adjusted to obtain the brain region structural deformation field. In this way, by narrowing the gap in the semantic subspace, the brain region voxel-level alignment effect of the brain image to the reference image is achieved, which is conducive to the subsequent construction of an accurate brain network.

[0041] in, Figure 3 It shows the specific process of downsampling the brain image and the reference image, which includes multiple convolutions, multiple batch normalizations and multiple downsampling operations. Figure 3 After 3 convolutions, 3 batch normalizations, and 2 downsampling operations, the size of 128×128×128 is converted to a feature map of 8×8×8 with 32 channels.

[0042] Step 130 , using deformable convolution to learn local deformations of brain images, adjust the local deformation differences of the brain region structural deformation field to obtain an optimized brain region structural deformation field, and construct and optimize the brain network based on the optimized brain region structural deformation field.

[0043] Specifically, if Figure 4 As shown, in the input feature Figure X Use deformable convolution kernel to sample, use deformable convolution to learn the rotation invariance and scale invariance features of image texture, and transform the input features Figure X The filter parameters W are multiplied by the convolution kernel and the convolution bias is added to obtain the output feature map Y. Figure X It is used to characterize the structural deformation field of the brain area before optimization, and the output feature map Y is used to characterize the structural deformation field of the brain area after optimization.

[0044] The specific calculation formula is as follows:

[0045]

[0046] Where p = p0 + p n +Δp n , p n is the position of each grid point in the convolution kernel, Δp n Represents the learnable position offset parameter, usually a floating point number, which needs to be resampled and mapped to integer coordinate positions.

[0047] In one possible implementation, bilinear interpolation is used to obtain the voxel value X(p) at the non-integer coordinate position on the feature map. The specific calculation formula is as follows:

[0048]

[0049] Among them, q is the input feature Figure X The integer coordinates on the , p is the input feature to be calculated Figure X The floating-point coordinates on , the function F is a bilinear interpolation function.

[0050] In one possible implementation, in order to ensure the differentiability of the neural network, the bilinear interpolation function F is selected as:

[0051] F(q,p)=g(q x , p x )·g(q y , p y )·g(q z , p z ).

[0052] Among them, function g is constructed by the maximum function, and the specific calculation formula of function g is as follows:

[0053] g(a, b) = max(0, 1-|q x -p x |).

[0054] Therefore, X(p) can be further calculated by the following formula:

[0055]

[0056] Through the above process, the local deformation of brain images is learned through deformable convolution, and the local deformation differences of the brain region structural deformation field are adjusted, thereby optimizing the brain region structural deformation field and obtaining a more accurate brain region structural deformation field, which is conducive to improving the accuracy of subsequent brain network construction.

[0057] In step 150 , the brain region structural deformation field is applied to the brain image to obtain a moving image indicating the distribution of brain regions, and regional information of multiple brain regions related to the disease is obtained by constraining the similarity between the moving image and the reference image.

[0058] In one possible implementation, a weighted encoding method with a cross-attention mechanism is used to jointly constrain the similarity between the moving image and the reference image.

[0059] In one possible implementation, multiple transformation functions composed of a neural network perform multiple cross-attention and self-attention operations with the moving image and the reference image respectively, and the moving image and the reference image are mapped to corresponding feature spaces respectively. The feature spaces corresponding to the moving image and the reference image are then spliced ​​to realize weighted encoding of the cross-attention mechanism, and finally obtain regional information related to the disease in each brain region.

[0060] Specifically, if Figure 5 As shown, M is the brain image, M' refers to the moving image formed by the brain area structural deformation field acting on the brain image, ψ is the brain area structural deformation field, F is the reference image, q, k, v respectively represent multiple transformation functions composed of neural networks, by matrix multiplication with the moving image and the reference image, the moving image and the reference image are mapped to the feature space of the corresponding dimension, and then the feature space Mi_i obtained by the moving image mapping and the feature space obtained by the reference image mapping are feature spliced ​​to obtain M' i .

[0061] In getting M' i After that, the disease-related regional information Mregion in each brain region can be calculated according to the following calculation formula: i .

[0062]

[0063] After the above process, the weighted encoding method of the cross-attention mechanism is used to jointly constrain the similarity between the moving image and the reference image, so that accurate disease-related brain area information of each brain region can be obtained.

[0064] In step 170 , an initial brain network is constructed based on the regional information of each brain region, and the initial brain network is optimized by performing graph comparison learning between the subgraphs of the brain network and their diffusion graphs to obtain a reconstructed brain network.

[0065] Among them, the reconstructed brain network is used to indicate the brain network characteristics of the sample.

[0066] In one possible implementation, such as Figure 6As shown, by sampling the initial brain network, a subgraph of the brain network is constructed, and a corresponding diffusion map is generated (Diffuse) based on the heat kernel features of the subgraph. The node features of the subgraph of the brain network and each node in its diffusion map are updated and aggregated to obtain the node embedding of the subgraph and the node embedding of the diffusion map respectively. The corresponding graph embedding is extracted from the node embedding of the subgraph and its diffusion map. Based on the node embedding and graph embedding of the subgraph and its diffusion map, the subgraph is compared with the diffusion map, and finally the initial brain network is optimized to a reconstructed brain network.

[0067] It is explained here that updating the node features of each node in the subgraph and its diffusion graph of the brain network and aggregating the node features are achieved through the graph neural network GCN. Specifically, GCN is used to learn the node embeddings of each node in the subgraph and its diffusion graph, and then the learned node embeddings are passed through a shared fully connected layer to obtain the set of node embeddings of the subgraph and its diffusion graph respectively. Finally, the readout function is used to obtain the respective graph embeddings.

[0068] First of all, the calculation formula for generating the corresponding diffusion map based on the heat kernel features of the sub-image is as follows:

[0069] S heat =exp(tAD -1 -t).

[0070] Where A represents the adjacency matrix of the subgraph, and D represents the degree matrix of the subgraph.

[0071] Secondly, the optimization objective of the brain network is expressed as the following calculation formula:

[0072]

[0073] in, and Node embeddings representing two views, and represents the graph embedding of two views, and MI represents the brain network mutual information function. The calculation formula of the brain network mutual information function MI is as follows:

[0074]

[0075] Through the above process, a graph comparative learning strategy was introduced. Graph diffusion technology was used to guide the model to learn the common connections between similar categories and the differences between different categories through the feature similarity between the node embedding and graph embedding of the brain network graph and its diffusion graph, thereby improving the performance of the brain network, optimizing the initial brain network, and obtaining a reconstructed brain network.

[0076] In one possible implementation, after obtaining the reconstructed brain network, the disease category of the sample is predicted based on the brain network representation indicated by the reconstructed brain network to obtain a disease prediction result for the sample.

[0077] Specifically, disease category prediction is implemented by a disease prediction module based on a unified brain network representation calculation method. The disease prediction module aggregates a graph representation containing rich pathological information through a multi-layer perceptron, and connects it with the Softmax function to output the probability that the sample belongs to different disease categories, and then obtains the disease prediction result of the sample based on the probability. The disease prediction module is composed of a three-layer BP neural network, a two-layer ReLu activation function, and a Softmax output layer; the Softmax output layer contains four neurons, which are used to map the brain network representation of the sample indicated by the reconstructed brain network into the probabilities of four different disease categories. In addition, the disease prediction module can also include a Dropout layer, which is used to adopt a Dropout strategy to prevent model overfitting.

[0078] Through the above process, an embodiment of the present invention provides an end-to-end unified brain network representation calculation method, which directly uses a deep learning model to map image data into brain network connections, avoiding the tedious image data processing steps in the existing technology, with a high degree of automation, simple and efficient operation, and learning the rotation invariance and scale invariance characteristics of standard brain space image texture, thereby improving the accuracy of brain network construction. By performing graph comparison learning between the subgraphs of the brain network and their diffusion maps, the stability of the brain network construction is improved. Through the multi-classification cross entropy loss function and the powerful nonlinear fitting ability of the multi-layer perceptron, the high-precision prediction results on the classification indicators are improved, further improving the accuracy of the unified brain network representation calculation.

[0079] Figure 7 and Figure 8 The overall framework and training process of the disease prediction model based on the unified representation computational method of brain networks are demonstrated. Figure 7 A schematic diagram showing the overall framework of a disease prediction model based on a unified brain network representation calculation method in one embodiment is shown. Figure 8 A schematic diagram showing the training process of a disease prediction model based on a unified brain network representation calculation method in one embodiment. Figure 7 The disease prediction model based on a unified brain network representation computational approach includes a global positioning module, a local perception module, a weighted coding module, a brain network reconstruction module, and a disease diagnosis module. Each of these modules in the disease prediction model is a trained machine learning model with corresponding predictive capabilities.

[0080] Now combined Figure 7 and Figure 8The training process of the disease prediction model based on the unified representation calculation method of brain networks is described in detail below:

[0081] like Figure 7 As shown, the method may include the following steps:

[0082] Step 310 : learning the regional position information of the brain image through the global positioning module so that the brain image is aligned with the standard brain space to obtain the brain region structural deformation field.

[0083] In one possible implementation, brain images are used as moving images. Based on the knowledge of anatomical brain regions, a manually aligned standard brain space is used as a reference image. The reference image is convolved and down-sampled by maximum pooling to obtain a low-dimensional image representation. The alignment of the moving image is guided by the low-dimensional image representation of the reference image, the gap in the semantic subspace is narrowed, the voxel-level alignment effect of the brain region in the moving image is achieved, and an accurate brain region structural deformation field is constructed.

[0084] Step 330: Use the local perception module to learn local micro-deformations and rotation invariance of brain images to optimize the brain region structural deformation field.

[0085] In step 350 , the optimized brain region structural deformation field is applied to the brain image, and the parameters of the global positioning module and the local perception module are updated according to the transformed moving image and the reference image through the weighted coding module.

[0086] In one possible implementation, the optimized brain region structural deformation field is applied to brain images, and the transformed moving image and the reference image are input into a weighted coding module based on brain abnormality representation learning. Weighted coding is performed based on the brain abnormality representation knowledge, and the feature spaces of the two are aligned. The similarity loss is back-propagated to guide the global positioning module and the local perception module to update their parameters.

[0087] Specifically, the loss function is as follows:

[0088]

[0089] Among them, the image texture feature similarity loss L similarity The mean square voxel difference is used for calculation, and the specific calculation formula is as follows:

[0090]

[0091] On the spatial deformation field φ, the gradient regularization function is used to promote the smoothness of the smooth displacement field L smooth , the calculation formula is as follows:

[0092]

[0093] Through the above process, the parameters of the global positioning module and the local perception module are updated.

[0094] Step 370: Use the brain network reconstruction module to optimize the brain network through graph comparison learning.

[0095] In one possible implementation, the brain network reconstruction module is trained using the brain network mutual information contrast loss, where the calculation formula for the brain network mutual information contrast loss is as follows:

[0096]

[0097] Backpropagation of the brain network mutual information contrast loss is performed to guide the brain network reconstruction module to update parameters.

[0098] Step 390: predict the disease category of the sample through the disease diagnosis module, and update the parameters according to the multi-classification cross entropy.

[0099] In one possible implementation, the multi-class cross entropy loss is calculated as follows:

[0100]

[0101] Where p is a probability distribution, each element p i represents the probability that the sample belongs to the i-th disease category, y i Represents the sample label, which indicates the actual disease category to which the sample belongs. C represents the number of sample labels, that is, the number of disease categories.

[0102] The multi-classification cross entropy loss is back-propagated to guide the parameter updates of the above modules in the disease prediction model.

[0103] Through the above process, after the disease prediction model based on the unified representation calculation method of the brain network is trained, the disease prediction model based on the unified representation calculation method of the brain network will have the ability to predict the disease category of the sample. Then, by inputting the brain image of the sample into the disease prediction model, the disease prediction result of the sample can be obtained. At the same time, combined with the brain network representation of the sample extracted from the brain network reconstruction module, the brain network connection differences of the sample can be analyzed, and by comparing with the group brain network, the abnormal brain connection of the sample can be obtained, thereby realizing intelligent assisted diagnosis and treatment.

[0104] See also Figure 9 In an exemplary embodiment, a brain network unified representation computing device 900 is provided.

[0105] The device 900 includes but is not limited to: a global positioning module 910 , a local perception module 930 , a weighted coding module 950 and a brain network reconstruction module 970 .

[0106] The global positioning module 910 is used to obtain a brain image of a sample and, based on the standard brain space provided by the reference image, guide the brain image to align with the standard brain space to obtain a brain region structural deformation field of the brain image.

[0107] The local perception module 930 is used to learn the local deformation of brain images through deformable convolution, adjust the local deformation differences of the brain region structural deformation field, and obtain an optimized brain region structural deformation field.

[0108] The weighted coding module 950 is used to apply the optimized brain region structural deformation field to the brain image to obtain a moving image indicating the distribution of brain regions, and to obtain regional information of multiple brain regions related to the disease by constraining the similarity between the moving image and the reference image.

[0109] The brain network reconstruction module 970 is used to construct an initial brain network based on the regional information of each brain region, and optimize the initial brain network by performing graph comparison learning between the subgraphs of the brain network and its diffusion graph to obtain a reconstructed brain network. It should be noted that the disease prediction device provided in the above embodiment only uses the division of the above functional modules as an example when performing disease prediction. In actual application, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the disease prediction device will be divided into different functional modules to complete all or part of the functions described above.

[0110] In addition, the embodiments of the brain network unified characterization calculation device and the brain network unified characterization calculation method provided in the above embodiments belong to the same concept, and the specific manner in which each module performs operations has been described in detail in the method embodiments and will not be repeated here.

[0111] In the present invention, the ADNI dataset was used for experimental simulation, and AD, EMCI, SMCI, and NC were selected to contain 50, 54, 44, and 44 samples respectively. All data were preprocessed to a unified size of 128×128×128 and aligned through a global positioning module. A 5-fold cross-validation was adopted, and the Python sklearn library was used to randomly divide the data into 5 folds. The average value of each indicator was calculated as the final result for comparison. The five evaluation indicators are: accuracy (ACC), sensitivity (SEN), specificity (SPE), and ROC curve area (AUC). The other two methods are: empirical method (Empirical) and benchmark method (Benchmark), and each method uses the same classifier to calculate the classification performance. Among them, the Empirical method uses a software toolbox to preprocess DTI to obtain a structural connection matrix, and the Empirical brain connection is obtained after taking the average value; the Benchmark method uses a simple 2-layer GCN network to obtain the Benchmark brain connection by fusing the structural connection matrix. The experimental results are as follows Figure 10 As shown in the figure, experiments have shown that the brain connections constructed by this model through the fusion of structural and functional images have great advantages in classification performance over other similar methods.

[0112] Figure 11 The following is a schematic diagram of the structure of an electronic device according to an exemplary embodiment.

[0113] It should be noted that the electronic device is only an example adapted for this application and cannot be considered to provide any limitation on the scope of use of this application. The electronic device cannot be interpreted as needing to rely on or must have Figure 11 One or more components of exemplary electronic device 2000 are shown.

[0114] The hardware structure of the electronic device 2000 may vary greatly due to different configurations or performances, such as Figure 11 As shown, the electronic device 2000 includes a power supply 210 , an interface 230 , at least one memory 250 , and at least one central processing unit (CPU) 270 .

[0115] Specifically, the power supply 210 is used to provide operating voltage for various hardware devices on the electronic device 2000 .

[0116] The interface 230 includes at least one wired or wireless network interface 231 for interacting with external devices.

[0117] Of course, in other examples adapted by this application, the interface 230 may further include at least one serial-to-parallel conversion interface 233, at least one input-output interface 235, and at least one USB interface 237, etc. Figure 9 As shown, this does not constitute a specific limitation.

[0118] The memory 250 serves as a carrier for resource storage and can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored thereon include an operating system 251, application 253 and data 255, etc. The storage method can be temporary storage or permanent storage.

[0119] Among them, the operating system 251 is used to manage and control the various hardware devices and application programs 253 on the electronic device 2000, so as to enable the central processing unit 270 to calculate and process the massive data 255 in the memory 250. It can be WindowsServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.

[0120] The application program 253 is a computer program that performs at least one specific task based on the operating system 251 and may include at least one module ( Figure 11 (not shown), each module can respectively include a computer program for the electronic device 2000. For example, the disease prediction device can be regarded as an application 253 deployed on the electronic device 2000.

[0121] The data 255 may be photos, pictures, etc. stored in a disk, or may be brain images, etc. stored in the memory 250 .

[0122] The central processing unit 270 may include one or more processors and is configured to communicate with the memory 250 via at least one communication bus to read the computer programs stored in the memory 250, thereby performing operations and processing on the massive amount of data 255 in the memory 250. For example, the unified brain network representation calculation method can be implemented by the central processing unit 270 reading a series of computer programs stored in the memory 250.

[0123] In addition, the present application can also be implemented through hardware circuits or hardware circuits combined with software. Therefore, the implementation of the present application is not limited to any specific hardware circuits, software, or a combination of the two.

[0124] See also Figure 12 In an embodiment of the present application, an electronic device 4000 is provided. The electronic device 400 may include: a desktop computer, a laptop computer, a server, etc.

[0125] exist Figure 12 In the embodiment, the electronic device 4000 includes at least one processor 4001, at least one communication bus 4002 and at least one memory 4003.

[0126] The processor 4001 and the memory 4003 are connected, for example, via a communication bus 4002. Optionally, the electronic device 4000 may further include a transceiver 4004, which may be used for data exchange between the electronic device and other electronic devices, such as data transmission and / or data reception. It should be noted that in actual applications, the number of transceivers 4004 is not limited to one, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of the present application.

[0127] Processor 4001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It may implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 4001 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and the like.

[0128] The communication bus 4002 may include a path for transmitting information between the above components. The communication bus 4002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The communication bus 4002 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 12 Only one thick line is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0129] The memory 4003 may be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited to these.

[0130] The memory 4003 stores a computer program, and the processor 4001 reads the computer program stored in the memory 4003 through the communication bus 4002 .

[0131] When the computer program is executed by the processor 4001, the brain network unified representation calculation method in the above embodiments is implemented.

[0132] In addition, an embodiment of the present application provides a storage medium on which a computer program is stored. When the computer program is executed by a processor, the unified brain network representation calculation method in the above embodiments is implemented.

[0133] In an embodiment of the present application, a computer program product is provided, comprising a computer program stored in a storage medium. A processor of a computer device reads the computer program from the storage medium and executes the computer program, causing the computer device to perform the unified brain network representation calculation method described in each of the above embodiments.

[0134] Compared with the related art, the present invention has the following beneficial effects:

[0135] 1. This paper proposes a new unified brain network representation calculation method. Compared with the existing technology, this paper is a completely end-to-end processing flow that directly uses a deep learning model to map brain imaging data into brain networks, avoiding the cumbersome image data preprocessing steps of traditional software. It has a high degree of automation and is simple and efficient to operate.

[0136] 2. This invention uses raw brain imaging data as the input of the model, combines the two tasks of brain network construction and graph representation learning, and utilizes the automatic feature extraction capabilities of deep learning and the idea of ​​graph neural network optimization based on brain networks to reconstruct the brain network corresponding to each sample, which is conducive to the analysis and diagnosis of abnormal brain connections in patients with brain diseases.

[0137] 3. The present invention proposes a new global positioning module guided by brain structure spatial information. Compared with the existing technology, the present invention is not limited to the registration or alignment of images of different modalities, but further learns the knowledge of brain region boundaries, so that the model can accurately locate the spatial position of the brain region divided by the template.

[0138] 4. The weighted coding module based on brain abnormality representation learning of the present invention uses texture feature similarity and gradient smoothness constraints. Compared with existing methods that rely solely on image consistency, the extracted features have higher-level semantics in the representation space and contain richer spatial information. Through local deformation fine-tuning of this module, the accuracy of regional positioning is improved.

[0139] 5. The brain network reconstruction module proposed in this paper, which is based on the embedding of brain disease topological features, uses graph diffusion operations to enhance data, and uses the differences between brain networks of different categories to strengthen the common connections between similar samples and highlight the differences between different categories, thereby improving the robustness of brain network reconstruction learning.

[0140] 6. The disease diagnosis module designed by the present invention based on brain pathology knowledge is not limited to the auxiliary diagnosis of Alzheimer's disease, but can also be applied to the medical diagnosis and treatment scenarios of other diseases: by modifying the corresponding disease sample data and model output labels during model training, it can be conveniently applied to the analysis and diagnosis of other brain diseases.

[0141] 7. The brain network mutual information comparison loss function proposed in this invention calculates the similarity between graph embedding features and node features of another view. Compared with the existing technology that compares node features and graph embedding features, it is more efficient and improves the stability of brain network representation learning.

[0142] 8. This paper proposes a multi-classification cross entropy loss function, which maps multiple input samples to corresponding categories and utilizes the powerful nonlinear fitting ability of the multi-layer perceptron to improve the high-precision prediction results on the classification indicators.

[0143] It should be understood that although the steps in the flowcharts of the accompanying drawings are shown in sequence as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some of the steps in the flowcharts of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0144] The above content is only a preferred exemplary embodiment of the present invention and is not intended to limit the implementation scheme of the present invention. Ordinary technicians in this field can easily make corresponding changes or modifications based on the main concept and spirit of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection required by the claims.

Claims

1. A unified brain network representation calculation method, characterized in that: The method comprises: Acquire a brain image of a sample, and guide the alignment of the brain image with the standard brain space provided by the reference image to obtain a brain region structural deformation field of the brain image; Learning the local deformation of the brain image through deformable convolution, adjusting the local deformation difference of the brain region structural deformation field, and obtaining an optimized brain region structural deformation field; Applying the optimized brain region structural deformation field to the brain image to obtain a moving image indicating brain region distribution, and obtaining regional information of multiple brain regions related to the disease by constraining the similarity between the moving image and the reference image; An initial brain network is constructed based on the regional information of each brain region, and the initial brain network is optimized by performing graph comparison learning between the subgraph of the brain network and its diffusion map to obtain a reconstructed brain network. The reconstructed brain network is used to indicate a unified brain network representation of the sample, which includes updating the node features of the subgraph of the brain network and each node in the diffusion map and aggregating the node features to obtain the node embedding of the subgraph and the node embedding of the diffusion map respectively, and extracting the corresponding graph embedding from each node embedding of the subgraph and its diffusion map; based on the node embedding and graph embedding of the subgraph and its diffusion map, the subgraph is compared with the diffusion map to optimize the initial brain network to the reconstructed brain network.

2. The method according to claim 1, wherein After obtaining the reconstructed brain network, the method further includes: Based on the unified brain network representation indicated by the reconstructed brain network, a disease category prediction is performed on the sample to obtain a disease prediction result of the sample.

3. The method according to claim 1, wherein The step of guiding the brain image to align with the standard brain space based on the reference image to obtain a brain region structural deformation field of the brain image includes: In the encoding stage, the brain image and the reference image are convolved and down-sampled respectively to obtain a low-dimensional representation of the brain image and a low-dimensional representation of the reference image; In the decoding stage, the low-dimensional representation of the reference image guides the alignment of the low-dimensional representation of the brain image to obtain the brain area structural deformation field of the brain image.

4. The method according to claim 3, wherein In the decoding stage, the low-dimensional representation of the reference image is used to guide the alignment of the low-dimensional representation of the brain image to obtain the brain region structural deformation field of the brain image, including: In the decoding stage, the low-dimensional features of the brain image and the reference image are up-sampled respectively to obtain high-dimensional features of the brain image and high-dimensional features of the reference image; Through skip connections between encoding and decoding stages, the low-dimensional features of the brain image and the reference image are spliced ​​and implicitly aligned with the corresponding high-dimensional features in the channel direction; The high-dimensional features of the brain image are fused with the high-dimensional features of the reference image, and the fused high-dimensional features of the brain image are transformed and adjusted to obtain the brain region structural deformation field.

5. The method according to claim 1, wherein Obtaining regional information of multiple brain regions related to the disease by constraining the similarity between the moving image and the reference image includes: Mapping the moving image and the reference image to corresponding feature spaces respectively through multiple transformation functions composed of neural networks and multiple cross-attention and self-attention with the moving image and the reference image; Feature splicing is performed on the feature space obtained by mapping the moving image and the reference image to obtain regional information related to the disease in each brain region.

6. The method according to claim 1, wherein The initial brain network is constructed based on the regional information of each brain region, and the initial brain network is optimized by graph comparison learning between the subgraphs of the brain network and their diffusion graphs to obtain a reconstructed brain network, including: By sampling the brain network, a subgraph of the brain network is constructed, and a corresponding diffusion map is generated based on the heat kernel feature of the subgraph.

7. The method according to any one of claims 1 to 6, wherein: The method also includes training a disease prediction model, which includes a global positioning module, a local perception module, a weighted coding module, a brain network reconstruction module and a disease diagnosis module; each module in the disease prediction model is a trained machine learning model with corresponding prediction capabilities.

8. A brain network unified representation computing device, characterized in that: The device comprises: A global positioning module is used to obtain a brain image of a sample and, based on a standard brain space provided by a reference image, guide the brain image to align with the standard brain space to obtain a brain region structural deformation field of the brain image; A local perception module, configured to learn the local deformation of the brain image through deformable convolution, adjust the local deformation difference of the brain region structural deformation field, and obtain an optimized brain region structural deformation field; a weighted coding module for applying the optimized brain region structural deformation field to a brain image to obtain a moving image indicating the distribution of brain regions, and obtaining regional information of multiple brain regions related to the disease by constraining the similarity between the moving image and a reference image; A brain network reconstruction module is used to construct an initial brain network based on the regional information of each brain region, and to optimize the initial brain network by performing graph comparison learning between the subgraph of the brain network and its diffusion map to obtain a reconstructed brain network. The reconstructed brain network is used to indicate a unified brain network representation of the sample, which includes updating the node features of the subgraph of the brain network and each node in its diffusion map and aggregating the node features to obtain the node embedding of the subgraph and the node embedding of the diffusion map respectively, and extracting the corresponding graph embedding from each node embedding of the subgraph and its diffusion map; based on the node embedding and graph embedding of the subgraph and its diffusion map, the subgraph is compared with the diffusion map to optimize the initial brain network to the reconstructed brain network.

9. An electronic device, characterized in that: include: At least one processor, at least one memory, and at least one communication bus, wherein the memory stores a computer program, and the processor reads the computer program from the memory via the communication bus; When the computer program is executed by the processor, the brain network unified representation calculation method according to any one of claims 1 to 7 is implemented.

10. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the brain network unified representation calculation method according to any one of claims 1 to 7 is implemented.

Images