A human brain digital twin 3D visualization method

Abstract

The present application belongs to the field of brain science and visualization technology, and specifically relates to a human brain digital twin 3D visualization method. The method comprises the following steps: S1: constructing a cerebral cortex outer surface model and an independent brain region model; S2: registering a voxelized brain atlas to a standard brain template space to obtain a correspondence relationship between the vertexes of the cerebral cortex outer surface model and the brain atlas region labels as a projection rule; S3: according to the projection rule, displaying BOLD signals and EEG backtracking signals to the cerebral cortex outer surface model; S4: constructing connection pathways between each brain region in the independent brain region model, and performing deterministic fiber bundle tracking, cleaning and integration on individual diffusion weighted imaging data to obtain biologically authentic and representative fiber bundles between any two brain regions as the connection pathways; and S5: integrating the connection pathways into the independent brain region model to visualize brain network and brain region function effects. The present application realizes dynamic and interactive visualization of multi-modal brain data in three-dimensional space.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of brain science and visualization technology, and particularly relates to a human brain digital twin 3D visualization method. BACKGROUND

[0002] With the continuous integration of brain science research and intelligent computing technology, the demand for visualization of human brain structure and function is increasing. Currently, such visualization technology has been widely used in the fields of auxiliary diagnosis of neurological diseases, development of brain-computer interface, and research of cognitive mechanism. The introduction of three-dimensional visual interaction, anatomical structure modeling, and multi-modal signal fusion has effectively promoted the fine modeling and dynamic analysis of the complex human brain system. Especially in emerging directions such as precision medicine and digital twin brain, how to intuitively present the spatial morphology, functional distribution, and interaction relationship between regions of brain tissue has become one of the core research topics.

[0003] However, the existing human brain three-dimensional visualization methods still have many deficiencies in practical application. In the visualization of brain atlas and brain function data (such as BOLD signal of FRMI image, EEG backtracking signal, etc.), traditional methods mainly use two-dimensional slices or three-dimensional volume rendering, which cannot realize flexible hierarchical control, region selection, and semantic highlighting, and cannot meet the needs of personalized cognition and analysis. Moreover, most traditional methods rely on desktop software (such as FreeSurfer, BrainSuite, SPM, etc.) for offline static rendering, which has complex operation process, poor interactivity, large computational overhead, and cannot realize efficient processing and rapid response of large-scale data, limiting its application in clinical real-time auxiliary decision-making and interactive teaching. Traditional visualization methods often focus on the presentation of structural information, and have limited expression ability for functional signals and brain connection group networks, resulting in a gap between structure and function, static and dynamic information. SUMMARY

[0004] The application provides a human brain digital twin 3D visualization method to solve the above problems.

[0005] The application adopts the following technical scheme: a human brain digital twin 3D visualization method, comprising:

[0006] S1: constructing a brain cortex outer surface model and an independent brain region model;

[0007] S2: registering the voxelized brain atlas to the standard brain template space, and performing coordinate conversion to obtain the correspondence relationship between the brain cortex outer surface model vertex and the brain atlas brain region label as a projection rule;

[0008] S3: according to the projection rule, displaying the BOLD signal and the EEG backtracking signal to the brain cortex outer surface model;

[0009] S4: Construct the connection pathways between brain regions in the independent brain region model, and perform deterministic fiber tract tracking, cleaning and integration of any two brain regions with biologically realistic and representative fiber tracts as connection pathways.

[0010] S5: Integrating connectivity pathways into independent brain region models to visualize brain networks and brain region functional effects, and using collision detection technology to discover potential brain regions that are linked by fiber bundles.

[0011] In some embodiments, step S1 includes obtaining reconstruction files of the left and right cerebral cortexes of an individual under a standard template brain space, and extracting model vertices and triangular faces from the reconstruction files of the left and right cerebral cortexes to construct a model of the outer surface of the cerebral cortex.

[0012] After registering the voxelized brain atlas to a standard template brain space, three-dimensional surface reconstruction is performed to construct independent brain region models.

[0013] In some embodiments, step S2 includes:

[0014] Voxelized brain atlases were registered from other spaces to standard brain template space, and the voxelized brain atlases in standard brain template space were projected onto the outer surface model of the cerebral cortex.

[0015] Interpolation was used to fill in the gaps inside the brain regions of the outer surface model of the cerebral cortex.

[0016] To address the issues of fragmented brain region boundaries and disordered sulci and gyri, the brain regions are finely divided.

[0017] In some embodiments, interpolation filling of gaps within brain regions of an outer surface model of the cerebral cortex includes:

[0018] Collect input data, which includes:

[0019] The set of vertex coordinates of the outer surface model of the cerebral cortex;

[0020] An array of brain region labels corresponding to the set of vertex coordinates, wherein unlabeled vertices are identified by specific label values;

[0021] The number of nearest neighbor vertices, K;

[0022] The percentage threshold T, where 0 <T≤1;

[0023] Identify all vertices in the brain region label array whose label value is the specific label value, as vacant vertices to be filled, specifically including:

[0024] For each of the aforementioned empty vertices:

[0025] Calculate the Euclidean distance between the missing vertex and all labeled vertices;

[0026] According to the Euclidean distance, K nearest vertices are selected from all labeled vertices as a nearest neighbor vertex set;

[0027] The frequency of occurrence of each brain region label in the nearest neighbor vertex set is counted;

[0028] It is judged whether the frequency of the brain region label with the highest frequency of occurrence is greater than the proportion threshold T;

[0029] If the judgment is yes, the brain region label with the highest frequency of occurrence is assigned to the current empty vertex;

[0030] The filled brain region label array is output.

[0031] In some embodiments, the fine division adopts the following formula:

[0032]

[0033]

[0034] wherein, is the brain region label of a vertex in the cerebral cortex outer surface model, mode is the mode operation, x, y, z are the corresponding voxel coordinates of a vertex in the cerebral cortex outer surface model in the voxelized brain atlas, and r is 1, is the brain region label of 26 voxels around the corresponding voxel, is the conversion matrix of the NIfTI header to the real space coordinates.

[0035] In some embodiments, step S3 comprises:

[0036] According to the projection rule, the visualization of the BOLD signal and the EEG backtracking signal on the cerebral cortex outer surface model is completed;

[0037] The BOLD signal and the EEG backtracking signal are maximum and minimum normalized to control the signal data in the range of 0-1, and are mapped to different colors through a color mapping table;

[0038] The vertices of the cerebral cortex outer surface model are colored to realize the coloring of the surface of the cerebral cortex outer surface model.

[0039] In some embodiments, step S4 comprises:

[0040] The individual diffusion weighted imaging data is fiber tract tracked;

[0041] The individual diffusion weighted imaging data is fiber tract cleaned to remove false positives or unreasonable trajectories generated in the fiber tracking process;

[0042] Fiber bundle integration is performed on individual diffusion weighted imaging data to aggregate a large number of discrete fiber bundles into anatomically reasonable whole fiber bundles.

[0043] In some embodiments, the fiber bundle cleaning comprises:

[0044] The fiber bundles are sorted by length, fiber bundle objects with a length being a mode of lengths of all fiber bundles are taken out, and outliers of all taken-out fiber bundles are calculated;

[0045] An outlier threshold is set;

[0046] The fiber bundles greater than or equal to the outlier threshold are filtered out.

[0047] In some embodiments, the fiber bundle integration comprises:

[0048] All remaining fiber bundles are integrated, and an integration formula is:

[0049]

[0050] The integrated fiber bundles are symmetrically averaged about a sagittal plane of a longitudinal fissure of the brain, and a symmetric averaging formula is:

[0051]

[0052] wherein, respectively represent three-dimensional coordinates of an i-th node of the integrated fiber bundle, C is a number of integrated fiber bundles, respectively represent three-dimensional coordinates of the i-th node of the j-th integrated fiber bundle, represents three-dimensional coordinates of the i-th node of the integrated fiber bundle when a first brain region of interest is a left brain region, represents three-dimensional coordinates of the i-th node of the integrated fiber bundle when the first brain region of interest is a right brain region, 、 respectively represent three-dimensional coordinates of the i-th node of the symmetric fiber bundle of the left and right brain regions after secondary processing.

[0053] In some embodiments, the step S5 comprises:

[0054] Based on the independent brain region model made in S1 and the connection path between brain regions constructed in S4, a brain network is constructed with each brain region in the independent brain region model as a point and with a fiber bundle as an edge;

[0055] A functional effect calculation is performed on brain function data, and a calculation result is displayed in the brain network; and through a collision detection technology, independent brain region models through which fiber bundles pass are detected, so that other potential brain regions associated with interaction between two brain regions in the independent brain region model are found.

[0056] Compared with the prior art, the present application has the following beneficial effects:

[0057] The present application realizes dynamic and interactive visualization of multi-modal brain data (such as BOLD signal, EEG trace signal) in three-dimensional space by constructing a double expression system of the cerebral cortex outer surface model and the independent brain region model, and combining high-precision voxel brain atlas projection rules; the brain region connection path with higher biological reality is constructed through fiber bundle tracking, cleaning and integration technology, which significantly improves the accuracy and intuitiveness of brain network and functional effect visualization; the limitations of complex operation and large calculation cost of traditional desktop software are overcome by using Web-side three-dimensional rendering and real-time interaction technology, which supports clinical real-time auxiliary decision-making and interactive teaching application; the potential brain region association is mined through collision detection technology, which enhances the ability of brain structure and function fusion analysis, and provides efficient and flexible visualization support for digital twin brain modeling and brain mechanism research. BRIEF DESCRIPTION OF DRAWINGS

[0058] Figure 1 is a flowchart of the present application;

[0059] Figure 2 is a flowchart of the present application brain cortex outer surface model and independent brain region model construction;

[0060] Figure 3 is a flowchart of the present application voxel brain atlas projection to the cerebral cortex outer surface model;

[0061] Figure 4 is an example diagram of the present application for BOLD signal and EEG trace signal visualization;

[0062] Figure 5 is a flowchart of the present application for constructing brain region connection path;

[0063] Figure 6 is an example diagram of the present application for brain network and brain region functional effect visualization. DETAILED DESCRIPTION

[0064] To make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below. Obviously, the described embodiments are part of, but not all of the embodiments of the present application; based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0065] As shown in Figure 1 A human brain digital twin 3D visualization method, comprising:

[0066] S1: constructing a cerebral cortex outer surface model and an independent brain region model;

[0067] S2: Register the voxelized brain atlas to the standard brain template space, and perform coordinate conversion to obtain the correspondence between the brain cortex outer surface model vertex and the brain atlas brain region label as the projection rule; the voxelized brain atlas used in the present application is AAL brain atlas, which is developed by University of Bordeaux, France and publicly released on its official website;

[0068] S3: According to the projection rule, the BOLD signal and the EEG backtracking signal are displayed to the brain cortex outer surface model;

[0069] S4: Construct the connection path between the independent brain region models, perform deterministic fiber bundle tracking, cleaning and integration of the most biologically realistic and representative fiber bundle between two brain regions as the connection path on the individual diffusion weighted imaging data; the individual diffusion weighted imaging data used in the present application is derived from the OpenNeuro platform, which is a public neuroimaging data sharing library;

[0070] S5: Integrate the connection path into the independent brain region model for visualizing the brain network and brain region functional effect, and use the collision detection technology to explore the potential brain regions associated with the fiber bundle between two brain regions.

[0071] In specific embodiments, the step S1 includes that the brain cortex outer surface model refers to a whole three-dimensional surface model constructed with the boundary of the cerebral cortex gray matter and cerebrospinal fluid, which reflects the complete morphological characteristics of the cerebral cortex. The independent brain region model is constructed by taking each brain region as a unit, and the partitioned representation of the cerebral cortex is formed by combining multiple independent brain region models.

[0072] The brain cortex outer surface model and the independent brain region model construction method includes: obtaining the left and right brain cortex reconstruction files of the individual in the standard template brain space, extracting the vertex coordinates (vertexs) and triangular faces (faces) in the file to construct the brain cortex outer surface model (type obj). After registering the voxelized brain atlas to the standard template brain space, the three-dimensional surface reconstruction is performed to construct the independent brain region model.

[0073] The brain cortex outer surface model is the basis for subsequent visualization. Considering its compatibility with various brain atlases and its universality in the field of brain science research, the brain cortex outer surface model of the present example selects the standard brain template space. The standard brain template space is an average model generated by spherical registration of the cerebral cortex of multiple healthy subjects, which has a consistent topological structure and standardized vertex distribution, and is one of the standard spaces widely used in current brain image processing and brain function visualization. The construction process is as follows Figure 2As shown, the pial surface data files (lh.pial and rh.pial) of the left and right hemispheres are loaded respectively using the pial surface file as the modeling basis, the vertex coordinates (vertexs) and triangular face (faces) information of the cerebral cortex surface are extracted by analyzing the pial surface data files in Python through the neuroimage processing library nibabel, and the mesh surface geometry is constructed. Subsequently, the above geometric information is organized and encoded in the Wavefront file (obj) format to generate a standard three-dimensional model file to support subsequent loading, rendering and interactive visualization operations in Three.js. This method has the advantages of format universality and strong compatibility, and can adapt to various visualization platforms and three-dimensional engines, facilitating cross-system integration and subsequent function expansion.

[0074] The construction of the independent brain region model needs to be based on the brain atlas, which is a tool for systematic partitioning and labeling of the brain. It divides the complex brain structure into several brain regions based on anatomy, function or connectivity, so as to better study its structure and function. The brain region is the basic unit of the brain atlas, and different brain regions undertake different perception, movement or high-level cognitive functions. Voxelized brain atlas refers to a kind of brain atlas expression method that divides the brain according to a regular three-dimensional grid and labels the brain region to which each voxel belongs.

[0075] The embodiment adopts the voxelized brain atlas to construct the independent brain region model, and the construction process is as shown in Figure 2 First, the voxelized brain atlas is spatially registered, and the target space is the standard brain template space. The registered label image is read by nibabel, and the voxel region corresponding to each label is extracted as the binary mask of the independent brain region. For each mask, the MarchingCubes algorithm is used to extract the isosurface to obtain the triangular mesh structure of the brain region, and the vertex coordinates are mapped from the voxel space to the real space through the affine matrix. Finally, the vertex and face information of each brain region is constructed into a three-dimensional model in the standard obj format, which is convenient for loading and rendering in the subsequent system. This realizes the precise separation modeling of the brain region structure and supports regional visualization and personalized interaction.

[0076] In the specific embodiment, the step S2 comprises:

[0077] S21: registering the voxelized brain atlas from other space to the standard brain template space, and initially projecting the voxelized brain atlas in the standard brain template space to the pial surface model;

[0078] In a specific embodiment, the brain mapping projection is to project the voxelized brain mapping to the brain cortex model surface. The projection involves the voxelized brain mapping and the brain cortex outer surface model, which can belong to different spaces. In this embodiment, the AAL voxelized brain mapping with 90 brain regions in the MNI space is projected to the brain cortex outer surface model in the standard brain template space. The specific steps are as shown in Figure 3 .

[0079] Since the voxelized brain mapping and the S1-made brain cortex outer surface model belong to different spaces, spatial registration is needed. First, prepare the T1 structural image of the voxelized brain mapping and the T1 structural image of the brain cortex outer surface model. To ensure the quality of registration, both need to be skull-removed. Then, perform rigid alignment to roughly position, avoid scale and deformation interference, and then perform affine registration to adjust the size and shape differences. Finally, perform nonlinear registration (SyN) to capture local differences in anatomical structure, obtain a set of transformation matrices of the T1 structural image from the MNI space to the standard brain template space, and apply the matrix to the voxelized brain mapping label file in the MNI space to obtain the brain mapping label of the voxelized brain mapping in the standard brain template space. Use the mri_vol2surf command to preliminarily project the voxelized brain mapping obtained in the previous step to the brain cortex outer surface model to obtain the preliminary mapping relationship between the brain cortex outer surface model vertex and the brain region. At this time, the brain cortex model is as shown in Figure 3 . The brain cortex region position is roughly correct, but there are serious internal vacancies, edge splitting, and sulcus and gyrus confusion, so further processing is needed.

[0080] S22: Interpolate and fill the internal vacancy position of the brain cortex outer surface model brain region;

[0081] Design a nearest neighbor majority voting vacancy filling algorithm to repair the internal vacancy of the brain region caused by the preliminary projection, thereby improving the continuity and integrity of the brain region label on the cortex surface. The process is as follows:

[0082] 1) Collect input data, which includes:

[0083] A set of vertex coordinates of the brain cortex outer surface model;

[0084] A brain region label array corresponding to the set of vertex coordinates, wherein the unmarked vertex is identified by a specific label value;

[0085] The number of nearest neighbor vertices K;

[0086] The proportion threshold T, where 0<T≤1;

[0087] 2) Identify all vertexes with the specific label value in the array of brain region labels as missing vertexes to be filled, specifically including:

[0088] For each of the missing vertexes:

[0089] Calculate the Euclidean distance between the missing vertex and all labeled vertexes;

[0090] According to the Euclidean distance, select the K nearest vertexes from all labeled vertexes as the nearest neighbor vertex set;

[0091] Statistical frequency of each brain region label in the nearest neighbor vertex set;

[0092] Determine whether the frequency of the highest frequency brain region label is greater than the proportion threshold T;

[0093] If yes, assign the highest frequency brain region label to the current missing vertex;

[0094] Output the filled brain region label array, where the intra-brain region vacancy is corrected.

[0095] S23: Fine division of brain region edges to solve the problem of brain region edge fragmentation and confusion of sulcal brain regions.

[0096] After solving the intra-brain region vacancy problem, fine division of brain region edges is performed to solve the problem of brain region edge fragmentation. The division idea is to find the voxel coordinates of the edge vertex in the voxelized brain atlas voxel space, and to determine the brain region label of the edge vertex according to the voxel position and its surrounding information. The formula for converting the real space coordinates of the brain cortex outer surface model vertex to the voxel coordinates of the voxelized brain atlas is as follows:

[0097]

[0098] Where x, y, z are the three-dimensional coordinates of the voxelized brain atlas with the center as the origin, and i, j, k represent the real space coordinates of the vertex in the brain cortex outer surface model. In the NIfTI format image file, the header (header) contains the key information of the image, is the conversion matrix from NIfTI header to real space coordinates.

[0099] After obtaining the voxel coordinates, the corresponding brain region of the vertex can be determined according to the brain atlas label. However, considering that the edge position is more sensitive to small errors in coordinate conversion, the surrounding voxels need to be considered when determining the label. The specific formula is as follows:

[0100]

[0101] Where, For the brain region label of a vertex in the cerebral cortex surface model, mode is the mode operation, x, y, z are the corresponding coordinates in the voxel brain atlas, and r is 1. That is, the mode of the brain region labels of the 26 voxels around the corresponding coordinates in the voxel space is taken as the final vertex brain region label.

[0102] After the above processing, the 2 164k vertex and brain region label correspondence relationship of the cerebral cortex surface model is obtained.

[0103] In specific embodiments, in step S3, the BOLD signal and EEG source signal visualization method is:

[0104] S31: Visualize the BOLD signal and EEG source signal on the cerebral cortex surface model according to the projection rule;

[0105] In specific embodiments, after processing steps S1 and S2, the vertex and brain region correspondence relationship of the cerebral cortex surface model has been obtained. Next, the BOLD signal and the like are visualized on the cerebral cortex surface model using the vertex and brain region correspondence relationship of the cerebral cortex surface model as the projection rule. However, since the obj model file allows multiple triangular faces to share the same vertex, and Three.js does not allow sharing due to normal problems, the number of vertices of the Mesh object (definition of the model in Three.js) increases significantly after loading the model. Therefore, the vertex and brain region correspondence relationship of the cerebral cortex surface model needs to be interpolated according to the vertex order of the Mesh object.

[0106] Digital twin brain is mostly constructed in brain region units to build dynamic models, so the BOLD signal generated by the Balloon-Windkessel model after simulation of the state variable is also in brain region units. EEG source signals may be traced in brain region units, or in vertex units, but there is no essential difference between the two in the visualization part.

[0107] After visualization, interaction with the model needs to be supported. Simple interaction capabilities can be achieved by adding a controller, such as rotating the model, changing the viewing angle, and scaling the model. Other interaction methods involving the meaning of the model itself are more complex, and the core is to recognize the triangular face or vertex clicked by the mouse. First, the coordinates of the mouse in the WebGL coordinate system are obtained, and the conversion formula is:

[0108]

[0109] wherein are the horizontal and vertical coordinates when the mouse is clicked, are the left boundary and upper boundary of the model display canvas, are the width and height of the model display canvas.

[0110] Subsequently, a ray is formed in WebGL according to the camera and the click coordinates to form a model, and it is detected whether it intersects with the model. If it intersects, it means that the point intersects with the model, the first intersection object is obtained, and thus the basis for interacting with the inside of the model is realized. On this basis, various flexible interaction modes can be realized, such as highlighting the selected brain area, positioning the stimulation position, and viewing the signal intensity at the specific position. Figure 4 The visualization effect and the interaction example are shown.

[0111] S32: The BOLD signal and the EEG backtracking signal are subjected to maximum-minimum standardization, so that the signal data is controlled in the range of 0-1, and is mapped to a specific color through a color mapping table;

[0112] The standardization formula is as follows:

[0113]

[0114] Where x is the original data, x' is the standardized data, x max is the maximum value in the original data, and x min is the minimum value in the original data.

[0115] A suitable color atlas is selected, the standardized source data of the visualization is converted into color representation through the color atlas, the color of all the vertices of the brain cortex outer surface model belonging to the brain area is determined through the correspondence between the brain cortex outer surface model vertices and the brain area, the vertex dyeing technology can be used to realize the dyeing of the vertices and the color change of the triangular faces, and all the pixel points in the triangular face are interpolated by the barycentric coordinate interpolation method according to the color of the three vertices of the triangular face.

[0116] S33: The vertices of the brain cortex outer surface model are colored, and the surface of the brain cortex outer surface model is colored.

[0117] In order to realize the complete visualization of the BOLD signal and the EEG backtracking signal or the real-time visualization of the digital twin brain, the vertex dyeing must be efficient and dynamic. Since the model itself has 2 164k vertices, and the vertices are increased again after the model is loaded, a memory buffer with a size of 3 times the number of vertices is created before coloring using the spatial caching technology, and the RGB color three components of each vertex are stored. The color data in the cache is modified and colored by using a timer or an event trigger, and finally the color switching speed of milliseconds can be realized.

[0118] In specific embodiments, in order to better visualize the brain network and the functional effect of brain regions, it is necessary to establish the connection path between each two brain regions in the independent brain region model. Since the fiber bundle is highly matched with the biological nature of brain connection in terms of structural authenticity, functional correlation and spatial continuity, and also takes into account the intuitiveness and scientific accuracy of visualization, the fiber bundle is used as the connection path between brain regions.

[0119] The construction process is roughly divided into three parts of fiber bundle tracking, fiber bundle cleaning and fiber bundle extraction and integration, Figure 6 The whole process. Since the extracted fiber bundle serves as an edge for brain network and brain region functional effect visualization, fiber bundle tracking needs to be based on brain atlas for region of interest tracking between brain regions.

[0120] In specific embodiments, the step S4 comprises:

[0121] S41: performing fiber bundle tracking on individual diffusion weighted imaging data;

[0122] Fiber bundle tracking: Before fiber bundle tracking, spatial noise reduction, Gibbs ring artifact correction, head motion distortion correction and other preprocessing are required for individual diffusion weighted imaging data. Then the individual diffusion weighted imaging data and the voxelized brain atlas are registered to the T1 structural image. The voxelized brain atlas is used to divide 90 regions of interest, and deterministic fiber bundle tracking is performed between each two regions of interest. The tracking algorithm is iFOD2, the threshold of FOD amplitude is set to 0.05, the maximum bending angle between adjacent tracking steps is set to 45 degrees, and the minimum fiber bundle length is set to 20 mm and the maximum length is set to 200 mm.

[0123] S42: cleaning the individual diffusion weighted imaging data to remove false positives or unreasonable trajectories generated during fiber tracking; avoid short, long or anatomically inconsistent fibers to interfere with subsequent fiber bundle integration; thereby improving the accuracy and purity of the results, providing a clearer and more anatomically meaningful basis for subsequent visualization presentation;

[0124] The fiber bundle tracking result is saved in a.tck (Track File) file. If all fiber bundles in each file are visualized as edges between brain regions, the number of fiber bundles between each two brain regions is about several hundred on average, which will make the network complex. Therefore, the fiber bundles between two brain regions need to be integrated into one representative fiber bundle. The fiber bundles are sorted by length, and the fiber bundle object with the mode length of all fiber bundles is taken out. The outlier value of all taken fiber bundles is calculated, which integrates the Euclidean distance and Z-Score standardization method. The calculation formula is as follows:

[0125]

[0126] Where N is the total number of fibers, This represents the average Euclidean distance between points on the i-th and j-th fibers. It is the average Euclidean distance between the i-th fiber bundle and all other fiber bundles, calculated as follows:

[0127]

[0128] Where M is the number of nodes in each fiber. These are the three-dimensional coordinates of the k-th node of the i-th fiber. This represents the Euclidean norm.

[0129] Then, the Z-score of this fiber bundle is calculated as the outlier value, as shown in the following formula:

[0130]

[0131]

[0132] in, and They are respectively The mean and variance, For the Z-score of the i-th fiber bundle, an outlier threshold of 2.0 is set. When the value exceeds 2.0, the i-th fiber bundle is filtered out.

[0133] S43: Fiber tract integration is performed on individual diffusion-weighted imaging data, aggregating numerous discrete fiber tracts into anatomically plausible overall fiber tracts. This reduces the uncertainty introduced by individual fibers, improving data stability and interpretability. The integrated fiber tracts morphologically more closely resemble the actual course of white matter, aiding not only in quantitative analysis and network research but also significantly enhancing visualization, making the overall structure of brain white matter connections more intuitive and clear.

[0134] Finally, all remaining fiber bundles are integrated using the following formula:

[0135]

[0136] in, Let represent the three-dimensional coordinates of the i-th node of the integrated fiber bundle, and C be the number of fiber bundles being integrated. Let represent the three-dimensional coordinates of the i-th node of the j-th integrated fiber bundle. This involves averaging the three-dimensional coordinates of the vertices of all remaining fiber bundles in the same order. Finally, to improve the distribution of fiber bundles between the left and right hemispheres, for each pair of brain regions, considering the symmetry of the fiber bundle distribution in the same brain regions of the other hemisphere, the two fiber bundles are symmetrically averaged about the sagittal plane of the longitudinal fissure of the brain.

[0137]

[0138] wherein, Xi, Yi, Zi represent the three-dimensional coordinates of the i-th node of the integrated fiber bundle when the first brain region of interest is the left brain region. Xi, Yi, Zi represent the three-dimensional coordinates of the i-th node of the integrated fiber bundle when the first brain region of interest is the right brain region. 、 Xi, Yi, Zi represent the three-dimensional coordinates of the i-th node of the symmetric fiber bundle after secondary processing of the left and right brain regions, respectively. The formula defines the three-dimensional coordinates of the two symmetric fiber bundles of the left and right hemispheres. For the x coordinate, the absolute values of the two are averaged, and then the left hemisphere takes the negative number, while the right hemisphere remains unchanged. For y and z coordinates, the absolute values of the corresponding coordinates of the left and right brain regions are directly taken as half.

[0139] After this step, except for individual brain regions between the two brain regions without tracking effective fiber bundles, the rest of the brain region pairs are integrated into a fiber bundle representing the independent brain region model between the two brain regions.

[0140] In specific embodiments, in step S5, the brain network and brain region function effect visualization method comprises:

[0141] S51: based on the independent brain region model made in S1 and the inter-brain region connection pathways constructed in S4, the brain network is constructed with each brain region model as a point and the fiber bundle object as an edge;

[0142] S52: function effect calculation is performed on the brain function data, and the calculation results are displayed in the brain network. The brain regions passed through by a certain fiber bundle are detected by collision detection technology, so as to discover other potential brain regions associated with the interaction of two brain regions.

[0143] In specific embodiments, the visualization of the brain network includes two parts of nodes and edges, wherein the points are represented by each independent brain region in the independent brain region model generated in S1, and the edges are represented by the fiber bundle object integrated in step four. The brain region function effect is based on the brain network, and the brain region function effect data (such as Pearson coefficient, etc.) is used to give the network edge weight, and the collision detection technology can be used to explore the potential intermediate brain regions when any two brain regions act.

[0144] Brain network visualization: Load each independent brain region model in the same virtual 3D space (Scene) in Three.js, and label the 3D geometry object (Mesh) generated after loading each model to detect which brain region the model belongs to. Load the representative fiber bundle of any two brain regions in the same virtual 3D space (Scene) in sequence. To ensure the accuracy of the subsequent collision detection structure, the fiber bundle and the independent brain region model need to belong to the same space. In this example, both are in the same space. Abstract the brain region as a small ball, and connect the small balls with straight lines to obtain the traditional visualization method. Using fiber bundles as edges is more realistic and effective, and the usability is higher when combined with brain region screening tools. The display effect is shown in Figure 6 .

[0145] Brain region functional effect visualization: Brain region functional effect gives weight to the fiber bundle object based on brain network. Analyze and calculate all data that can calculate brain region functional effect, such as EEG backtracking data and BOLD signal, and use the result as edge weight. In visualization, use fiber bundle color, thickness, and other features to represent functional effect strength, and set weight threshold, brain region screening, and other tools for use. For the effect between two brain regions, the potential brain region is explored by using the fiber bundle object in the scene for collision detection. The collision detection process is as follows:

[0146] Any fiber bundle can be represented as , where represents the coordinates of the i+1th node of the fiber bundle, i takes values from 0 to n, and a minimum sub-segment is formed by taking two adjacent nodes , which is represented as:

[0147]

[0148] where t represents the position ratio of a point in the line segment L in the line segment, and each forms a ray , which is represented as:

[0149]

[0150] where is the unit direction vector of the line segment , and t is a variable within the length range of , representing the reachable range of the ray . Then get all the triangular faces in each independent brain region model, and detect whether the ray intersects the triangular face according to the Moller-Trumbore algorithm. If it intersects, add the brain region where the triangular face is located to the potential brain region list. Repeat the above operation to detect all sub-segments, and remove the potential brain region list to obtain all potential brain regions. Highlight the start point, end point, and potential brain region in the scene.

[0151] In this embodiment, the brain cortex outer surface model and the independent brain region model are extracted and converted into a Three.js loadable mode, the model visualization is loaded on the web side, and the complexity of the previous visualization is reduced; a voxel brain atlas to brain cortex outer surface model projection method is designed to realize brain atlas, BOLD signal and EEG trace signal visualization, improve the interactivity and real-time performance; and the integrated fiber bundle object is integrated into the independent brain region model to make the brain network and brain function effect visualization expression stronger and more biologically realistic.

[0152] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for 3D visualization of a human brain digital twin, characterized in that, include: S1: Construct models of the outer surface of the cerebral cortex and independent brain regions; S2: Register the voxelized brain map to the standard brain template space and perform coordinate transformation to obtain the correspondence between the vertices of the outer surface model of the cerebral cortex and the brain region labels of the brain map as the projection rule; Step S2 includes: Voxelized brain atlases were registered from other spaces to standard brain template space, and the voxelized brain atlases in standard brain template space were projected onto the outer surface model of the cerebral cortex. Interpolation was used to fill in the gaps inside the brain regions of the outer surface model of the cerebral cortex. To address the issues of fragmented brain region boundaries and disordered sulci and gyri, finely subdividing the brain region boundaries is necessary. The interpolation filling of gaps within brain regions of the external surface model of the cerebral cortex includes: Collect input data, which includes: The set of vertex coordinates of the outer surface model of the cerebral cortex; An array of brain region labels corresponding to the set of vertex coordinates, wherein unlabeled vertices are identified by specific label values; The number of nearest neighbor vertices, K; The percentage threshold T, where 0 <T≤1; Identify all vertices in the brain region label array whose label value is the specific label value, as vacant vertices to be filled, specifically including: For each of the aforementioned empty vertices: Calculate the Euclidean distance between the missing vertex and all labeled vertices; Based on the Euclidean distance, select the K nearest vertices from all labeled vertices as the nearest neighbor set; Statistically analyze the frequency of occurrence of each brain region label in the nearest neighbor vertex set; Determine whether the frequency of the brain region label with the highest occurrence frequency is greater than the percentage threshold T; If the determination is yes, then the label of the brain region with the highest frequency of occurrence is assigned to the current vacant vertex; Output the array of brain region labels after imputation; S3: According to the projection rules, the BOLD signal and the EEG source signal are displayed on the outer surface model of the cerebral cortex; S4: Construct the connection pathways between brain regions in the independent brain region model, and perform deterministic fiber tract tracking, cleaning and integration of any two brain regions with biologically realistic and representative fiber tracts as connection pathways. S5: Integrating connectivity pathways into independent brain region models to visualize brain networks and brain region functional effects, and using collision detection technology to discover potential brain regions that are linked by fiber bundles.

2. The method for 3D visualization of a human brain digital twin according to claim 1, characterized in that, Step S1 includes obtaining reconstruction files of the left and right cerebral cortexes of an individual under a standard brain template space, and extracting model vertices and triangular faces from the reconstruction files of the left and right cerebral cortexes to construct a model of the outer surface of the cerebral cortex. After registering the voxelized brain atlas to the standard brain template space, three-dimensional surface reconstruction is performed to construct independent brain region models.

3. The method for 3D visualization of a human brain digital twin according to claim 1, characterized in that, The fine division is based on the following formula: in, This represents the brain region label for a vertex in the external cortical surface model, where `mode` is the mode operator, `x`, `y`, and `z` are the voxel coordinates of that vertex in the voxelized brain atlas, and `r` is 1. This refers to the brain region labels for the 26 voxels surrounding the corresponding voxel in the voxelized brain atlas. affine -1 This is the transformation matrix from the NIfTI header to real space coordinates.

4. The method for 3D visualization of a human brain digital twin according to claim 1 or 3, characterized in that, Step S3 includes: Visualize the BOLD signal and EEG source signal on the outer surface model of the cerebral cortex according to the projection rules; The BOLD signal and EEG traceability signal are standardized to the maximum and minimum values ​​to control the signal data within the range of 0-1, and then mapped to different colors using a color mapping table. Coloring the vertices of the outer surface model of the cerebral cortex enables surface coloring of the outer surface model of the cerebral cortex.

5. The method for 3D visualization of a human brain digital twin according to claim 1, characterized in that, Step S4 includes: Fiber bundle tracking of individual diffusion-weighted imaging data; Fiber bundle cleaning is performed on individual diffusion-weighted imaging data to remove false positives or unreasonable trajectories generated during fiber tracking. Fiber bundle integration is performed on individual diffusion-weighted imaging data, aggregating a large number of discrete fiber bundles into an anatomically reasonable whole fiber bundle.

6. The method for 3D visualization of a human brain digital twin according to claim 5, characterized in that, The fiber bundle cleaning includes: Sort the fiber bundles by length, extract the fiber bundle objects whose length is the mode of all fiber bundle lengths, and calculate the outlier value of all extracted fiber bundles; Set an outlier threshold; Filter out fiber bundles that are greater than or equal to the outlier threshold.

7. The method for 3D visualization of a human brain digital twin according to claim 5, characterized in that, The fiber bundle integration includes: All remaining fiber bundles are integrated using the following formula: The integrated fiber bundles are symmetrically averaged in the sagittal plane about the longitudinal fissure of the brain. The formula for symmetrical averaging is: in, Let represent the three-dimensional coordinates of the i-th node of the integrated fiber bundle, and C be the number of fiber bundles being integrated. Let represent the three-dimensional coordinates of the i-th node of the j-th integrated fiber bundle, respectively. This represents the three-dimensional coordinates of the i-th node of the integrated fiber tract in the first brain region of interest, which is the left hemisphere. This represents the three-dimensional coordinates of the i-th node of the integrated fiber tract in the first brain region of interest, which is the right brain region. , These represent the three-dimensional coordinates of the i-th node of the symmetrical fiber bundles in the left and right hemispheres after secondary processing.

8. The method for 3D visualization of a human brain digital twin according to claim 1, characterized in that, Step S5 includes: The connection pathways between the independent brain region model created by S1 and the brain region constructed by S4 are used to construct a brain network with each brain region in the independent brain region model as a point and fiber bundles as edges. Functional effect calculations are performed on brain function data, and the results are displayed on the brain network. Collision detection technology is used to detect independent brain region models through which fiber tracts pass, thereby discovering other potential brain regions associated with the interaction of two brain regions in the independent brain region model.

Images