Adversarial generative network model for three-dimensional reconstruction of brain cell-level vascular network

Abstract

The invention relates to the technical field of medical image processing, and discloses an adversarial generative network model for three-dimensional reconstruction of a brain cell-level vascular network. The model comprises a three-dimensional blood vessel feature extraction module, a dynamic attention generator, a multi-scale discriminator and the like. Characteristics are obtained through multi-modal medical image data (magnetic resonance angiography and confocal microscope scanning data), and high-precision vascular network three-dimensional reconstruction is realized by utilizing cooperative work of all the modules. The dynamic attention generator calculates the topological connection probability of the vascular branches, and the multi-scale discriminator globally and locally evaluates the reconstruction result. Meanwhile, model training is optimized through an adversarial training controller, and the blood vessel network is perfected through a capillary network completion module. According to the method, the advantages of multi-modal data are effectively fused, the reconstruction precision is improved, and powerful support is provided for research and diagnosis of brain vascular diseases.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and in particular to a generative adversarial network model for three-dimensional reconstruction of brain cell-level vascular networks. Background Art

[0002] In medical research and clinical practice, accurate three-dimensional reconstruction of brain vascular networks is crucial for understanding brain physiological functions and diagnosing and treating various brain diseases. With the continuous advancement of medical imaging technologies, magnetic resonance angiography (MRA) and confocal microscopy have provided effective means for obtaining brain vascular information. However, current efforts to reconstruct brain vascular networks at the cellular level in three dimensions still face numerous challenges.

[0003] On the one hand, while MRA can visualize a wide range of vascular structures, its ability to display microscopic details of tiny vessels and their walls is limited, and its resolution struggles to achieve the precision required at the level of brain cells. Its imaging principle is based on the flow characteristics of blood and the differences in magnetic resonance signals from tissues. In the complex brain vascular environment, it is susceptible to interference from various factors, resulting in the loss or blurring of some tiny vessel signals, affecting the integrity and accuracy of reconstruction.

[0004] On the other hand, while confocal microscopy can provide high-resolution local vascular details, its limited imaging range makes it difficult to capture a complete picture of the entire brain vascular network. Furthermore, reconstruction using confocal microscopy data alone cannot effectively link local information with the overall vascular network, failing to meet the needs of holistic brain vascular network analysis.

[0005] Existing 3D reconstruction methods often fail to fully integrate the advantages of different modalities when processing multimodal data. Traditional feature fusion methods are simple and direct, but they do not take into account the differences in feature dimensions, scales, and data distribution between different modalities. As a result, the fused features cannot accurately reflect the true characteristics of the vascular network. During the reconstruction process, the detection of vascular bifurcation points and the completion of microvascular networks also suffer from low accuracy. This causes the reconstructed vascular network topology to deviate from the actual situation and cannot provide a reliable basis for subsequent medical research and clinical diagnosis.

[0006] Furthermore, current reconstruction models lack effective optimization strategies during training, resulting in slow model convergence and low reconstruction accuracy. Furthermore, there is a lack of comprehensive and accurate methods for evaluating reconstruction results, making it difficult to ensure the reliability and stability of the reconstructed models. These issues severely restrict the further application and development of 3D reconstruction technology for brain vascular networks in medicine, and a new technology is urgently needed to address these challenges. Summary of the Invention

[0007] The purpose of the present invention is to provide a generative adversarial network model for three-dimensional reconstruction of brain cell-level vascular networks to solve the problems raised in the above-mentioned background technology.

[0008] To achieve the above objectives, the present invention provides the following technical solution: a generative adversarial network model for three-dimensional reconstruction of brain cell-level vascular networks, the model comprising:

[0009] a 3D vascular feature extraction module for acquiring initial 3D voxel features of the brain vascular network from multimodal medical imaging data, the multimodal medical imaging data including magnetic resonance angiography and confocal microscopy scanning data, and mapping the initial 3D voxel features into a high-dimensional feature tensor;

[0010] A dynamic attention generator, configured to generate a three-dimensional reconstruction of a candidate vascular network based on the high-dimensional feature tensor, wherein the dynamic attention generator includes a spatiotemporal convolution kernel and a channel attention module, and calculates the topological connection probability of vascular branches through an adaptive weight allocation mechanism;

[0011] a multi-scale discriminator for performing resolution classification on the three-dimensional reconstruction results of the candidate vascular network, comprising a low-resolution global discriminant unit and a high-resolution local discriminant unit. The low-resolution global discriminant unit extracts the overall topological features of the vascular network by downsampling, and the high-resolution local discriminant unit extracts the microstructural features of the vascular wall by sliding window segmentation;

[0012] a vascular topology optimization module, configured to iteratively adjust the parameters of the dynamic attention generator according to the output of the multi-scale discriminator, and optimize the continuity of the 3D reconstruction results by minimizing the vascular branch breakage loss function;

[0013] a vascular wall morphology correction module, configured to generate vascular wall morphology correction parameters based on the vascular wall thickness distribution in the confocal microscope scanning data, and adjust the geometric shape of the vascular wall in the three-dimensional reconstruction result of the candidate vascular network through a dynamic convolution kernel;

[0014] An adversarial training controller, configured to coordinate the training process of the dynamic attention generator and the multi-scale discriminator, adopt an alternating optimization strategy, and dynamically adjust the learning rate decay factor according to the vascular network reconstruction accuracy after each round of training;

[0015] a blood vessel bifurcation point detection unit, configured to extract the spatial coordinates of the blood vessel bifurcation points from the high-dimensional feature tensor and calculate the optimal solution for the bifurcation point connection direction using a probability density estimation model;

[0016] The microvascular network completion module is used to generate candidate microvascular pathways based on the variational autoencoder for missing microvascular regions, and to screen the completion results that meet the hemodynamic constraints through the path energy function;

[0017] A multimodal feature fusion module is used to perform cross-modal alignment of the features of the magnetic resonance angiography data and the confocal microscopy scanning data, and to fuse the vascular wall thickness and blood flow velocity information of different modalities through a feature gating mechanism.

[0018] Preferably, the specific implementation method of the spatiotemporal convolution kernel and channel attention module is:

[0019] Extract spatiotemporal feature slices from the input high-dimensional feature tensor and normalize each slice by channel dimension;

[0020] The attention weight of each channel is calculated through a learnable parameter matrix, and the spatiotemporal features of different channels are weightedly fused;

[0021] The dynamic convolution kernel is used to adjust the expansion coefficient of the convolution kernel according to the curvature radius of the blood vessel branch. The expansion coefficient calculation formula is:

[0022] D=α·log(1+τ)+β

[0023] Where D is the dilation coefficient of the convolution kernel, τ is the local curvature of the blood vessel branch, and α and β are trainable parameters.

[0024] Preferably, the implementation method of the low-resolution global discriminant unit of the multi-scale discriminator includes:

[0025] The input 3D reconstruction result is downsampled by maximum pooling to generate low-resolution voxel blocks; the main topological structure features of the vascular network are extracted through the 3D convolution layer;

[0026] Calculate the global discriminant score, the formula is:

[0027] S global =MLP(Flatten(F global ))

[0028] Among them, S global is the global discrimination score at low resolution, F global is the global feature vector, and MLP is a multi-layer perceptron.

[0029] Preferably, the implementation method of the high-resolution local discrimination unit includes:

[0030] Segment the input 3D reconstruction into overlapping local voxel sub-blocks;

[0031] The surface texture features of the blood vessel wall are extracted through a three-dimensional residual network;

[0032] Calculate the local discriminant score, the formula is:

[0033]

[0034] Among them, S local is the local discrimination score at high resolution, P j is the jth sub-block, φ is the residual network parameter, and M is the total number of sub-blocks.

[0035] Preferably, the probability density estimation model of the blood vessel bifurcation point detection unit adopts a Gaussian mixture model, and the specific steps include:

[0036] Extracting a set of spatial coordinates of candidate bifurcation points from the high-dimensional feature tensor;

[0037] Fitting Gaussian mixture distribution parameters by expectation maximization algorithm;

[0038] The K bifurcation points with the highest probability density are selected as the connection direction candidate set, where K is a preset positive integer.

[0039] Preferably, the path energy function of the microvascular network completion module is defined as:

[0040] E=λ1·L length +λ2·L curvature +λ3·L pressure

[0041] Among them, E is the total energy of the completed path, L length is the path length constraint, L curvature is the curvature smoothness constraint, L pressure To simulate the physical constraints of blood flow pressure, λ1, λ2, and λ3 are weight coefficients.

[0042] Preferably, the cross-modal alignment method of the multimodal feature fusion module includes:

[0043] Voxel-wise registration of MRI data and confocal microscopy data was performed. The cross-modal feature map matrix was calculated using the mutual information maximization algorithm.

[0044] A gated recurrent unit is used to fuse the temporal hemodynamic features of different modalities.

[0045] Preferably, the calculation formula of the topological connection probability is:

[0046]

[0047] Among them, σ is the Sigmoid function, W k is the kth attention weight, F is the input feature tensor, θ k is the three-dimensional convolution kernel parameter, b is the bias term, and K is the total number of spatiotemporal convolution kernels;

[0048] The fracture loss function is defined as:

[0049]

[0050] Among them, S i represents the voxel gradient field of the i-th blood vessel segment, N is the total number of segments, is the divergence operator;

[0051] Preferably, the learning rate attenuation factor adjustment method of the adversarial training controller is:

[0052] The attenuation coefficient is dynamically calculated based on the accuracy of vascular branch connections on the validation set. The formula is:

[0053] η t+1 =η t ·exp(-γ·(1-Acc t ))

[0054] Among them, η t+1 is the learning rate for the t+1th round, η t is the learning rate for the tth round, γ is the decay rate, Acc t is the verification accuracy of the tth round.

[0055] Preferably, the dynamic convolution kernel parameter adjustment method of the blood vessel wall morphology correction module includes:

[0056] The vascular wall thickness distribution histogram was extracted based on confocal microscopy data;

[0057] The mapping relationship between thickness distribution and convolution kernel size is established through nonlinear regression model;

[0058] The adaptive step size gradient descent algorithm is used to optimize the expansion factor and contraction factor of the convolution kernel.

[0059] Compared with the prior art, the present invention has the following beneficial effects:

[0060] In terms of data utilization, a multimodal feature fusion module is used to cross-modally align and fuse the features of MRI and confocal microscopy data. This process not only fully leverages the advantages of MRI data in displaying the overall structure of blood vessels, but also takes advantage of the strengths of confocal microscopy data in presenting microscopic details of the vessel walls. For example, when analyzing the relationship between brain tumors and surrounding blood vessels, the distribution of tiny blood vessels around the tumor and the morphological changes of the vessel walls can be clearly seen, providing more comprehensive and accurate information for tumor diagnosis and treatment planning.

[0061] In terms of reconstruction accuracy, the initial three-dimensional voxel features obtained by the three-dimensional vascular feature extraction module are mapped into high-dimensional feature tensors, providing a rich information basis for subsequent processing. The dynamic attention generator combines the spatiotemporal convolution kernel with the channel attention module to adaptively calculate the topological connection probability of vascular branches, greatly improving the accuracy of the reconstruction results. The multi-scale discriminator discriminates at two levels: low-resolution global and high-resolution local. The low-resolution global discriminant unit extracts the overall topological features of the vascular network, and the high-resolution local discriminant unit captures the microstructural features of the vascular wall. The two work together to effectively improve the reconstruction model's ability to recognize and reconstruct features at different levels of the vascular network. Taking the diagnosis of cerebral vascular malformations as an example, it can more accurately present the overall morphology of the malformed blood vessels and the abnormal changes in the local vascular walls, assisting doctors in making more accurate judgments.

[0062] Targeting key structures within the vascular network, the vascular bifurcation point detection unit uses a Gaussian mixture model to accurately extract bifurcation point coordinates and calculate the optimal connection direction. The microvascular network completion module, based on a variational autoencoder and a path energy function, completes missing microvascular regions, ensuring the integrity and rationality of the vascular network topology. This complete and accurate vascular network model contributes to a deeper understanding of the interaction between nerves and blood vessels when studying the brain's neurovascular unit.

[0063] In terms of model training and optimization, the adversarial training controller employs an alternating optimization strategy to coordinate the training of the dynamic attention generator and the multi-scale discriminator. It also dynamically adjusts the learning rate attenuation factor based on the accuracy of vascular network reconstruction, accelerating model convergence and improving training efficiency. The vascular wall morphology correction module generates correction parameters based on confocal microscopy scan data, adjusting the vascular wall geometry to make the reconstruction more consistent with the actual physiological structure. In studies simulating brain vascular hemodynamics, more realistic vascular wall morphology can provide more accurate blood flow simulation results, providing strong support for pathological research on related diseases. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 This is a diagram showing the working principle of the adversarial generative network model described in the present invention;

[0065] Figure 2 This is a diagram of the working principle of the spatiotemporal convolution kernel and channel attention module;

[0066] Figure 3 This is a diagram showing the working principle of the low-resolution global discriminant unit of the multi-scale discriminator;

[0067] Figure 4 This is a diagram showing the working principle of dynamic convolution kernel parameter adjustment in the vascular wall morphology correction module. DETAILED DESCRIPTION

[0068] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0069] See also Figure 1-4 The present invention provides a generative adversarial network model for three-dimensional reconstruction of brain cell-level vascular networks, the model comprising:

[0070] 3D Vascular Feature Extraction Module: This module uses multimodal medical imaging data, namely magnetic resonance angiography and confocal microscopy data, to obtain initial 3D voxel features of the brain's vascular network and map them into high-dimensional feature tensors. This process provides foundational data for subsequent analysis and processing.

[0071] Dynamic Attention Generator: Based on the high-dimensional feature tensor obtained by the 3D Vessel Feature Extraction module, it generates a 3D reconstruction of the candidate vascular network. This generator includes a spatiotemporal convolution kernel and a channel attention module. Using an adaptive weight allocation mechanism, it calculates the topological connection probability of vascular branches to determine the connectivity between them.

[0072] Multi-scale discriminator: This discriminates the 3D reconstruction results of the candidate vascular networks generated by the dynamic attention generator at different resolution levels. A low-resolution global discriminator extracts the overall topological features of the vascular network through downsampling, while a high-resolution local discriminator extracts the microstructural features of the vascular wall through sliding window segmentation. The accuracy of the reconstruction results is evaluated at different resolution levels.

[0073] Vascular Topology Optimization Module: Based on the output of the multi-scale discriminator, the parameters of the dynamic attention generator are iteratively adjusted. By minimizing the vascular branch breakage loss function, the continuity of the 3D reconstruction results is optimized to ensure the topology of the vascular network is reasonable.

[0074] The vessel wall morphology correction module generates vessel wall morphology correction parameters based on the vessel wall thickness distribution in the confocal microscope scan data. It uses a dynamic convolution kernel to adjust the geometry of the vessel wall in the 3D reconstruction of the candidate vascular network, making the reconstructed vessel wall morphology more consistent with the actual situation.

[0075] Adversarial training controller: This controller coordinates the training of the dynamic attention generator and the multi-scale discriminator, using an alternating optimization strategy. After each round of training, the learning rate decay factor is dynamically adjusted based on the accuracy of vascular network reconstruction to improve model training efficiency and reconstruction accuracy.

[0076] Vascular bifurcation point detection unit: extracts the spatial coordinates of vascular bifurcation points from the high-dimensional feature tensor, and calculates the optimal solution for the bifurcation point connection direction through the probability density estimation model, providing a basis for accurately reconstructing the bifurcation structure of the vascular network.

[0077] Microvascular Network Completion Module: Based on the variational autoencoder, candidate microvascular pathways are generated for missing microvascular regions. Path energy functions are used to filter completion results that meet hemodynamic constraints, thereby improving the reconstruction of the vascular network.

[0078] Multimodal feature fusion module: Cross-modally aligns the features of magnetic resonance angiography data and confocal microscopy scanning data, and fuses the vascular wall thickness and blood flow velocity information of different modalities through a feature gating mechanism, fully leveraging the advantages of multimodal data to improve reconstruction effects.

[0079] The implementation of the present invention will be further described below with reference to Examples 1 to 6.

[0080] Example 1:

[0081] This example illustrates the specific implementation of the spatiotemporal convolution kernel and channel attention module in the dynamic attention generator, as well as the calculation method for topological connection probability. Its purpose is to improve the accuracy of the 3D reconstruction results of candidate vascular networks through more precise feature extraction and weight assignment.

[0082] In the dynamic attention generator, the spatiotemporal convolution kernel and channel attention module extract spatiotemporal feature slices from the input high-dimensional feature tensor. This process decompiles high-dimensional data, slicing the complex high-dimensional information along the spatiotemporal dimensions for subsequent analysis. Each slice is normalized to the channel dimension, ensuring uniform scale for data across different channels, eliminating the effects of data scale differences and ensuring the accuracy of subsequent calculations.

[0083] The attention weights for each channel are calculated using a learnable parameter matrix. This learnable parameter matrix is ​​a key component of continuous optimization during model training. It automatically adjusts the focus on different channel features based on the data characteristics and the model's learning objectives. The calculated attention weights are then weighted and fused with the spatiotemporal characteristics of the different channels to highlight features that are important to the reconstruction and suppress less important features, thereby ensuring that the 3D reconstructions generated by the model are more consistent with the actual vascular network structure.

[0084] When calculating the topological connection probability, the formula is used:

[0085]

[0086] Among them, σ is the Sigmoid function, which maps the weighted result to between 0 and 1 to represent the probability of topological connection; W k is the kth attention weight, reflecting the importance of different convolution kernels in calculating the connection probability; F is the input feature tensor, which contains various feature information of the brain vascular network; θ k is the 3D convolution kernel parameter, which determines the method and range of the convolution operation; b is the bias term, which adjusts the offset of the calculated result; and K is the total number of spatiotemporal convolution kernels. This formula comprehensively considers the effects of multiple convolution kernels and more accurately calculates the topological connection probability between vascular branches.

[0087] Example 2:

[0088] This embodiment details the implementation of the low-resolution global discriminant unit and the high-resolution local discriminant unit in the multi-scale discriminator. Its function is to comprehensively evaluate the 3D reconstruction results of the candidate vascular network at different resolution levels, improve the discrimination accuracy, and thus optimize the reconstruction results.

[0089] For the low-resolution global discriminant unit, the input 3D reconstruction result is first downsampled by maximum pooling. The maximum pooling operation is a commonly used downsampling technique. It reduces the resolution of the data while maintaining the main features, converting the 3D reconstruction results into low-resolution voxel blocks, reducing the amount of data, and facilitating the subsequent rapid extraction of overall topological features. The low-resolution voxel blocks are convolved through the 3D convolution layer to extract the main topological structural features of the vascular network. The 3D convolution layer can convolve the data in 3D space to capture the spatial structural information of the vascular network. The global discriminant score is calculated using the formula:

[0090] S global =MLP(Flatten(F global ))

[0091] Among them, S global is the global discrimination score at low resolution, F global is a global feature vector that contains the overall topological information of the vascular network; MLP is a multi-layer perceptron, which is a neural network structure containing multiple neurons. It performs nonlinear transformation on the global feature vector and outputs a comprehensive evaluation score to measure the accuracy of the reconstruction result in the overall topological structure.

[0092] The high-resolution local discriminant unit segments the input 3D reconstruction results into overlapping local voxel sub-blocks. This segmentation method allows for a more detailed observation of the microstructure of the vessel wall, avoiding missing local details. The 3D residual network is used to extract the surface texture features of the vessel wall. The 3D residual network can effectively solve the gradient vanishing problem in deep neural networks, allowing the network to learn more complex features and has a good ability to capture subtle texture and structural changes on the surface of the vessel wall. The local discriminant score is calculated using the formula:

[0093]

[0094] Among them, S local is the local discrimination score at high resolution, P j is the jth sub-block, φ is the residual network parameter, and M is the total number of sub-blocks. This formula performs convolution calculations on all sub-blocks and takes the average to obtain a score that reflects the accuracy of local features, thereby evaluating the quality of the reconstruction results from a local level.

[0095] Example 3:

[0096] This example focuses on the specific implementation steps of a Gaussian mixture model in the probability density estimation model of the vascular bifurcation point detection unit. Its key role is to accurately locate vascular bifurcations and determine the optimal solution for their connection directions, which is crucial for accurately reconstructing the bifurcation structure of the vascular network and restoring the true topology of the entire vascular network.

[0097] When extracting the spatial coordinate set of candidate bifurcation points from a high-dimensional feature tensor, a specific image feature recognition algorithm is used to traverse the high-dimensional feature tensor. The high-dimensional feature tensor is obtained by processing multimodal medical imaging data using a three-dimensional vascular feature extraction module. It contains rich vascular network information, including not only the location of the vessels but also their morphology, direction, and other characteristics. The algorithm analyzes and judges each voxel in the tensor based on preset bifurcation point feature rules, such as changes in the number of vascular branches within a certain neighborhood and sudden changes in local curvature. Once a voxel that meets the bifurcation point characteristics is detected, its spatial coordinates are recorded, gradually forming a set of spatial coordinates for candidate bifurcation points.

[0098] Next, the expectation-maximization algorithm is used to fit the parameters of the Gaussian mixture distribution. The expectation-maximization algorithm is an iterative optimization algorithm. In this scenario, the initial parameters of each Gaussian distribution in the Gaussian mixture model are first set, such as the mean, variance, and mixing coefficient. The mean represents the center position of each Gaussian distribution, the variance reflects the dispersion of the distribution, and the mixing coefficient indicates the relative importance of each Gaussian distribution in the overall mixture model. After setting the initial parameters, the iterative process begins. In the expectation step, the probability of each candidate bifurcation point belonging to each Gaussian distribution is calculated based on the current model parameters. This step is based on the probability density function of the Gaussian distribution. In the maximization step, these probabilities are used to re-estimate the parameters of the Gaussian mixture model. By calculating and analyzing all candidate bifurcation points, the mean, variance, and mixing coefficient are continuously adjusted to improve the model's fit to the data. After multiple rounds of iteration, when the change in model parameters is less than a preset threshold, it is considered that a relatively suitable Gaussian mixture distribution parameter has been found.

[0099] Finally, the K bifurcation points with the highest probability density are selected as the candidate connection direction set. Based on the fitted Gaussian mixture model, the probability density of each candidate bifurcation point is calculated. The higher the probability density, the more likely the bifurcation point is in the model and the more likely it is to be a true bifurcation point. All candidate bifurcation points are sorted from highest to lowest probability density, and the top K bifurcation points are selected. K is a positive integer pre-set based on actual needs and experience. These K bifurcation points constitute the candidate connection direction set, providing high-quality candidate data for subsequent calculation of the optimal solution for the bifurcation point connection direction.

[0100] Example 4:

[0101] This example focuses on the specific application and significance of the path energy function in the microvascular network completion module. Its core purpose is to select completion paths that conform to hemodynamic laws and actual physiological needs when completing missing microvascular regions by comprehensively considering multiple key factors, thereby significantly improving the accuracy and reliability of microvascular network reconstruction.

[0102] The path energy function used by the microvascular network completion module is defined as:

[0103] E=λ1·L length +λ2·L curvature +λ3·L pressure

[0104] In this formula, E represents the total energy of the completion path, which is a comprehensive evaluation indicator used to measure the quality of the generated completion path. lengthRepresents the path length constraint. In the actual microvascular network, the length of the blood vessels does not change arbitrarily, but is limited by physiological functions and metabolic needs. If the completion path is too long, it will increase the resistance to blood flow and consume too much energy; if it is too short, it may not be able to effectively connect the various vascular regions, affecting material exchange and blood circulation. Therefore, by introducing the path length constraint, it is ensured that the length of the generated completion path conforms to the actual situation of the microvascular network. For example, when completing the microvessels in a specific area, according to the prior knowledge of the blood vessel distribution density and average blood vessel length in the area, a reasonable length range can be set so that L length Take the value within this range.

[0105] L curvature It is a curvature smoothness constraint. The path of the microvessel is usually smooth at the microscopic level, without sudden sharp turns or sharp bends. Because too sharp bends will lead to uneven blood flow velocity, increase the risk of thrombosis, and are not in line with physiological laws. The curvature smoothness constraint ensures the smoothness of the completed path by limiting the curvature of the path. During the calculation process, numerical calculation methods can be used to calculate the curvature of each point on the path. For example, the curvature can be approximated by performing differential operations on the coordinates of adjacent points on the path. Then, according to the preset curvature threshold, the path is adjusted so that L curvature Meet the smoothness requirements.

[0106] L pressure It is a physical constraint that simulates blood flow pressure. Considering the flow characteristics of blood flow in microvessels, the pressure distribution has an important influence on the morphology and path of blood vessels. In the actual physiological environment, the blood flow pressure in microvessels is dynamically changing and interacts with the structure of blood vessels. By introducing physical constraints that simulate blood flow pressure, the completion path is made more reasonable in terms of hemodynamics. For example, based on the principles of fluid mechanics, a relationship model between blood flow pressure and blood vessel radius, length and blood flow velocity can be established, and L can be calculated based on this model. pressure The value of ensures that the generated completion path can adapt to changes in blood flow pressure.

[0107] λ1, λ2, and λ3 are weight coefficients used to adjust the relative importance of different constraints in the total energy calculation. In practical applications, the values ​​of these weight coefficients are determined through experimentation and optimization based on specific needs and data characteristics. For example, in some cases, where blood pressure has a significant impact on microvasculature, the value of λ3 can be appropriately increased to emphasize the role of the physical constraint of simulated blood pressure in path selection. In other cases, if path smoothness is more important, the value of λ2 can be increased. By properly adjusting the weight coefficients, more realistic completion results can be obtained.

[0108] Example 5:

[0109] This example illustrates the cross-modal alignment method of the multimodal feature fusion module. Its key role is to effectively integrate the features of magnetic resonance angiography data and confocal microscopy data, leveraging the advantages of both modalities to improve the accuracy and reliability of 3D vascular network reconstruction.

[0110] Voxel-wise registration is performed on MRI data and confocal microscopy data. Magnetic resonance angiography (MRA) provides a wide range of global vascular structural information, while confocal microscopy data provides clearer visualization of vascular wall details at the microscopic level. The goal of voxel-wise registration is to precisely align these two different modalities so that they share the same coordinate system and spatial resolution. During the registration process, several distinct feature points are first selected, such as vascular branch points and intersections, that are clearly identifiable in both modalities. These feature points are then used to calculate a spatial transformation matrix from one modality to the other using a specific registration algorithm, such as one based on rigid or elastic transformations. Rigid transformations perform translation and rotation operations to correct for differences in spatial position and orientation between the two data sets; elastic transformations further locally stretch and distort the data to better match subtle structural differences between the two data sets. By applying this transformation matrix, the MRI and confocal microscopy data are spatially transformed, achieving precise voxel-wise registration.

[0111] The cross-modal feature mapping matrix is ​​calculated by the mutual information maximization algorithm. Mutual information is an indicator that measures the correlation between two random variables. In the present invention, it is used to measure the degree of information overlap between magnetic resonance angiography data and confocal microscope scanning data. The goal of the mutual information maximization algorithm is to find an optimal mapping matrix so that the mutual information between the two modal data after mapping is maximized. In the specific implementation process, the features of the two modal data are first extracted, for example, the morphological features and texture features of the blood vessels can be extracted. Then, these features are represented in vector form. By continuously adjusting the parameters of the mapping matrix, the mutual information between the mapped feature vectors is calculated. Using an optimization algorithm, such as a gradient descent algorithm, the mapping matrix that maximizes the mutual information is gradually searched. This mapping matrix can effectively align and fuse the features of the two modal data, and retain the common information between the two data to the maximum extent.

[0112] A gated recurrent unit (GRU) is used to fuse time-series hemodynamic features from different modalities. The Gated Recurrent Unit (GRU) is a special recurrent neural network architecture with unique advantages in processing time series data. In vascular networks, hemodynamic features vary over time. For example, parameters such as blood velocity and pressure change periodically with the heartbeat and physiological activity of the blood vessels. By inputting time-series hemodynamic features from magnetic resonance angiography (MRI) and confocal microscopy (CM) data into the GRU, the GRU can adaptively learn and fuse these features based on the time series information of the data. The gating mechanism in the GRU controls the flow of information, determining which information to retain and which to update. For example, at a given moment, when MRI data provides important information about global blood flow velocity changes in a vessel, the GRU's gating mechanism can prioritize retaining this information and integrate it with features from the confocal microscopy data regarding the influence of the local vessel wall on blood flow. In this way, the GRU can effectively fuse time-series hemodynamic features from different modalities, providing more accurate and comprehensive information for 3D reconstruction of vascular networks.

[0113] Example 6:

[0114] This example describes a method for adjusting the learning rate attenuation factor of the adversarial training controller and a method for adjusting the dynamic convolution kernel parameters of the vascular wall morphology correction module. This method optimizes the model training process, improving training efficiency and accuracy while also making the reconstructed vascular wall morphology more realistic.

[0115] The adversarial training controller dynamically calculates the attenuation coefficient based on the accuracy of vascular branch connections on the validation set. The formula is:

[0116] η t+1 =η t ·exp(-γ·(1-Acc t ))

[0117] Among them, η t+1 is the learning rate for the t+1th round, η t is the learning rate for the tth round, which determines the step size of the parameter update during the training process. γ is the decay rate, which is a preset parameter used to control the speed of the learning rate decay. t is the validation accuracy in round t, obtained by evaluating on the validation set. A high validation accuracy indicates good model training results. In this case, it is advisable to reduce the learning rate appropriately to stabilize model training and avoid over-tuning parameters that could degrade model performance. A low validation accuracy can be achieved by increasing the learning rate appropriately to accelerate model convergence.

[0118] The vascular wall morphology correction module first extracts a vascular wall thickness distribution histogram based on the confocal microscopy data. Confocal microscopy data provides detailed information on vascular wall thickness. By statistically analyzing this data, the distribution of vascular wall thickness can be obtained and presented as a histogram. A nonlinear regression model is used to establish a mapping between thickness distribution and convolution kernel size. This model captures the complex nonlinear relationship between thickness distribution and convolution kernel size, providing a basis for subsequent adjustment of the convolution kernel size. An adaptive step-size gradient descent algorithm is used to optimize the dilation and contraction factors of the convolution kernel. This algorithm automatically adjusts the step size based on the model's training progress. When optimizing the convolution kernel parameters, it more efficiently finds the optimal dilation and contraction factors, enabling the dynamic convolution kernel to more accurately adjust the geometric shape of the vascular wall, making the reconstructed vascular wall morphology more realistic.

[0119] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "includes," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus.

[0120] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A generative adversarial network model for 3D reconstruction of brain cell-level vascular networks, characterized by: include: a 3D vascular feature extraction module for acquiring initial 3D voxel features of the brain vascular network from multimodal medical imaging data, the multimodal medical imaging data including magnetic resonance angiography and confocal microscopy scan data, and mapping the initial 3D voxel features into a high-dimensional feature tensor; A dynamic attention generator, configured to generate a three-dimensional reconstruction of a candidate vascular network based on the high-dimensional feature tensor, wherein the dynamic attention generator includes a spatiotemporal convolution kernel and a channel attention module, and calculates the topological connection probability of vascular branches through an adaptive weight allocation mechanism; a multi-scale discriminator for performing resolution classification on the three-dimensional reconstruction results of the candidate vascular network, comprising a low-resolution global discriminant unit and a high-resolution local discriminant unit. The low-resolution global discriminant unit extracts the overall topological features of the vascular network by downsampling, and the high-resolution local discriminant unit extracts the microstructural features of the vascular wall by sliding window segmentation; a vascular topology optimization module, configured to iteratively adjust the parameters of the dynamic attention generator according to the output of the multi-scale discriminator, and optimize the continuity of the 3D reconstruction results by minimizing the vascular branch breakage loss function; a vascular wall morphology correction module, configured to generate vascular wall morphology correction parameters based on the vascular wall thickness distribution in the confocal microscope scanning data, and adjust the geometric shape of the vascular wall in the three-dimensional reconstruction result of the candidate vascular network through a dynamic convolution kernel; An adversarial training controller, configured to coordinate the training process of the dynamic attention generator and the multi-scale discriminator, adopt an alternating optimization strategy, and dynamically adjust the learning rate decay factor according to the vascular network reconstruction accuracy after each round of training; a blood vessel bifurcation point detection unit, configured to extract the spatial coordinates of the blood vessel bifurcation points from the high-dimensional feature tensor and calculate the optimal solution for the bifurcation point connection direction using a probability density estimation model; The microvascular network completion module is used to generate candidate microvascular pathways based on the variational autoencoder for missing microvascular regions, and to screen the completion results that meet the hemodynamic constraints through the path energy function; A multimodal feature fusion module is used to perform cross-modal alignment of the features of the magnetic resonance angiography data and the confocal microscopy scanning data, and to fuse the vascular wall thickness and blood flow velocity information of different modalities through a feature gating mechanism.

2. The generative adversarial network model for 3D reconstruction of brain cell-level vascular networks according to claim 1, characterized in that: The specific implementation method of the spatiotemporal convolution kernel and channel attention module is: Extract spatiotemporal feature slices from the input high-dimensional feature tensor and normalize each slice by channel dimension; The attention weight of each channel is calculated through a learnable parameter matrix, and the spatiotemporal features of different channels are weightedly fused; The dynamic convolution kernel is used to adjust the expansion coefficient of the convolution kernel according to the curvature radius of the blood vessel branch. The expansion coefficient calculation formula is: D = α·log(1+τ)+β; Where D is the dilation coefficient of the convolution kernel, τ is the local curvature of the blood vessel branch, and α and β are trainable parameters.

3. The generative adversarial network model for 3D reconstruction of brain cell-level vascular networks according to claim 2, characterized in that: The implementation method of the low-resolution global discriminant unit of the multi-scale discriminator includes: The input 3D reconstruction result is downsampled by maximum pooling to generate low-resolution voxel blocks; the main topological structure features of the vascular network are extracted through the 3D convolution layer; Calculate the global discriminant score, the formula is: S global =MLP(Flatten(F global )); Among them, S global is the global discrimination score at low resolution, F global is the global feature vector, and MLP is a multi-layer perceptron.

4. The generative adversarial network model for 3D reconstruction of brain cell-level vascular networks according to claim 3, characterized in that: The implementation method of the high-resolution local discrimination unit includes: Segment the input 3D reconstruction into overlapping local voxel sub-blocks; The surface texture features of the blood vessel wall are extracted through a three-dimensional residual network; Calculate the local discriminant score, the formula is: Among them, S local is the local discrimination score at high resolution, P j is the jth sub-block, φ is the residual network parameter, and M is the total number of sub-blocks.

5. The generative adversarial network model for 3D reconstruction of brain cell-level vascular networks according to claim 1, characterized in that: The probability density estimation model of the blood vessel bifurcation point detection unit adopts a Gaussian mixture model, and the specific steps include: Extracting a set of spatial coordinates of candidate bifurcation points from the high-dimensional feature tensor; Fitting Gaussian mixture distribution parameters by expectation maximization algorithm; The K bifurcation points with the highest probability density are selected as the connection direction candidate set, where K is a preset positive integer.

6. The generative adversarial network model for 3D reconstruction of brain cell-level vascular networks according to claim 1, characterized in that: The path energy function of the microvascular network completion module is defined as: E=λ1·L length +λ2·L curvature +λ3·L pressure ; Among them, E is the total energy of the completed path, L length is the path length constraint, L curvature is the curvature smoothness constraint, L pressure To simulate the physical constraints of blood flow pressure, λ1, λ2, and λ3 are weight coefficients.

7. The generative adversarial network model for 3D reconstruction of brain cell-level vascular networks according to claim 1, characterized in that: The cross-modal alignment method of the multimodal feature fusion module includes: Voxel-wise registration of MRI data and confocal microscopy data was performed. The cross-modal feature map matrix was calculated using the mutual information maximization algorithm. A gated recurrent unit is used to fuse the temporal hemodynamic features of different modalities.

8. The generative adversarial network model for 3D reconstruction of brain cell-level vascular networks according to claim 1, characterized in that: The calculation formula of the topological connection probability is: Among them, σ is the Sigmoid function, W k is the kth attention weight, F is the input feature tensor, θ k is the three-dimensional convolution kernel parameter, b is the bias term, and K is the total number of spatiotemporal convolution kernels; The fracture loss function is defined as: Among them, S i represents the voxel gradient field of the i-th blood vessel segment, N is the total number of segments, is the divergence operator.

9. The generative adversarial network model for 3D reconstruction of brain cell-level vascular networks according to claim 1, characterized in that: The learning rate attenuation factor adjustment method of the adversarial training controller is: The attenuation coefficient is dynamically calculated based on the accuracy of vascular branch connections on the validation set. The formula is: or t+1 =the t ·exp(-γ·(1-Acc t )); Among them, η t+1 is the learning rate for the t+1th round, η t is the learning rate for the tth round, γ is the decay rate, Acc t is the verification accuracy of the tth round.

10. The generative adversarial network model for 3D reconstruction of brain cell-level vascular networks according to claim 1, characterized in that: The dynamic convolution kernel parameter adjustment method of the blood vessel wall morphology correction module includes: The vascular wall thickness distribution histogram was extracted based on confocal microscopy data; The mapping relationship between thickness distribution and convolution kernel size is established through nonlinear regression model; The adaptive step size gradient descent algorithm is used to optimize the expansion factor and contraction factor of the convolution kernel.

Images