Visual disorder dynamic evolution simulation system and method

Abstract

The invention relates to the technical field of visual impairment simulation, in particular to a visual impairment dynamic evolution simulation system and method. Multi-modal medical image data of a patient is collected and subjected to preprocessing and focus positioning, an individual visual pathway fiber bundle is constructed by adopting a deterministic fiber tracking algorithm based on diffusion tensor imaging data, non-rigid registration is performed on the individual fiber bundle and a standard visual pathway template, a local deformation field is calculated, and the focus positioning accuracy is improved. Based on a local deformation field, analyzing a spatial geometrical relationship between a focus mask and each section of nerve fiber to obtain an individualized three-dimensional visual pathway model and an SDI scoring matrix of each section corresponding to the individualized three-dimensional visual pathway model, and judging a damaged pathway section and a typical visual field defect mode matched with the damaged pathway section in degree; according to the method, individualized visual simulation of a patient is realized, a visual dynamic evolution prediction model is constructed by utilizing diffusion Transform, and dynamic evolution of visual impairment is realized.

Description

Technical Field

[0001] This invention relates to the field of visual impairment simulation technology, and more specifically to a dynamic evolution simulation system and method for visual impairment. Background Technology

[0002] Existing visual impairment simulation systems typically model visual field defects based on static image data, lacking the ability to finely characterize the variability of individual neuroanatomical structures. Traditional methods often employ standardized template matching or coarse partitioning mapping, which struggles to accurately reflect the patient-specific spatial pathways and their relationship to lesion compression. Especially when dealing with complex lesions (such as sellar region tumors, occipital lobe infarction, and stroke), the lack of sufficient integration of multimodal medical imaging information limits the accuracy of mapping from structural damage to visual field defect patterns. This prevents high-fidelity reconstruction of subjective visual experience, hindering patients' and doctors' ability to intuitively observe and understand the development of related lesions or the visual impact of surgical treatment, resulting in a lack of direct perception of visual impairment. Summary of the Invention

[0003] To address the shortcomings of the existing technologies, this invention aims to provide a dynamic evolution simulation system and method for visual impairment, thereby solving the problem of inaccurate intuitive experience of visual impairment in existing technologies.

[0004] To solve the above problems, the present invention adopts the following technical solution:

[0005] On the one hand, the present invention provides a dynamic evolution simulation system for visual impairment, including a data acquisition module, a visual pathway construction module, and a static simulation module for the current visual impairment;

[0006] The data acquisition module is used to acquire patients' multimodal medical imaging data and perform preprocessing and lesion localization. The multimodal medical imaging data includes structural MRI data, diffusion tensor imaging data, functional magnetic resonance imaging data, optical coherence tomography data, and / or visual evoked potential data.

[0007] The visual pathway construction module is used to construct individual visual pathway fiber bundles based on the diffusion tensor imaging data using a deterministic fiber tracing algorithm, perform non-rigid registration between the individual fiber bundles and the standard visual pathway template, calculate the local deformation field, and analyze the spatial geometric relationship between the lesion mask and each segment of nerve fiber based on the local deformation field to obtain an individualized three-dimensional visual pathway model and its corresponding SDI score matrix for each segment.

[0008] The current visual impairment static simulation module is used to determine the damaged pathway segments and their degree of damage to match typical visual field defect patterns based on the individualized three-dimensional visual pathway model and its corresponding SDI score matrix.

[0009] As one possible implementation, the acquisition of multimodal medical image data from patients, followed by preprocessing and lesion localization, includes:

[0010] A brain anatomical framework was constructed based on structural MRI data;

[0011] Microscopic diffusion information of white matter fiber bundles was obtained based on diffusion tensor imaging data, and the optic nerve-optic chiasm-optic tract-lateral geniculate body-optic radiation pathway was analyzed.

[0012] Individualized visual cortical functional areas are located based on functional magnetic resonance imaging data;

[0013] The thickness of the retinal nerve fiber layer is quantified based on optical coherence tomography data to reflect the degree of anterior visual pathway degeneration.

[0014] Determining the optic nerve conduction latency and amplitude based on visual evoked potential data;

[0015] Multimodal medical image data is uniformly converted into the NIfTI standard format, and denoising, correction and spatial alignment are performed. U-Net++ deep network is used to automatically perform semantic segmentation of tumor or infarct areas, and output three-dimensional lesion masks and their subvoxel-level spatial coordinates.

[0016] As one possible implementation, the construction of individual visual pathway fiber bundles based on the diffusion tensor imaging data using a deterministic fiber tracing algorithm includes:

[0017] Based on diffusion tensor imaging data, the fiber orientation distribution within each voxel is estimated using a spherical deconvolution algorithm to obtain a FOD map. The FOD map is stably solved using the recursive spherical harmonic expansion method. A deterministic fiber tracing algorithm is applied, using the retinal projection area, lateral geniculate body, and primary visual cortex as dual-end seed points. Fiber tracing is performed by combining FA threshold and angle recursion threshold to reconstruct the fiber bundles of the entire visual pathway. The tracked fiber bundles are mapped to the population average skeleton space using white matter skeletal projection technology to complete the standardization and alignment, thus obtaining the individual visual pathway fiber bundles.

[0018] As one possible implementation, the spatial geometric relationship includes the shortest Euclidean distance, the compression angle, and the spatial overlap integral.

[0019] As one possible implementation, determining the damaged pathway segments and their severity to match typical visual field defect patterns based on the individualized three-dimensional visual pathway model and its corresponding SDI score matrix includes:

[0020] Based on the SDI scoring matrix, clinically interpretable threshold rules are set to automatically identify damaged pathway segments and match typical visual field defect patterns with the compression location: optic nerve compression corresponds to ipsilateral central scotoma; compression of the middle of the optic chiasm causes bitemporal hemianopsia; damage to the optic tract or optic radiation leads to contralateral homonymous hemianopsia; occipital cortical lesions present with macular avoidance. A lookup table mechanism for mapping damage location to visual field defect is established to support dynamic recall and clinical correction.

[0021] A physiological perception modeling mechanism is introduced, and an adaptive Gaussian blur kernel is used to simulate the local visual blur effect. A contrast attenuation model is designed, whose attenuation coefficient has a positively correlated exponential decay relationship with SDI. A VR / AR real-time distortion rendering platform based on the Unity engine is established to support real-time input of external scene images. In the rendering platform, a spatial transformation matrix is ​​applied to dynamically occlude and distort local areas of the image: high SDI areas are subjected to strong distortion and blurring, while low SDI areas retain their original sharpness. As one possible implementation, it also includes a multimodal longitudinal dataset construction module and a dynamic evolution prediction module. The multimodal longitudinal dataset construction module is used to collect multimodal medical image data, visual field examination data, and treatment data of patients over a period of time to construct a multimodal longitudinal dataset.

[0022] The dynamic evolution prediction module is used to construct a visual dynamic evolution prediction model using a diffusion Transformer and train it based on the multimodal longitudinal dataset. The visual dynamic evolution prediction model includes a forward diffusion process, a Transformer backbone network, and a reverse denoising generation process. The forward diffusion process is defined as a Markov chain, which progressively adds Gaussian noise to the real historical state sequence to transform it into pure noise samples. The noise scheduling table adopts a cosine annealing strategy. The reverse denoising generation process is implemented by a stacked Transformer decoder, which recovers the latent state from the noise at each step, and its transition probability is parameterized by the neural network. The Transformer backbone network adopts a multi-head self-attention mechanism to capture long-range temporal dependencies, introduces a fiber orientation-guided sparse attention mask, restricts information transmission across non-adjacent anatomical regions, and enhances the model's ability to model visual pathway-specific propagation paths. The visual dynamic evolution prediction model is used to predict the visual dynamic evolution of patients by inputting their multimodal longitudinal data.

[0023] As one possible implementation, constructing the multimodal longitudinal dataset includes:

[0024] Multimodal medical imaging data, visual field examination data, and treatment data were time-aligned and an initial matrix was constructed. The initial matrix included:

[0025] Visual field indicators: mean sensitivity in the upper quadrant, lower quadrant, nasal quadrant, and temporal quadrant; RNFL thickness: mean, superior, inferior, and temporal quadrant thickness; VEP parameters: left / right eye P100 latency and amplitude; fMRI activation: mean and maximum activation intensity in the V1 region; lesion volume: cumulative tumor volume; treatment status coding: postoperative time, cumulative radiotherapy dose, targeted drug use status, and hormone use status; calculate the rate of change and acceleration of the initial matrix indicators and add them as new columns to the initial matrix;

[0026] Add static clinical variables for patients to construct a multimodal longitudinal dataset.

[0027] As one possible implementation, the forward diffusion process is defined as a Markov chain, which progressively adds Gaussian noise to the real historical state sequence to transform it into pure noise samples. The noise scheduling table employs a cosine annealing strategy, including:

[0028] To the true historical state sequence Gaussian noise is added gradually, and the noise process satisfies the Markov property, with the following specific distribution:

[0029] ;

[0030] in, It is a multivariate Gaussian distribution. It is the mean vector. Let covariance matrix be the variance matrix. Let t be the state sequence after adding noise. Let be the state sequence at step t-1. Let be the noise intensity parameter at step t. The unit covariance matrix;

[0031] After T-step noise addition, the original state Completely degraded into pure noise samples that follow a standard normal distribution:

[0032] ;

[0033] in, It is a vector with zero mean;

[0034] To adapt to the nonlinear characteristics of biological degradation, The cosine annealing strategy is used for definition, and intermediate parameters are introduced first. The specific formula is as follows:

[0035] ;

[0036] in, The state retention coefficient at step t has a value range of [0,1], and decreases monotonically from 1 to 0 as t increases. When the noise level is low - in the early stages of noise addition, The descent was slow. Smaller - weaker noise, slower degradation; when Approaching 1 - Later stage of noise addition, Sudden drop, Rapid growth - high noise, rapid degradation;

[0037] The stacked Transformer decoder is used to recover the latent state from noise, and the state transition probability is parameterized by a neural network. To achieve the transition from the noise state at step t Restore the potential state at step t-1 The network configuration consists of 12 stacked self-attention modules, 768 hidden layers with 768 dimensions, and 12 multi-head attention heads.

[0038] As one possible implementation, the loss function of the visual dynamic evolution prediction model is:

[0039] ;

[0040] in, For loss function, , The regularization coefficient is . Measure the true value of L2 reconstruction loss Compared with the estimated value The Euclidean distance between them KL divergence is used to measure variational distributions. With model distribution The degree of approximation Let be a sequence of random variables. For parameterized model distribution, It is a variational approximation distribution.

[0041] On the other hand, the present invention provides a method for simulating the dynamic evolution of visual impairment, comprising:

[0042] The patient's multimodal medical imaging data is collected and preprocessed and lesion localized. The multimodal medical imaging data includes structural MRI data, diffusion tensor imaging data, functional magnetic resonance imaging data, optical coherence tomography data and / or visual evoked potential data.

[0043] Based on the diffusion tensor imaging data, a deterministic fiber tracing algorithm is used to construct individual visual pathway fiber bundles. The individual fiber bundles are non-rigidly registered with the standard visual pathway template, the local deformation field is calculated, and the spatial geometric relationship between the lesion mask and each segment of nerve fiber is analyzed based on the local deformation field to obtain an individualized three-dimensional visual pathway model and its corresponding SDI score matrix for each segment.

[0044] Based on the individualized three-dimensional visual pathway model and its corresponding SDI score matrix for each segment, the damaged pathway segments and their degree are determined to match typical visual field defect patterns.

[0045] The beneficial effects of this invention are as follows: The visual impairment dynamic evolution simulation system of this invention realizes the static simulation of visual impairment through multimodal medical image data function, and constructs a visual dynamic evolution prediction model based on diffusion Transformer to realize the dynamic evolution of visual impairment, so that doctors, patients and others can intuitively and accurately reflect the specific situation of visual impairment. Attached Figure Description

[0046] Figure 1 This is a schematic diagram of a dynamic evolution simulation system for visual impairment.

[0047] Figure 2 This is a schematic diagram of a method for simulating the dynamic evolution of visual impairment. Detailed Implementation

[0048] The present invention will be further described in detail below with reference to specific embodiments.

[0049] It should be noted that these embodiments are only used to illustrate the present invention and are not intended to limit the present invention. Simple improvements to the method under the premise of the present invention are all within the scope of protection claimed by the present invention.

[0050] See Figure 1 A dynamic evolution simulation system for visual impairment includes a data acquisition module 100, a visual pathway construction module 200, and a static simulation module for current visual impairment 300.

[0051] The data acquisition module 100 is used to acquire multimodal medical image data of patients and perform preprocessing and lesion localization. The multimodal medical image data includes structural MRI data, diffusion tensor imaging data, functional magnetic resonance imaging data, optical coherence tomography data and / or visual evoked potential data.

[0052] Specifically, collecting and preprocessing multimodal medical imaging data from patients and lesion localization includes:

[0053] A brain anatomical framework was constructed based on structural MRI data;

[0054] Microscopic diffusion information of white matter fiber bundles was obtained based on diffusion tensor imaging data, and the optic nerve-optic chiasm-optic tract-lateral geniculate body-optic radiation pathway was analyzed.

[0055] Individualized visual cortical functional areas are located based on functional magnetic resonance imaging data;

[0056] The thickness of the retinal nerve fiber layer is quantified based on optical coherence tomography data to reflect the degree of anterior visual pathway degeneration.

[0057] Determining the optic nerve conduction latency and amplitude based on visual evoked potential data;

[0058] Multimodal medical image data is uniformly converted into the NIfTI standard format, and denoising, correction and spatial alignment are performed. U-Net++ deep network is used to automatically perform semantic segmentation of tumor or infarct areas, and output three-dimensional lesion masks and their subvoxel-level spatial coordinates.

[0059] for example:

[0060] Structural MRI: T1-weighted 3D MPRAGE sequence images (spatial resolution 0.9 × 0.9 × 1.0 mm³) and T2-FLAIR sequence images (1.0 × 1.0 × 1.2 mm³) were acquired to construct a high-precision brain anatomical framework; Diffusion tensor imaging (DTI): A single-shot EPI sequence was used with a b-value of 3000 s / mm², acquiring 128 diffusion directions at a slice thickness of 1.5 mm and TR / TE = 7200 / 85 ms to fully resolve the microscopic diffusion anisotropy of visual pathway fibers; Functional magnetic resonance imaging (fMRI): BOLD signals were acquired under a checkerboard flipping visual stimulus task with a spatial resolution of 2 mm³ and a temporal resolution of 2 s to accurately locate individualized primary visual cortex (V1) activation clusters; Optical coherence tomography (OCT): Circular and macula cube scans were performed using a Spectralis HRA system, automatically outputting RNFL thickness distribution maps, with an average thickness of 78 mm in the left eye. The right eye measured 81 μm, both below the normal reference range (>90 μm), indicating anterior visual pathway degeneration. Visual evoked potentials (VEP): P100 waveforms were recorded according to the International Society for Clinical Visual Electrophysiology (ISCEV) standard protocol. The latency was prolonged (128 ms in the left eye, 122 ms in the right eye, normal <115 ms), and the amplitude decreased to 65% of the normal value, reflecting impaired optic nerve conduction function. The data was imported into the system and uniformly converted to the NIfTI-1 standard format to eliminate format heterogeneity between different devices. For DTI data, three-dimensional nonlocal mean filtering (NL-Means) was used for volume data denoising, significantly improving the image signal-to-noise ratio (SNR improvement of approximately 32%) while preserving minor directional variations in the crossing fiber region. The FSL toolkit was used to perform head motion correction and eddy current distortion correction. Based on the FLIRT (linear) and FNIRT (nonlinear) algorithms, DTI, fMRI, and T1-weighted images were jointly registered to the MNI152 standard space, achieving spatial alignment of multimodal data with registration errors controlled within 0.5 mm. A network was then used for automatic semantic segmentation of the tumor region. This network introduced nested skip connections and a multi-scale attention mechanism. The input consisted of a multi-channel combination of T1 enhancement, T2-FLAIR, and DTI_FA maps, and the output was a three-dimensional lesion mask and its sub-voxel-level spatial coordinates. After manual review by radiologists, the segmentation results achieved a Dice similarity coefficient of 0.91 and a boundary error of less than 0.8 mm. Finally, a standardized multimodal image dataset was generated, including spatially aligned T1, DTI_FA, fMRI_BOLD, OCT_thickness maps and VEP source localization maps, along with a precise lesion mask, serving as a unified input source for subsequent individualized visual pathway reconstruction.

[0061] The visual pathway construction module 200 is used to construct individual visual pathway fiber bundles based on the diffusion tensor imaging data using a deterministic fiber tracing algorithm, perform non-rigid registration between the individual fiber bundles and the standard visual pathway template, calculate the local deformation field, and analyze the spatial geometric relationship between the lesion mask and each segment of nerve fiber based on the local deformation field to obtain an individualized three-dimensional visual pathway model and its corresponding SDI score matrix for each segment.

[0062] Specifically, constructing individual visual pathway fiber bundles using a deterministic fiber tracing algorithm based on the diffusion tensor imaging data includes:

[0063] Based on diffusion tensor imaging data, the fiber orientation distribution within each voxel is estimated using a spherical deconvolution algorithm to obtain a FOD map. The FOD map is stably solved using the recursive spherical harmonic expansion method. A deterministic fiber tracing algorithm is applied, using the retinal projection area, lateral geniculate body, and primary visual cortex as dual-end seed points. Fiber tracing is performed by combining FA threshold and angle recursion threshold to reconstruct the fiber bundles of the entire visual pathway. The tracked fiber bundles are mapped to the population average skeleton space using white matter skeletal projection technology to complete the standardization and alignment, thus obtaining the individual visual pathway fiber bundles.

[0064] for example:

[0065] Based on preprocessed DTI data, the fiber orientation distribution (FOD) within each voxel is estimated using the spherical deconvolution (SD) algorithm, overcoming the limitations of traditional single-tensor models in complex regions such as the optic chiasm. Combined with a high-angle-resolution diffusion imaging (HARDI) sampling scheme (b=3000 s / mm², 128 directions), the FOD map is stably solved using the recursive spherical harmonic expansion method, achieving a spatial resolution of 1.5 mm³, effectively resolving the multi-peak orientation of crossing and non-crossing fibers in the left and right eyes within the optic chiasm. Building upon this, a deterministic fiber tracing algorithm (FACT-type streamlinetracking) is applied, using the bilateral retinal projection areas, the lateral geniculate nucleus (LGN), and the primary visual cortex (V1) as seed points. Fiber tracing is performed using an FA threshold (0.2) and an angle recursion threshold (60° / step) to reconstruct the fiber bundles of the entire visual pathway. To improve cross-individual comparability, white matter skeleton projection (TBSS) technology is used to map the tracked fiber bundles to the population average skeleton space, achieving standardized alignment. A standard visual pathway template was further constructed based on high-precision DTI data from 50 healthy adults. Through population registration and fiber bundle clustering analysis, the topological connectivity prior paths of sub-segments such as the optic nerve, optic chiasm, optic tract, and optic radiation were extracted. Individual patient fiber bundles were non-rigidly registered with this standard template, and the local deformation field was calculated to quantify individual anatomical variations. For example, in the optic chiasm region, the patient's fiber bundle shifted posteriorly and superiorly by 2.3 mm, consistent with the tumor mass effect. Combined with the lesion mask, the spatial geometric relationship between the lesion and each segment of nerve fiber was analyzed: Shortest Euclidean distance: the minimum distance from the lesion edge to each fiber bundle segment was calculated, such as 1.1 mm for the optic chiasm segment and 4.7 mm for the right optic tract segment; Compression angle: defined as the angle between the fiber tangent direction and the compression vector; less than 30° is considered tangential compression (e.g., sliding shear), and greater than 60° is radial compression (e.g., vertical compression); Spatial overlap integral: the ratio of voxels occupied by the lesion and fiber bundles was statistically analyzed, reflecting the degree of direct invasion. The Structural Damage Index (SDI), with a value range of [0,1], represents the probability of damage to a certain fiber segment. Riemannian manifold distance weighting is incorporated into the SDI calculation to enhance sensitivity to curved fiber segments. The specific formula is as follows:

[0066] SDI: Structural Damage Index, used to quantify the degree of compression damage in different segments of the visual pathway;

[0067] , The ogistic function parameter (d is the shortest distance) controls the nonlinear relationship between damage and distance.

[0068] Compression angle, reflecting the angle between the direction of compression and the fiber direction;

[0069] The volume of overlap between the compression area and the fiber bundle;

[0070] The total volume of the fiber bundles in this segment of the visual pathway;

[0071] The ratio of local anisotropy fraction (FA) to healthy template reflects fiber integrity;

[0072] Weight parameters: , , .

[0073] The current visual impairment static simulation module 300 is used to determine the damaged pathway segments and their degree to match typical visual field defect patterns based on the individualized three-dimensional visual pathway model and its corresponding SDI score matrix.

[0074] Based on the SDI scoring matrix, clinically interpretable threshold rules are set to automatically identify damaged pathway segments and match typical visual field defect patterns with the compression location: optic nerve compression corresponds to ipsilateral central scotoma; compression of the middle of the optic chiasm causes bitemporal hemianopsia; damage to the optic tract or optic radiation leads to contralateral homonymous hemianopsia; occipital cortical lesions present with macular avoidance. A lookup table mechanism for mapping damage location to visual field defect is established to support dynamic recall and clinical correction.

[0075] A physiological perception modeling mechanism is introduced, simulating local visual blurring effects through an adaptive Gaussian blur kernel. A contrast attenuation model is designed, with its attenuation coefficient exhibiting an exponentially decreasing relationship with SDI (Spectrum Dispersion Index). A real-time VR / AR distortion rendering platform based on the Unity engine is established, supporting real-time input of external scene images. Within the rendering platform, a spatial transformation matrix is ​​applied to dynamically mask and distort local image regions: high SDI regions are subjected to strong distortion and blurring, while low SDI regions retain their original sharpness. For example:

[0076] Compression of the middle of the optic chiasm → Bitemporal hemianopsia;

[0077] Mild compression of the left optic nerve → ipsilateral central scotoma superimposed; prolonged VEP latency and OCT showing thinning of the RNFL corroborate each other, supporting anterior conduction dysfunction.

[0078] A lookup table mechanism mapping lesion location to visual field defect is established, supporting dynamic recall and clinical correction. For example, when the SDI exceeds 0.8 at the optic chiasm, a "bitemporal hemianopsia" mode is automatically triggered, allowing physicians to fine-tune the defect boundary based on actual visual field examination results. Furthermore, a physiological perception modeling mechanism is introduced: visual blur simulation: an adaptive Gaussian blur kernel is designed, with a standard deviation... The SDI value is dynamically adjusted from 0.5 to 3.0 pixels, with the following relationship: When the SDI of the view intersection segment is 0.87, the calculation yields: = 0.5 + 2.5 * 0.87 = 2.68, at which point a noticeable local visual blurring effect will occur. Contrast attenuation model: The attenuation coefficient C has a positively correlated exponential decay relationship with SDI. The contrast ratio is preserved at only 35.7% of its original value, restoring the patient's subjective experience of decreased contrast sensitivity. The VR / AR real-time distortion rendering platform supports real-time input of external scene images (such as text, faces, and street scenes). In the rendering engine, a spatial transformation matrix is ​​applied to dynamically mask and distort local image regions: high SDI regions (>0.7) are subjected to strong nonlinear B-spline distortion and Gaussian blur; medium SDI regions (0.4–0.7) are subjected to affine distortion and moderate blur; low SDI regions maintain their original sharpness. Distortion parameters are obtained by bilinear interpolation of the SDI map, ensuring smooth spatial transitions. For example, in a simulated reading scenario, a bilateral blur band appears in the central horizontal visual field, approximately 15° wide, consistent with bitemporal hemianopsia; while in face recognition mode, the facial contours on both sides are blurred, affecting identification. The system supports real-time playback on head-mounted displays (HMDs), outputting interactive VR / AR video streams for clinicians to immerse themselves in the patient's visual impairment state. In this way, attending physicians can intuitively understand the reasons for patients' chief complaints that they can see but cannot recognize, providing intuitive evidence for diagnosis and treatment decisions. This system also includes a multimodal longitudinal dataset construction module 400 and a dynamic evolution prediction module 500. The multimodal longitudinal dataset construction module 400 is used to collect patients' multimodal medical imaging data, visual field examination data, and treatment data over a period of time to construct a multimodal longitudinal dataset.

[0079] For example, the system automatically connects to the hospital's PACS image archiving system and electronic medical record system (EMR) via API interface, periodically collecting multimodal clinical data from the patient's six follow-up visits since enrollment (baseline, 3 months, 6 months, 9 months, 12 months, and 15 months). Specifically, this includes: visual field examination: using the Humphrey perimeter 10-2 program, visual field sensitivity distribution maps (unit: dB) are obtained at each time point, with a focus on extracting the mean defect depth (MD) and pattern standard deviation (PSD); OCT measurement: the mean thickness (μm) of the retinal nerve fiber layer (RNFL) and the subfoveal thickness (SCT) in the macular cube scan are recorded; VEP recording: the P100 wave latency (ms) and amplitude (μV) are extracted to reflect the dynamic changes in optic nerve conduction function.

[0080] fMRI activation intensity: Under visual stimulation task, the spatial intensity mean (BOLD signal change percentage) of activation clusters in the primary visual cortex (V1) is calculated.

[0081] Follow-up MRI lesion volume: The tumor region was automatically segmented using the U-Net++ network, and the volume growth rate (mm³ / month) was calculated.

[0082] Treatment intervention information: Gamma knife radiotherapy (12 Gy) was administered in month 6; oral everolimus (10 mg / day) was started in month 9. A data fusion engine was used to generate a cross-modal aligned time series dataset. Feature engineering and time alignment were performed.

[0083] The patient time-series feature matrix PTFM was constructed with dimensions T×F, where T = 6 valid follow-up time points and F > 50 features. The specific steps are as follows: (1) Time alignment and matrix construction

[0084] The data for each modality were aligned to the nearest whole month node according to the examination date (e.g., day 89 aligned to month 3). Missing observations (e.g., no OCT performed in month 9) were marked as "NaN" and subsequently filled in using Gaussian process regression (GPR) interpolation. The initial PTFM matrix was then constructed, containing the following feature categories: visual field indicators (6-dimensional): MD, PSD, and average sensitivity in the superior / inferior / nasal / temporal quadrants; RNFL thickness (4-dimensional): average, superior, inferior, and temporal quadrant thickness; VEP parameters (2-dimensional): P100 latency and amplitude for left / right eyes; fMRI activation (2-dimensional): mean and maximum activation intensity in the V1 region; lesion volume (1-dimensional): cumulative tumor volume (mm³).

[0085] Treatment status coding (5 dimensions): cumulative radiotherapy dose, targeted drug use status (0 / 1), hormone use status, etc.

[0086] (2) Differential feature extraction

[0087] Locally weighted regression (LOESS) was used to fit the continuous trend of key physiological parameters, and the first derivative (rate of change) and second derivative (acceleration) were calculated to enhance the model's sensitivity to early regression trends. Taking the average thickness of the RNFL as an example, the observed values ​​at six time points were: 92.1, 89.5, 87.3, 85.6, 83.9, and 81.7 μm. The LOESS curve h(t) was fitted, and its first derivative (shrinkage rate) was calculated. During the period from June to September, the average atrophy rate was calculated to be 0.81 μm / month, significantly faster than the previous period (0.42 μm / month), indicating that the disease has entered an accelerated atrophy phase. The "acceleration" of the VEP latency period was then calculated: .when An acceleration of >0.15 ms / month² is considered a warning signal of conduction function mutation. In this case, the acceleration reached 0.18 ms / month² from month 9 to month 12, triggering an early intervention alert. These continuous-time derivative features were added as new columns to PTFM, forming an enhanced feature space containing dynamic information on trend changes. The following static clinical covariates were added: age (52), sex (male), disease onset time (-6 months, retrospective estimation), and genetic risk score (APOEε4 carrier, score = 1.8). For non-uniform sampling or missing observations (e.g., no OCT data in month 9), Gaussian process regression (GPR) combined with physiological prior kernel function was used for interpolation.

[0088] The dynamic evolution prediction module 500 is used to construct a visual dynamic evolution prediction model using a diffusion Transformer and train it based on the multimodal longitudinal dataset. The visual dynamic evolution prediction model includes a forward diffusion process, a Transformer backbone network, and a reverse denoising generation process. The forward diffusion process is defined as a Markov chain, progressively adding Gaussian noise to the real historical state sequence to transform it into pure noise samples. The noise scheduling table uses a cosine annealing strategy. The reverse denoising generation process is implemented by a stacked Transformer decoder, recovering the latent state from the noise at each step, with its transition probability parameterized by the neural network. The Transformer backbone network employs a multi-head self-attention mechanism to capture long-range temporal dependencies, introducing a fiber-guided sparse attention mask to restrict information transmission across non-adjacent anatomical regions, enhancing the model's ability to model visual pathway-specific propagation paths. The visual dynamic evolution prediction model is used to predict the patient's visual dynamic evolution by inputting the patient's multimodal longitudinal data. Visual simulation can also be performed using the current visual impairment static simulation module based on the patient's visual dynamic evolution prediction results.

[0089] The forward diffusion process is defined as a Markov chain, which gradually adds Gaussian noise to the real historical state sequence to transform it into pure noise samples. The noise scheduling table uses a cosine annealing strategy, including:

[0090] To the true historical state sequence Gaussian noise is added gradually, and the noise process satisfies the Markov property, with the following specific distribution:

[0091] ;

[0092] in, It is a multivariate Gaussian distribution. It is the mean vector. Let covariance matrix be the variance matrix. Let t be the state sequence after adding noise. Let be the state sequence at step t-1. Let be the noise intensity parameter at step t. The unit covariance matrix;

[0093] After T-step noise addition, the original state Completely degraded into pure noise samples that follow a standard normal distribution:

[0094] ;

[0095] in, It is a vector with zero mean;

[0096] To adapt to the nonlinear characteristics of biological degradation, The cosine annealing strategy is used for definition, and intermediate parameters are introduced first. The specific formula is as follows:

[0097] ;

[0098] in, The state retention coefficient at step t has a value range of [0,1], and decreases monotonically from 1 to 0 as t increases. When the noise level is low - in the early stages of noise addition, The descent was slow. Smaller - weaker noise, slower degradation; when Approaching 1 - Later stage of noise addition, Sudden drop, Rapid growth - high noise, rapid degradation;

[0099] The stacked Transformer decoder is used to recover the latent state from noise, and the state transition probability is parameterized by a neural network. To achieve the transition from the noise state at step t Restore the potential state at step t-1 The network configuration consists of 12 stacked self-attention modules, 768 hidden layers with 768 dimensions, and 12 multi-head attention heads.

[0100] The loss function of the visual dynamic evolution prediction model is:

[0101] ;

[0102] in, For loss function, , The regularization coefficient is . Measure the true value of L2 reconstruction loss Compared with the estimated value The Euclidean distance between them KL divergence is used to measure variational distributions. With model distribution The degree of approximation Let be a sequence of random variables. For parameterized model distribution, It is a variational approximation distribution.

[0103] Multi-task learning paradigm

[0104] Main task: Regression task, predicting visual field sensitivity distribution over the next 6-12 months (76 test points in a 10-2 program); Auxiliary task 1: Binary classification task, determining whether the patient will enter a "rapid deterioration phase" (defined as a decrease in MD > 2 dB / year) within the next 6 months; Auxiliary task 2: Treatment response prediction, predicting whether targeted therapy will significantly delay progression based on a Cox proportional hazards model (HR < 0.7 is considered effective). Sformer backbone, independently designed output heads, and joint optimization of total loss.

[0105] Closed-loop feedback and system optimization module

[0106] Dynamic calibration mechanism

[0107] When the patient completes the 18-month follow-up examination, the system automatically acquires new visual field examination and VEP test data and triggers the feedback process:

[0108] (1) Prediction-observation alignment

[0109] The actual field-of-view map and VEP parameters for month 18 are extracted and spatiotemporally aligned with the model's predictions for month 15. The prediction-observation residual matrix is ​​calculated pixel-wise or channel-wise to generate a residual bias map.

[0110] (2) Bayesian parameter update

[0111] Using the initial state distribution of the original diffusion model as the prior probability Using residual likelihood function Iteratively solve for the initial conditions of posterior optimum With noise scheduling parameters . Jointly update latent variables and hyperparameters to avoid the high computational cost of traditional MCMC methods. The updated... The reweighted noise variance was injected into the inverse denoising process to regenerate individualized evolution prediction curves. After calibration, the predicted RNFL value for month 24 was reduced from 72.1 μm to 68.5 μm, which better reflects the actual degradation rate.

[0112] The human-computer interaction and decision support system is equipped with a web-based interface for doctors, supporting multi-dimensional visualization and hypothetical intervention analysis.

[0113] (1) Visualization function: View individualized 3D visual pathway model (based on DTI reconstruction); Play current visual impairment simulation video (based on static SDI mapping);

[0114] Review the dynamic evolution prediction map for the next 6-12 months (showing the trend of field of view expansion in the form of a heat map); compare the residual map of historical predictions with the latest observations to identify systematic biases.

[0115] (2) Hypothetical intervention analysis

[0116] Doctors can input a "hypothetical intervention" command on the interface, such as: "If 80% of the tumor volume is surgically removed in the 3rd month." The system calls a pre-trained treatment response dynamics model to generate the expected recovery trajectory after intervention in real time: the model predicts that the postoperative RNFL shrinkage rate will decrease from 0.8 μm / month to 0.3 μm / month; the VEP latency growth trend will slow down, and the P100 amplitude is expected to rebound; the overall probability of entering the "stable period" will increase to 82% (from 41%). Two evolution curves before and after intervention are presented in a comparative form, supporting dynamic sliding of the timeline to view differences at different stages. Risk assessments at key time points (e.g., "Probability of rapid deterioration in the 12th month: 67%)"; treatment window suggestions (e.g., "Optimal intervention window: 6-9 months"); uncertainty quantification indicators (e.g., prediction interval, Monte Carlo standard deviation); and a hypothetical intervention effect comparison chart. This fully supports personalized clinical diagnosis and treatment decisions, achieving an end-to-end closed loop from data collection to intelligent decision-making. See also Figure 2 This is a method for simulating the dynamic evolution of visual impairment, including:

[0117] S100. Collect multimodal medical imaging data of patients and perform preprocessing and lesion localization. Multimodal medical imaging data includes structural MRI data, diffusion tensor imaging data, functional magnetic resonance imaging data, optical coherence tomography data and / or visual evoked potential data.

[0118] S200. Based on diffusion tensor imaging data, a deterministic fiber tracing algorithm is used to construct individual visual pathway fiber bundles. The individual fiber bundles are non-rigidly registered with the standard visual pathway template. The local deformation field is calculated, and the spatial geometric relationship between the lesion mask and each segment of nerve fiber is analyzed based on the local deformation field to obtain an individualized three-dimensional visual pathway model and its corresponding SDI score matrix for each segment.

[0119] S300. Based on the individualized three-dimensional visual pathway model and its corresponding SDI scoring matrix for each segment, determine the damaged pathway segment and its degree to match typical visual field defect patterns. Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described with reference to preferred embodiments, those skilled in the art should understand that various changes in form and detail can be made without departing from the spirit and scope of the invention as defined in the appended claims.

Claims

1. A dynamic visual impairment evolution simulation system, comprising: The system comprises a data acquisition quantity module, a visual pathway construction module and a current visual impairment static simulation module. The data acquisition quantity module is used for acquiring and pre-processing multi-modal medical image data of a patient and locating a lesion, and the multi-modal medical image data comprises structural MRI data, diffusion tensor imaging data, functional magnetic resonance data, optical coherence tomography data and / or visual evoked potential data. The visual pathway construction module is used for constructing individual visual pathway fiber bundles based on the diffusion tensor imaging data by using a deterministic fiber tracking algorithm, performing non-rigid registration of individual fiber bundles with a standard visual pathway template, calculating a local deformation field, and analyzing a spatial geometric relationship between a lesion mask and each segment of neural fibers based on the local deformation field to obtain an individualized three-dimensional visual pathway model and an SDI score matrix corresponding to each segment. The current visual impairment static simulation module is used for determining a damaged pathway segment and its degree matching a typical visual field defect pattern according to the individualized three-dimensional visual pathway model and the SDI score matrix corresponding to each segment.

2. The dynamic visual impairment simulation system of claim 1, wherein, The acquisition and pre-processing of the multi-modal medical image data of the patient and the lesion location comprise: constructing a brain anatomical framework based on the structural MRI data; obtaining micro diffusion information of white matter fiber bundles based on the diffusion tensor imaging data, and analyzing a visual nerve-optic chiasm-optic tract-lateral geniculate body-optic radiation pathway; locating individualized visual cortex functional areas based on the functional magnetic resonance data; quantifying retinal nerve fiber layer thickness based on the optical coherence tomography data to reflect the degree of degeneration of the anterior segment of the visual pathway; determining visual nerve conduction latency and amplitude based on the visual evoked potential data; unifying the multi-modal medical image data into a NIfTI standard format, and performing denoising, correction and spatial alignment, automatically segmenting a tumor or infarction area by using a U-Net++ deep network, and outputting a three-dimensional lesion mask and sub-voxel level spatial coordinates.

3. The dynamic visual impairment simulation system of claim 1, wherein, The construction of the individual visual pathway fiber bundles based on the diffusion tensor imaging data by using the deterministic fiber tracking algorithm comprises: based on the diffusion tensor imaging data, estimating the fiber direction distribution in each voxel by using a spherical deconvolution algorithm to obtain an FOD atlas, stably solving the FOD atlas by using a recursive spherical harmonic expansion method, applying the deterministic fiber tracking algorithm, taking the retinal projection area, the lateral geniculate body and the primary visual cortex as double-end seed points, combining the FA threshold and the angle recursive threshold to perform fiber tracking, and reconstructing the full-segment visual pathway fiber bundle; and mapping the tracked fiber bundle to a population average skeleton space by using a white matter skeleton projection technology to complete standardized alignment and obtain the individual visual pathway fiber bundle.

4. The dynamic visual impairment simulation system of claim 3, wherein, The spatial geometric relationship comprises a shortest Euclidean distance, a compression angle and a spatial overlap integral.

5. The dynamic visual impairment simulation system of claim 4, wherein, The determination of the damaged pathway segment and its degree matching the typical visual field defect pattern according to the individualized three-dimensional visual pathway model and the SDI score matrix corresponding to each segment comprises: According to the SDI score matrix, a clinically interpretable threshold rule is set to automatically identify damaged pathway segments, and combined with the compression position, a typical visual field defect pattern is matched: optic nerve compression corresponds to ipsilateral central scotoma; mid-optic chiasm compression causes bilateral temporal hemianopia; optic tract or optic radiation damage leads to contralateral homonymous hemianopia; occipital lobe lesion presents macular avoidance phenomenon, establishes an injury location-visual field defect mapping table mechanism to support dynamic calling and clinical correction; A physiological perception modeling mechanism is introduced to simulate local visual blur effects through an adaptive Gaussian blur kernel; a contrast attenuation model is designed, whose attenuation coefficient has a positive exponential attenuation relationship with SDI, and a VR / AR real-time distortion rendering platform based on Unity engine is established to support real-time input of external scene images; In the rendering platform, a spatial transformation matrix is applied to dynamically mask and distort the local area of the image: strong distortion and blur are applied to high SDI areas, and original clarity is preserved in low SDI areas.

6. The dynamic visual impairment simulation system of claim 5, wherein, It also includes a multi-modal longitudinal data set construction module and a dynamic evolution prediction module. The multi-modal longitudinal data set construction module is used to collect multi-modal medical image data, visual field examination data and treatment data of a patient within a period of time to construct a multi-modal longitudinal data set. The dynamic evolution prediction module is used to construct a visual dynamic evolution prediction model using a diffusion Transformer, and train it based on the multi-modal longitudinal data set. The visual dynamic evolution prediction model includes a forward diffusion process, a Transformer backbone network and a reverse denoising generation process. The forward diffusion process is defined as a Markov chain, which gradually adds Gaussian noise to the real historical state sequence to convert it into a pure noise sample. The noise scheduling table uses a cosine annealing strategy. The reverse denoising generation process is implemented by a stacked Transformer decoder, which recovers the latent state from noise at each step, and its transition probability is parameterized by a neural network. The Transformer backbone network uses a multi-head self-attention mechanism to capture long-range temporal dependencies, introduces a sparse attention mask guided by fiber orientation to limit information transfer across non-adjacent anatomical regions, and enhances the model's ability to model specific propagation paths in the visual pathway. The visual dynamic evolution prediction model is used to input the patient's multi-modal longitudinal data to predict the patient's visual dynamic evolution.

7. The dynamic visual impairment simulation system of claim 6, wherein, The multi-modal longitudinal data set construction includes: Time alignment and initial matrix construction of multi-modal medical image data, visual field examination and treatment data, the initial matrix includes: Visual field indicators: MD, PSD, average sensitivity of upper quadrant, lower quadrant, nasal side and temporal side; RNFL thickness: average, upper, lower, temporal quadrant thickness; VEP parameters: left / right eye P100 latency and amplitude; fMRI activation: mean and maximum V1 region activation intensity; Lesion volume: cumulative tumor volume; Treatment state encoding: postoperative time, cumulative radiotherapy dose, targeted drug use status, hormone use status; Calculate the change rate and acceleration of the initial matrix indicators and add them as new columns to the initial matrix; Add the patient's static clinical variables to construct the multi-modal longitudinal data set.

8. The dynamic visual impairment simulation system of claim 7, wherein, The forward diffusion process is defined as a Markov chain, gradually adding Gaussian noise to the real historical state sequence, converting into pure noise samples, and the noise schedule adopts a cosine annealing strategy including: To the real historical state sequence Gaussian noise is added step by step, and the noise process satisfies Markov property, and the specific distribution is: ； wherein, is a multivariate Gaussian distribution, is a mean vector, is a covariance matrix, is a state sequence after adding noise at the t-th step, is a state sequence at the t-1-th step, is a noise intensity parameter at the t-th step, is a unit covariance matrix; After T-step noise addition, the original state Completely degenerate into pure noise samples following a standard normal distribution: ； wherein is a zero-mean vector; To adapt the nonlinear characteristics of biological degeneration, An intermediate parameter is introduced by using the cosine annealing strategy definition The specific formula is: ； wherein, is the state reserved coefficient of the t-th step, taking a value in the range [0, 1], monotonically decreasing from 1 to 0 as t increases, and is small - initial stage of noise addition, is slow, is small - weak noise, slow degradation; when is close to 1 - late stage of noise addition, is sudden, is rapidly increased - strong noise, fast degradation; The stacked Transformer decoder is used to recover the latent state from noise, and the state transition probability is parameterized by a neural network. To achieve the transition from the noise state at step t Restore the potential state at step t-1 The network configuration consists of 12 stacked self-attention modules, 768 hidden layers with 768 dimensions, and 12 multi-head attention heads.

9. The dynamic visual impairment simulation system of claim 8, wherein, The loss function of the visual dynamic evolution prediction model is: ； wherein, is a loss function, , is a regularization coefficient, is an L2 reconstruction loss measuring the Euclidean distance between a true value and an estimate , is a KL divergence used to measure how well a variational distribution approximates a model distribution , is a sequence of random variables, is a parameterized model distribution, is a variational approximation distribution.

10. A method of dynamic evolution simulation of visual impairment, characterized in that, Including: Collecting and preprocessing multi-modal medical image data of a patient and locating lesions, the multi-modal medical image data including structural MRI data, diffusion tensor imaging data, functional magnetic resonance data, optical coherence tomography data and / or visual evoked potential data; Based on the diffusion tensor imaging data, a deterministic fiber tracking algorithm is used to construct individual visual pathway fiber bundles, individual fiber bundles are non-rigidly registered with a standard visual pathway template, a local deformation field is calculated, and based on the local deformation field, the spatial geometric relationship between the lesion mask and each segment of the neural fiber is analyzed to obtain an individualized three-dimensional visual pathway model and its corresponding SDI score matrix for each segment; According to the individualized three-dimensional visual pathway model and its corresponding SDI score matrix for each segment, the damaged pathway segment and its degree are matched with a typical visual field defect pattern.

Images