A method, device, medium, and program product for generating myopia risk assessment information

Abstract

This specification provides a method, device, medium, and program product for generating myopia risk assessment information, relating to the field of intelligent healthcare. This application innovatively proposes an interpretable deep learning framework, MIRAGE, which integrates fundus imaging and whole-exome genotyping to predict the risk of high myopia (HM). By combining lifelong genomic susceptibility with current retinal characteristics, MIRAGE achieves higher accuracy than single-modal models through gated attention mechanisms and adaptive noise reduction techniques.

Description

Technical Field

[0001] This invention relates to the field of intelligent healthcare, and more specifically, to a method, device, medium, and program product for generating myopia risk assessment information. Background Technology

[0002] High myopia (HM) is a multifactorial disease caused by a complex interaction between genetic and environmental factors. Currently, numerous molecular genetic studies have identified hundreds of gene loci associated with myopia and HM, providing a genetic basis for constructing polygenic risk scores (PRS). Large-scale genome-wide association studies (GWAS) in European populations have yielded receiver operating characteristic (AUROC) values ​​ranging from 0.670 to 0.783 for myopia prediction, with the best-performing PRS explaining approximately 19% of phenotypic variation. However, the predictive performance of PRS drops significantly when applied to different populations, indicating a lack of universality in non-European populations. While previous studies have developed a whole-exome polygenic risk score (ExPRS) using a large Chinese cohort, which integrates common and rare variants and significantly improves the accuracy of myopia risk prediction, highlighting the potential of genetic information in identifying high-risk individuals for myopia, this predictor is limited to simulating additive effects and the types of data it can incorporate, thus leaving considerable room for further research.

[0003] Myopia in children (HM) is often accompanied by choroidal retinal atrophy and optic disc deformity, making fundus imaging crucial for disease assessment. Recent studies have shown that deep learning models trained on baseline fundus images can accurately predict the onset of myopia in children. Qi et al. constructed a deep learning model based on fundus images to predict myopia onset and intervention response (see: https: / / doi.org / 10.1038 / s41746-024-01204-7). However, the significant heterogeneity and temporal variability of fundus images prevent their direct application in routine clinical practice.

[0004] Combining lifetime risk genetic variants with fundus images may improve the accuracy of predicting strabismus progression. However, constructing multimodal models remains challenging because high-dimensional genotypic data (with more variants than individuals) introduces the "large-p-small-n" problem into standard statistical models. Furthermore, the fusion of molecular biomarkers with image-based morphological features still lacks interpretability, as current multimodal models struggle to distinguish cross-modal relationships, limiting robust prognostic biomarker identification and accurate patient risk stratification. Summary of the Invention

[0005] This invention aims to address at least one of the technical problems existing in the prior art. To this end, this invention provides a method, device, medium, and program product for generating myopia risk assessment information; the method of this invention utilizes MIRAGE, a deep learning framework integrating genetic and fundus imaging data, to improve the prediction of high myopia risk and achieve interpretable biomarker discovery. This application uses a custom neural network to capture common and rare variant effects superior to the PRS method, while attribution-based analysis highlights key morphological and molecular features for identifying clinically relevant biomarkers.

[0006] The first aspect of this application discloses a method for generating myopia risk assessment information, including the following steps: Acquire whole-exome sequencing data and fundus image data of the target subject; The whole exome sequencing data were subjected to single nucleotide polymorphism encoding and feature extraction to generate gene feature vectors; the fundus image data were subjected to convolutional feature extraction to generate image feature vectors. The gene feature vector and the image feature vector are fused using a gated attention mechanism to generate multimodal fusion features. The gated attention mechanism includes mapping the dimension-aligned gene feature vector and the image feature vector to gate weights, and then using the gate weights to perform weighted calibration on the corresponding feature vectors. Based on the multimodal fusion features, a classification model is used to calculate and output the myopia risk assessment probability value of the target object.

[0007] A second aspect of this application discloses a computer device, comprising: a memory and a processor; the memory is used to store a computer program; the processor executes the computer program to implement the steps of the above-described method.

[0008] A third aspect of this application discloses a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described above.

[0009] The fourth aspect of this application discloses a computer program product, including a computer program that, when executed by a processor, implements the steps of the above-described method.

[0010] This application has the following beneficial effects: This application innovatively proposes an interpretable deep learning framework, MIRAGE, which integrates fundus imaging and whole-exome genotyping to predict the risk of high myopia (HM). By combining lifetime genomic susceptibility with current retinal features, MIRAGE achieves higher accuracy (ACC = 0.903, F1 = 0.910) than unimodal models through gated attention mechanisms and adaptive noise reduction techniques. This framework enables personalized risk assessment, elucidates the contributions of genetic and imaging factors, and reveals gene-gene interaction networks.

[0011] MIRAGE demonstrates how multimodal artificial intelligence can advance precise risk stratification and mechanistic insights in genomic medicine by achieving high precision and revealing the underlying molecular networks of high myopia susceptibility. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 This is a schematic diagram of the method flow provided in the first aspect of the present invention; Figure 2 This is a schematic diagram of a system for generating myopia risk assessment information provided in the second aspect of the present invention; Figure 3 This is a schematic diagram of a computer device provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the architecture of an exemplary computing device provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of the storage medium provided in an embodiment of the present invention; Figure 6 This is a schematic diagram summarizing the MIRAGE method provided in an embodiment of the present invention, wherein, Figure 6 A represents a one-time encoding of the WES data to generate an input matrix with SNPs as rows and samples as columns, which is then input into the genetic model DeepExGRS for feature extraction. Figure 6 B indicates that the fundus image is center-cropped and pixel-normalized before being input into the image model CNN for feature extraction. Figure 6 C represents the feature fusion module of the MIRAGE model. SNPs and image features are dynamically weighted through a gating mechanism, and then deeply interacted through Kronecker products. Initial features and interaction terms are fused using skip connections, and the final output is generated through a custom multimodal loss function. Figure 6D represents the evaluation of the MIRAGE model, including classification performance analysis, modality-specific contribution interpretation, and interpretability of multimodal representation; Figure 7 This is a schematic diagram illustrating the performance and multimodal interpretability analysis results of MIRAGE provided in this embodiment of the invention; wherein, Figure 7 A shows the ROC curves for MIRAGE ConvNeXt-Tiny and DeepExGRS. Performance was evaluated by repeatedly performing random training / validation / test splits and model training using five different random seeds. The solid line represents the average ROC curve over five runs, and the shaded area represents the corresponding standard deviation. The area under the curve (AUC) is reported as mean ± standard deviation. Figure 7 B represents the accuracy (ACC), precision, recall, and F1 score of DeepExGRS ConvNeXt-Tiny and MIRAGE, reported based on five independent runs using different random seeds. The bar height represents the average performance, and the superimposed points represent the results of a single run. Figure 7 C represents the sample-level analysis of the relative contributions of genetic and image modalities to the performance of the multimodal model. Contribution values ​​are specified as -0.5 (negative), 0 (neutral), 0.5 (moderate), 1 (cooperative), and 1.5 (dominant). Figure 7 D represents a bubble plot, showing the top 10 SNPs ranked by their contribution to the MIRAGE model. The bubble size reflects the magnitude of feature attribution. Figure 7 E shows a heatmap of attention weights for different image regions in the MIRAGE model; brighter colors indicate higher attention levels. Figure 8 This is a performance evaluation diagram of the DeepExGRS model provided in an embodiment of the present invention, wherein, Figure 8 A shows the ROC curves for MIRAGE ConvNeXt-Tiny and DeepExGRS. Performance was evaluated by repeatedly performing random training / validation / test splits and model training using five different random seeds. The solid line represents the average ROC curve over five runs, and the shaded area represents the corresponding standard deviation. The area under the curve (AUC) is reported as mean ± standard deviation. Figure 8 B represents the AUC comparison of rare variants (DeeprvGRS vs. rvGRS) and combined variants (DeepExGRS vs. ExGRS). Figure 8 C represents the AUC comparison of DeepCVGR with CNN MLP and Lasso models using a common variant dataset; Figure 9 This is a schematic diagram of gene interaction analysis using DeepExGRS provided in an embodiment of the present invention; wherein, Figure 9A provides an overview of the gene interaction analysis framework. The left figure shows a real gene interaction assessment, while the right figure illustrates the permutation test framework. SNPs within the same gene are grouped into gene blocks. During the permutation test, single-hot encoded SNPs within each gene block are randomly shuffled to disrupt the original structure. Figure 9 B represents the comparison between the true gene interaction score (x-axis) and the permutation-derived score (y-axis). Blue dots represent gene pairs with significant interactions, blue error bars represent the 95% confidence interval of the permutation, and red dashed lines represent the null hypothesis of no interaction. Figure 9 C represents the gene interaction network for gene pairs with significant interactions. Node colors correspond to p-values, with redder nodes indicating more significant interactions. Figure 9 D shows a bar graph displaying the number of known (left) and unknown (right) gene pairs in different p-value bins. Detailed Implementation

[0014] To enable those skilled in the art to better understand the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings.

[0015] In some of the processes described in the specification, claims, and accompanying drawings of this invention, multiple operations appearing in a specific order are included. However, it should be clearly understood that these operations may not be executed in the order they appear herein, or may be executed in parallel. The operation numbers, such as 101, 102, etc., are merely used to distinguish different operations and do not represent any execution order. Furthermore, these processes may include more or fewer operations, and these operations may be executed sequentially or in parallel. It should be noted that the descriptions such as "first," "second," etc., in this document are used to distinguish different messages, devices, modules, etc., and do not represent a sequential order, nor do they limit "first" and "second" to different types.

[0016] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0017] Figure 1 This is a schematic flowchart of a method for generating myopia risk assessment information according to an embodiment of the present invention. Specifically, the method includes the following steps: S101: Acquire whole exome sequencing data and fundus image data of the target object; In some embodiments, the terms “subject” or “test subject” or “sample to be tested” as used herein refer to any animal (e.g., a mammal), including but not limited to humans, non-human primates, rodents, etc., which will become the recipient of a particular treatment. Generally, the terms “subject” and “patient” are used interchangeably herein when referring to human subjects. Preferably, the subject is a human. In some embodiments, the target is a patient clinically undergoing prognostic assessment.

[0018] S102: Perform single nucleotide polymorphism encoding and feature extraction on the whole exome sequencing data to generate gene feature vectors; perform convolution feature extraction on the fundus image data to generate image feature vectors; In some embodiments, the single nucleotide polymorphism encoding and feature extraction of the whole exome sequencing data includes: Identify single nucleotide polymorphism (SNP) sites in the whole exome sequencing data; construct a four-dimensional coding vector for each SNP site; the four dimensions of the four-dimensional coding vector correspond to homozygous reference genotype, heterozygous genotype, homozygous substitution genotype, and deletion genotype, respectively; when a data deletion is detected for a certain SNP site, the four-dimensional coding vector is set to a preset deletion state one-hot encoding; input the encoded data into a grouped fully connected network to extract the gene feature vector.

[0019] S103: The gene feature vector and the image feature vector are fused using a gated attention mechanism to generate multimodal fusion features; the gated attention mechanism includes: mapping the dimension-aligned gene feature vector and the image feature vector to gate weights, and using the gate weights to perform weighted calibration on the corresponding feature vectors respectively; In some embodiments, the feature fusion of the gene feature vector and the image feature vector through a gated attention mechanism includes: The gene feature vector and the image feature vector are concatenated and input into a multilayer perceptron to output a joint feature representation. The joint feature representation is then processed using an activation function to generate a first gating coefficient for the gene modality and a second gating coefficient for the image modality. The Hadamard product of the first gating coefficient and the gene feature vector is calculated to obtain the gated gene feature. The Hadamard product of the second gating coefficient and the image feature vector is calculated to obtain the gated image feature. The outer product of the gated gene feature and the gated image feature is calculated to generate the multimodal fusion feature containing nonlinear interaction information.

[0020] In some embodiments, the generation of multimodal fusion features further includes: Construct cross-modal residual connections; concatenate the gated gene features, the gated image features, and the nonlinear interaction information along the channel dimension, and use them as the multimodal fusion features input to the final classification model.

[0021] S104: Based on the multimodal fusion features, calculate and output the myopia risk assessment probability value of the target object using a classification model.

[0022] In some embodiments, the classification model and the feature extraction network used in the gene feature vector generation step are obtained through joint training, and the loss function used in the joint training includes a weighted sum of the main prediction task loss term, the consistency constraint loss term, and the sparse regularization loss term. The consistency constraint loss term is used to constrain the directional consistency between the gene feature extraction branch and the image feature extraction branch in the prediction trend.

[0023] In some embodiments, after outputting the myopia risk assessment probability value, the method further includes: The contribution score of each single nucleotide polymorphism site and fundus image pixel region at the input end to the myopia risk assessment probability value is calculated using the integral gradient algorithm. Based on the contribution score, an interactive report is generated that displays a list of key gene loci or an overlaid heatmap of the fundus image.

[0024] In some embodiments, the interactive report generation step further includes performing statistical validation based on a permutation test: Multiple sets of random background data are generated by randomly shuffling single nucleotide polymorphism sites in the whole exome sequencing data; Calculate the distribution of the interaction scores of the random background data in the classification model; The interaction scores of real data are compared with the distribution of interaction scores to filter out gene-gene interaction pairs or gene-image interaction regions with statistical significance higher than a preset threshold, and these regions are marked in the interactive report.

[0025] In some embodiments, the probability value of myopia risk assessment includes, but is not limited to, paper or electronic reports. This result is obtained by intelligent machines based on the relevant data of the subjects and is only used as a reference for medical staff, and is not used as the final diagnosis result of the subjects.

[0026] Figure 3 This is a schematic diagram of a computer device provided in an embodiment of the present invention, such as... Figure 3As shown, the device 2000 may include: one or more processors 2010 and one or more memories 2020; wherein the memories store computer-readable code that, when run by the one or more processors, can perform the methods described above.

[0027] The processor in this embodiment can be an integrated circuit chip with signal processing capabilities. The processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, operations, and logic block diagrams disclosed in this embodiment. The general-purpose processor can be a microprocessor or any conventional processor, and can be based on an x86 or ARM architecture.

[0028] In general, the various exemplary embodiments of this disclosure can be implemented in hardware or dedicated circuitry, software, firmware, logic, or any combination thereof. Some aspects can be implemented in hardware, while others can be implemented in firmware or software that can be executed by a controller, microprocessor, or other computing device. When aspects of embodiments of this disclosure are illustrated or described as block diagrams, flowcharts, or using some other graphical representation, it will be understood that the blocks, apparatuses, systems, techniques, or methods described herein can be implemented as non-limiting examples in hardware, software, firmware, dedicated circuitry or logic, general-purpose hardware or controllers or other computing devices, or some combination thereof.

[0029] For example, the method or apparatus according to embodiments of this disclosure can also be used by means of Figure 4 The architecture of the computing device 3000 shown is used for implementation. For example... Figure 4 As shown, the computing device 3000 may include a bus 3010, one or more CPUs 3020, a read-only memory (ROM) 3030, a random access memory (RAM) 3040, a communication port 3050 connected to a network, an input / output component 3060, a hard disk 3070, etc. The storage devices in the computing device 3000, such as the ROM 3030 or the hard disk 3070, may store various data or files used for processing and / or communication of the methods provided in this disclosure, as well as program instructions executed by the CPU. The computing device 3000 may also include a user interface 3080. Of course, Figure 4 The architecture shown is merely exemplary and can be omitted as needed when implementing different devices. Figure 4 One or more components in the computing device shown.

[0030] This invention also includes a computer-readable storage medium, such as... Figure 5The diagram illustrates a storage medium 4000 provided in an embodiment of the present invention. The computer storage medium 4020 stores computer-readable instructions 4010. When the computer-readable instructions 4010 are executed by a processor, the method described above according to embodiments of the present disclosure can be performed. The computer-readable storage medium in the embodiments of the present disclosure may be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. Non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory may be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM), Enhanced Synchronous Dynamic Random Access Memory (ESDRAM), Synchronous Link Dynamic Random Access Memory (SLDRAM), and Direct Memory Bus Random Access Memory (DRRAM). It should be noted that the memory used in the methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0031] This disclosure also provides a computer program product or system, including a computer program that, when executed by a processor, implements the steps of the above-described method, such as... Figure 2 As shown, the computer program product or computer program is also used to execute a system for generating myopia risk assessment information, the system comprising: The multimodal data acquisition module 201 is used or configured to acquire whole exome sequencing data and fundus image data of the target object; The feature vector generation module 202 is used or configured to perform single nucleotide polymorphism encoding and feature extraction on the whole exome sequencing data to generate gene feature vectors; and to perform convolution feature extraction on the fundus image data to generate image feature vectors. The feature fusion module 203 is used or configured to perform feature fusion on the gene feature vector and the image feature vector through a gated attention mechanism to generate multimodal fusion features; the gated attention mechanism includes: mapping the dimension-aligned gene feature vector and the image feature vector to gate weights, and using the gate weights to perform weighted calibration on the corresponding feature vectors respectively; Risk assessment module 204 is used or configured to calculate and output the myopia risk assessment probability value of the target object based on the multimodal fusion features using a classification model.

[0032] In this embodiment, to more clearly describe the acquisition, processing, and training process of the deep learning framework that integrates genetic and fundus imaging data, a case study of high myopia is used as an example. However, this description is only one application scenario and is not intended to limit the scope of protection of this data processing method. Those skilled in the art will understand that the content of this embodiment, in addition to using the following acquisition method for the auxiliary prediction of high myopia, can be extended to other fundus image and genetic auxiliary prediction scenarios. Specific exemplary descriptions are as follows: Materials and Methods Dataset Description: The Myopia-Related Genetics and Intervention Consortium (MAGIC) is a large genomic consortium that integrates myopia cohort and sequencing data from numerous researchers. Over the past few years, MAGIC has collected samples through the Institute of Biomedical Big Data (Zhejiang Eye Hospital, Affiliated to Wenzhou Medical University). Sample recruitment, quality control, and phenotypic definition followed previous study definitions. In the first phase of the MAGIC cohort study, 9613 Han Chinese patients aged 6 to 18 years with myopia (HM) and 9606 controls were recruited. In the second phase, 1069 HM patients and 922 age-matched controls aged 18 to 25 years were recruited. All participants in the first phase underwent whole-exome sequencing (WES), while the second phase simultaneously collected WES and fundus imaging data to integrate genomic and morphological features.

[0033] In this study, high myopia was defined as an equivalent spherical power (SER, spherical power + cylindrical power²) ≤ -6.00 diopters (D) in at least one eye, consistent with the standards and demographic characteristics detailed in previous literature. This study was approved by the Ethics Committee of the Affiliated Eye Hospital of Wenzhou Medical University (approval numbers: Wmu191204 and Wmu191205). All participants or their legal guardians signed written informed consent forms prior to the start of the study, conforming to the principles of the Declaration of Helsinki and following the "Guidelines for Human Genetic Disease Sample Collection" (2021SQCJ5721) issued by the National Health Commission of China. All procedures were strictly performed in accordance with the "Regulations on the Management of Human Genetic Resources" (issued by the Ministry of Science and Technology of China, numbers: BF2022060511307 and 197BF2022060611309, effective from November 8, 2021).

[0034] Genotyping quality control: This study adopted the sample quality control and variant quality control methods of MAGIC from previous studies. First, samples with phenotypic data were selected. High-quality variants that passed the GATK variant quality score recalibration (VQSR) and were located outside low-complexity genomic regions were retained. Heterozygous variants with genotype depth (DP) <10, genotype quality (GQ) <20, or allele balance >0.8 or <0.2 were marked as missing values. Subsequently, PLINKv1.9 based on (GRCh37 human genome assembly) was used to exclude genotype missing rates >5% and Hardy-Weinberg balance (HWE) test p-values ​​<1×10⁻⁶. -6 Variations with a minor allele count (MAC) <3.

[0035] Variance Annotation: Similar to previous studies, this application uses Ensembl's Variance Effect Predictor (VEP v.99) for variant annotation. In addition to standard annotation, VEP was used to generate extended bioinformatics predictions of variant harmfulness. Protein-coding variants were classified into four functional categories: (1) synonymous variants; (2) benign missense variants; (3) harmful missense variants; and (4) protein truncation variants (PTV). Specifically, VEP annotated missense variants as "in-box deletion," "in-box insertion," and "missense variant."

[0036] MIRAGE architecture and model building: MIRAGE (Multimodal Interpretable Risk Assessment Based on Genetic and Ocular Imaging Data) is a high-throughput and interpretable deep learning framework designed to integrate genotypic data and fundus images for individual-level risk prediction of hemangiomas. Figure 6 As shown, we first construct two unimodal models to learn modality-specific representations: a multilayer perceptron (MLP) network called DeepExGRS is used to model common and rare variants across the exome range, while a convolutional neural network (ConvNeXt) is used to extract hierarchical morphological features from fundus images. To achieve cross-modal interaction, a fusion module based on a Kronecker product is used to integrate latent features from both modalities. This fusion module is further improved through a gated attention mechanism and skip connections to preserve complementary modality-specific information. This architecture enables MIRAGE to capture the nonlinear relationship between genetic variation and retinal phenotype, thereby improving the accuracy and interpretability of HM risk prediction.

[0037] To effectively extract predictive signals from high-dimensional genotype data, the DeepeExGRS module employs a hot encoding strategy followed by embedding-based representation learning, enabling the model to capture complex patterns without imposing artificial ordering relationships. Specifically, each genotype is encoded as a 4-dimensional binary vector, where different combinations represent homozygous reference, heterozygous, homozygous alternation, and deletion genotypes (encoded as [0 0 0 1]). This approach replaces traditional additive encoding (genotype dose 01 2), which can introduce implicit ordering assumptions inconsistent with the underlying biological mechanisms. DeepExGRS is built on a standard MLP architecture, combining modular block layers, normalization layers, Swish activation functions, and fully connected layers. In the feature extraction stage, four adjacent non-overlapping SNPs are grouped in the first layer, and then eight adjacent non-overlapping SNPs are grouped in the second and third layers, effectively capturing local genomic patterns. This hierarchical grouping produces a 704-dimensional feature vector, which is then passed through fully connected layers to generate the final prediction. Dropout and random depth are used for regularization, enhancing generalization and mitigating overfitting.

[0038] The image model employs a ConvNeXt-Tiny architecture, comprising a Conv2dNormActivation layer, a CNBlock module, a Conv2d layer, an AdaptiveAvgPool2d layer, and a classifier. The original 1000-class output layer is replaced with a single output node for binary classification. The RGB input image is expanded from 96 to 192, 384, and 768 feature map channels, followed by global average pooling. Then, a fully connected layer projects the 768-dimensional vector onto a single output representing the prediction.

[0039] DeepExGRS and ConvNeXt-Tiny were used to extract feature representations from genotype and fundus image data, generating vectors XSNP ∈ 704 × 1 and XImage ∈ 768 × 1, respectively. Connective layers were applied to map the two feature vectors to the same dimension ^32 × 1, thus facilitating modality alignment. A gated attention mechanism was then introduced to adaptively weight the contribution of each modality, allowing the model to emphasize informative features while reducing noise and redundancy in single-modal inputs. This is illustrated by the following equation: (1) (2) (3) Formula (1) is used to extract unimodal features. It represents the extraction of unimodal feature representation by performing a linear transformation + nonlinear activation on unimodal features (genotype SNP or fundus image). Matrix x i This represents the dimension-aligned feature vector. Linear transformation weight matrices for single-modal features designed for SNPs or Images respectively. ReLU is the bias term for the single-modal features (compensating for the deviation of the linear transformation), and it is the activation function that introduces nonlinearity. To form a joint feature vector, the genotype feature X is... SNP and image feature vector X Image The features are concatenated to form a joint representation, where W is the linear transformation weight matrix of the joint features, b is the bias term of the joint features, and σ (sigmoid) is the activation function. The weight matrix is ​​calculated based on this joint representation. This matrix is ​​then passed through the sigmoid activation function to produce a score matrix Z, where each element reflects the importance of the corresponding pair of unimodal features. The score matrix is ​​then applied to X using the Hadamard product (element-wise multiplication). SNP and X Image The score matrix z is used as the gate signal, and element-wise multiplication is performed with the single-modal feature xi. Then, through linear transformation, ReLU activation, and Dropout, the gated feature representation Xi is obtained, which is gated (preserving important features and suppressing irrelevant features). After the gate mechanism, Kronecker product is used to fuse the multimodal features. The fused representation is calculated as follows, where... Representing the Kronecker product: (4) To preserve information signals during network propagation and avoid over-reliance on single modalities, a skip connection approach is adopted by directly linking the fused representation with single-modal features before fusion. (5) Where Contact is the connection operation, X fusion These are the characteristics after the connection.

[0040] Loss function: Capturing the inter-modal interactions in multimodal models remains challenging when applying standard loss functions, as they often overlook potential and complex dependencies across feature spaces. To overcome this limitation, this application proposes a custom loss function designed to enhance modal interactions and improve prediction accuracy. The objective function is defined as follows.

[0041] (6) Where Y represents the binary phenotypic label, 0 represents the control, and 1 represents the case status. The first two terms are mean squared error (MSE) losses: the first term captures the reconstruction error of the fused multimodal representation reflecting cross-modal interactions; the second term maintains the integrity of the unimodal representation by independently minimizing the prediction errors from the genotype and image branches. The third term introduces L1 norm regularization on the model weights, which are controlled by hyperparameters, to promote sparsity and perform implicit feature selection. By penalizing small weights, the model suppresses non-informative features and reduces overfitting. The fourth term imposes a consistency constraint between the unimodal outputs through MSE, encouraging the genotype and image branches to converge to similar predictions. This term is scaled by a coefficient. In this experiment, and are set.

[0042] Evaluation scheme and hyperparameter selection: The dataset was randomly divided into training, validation, and testing partitions in a 7:2:1 ratio. To mitigate potential evaluation bias caused by distribution variations resulting from a single data split, this application used five different random seeds to generate multiple dataset partitions for each model. Training and evaluation procedures were performed independently on each partition, and the final performance was reported as the mean and variance of the evaluation metrics across the five datasets.

[0043] To determine the learning rate for different models, a learning rate sensitivity analysis was performed under the same multi-split setting. Specifically, while keeping other hyperparameters constant, multiple candidate learning rates were evaluated for each module. The learning rate that achieved the best average performance across all five splits and exhibited stable convergence behavior was selected as the optimal configuration for the corresponding model.

[0044] Training Details: During model development, genotypic and fundus image data were randomly divided into training (70%), validation (20%), and test (10%) sets to ensure an approximately 1:1 case-control ratio and no overlapping samples in segmentation. Genotypic data were encoded from PLINK binary format files (.bim.bed.fam) into a sample-level one-hot encoding matrix. Prior to training, input images were cropped to 224 × 224 pixels and normalized.

[0045] All models were implemented using PyTorch (version 1.13) and trained on a workstation equipped with an NVIDIA GeForce RTX 3060 GPU (12GB RAM). Training was performed using a single GPU with a batch size of 32 without distributed parallelism. The Adam optimizer was used for gradient updates, and the learning rate of DeepExGRS was set to 1e. -5 The learning rate for both the ConvNeXt and fusion models was set to 1e. -4The optimizer and parameters are left at their default values. The StepLR scheduler is applied to dynamically adjust the learning rate during training. To mitigate overfitting, early stopping is implemented for up to 200 epochs; training stops if the validation loss fails to improve within a fixed number of consecutive epochs.

[0046] For DeepExGRS and ConvNeXt, use binary cross-entropy with Logits loss (BCE WithLogits Loss): (7) Where N represents the number of samples, y i z represents the basic true phenotype of the i-th sample. i This is the original model output, the Sigmoid function. Defined as: (8) The MIRAGE model is trained using the custom multimodal loss function proposed above.

[0047] Multimodal Interpretability and Visualization: To interpret the contributions of genetic and imaging features in a multimodal model, we employ the Integral Gradient (IG) method to evaluate the model's attribution during training. IG is able to identify information-rich genetic loci and key image regions driving model predictions. As a gradient-based attribution technique, IG quantifies the contribution of each input feature by computing the path integral gradient between the baseline input and the actual input: (9) Where x represents the model input, x i This represents the i-th input feature. The baseline input is used. The integral calculation determines how the predicted output F(x) changes as the input transitions from the baseline to the actual input, thus assigning importance scores to each feature. Using the trained model, the contributions of genetic and imaging data to the validation set are evaluated. For each SNP, the mean absolute IG attribution value of the sample is calculated as its contribution to the model's predictions. For image data, pixel-wise attribute values ​​are calculated and overlaid as heatmaps on the original images, visually highlighting regions that influence the model's output. This approach helps to delve into understanding the model's decision-making process and the predictive relevance of each modality.

[0048] Genetic Attribution and Image Features: To quantitatively evaluate the contribution of each modality at the individual sample level, this application introduces a fine-grained modality attribution metric based on Shapley values. This approach enables the evaluation of the contribution of each modality to the performance improvement of the multimodal model (on a per-sample basis). Given a sample with two available modalities... First, the predictive utility of a model using a subset of modalities is defined as: (10) Where M represents the subset of modes used for prediction, and if g(M) predicts correctly, it is assigned a utility score equal to the number of modes used, otherwise it is 0.

[0049] To capture the marginal contribution of each mode under all possible conditions, let This represents the set of all possible permutations of n modes. For a given permutation... And mode i, let The modal geometry i prior to the mode is represented in the permutation test k, and the marginal contribution i of the mode is defined as: (11) This formula quantifies the additional value i of a mode when the permutation k follows all previous modes, and the average value over all possible permutations is the Shapley value. This value represents the overall contribution x of mode i to the model's prediction of the sample: (12) Gene-Gene Interaction Score: The Shapley interaction score is a Shapley value-based method used to quantify the interaction effect between two features. The core idea is to assess the joint contribution of a specific feature pair by considering the impact of a subset of features on the model's predictions. In this study, this application annotated the top 10,000 SNPs selected by the genetic model DeepExGRS that contributed the most to their respective genes. The Shapley interaction score was then used to calculate the interactions between gene pairs. A gene block layer was added on top of the DeepExGRS model, grouping SNPs within the same gene into blocks. The block size of the first layer in DeepExGRS was set to 1, enabling the model to calculate interactions between gene pairs at the gene level. Let the gene set be F, where i and j are two distinct genes in F, and S is a subset of F excluding i and j. Let f denote the genetic model DeepExGRS. The formula for calculating the Shapley interaction score is as follows: (13) in, This represents the prediction result of model f for the k-th sample in the genetic data under different feature conditions. This represents the interaction value between genes i and j.

[0050] Identifying Significant Gene Interactions Using Permutation Tests: To further evaluate whether the calculated gene interactions are significant, this application proposes a method to determine the statistical significance of gene interactions by calculating an empirical distribution of the values ​​of each pair of gene interactions, rather than using an arbitrary fixed cutoff value. Here, permutation tests are used to generate the empirical distribution of gene interactions. Each gene contains a corresponding SNP locus. To study gene-gene interactions, the SNP data within each gene are randomly shuffled to generate a new gene dataset. To more accurately capture the relationships between genes and avoid label bias, phenotype Y is kept constant. Significant gene interactions should exhibit larger interaction values ​​compared to insignificant gene interactions. Therefore, the empirical p-value is defined as follows: (14) Where N represents the number of permutations (N=1000), and I is an indicator function. These are the actual gene interaction values. These are the interaction values ​​obtained from the permutation test. When the conditions are met... Under certain conditions, I equals 1; otherwise, it is 0. The empirical p-value is calculated as the proportion of gene interactions in the permutation that are greater than the actual interaction value. Gene pairs with p-values ​​less than 0.05 are defined as having significant interactions.

[0051] result: Improve the multi-modal integration of risk assessment: To address the challenge of creating joint genetic image features for disease prediction, MIRAGE was developed as a high-throughput and interpretable deep learning framework that integrates fundus images and whole-exome genotypic data for patient-level HM risk prediction. Figure 6 For genomic data that provide lifetime risk information, this application developed a module called DeepExGRS, a neural network-based multilayer perceptron (MLP) model designed to efficiently capture exome genetic variations in large cohorts. Figure 6 A). In model construction, candidate SNPs were selected based on previous whole-exome association studies (ExWAS), particularly those exhibiting p-values ​​<0.05 and odds ratios (OR) >1, resulting in 25,190 SNPs significantly associated with HM susceptibility. Image features were progressively extracted using hierarchical convolutions of ConvNeXt-Tiny and the CNBlock module, ultimately forming a pooling representation for classification. Figure 6 B). During multimodal fusion, the gated attention mechanism adaptively assigns weights to each modality, enhancing informative features while reducing noise and redundancy. Figure 6C). Following model training and evaluation, a detailed interpretability analysis was conducted to investigate intra- and inter-modal feature dependencies, modality-specific contributions, and the underlying mechanisms of multimodal integration. Figure 6 D).

[0052] The MIRAGE framework was further applied to the internal cohort of this application, comprising 1069 patients with high myopia and 922 controls. Participants were randomly assigned to training (70%), validation (20%), and test (10%) groups, ensuring no overlap in samples across partitions. The model was trained using the Adam optimizer, with β1 and β2 set to default values. The learning rate was determined based on validation-guided hyperparameter tuning. Specifically, multiple candidate learning rates were evaluated for the training-validation split for each module. DeepExGRS was used with a range of 1e. -2 up to 1e -6 The learning rate is used for training, while ConvNeXt-Tiny and fusion networks use a range of 1e. -3 up to 1e -6 The learning rate was evaluated. Based on validation performance, DeepExGRS's learning rate was set to 1e. -5 The learning rate of the image encoder and the fusion model is 1e -4 The selected model was used for the final framework. The StepLR scheduler dynamically adjusted the learning rate and stopped training early after 200 epochs if the validation loss did not improve. Overall, internal validation showed that MIRAGE achieved robust predictive performance against HM, with an average AUC of 0.958 on the validation set and an average AUC of 0.963 on the independent test set across five random splits. Figure 7 A).

[0053] To evaluate the effectiveness of multimodal fusion, this application conducted a comparative analysis of unimodal deep learning models, including a pure genetic model (DeepExGRS) and a pure image model. Performance evaluation on five random splits showed that the pure genetic model had a mean accuracy (ACC) of 0.801, precision of 0.788, recall of 0.882, and an F1 score of 0.831. In contrast, the pure image model showed improved performance, with a mean ACC of 0.866, precision of 0.873, recall of 0.853, and an F1 score of 0.862. Notably, the multimodal fusion model consistently outperformed the two unimodal models, achieving a mean ACC of 0.897, precision of 0.880, recall of 0.909, and an F1 score of 0.894 across five runs, highlighting the significant advantages of integrating genetic and imaging information for disease prediction. Figure 7 B).

[0054] Post-hoc multimodal interpretation: This application further clarifies the role of each modality in the fusion model through fine-grained analysis of the marginal contributions of genetic and imaging data at the individual sample level. Marginal contribution scores of -0.5, 0, 0.5, 1.0, and 1.5 represent negative, neutral, moderately synergistic, and dominant effects, respectively. The two modalities were the most frequent, with a score of 1.0 (59.1%), indicating strong complementarity. However, genetic traits more frequently showed moderate contributions (0.5: 29.3% vs. 7.4% for imaging), while imaging traits more frequently dominated (1.5: 26.8% vs. 4.8% for genetics), suggesting that imaging has a stronger impact on model performance in a larger proportion of high-confidence cases. Figure 7 C).

[0055] Following modal-level contribution assessment, integral gradients (IG) are applied to the trained model to identify key genetic features and image regions that have the greatest impact on individual predictions. IG attribution scores for genetic branches preferentially consider several SNPs with known biological relevance. Specifically, the second SNP, rs651724, is a synonymous variant in the GJD2 gene, a locus previously associated with myopia, refractive errors, and axial length. This application also identified two synonymous variants, rs7654255 in TENM3 and rs2066721 in HHAT, which showed associations with refractive errors and related phenotypes. Figure 7 D). For the imaging branch, saliency based on IG. Figure 1 The model highlights clinically relevant areas, such as the optic disc, macula, and mosaic fundus, demonstrating that it effectively captures meaningful imaging biomarkers. Figure 7 E).

[0056] Genome deep learning outperforms PRS in HM risk prediction: Building upon earlier ExPRS research, this application further explores whether deep learning can still extract additional HM signals from common variants, despite the smaller contribution of genetics to multimodal fusion compared to imaging factors. Using 12,600 MAGIC Phase 1 samples and published ExWAS pooled statistics, this application first constructs and evaluates a deep learning-based common variant genetic risk score (DeepCVGR), where SNPs across pruning r 2 Threshold selection. Based on a linear PR with multiple P-value thresholds, the neural network consistently outperforms the traditional scoring method, producing an AUC gain of 0.94% to 3.60%. Figure 8 A). As weaker SNPs are included, in r 2 With p=0.4 and p=1.0, the AUC peaked at 0.918, demonstrating improved performance and confirming that the deep model captures nonlinear patterns in common variants to enhance HM risk stratification.

[0057] Furthermore, this application also evaluated the predictive power of the deep learning-based rare variant genetic risk score (DeepRVGR) and its integration with DeepCVGR. To construct the model, this application incorporated rare PTV and D-mis from HM-related genes from 12,600 training participants. After applying nonlinear modeling, a significant improvement in HM genetic risk prediction was observed, leading to an increase in AUC from 0.786 to 0.901. Figure 8 B). This enhanced predictive power likely stems from deep learning's ability to capture the higher heritability and strong effect size of rare harmful variants in the MAGIC cohort, variants that may not be effectively modeled by linear methods. To enhance HM risk stratification, rare variants were combined with common variants to develop a deep learning-based whole-exome genetic risk score (DeepExGRS). The DeepExGRS model achieved the highest predictive accuracy with an AUC of 0.921, outperforming models for both rare variants (AUC=0.901) and common variants (AUC=0.918). It also surpassed the performance of the conventional PRS model reported an AUC of 0.897 in previous studies. Figure 8 B). DeepeExGRS consistently outperforms both traditional and deep learning-based baselines. It captures the complex nonlinear interactions between common genetic variations more effectively than CNN (AUC=0.903) and MLP (AUC=0.899). In contrast, the traditional machine learning method LASSO produces a much lower AUC of 0.756, highlighting the limitations of linear models in representing the polygenic structure of HM. Figure 8 C).

[0058] DeepExGRS identifies significant gene-gene interactions: The superior performance of DeepExGRS implies that linear PRS misses epistatic effects between variants. To reveal these effects, this application maps the top 10,000 most influential SNPs to genes and calculates pairwise Shapley interaction scores; significance is assessed by phenotypic-preserved arrangement of SNPs within each gene, generating empirical p-values. Gene pairs with p < 0.05 are considered to exhibit significant nonlinear interactions, contributing to HM risk ( Figure 9 A).

[0059] From a total of 2,775 candidate gene pairs from the top 10,000 SNPs, this application identified 314 pairs with significant interaction effects (p<0.05). To assess whether these observed interactions were non-random, the true interaction scores were compared with those generated by permutation tests, as shown in the QQ plot. Figure 9B). Each blue dot represents a gene pair, and most significant interactions fall entirely outside the 95% confidence interval derived from the null distribution (blue area), with interaction scores consistently below the red dashed line representing the null hypothesis. This suggests that the observed interactions are unlikely to be accidental. Next, 314 significant gene-gene interactions were visualized using a network graph, where each edge corresponds to an interaction, and the color intensity reflects the p-value (…). Figure 9 C). The resulting network reveals clear interaction centers and clusters, suggesting possible synergistic effects and functional convergence among HM-related genes.

[0060] To further evaluate the biological relevance we discovered, this application compared the identified interactions with the GeneMANIA database. Of the 314 gene pairs, 165 were directly or indirectly associated with previously reported interactions, while 149 represented potential novel gene-gene interactions. Figure 9 (D). Notably, most known interactions correspond to strong statistical signals, and many novel interactions also demonstrate high importance, highlighting the ability of the method in this application to reveal previously unidentified genetic relationships. Among these significant interactions, several gene pairs stand out due to their potential relevance to HM.

[0061] It should be noted that the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0062] In general, the various exemplary embodiments of this disclosure can be implemented in hardware or dedicated circuitry, software, firmware, logic, or any combination thereof. Some aspects can be implemented in hardware, while others can be implemented in firmware or software that can be executed by a controller, microprocessor, or other computing device. When aspects of embodiments of this disclosure are illustrated or described as block diagrams, flowcharts, or using some other graphical representation, it will be understood that the blocks, apparatuses, systems, techniques, or methods described herein can be implemented as non-limiting examples in hardware, software, firmware, dedicated circuitry or logic, general-purpose hardware or controllers or other computing devices, or some combination thereof.

[0063] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0064] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between apparatuses or units through some interfaces, and may be electrical, mechanical, or other forms.

[0065] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0066] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0067] The exemplary embodiments of this disclosure described in detail above are merely illustrative and not restrictive. Those skilled in the art will understand that various modifications and combinations can be made to these embodiments or their features without departing from the principles and spirit of this disclosure, and such modifications should fall within the scope of this disclosure.

Claims

1. A method for generating myopia risk assessment information, characterized in that, Includes the following steps: Acquire whole-exome sequencing data and fundus image data of the target subject; The whole exome sequencing data were subjected to single nucleotide polymorphism encoding and feature extraction to generate gene feature vectors; the fundus image data were subjected to convolutional feature extraction to generate image feature vectors. The gene feature vector and the image feature vector are fused using a gated attention mechanism to generate multimodal fusion features. The gated attention mechanism includes mapping the dimension-aligned gene feature vector and the image feature vector to gate weights, and then using the gate weights to perform weighted calibration on the corresponding feature vectors. Based on the multimodal fusion features, a classification model is used to calculate and output the myopia risk assessment probability value of the target object.

2. The method for generating myopia risk assessment information according to claim 1, characterized in that, The single nucleotide polymorphism encoding and feature extraction of the whole exome sequencing data includes: Identify single nucleotide polymorphism (SNP) sites in the whole exome sequencing data; construct a four-dimensional coding vector for each SNP site; the four dimensions of the four-dimensional coding vector correspond to homozygous reference genotype, heterozygous genotype, homozygous substitution genotype, and deletion genotype, respectively; when a data deletion is detected for a certain SNP site, the four-dimensional coding vector is set to a preset deletion state one-hot encoding; input the encoded data into a grouped fully connected network to extract the gene feature vector.

3. The method for generating myopia risk assessment information according to claim 1, characterized in that, The feature fusion of the gene feature vector and the image feature vector through a gated attention mechanism includes: The gene feature vector and the image feature vector are concatenated and input into a multilayer perceptron to output a joint feature representation. The joint feature representation is then processed using an activation function to generate a first gating coefficient for the gene modality and a second gating coefficient for the image modality. The Hadamard product of the first gating coefficient and the gene feature vector is calculated to obtain the gated gene feature. The Hadamard product of the second gating coefficient and the image feature vector is calculated to obtain the gated image feature. The outer product of the gated gene feature and the gated image feature is calculated to generate the multimodal fusion feature containing nonlinear interaction information.

4. The method for generating myopia risk assessment information according to claim 3, characterized in that, The generation of multimodal fusion features also includes: Construct cross-modal residual connections; concatenate the gated gene features, the gated image features, and the nonlinear interaction information along the channel dimension, and use them as the multimodal fusion features input to the final classification model.

5. The method for generating myopia risk assessment information according to claim 1, characterized in that, The classification model is obtained through joint training, and the loss function used in the joint training includes a weighted sum of the main prediction task loss term, the consistency constraint loss term, and the sparse regularization loss term. The consistency constraint loss term is used to constrain the directional consistency between the gene feature extraction branch and the image feature extraction branch in the prediction trend.

6. The method for generating myopia risk assessment information according to claim 1, characterized in that, After outputting the myopia risk assessment probability value, the following is also included: The contribution score of each single nucleotide polymorphism site and fundus image pixel region at the input end to the myopia risk assessment probability value is calculated using the integral gradient algorithm. Based on the contribution score, an interactive report is generated that displays a list of key gene loci or an overlaid heatmap of the fundus image.

7. The method for generating myopia risk assessment information according to claim 6, characterized in that, The interactive report generation step also includes performing statistical validation based on permutation tests: Multiple sets of random background data are generated by randomly shuffling single nucleotide polymorphism sites in the whole exome sequencing data; Calculate the distribution of the interaction scores of the random background data in the classification model; The interaction scores of real data are compared with the distribution of interaction scores to filter out gene-gene interaction pairs or gene-image interaction regions with statistical significance higher than a preset threshold, and these regions are marked in the interactive report.

8. A computer device, characterized in that, The device includes: a memory and a processor; the memory is used to store a computer program; the processor executes the computer program to implement the method according to any one of claims 1-7.

9. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by a processor, implements the method as described in any one of claims 1-7.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1-7.

Images