Gene image set-based prediction method, device and equipment and storage medium

Abstract

The application discloses a prediction method and device based on a gene image group, equipment and a storage medium, and the method comprises the steps that MRI data and SNP data of a subject are acquired; the MRI data and the SNP data are input into a preset prediction model, the MRI data and the SNP data are processed based on the preset prediction model, and a target prediction result is obtained; wherein the preset prediction model has a cross network structure, each substructure in the cross network structure fuses modal data corresponding to the MRI data and the SNP data, and different modal data is fused in each substructure, and the modal data corresponding to the MRI data and the SNP data is obtained by respectively performing corresponding low-dimensional conversion on the MRI data and the SNP data. The application increases the analysis on the patient characteristic data, thereby improving the accuracy of the AD risk prediction result.

Description

Technical Field

[0001] This application relates to the field of medical testing technology, and in particular to a prediction method, device, equipment and storage medium based on gene imaging. Background Technology

[0002] Alzheimer's disease (AD) is a common, progressive degenerative disease of the central nervous system, accounting for approximately 60%-80% of all dementia cases. Current research indicates that brain atrophy in AD patients often occurs before clinical manifestations. The risk of developing AD can be predicted using magnetic resonance imaging (MRI) or single nucleotide polymorphism (SNP) testing, further improving the effectiveness of early intervention for AD.

[0003] Currently, most early diagnostic techniques for Alzheimer's disease based on MRI and SNP employ statistical methods such as independent component analysis or canonical correlation analysis. These methods typically select imaging features such as gray matter volume in the region of interest on MRI as quantitative traits, establish regression models to analyze the association between imaging features and risk sites, and determine features associated with AD. However, correlation analysis is greatly affected by the dataset and statistical methods, making it difficult to accurately predict the risk of developing the disease in the early stages. Summary of the Invention

[0004] The main objective of this application is to provide a prediction method, device, equipment, and storage medium based on genomic imaging, aiming to solve the technical problem in related technologies that cannot accurately predict the risk of a patient's disease.

[0005] To achieve the above objectives, embodiments of this application provide a prediction method based on genomic imaging, the method comprising: Acquire MRI and SNP data from the subjects; The MRI data and the SNP data are input into a preset prediction model. Based on the preset prediction model, the MRI data and the SNP data are processed for prediction to obtain the target prediction result. The preset prediction model has a cross-network structure, and each substructure in the cross-network structure integrates modal data corresponding to the MRI data and the SNP data. Each substructure integrates different modal data, and the modal data corresponding to the MRI data and the SNP data are obtained by performing corresponding low-dimensional transformations on the MRI data and the SNP data respectively.

[0006] In one possible implementation of this application, the step of performing predictive processing on the MRI data and the SNP data based on the preset prediction model includes: Based on the preset prediction model, features of the MRI data are extracted to obtain image feature data, and features of the SNP data are extracted to obtain gene feature data. The image feature data and the gene feature data are respectively subjected to low-dimensional transformation processing to obtain image modality data and gene modality data; Based on the cross-network structure of the preset prediction model, the image modal data and gene modal data are fused to obtain the target fused data, wherein different modal data are fused in each substructure of the cross-network structure; Based on the target fusion data, predictive processing is performed on the MRI data and the SNP data.

[0007] In one possible implementation of this application, the step of fusing the image modality data and gene modality data based on the cross-network structure of the preset prediction model to obtain the target fused data includes: Based on the cross-network structure of the preset prediction model, the image modality data and gene modality data are fused to obtain multiple hybrid features; Key features are extracted from the mixed features, and target fused data is obtained based on the key features.

[0008] In one possible implementation of this application, the step of extracting features from the MRI data based on the preset prediction model to obtain image feature data includes: Based on the preset prediction model, convolution and pooling operations are performed on the MRI data to obtain the first extracted features; Determine the residual characteristics of the MRI data; Based on the first extracted features and the residual features, the image feature data is determined.

[0009] In one possible implementation of this application, the step of extracting features from the SNP data to obtain gene feature data includes: Based on the preset prediction model, the SNP data is transformed in a low dimension to obtain the transformation features; Determine the attention value of the transformation feature, and multiply the attention value by the transformation feature to obtain the second extracted feature; Based on the transformation features and the second extraction features, the gene feature data is determined.

[0010] In one possible implementation of this application, before the step of inputting the MRI data and the SNP data into a preset prediction model, the following steps are included: The MRI data and the SNP data are preprocessed to obtain preprocessed MRI data and the SNP data; The step of inputting the MRI data and the SNP data into a preset prediction model, and performing prediction processing on the MRI data and the SNP data based on the preset prediction model to obtain the target prediction result includes: The preprocessed MRI data and SNP data are input into a preset prediction model. Based on the preset prediction model, the preprocessed MRI data and SNP data are subjected to prediction processing to obtain the target prediction result.

[0011] In one possible implementation of this application, the step of preprocessing the MRI data and the SNP data to obtain preprocessed MRI data and the SNP data includes: The MRI data is subjected to bias field correction and skull dissection to obtain corrected data, and the corrected data is registered to a preset standard template to obtain the preprocessed MRI data. The SNP sites in the SNP data are filled in, and the risk sites associated with the disease are extracted from the SNP sites to obtain the preprocessed SNP data.

[0012] This application also provides a prediction device based on genomic imaging, the prediction device based on genomic imaging includes: The acquisition module is used to acquire the MRI and SNP data of the subjects. The processing module is used to input the MRI data and the SNP data into a preset prediction model, and perform prediction processing on the MRI data and SNP data based on the preset prediction model to obtain the target prediction result.

[0013] This application also provides a prediction device based on a genome imaging group. The prediction device based on a genome imaging group is a physical node device. The prediction device based on a genome imaging group includes: a memory, a processor, and a program of the prediction method based on a genome imaging group stored in the memory and executable on the processor. When the program of the prediction method based on a genome imaging group is executed by the processor, it can implement the steps of the prediction method based on a genome imaging group as described above.

[0014] To achieve the above objectives, a storage medium is also provided, on which a genomic imaging prediction program is stored, wherein when the genomic imaging prediction program is executed by a processor, the steps of any of the genomic imaging prediction methods described above are implemented.

[0015] This application provides a prediction method, device, equipment, and storage medium based on genomic imaging. Compared with related technologies that use correlation analysis to determine features related to Alzheimer's disease (AD) but cannot accurately predict the risk of developing the disease, this application acquires the subject's MRI data and SNP data; inputs the MRI data and SNP data into a preset prediction model; and performs prediction processing on the MRI data and SNP data based on the preset prediction model to obtain a target prediction result. The preset prediction model has a cross-network structure, and each substructure in the cross-network structure integrates modal data corresponding to the MRI data and SNP data. Each substructure integrates different modal data, and the modal data corresponding to the MRI data and SNP data are obtained by performing corresponding low-dimensional transformations on the MRI data and SNP data respectively. In this application, it is understood that by acquiring the subject's MRI data and SNP data, and inputting the MRI data and SNP data into a preset prediction model for processing, the modal data corresponding to the MRI data and the SNP data are fused through the cross-network structure in the preset prediction model, thereby increasing the correlation between the two types of data, thus determining the features related to AD, and then assessing the risk of predicting the patient's development of AD based on the target prediction results. In this application, by fusing SNP sample data and MRI sample data, the accuracy of the target prediction results is increased. Attached Figure Description

[0016] Figure 1 This is a flowchart illustrating the first embodiment of the prediction method based on genomic imaging in this application; Figure 2 This is a schematic diagram illustrating the implementation process of the prediction method based on genomic imaging in this application; Figure 3 This is a schematic diagram of the device structure of the hardware operating environment involved in the embodiments of this application; Figure 4 This is a schematic diagram of the MRI feature extraction process involved in the genomic imaging prediction method of this application; Figure 5 This is a schematic diagram of the SNP feature extraction process involved in the prediction method based on genomic imaging in this application; Figure 6 This is a schematic diagram of the Transformer network structure involved in the prediction method based on genomic imaging in this application; Figure 7 This is a flowchart illustrating the process of fusing MRI features and SNP features involved in the prediction method based on genomic imaging in this application. Detailed Implementation

[0017] It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit this application.

[0018] This application provides a prediction method based on genomic imaging. In the first embodiment of the prediction method based on genomic imaging in this application, referring to... Figure 1 The method includes: Step S10: Obtain the subject's MRI data and SNP data; Step S20: Input the MRI data and the SNP data into a preset prediction model, and perform prediction processing on the MRI data and the SNP data based on the preset prediction model to obtain the target prediction result; The preset prediction model has a cross-network structure, and each substructure in the cross-network structure integrates modal data corresponding to the MRI data and the SNP data. Each substructure integrates different modal data, and the modal data corresponding to the MRI data and the SNP data are obtained by performing corresponding low-dimensional transformations on the MRI data and the SNP data respectively.

[0019] This embodiment aims to increase the accuracy of target prediction results by fusing SNP sample data and MRI sample data.

[0020] The specific steps are as follows: Step S10: Obtain the subject's MRI data and SNP data; As an example, a genomic imaging-based prediction method can be applied to a genomic imaging-based prediction device, which is subordinate to a genomic imaging-based prediction system, which is a genomic imaging-based prediction device.

[0021] As an example, one way to obtain MRI data from a subject is to perform an MRI scan on the subject to obtain MRI images.

[0022] As an example, one way to obtain SNP data from subjects is to perform genetic testing on the subjects and extract the SNP data from their genes.

[0023] As an example, the subjects can be either AD patients or healthy individuals.

[0024] As an example, magnetic resonance imaging (MRI) is a commonly used non-invasive imaging method. Since brain atrophy in AD patients usually occurs before clinical manifestations, significant changes occur in the hippocampus, amygdala, and entorhinal cortex during the progression of AD. Therefore, AD can be diagnosed through imaging features. MRI data can reveal structural changes such as cortical thinning, brain atrophy, and changes in regional tissue density caused by neurodegenerative diseases. MRI data can achieve good results in the early diagnosis of AD.

[0025] As an example, single nucleotide polymorphism (SNP) refers to DNA sequence polymorphism caused by a single nucleotide variation at the genomic level. There are approximately four to five million SNPs in a person's genome. Alzheimer's disease (AD) is usually the result of the combined effects of multiple small-effect SNP risk sites and environmental factors. Therefore, using a patient's SNP data can predict the risk of developing AD before significant changes occur in the structure or function of the patient's brain, further improving the effectiveness of early intervention for AD.

[0026] Step S20: Input the MRI data and the SNP data into a preset prediction model, and perform prediction processing on the MRI data and the SNP data based on the preset prediction model to obtain the target prediction result; The preset prediction model has a cross-network structure, and each substructure in the cross-network structure integrates modal data corresponding to the MRI data and the SNP data. Each substructure integrates different modal data, and the modal data corresponding to the MRI data and the SNP data are obtained by performing corresponding low-dimensional transformations on the MRI data and the SNP data respectively.

[0027] As an example, the preset prediction model includes multiple modules, and the specific implementation process is as follows: Figure 2 As shown, in Figure 2 In the process, the preset prediction model first processes the input MRI data and SNP data, extracts features from the MRI data and SNP data respectively, and then fuses the feature data from the MRI data and SNP data to output the target prediction result.

[0028] As an example, cross-network structures such as Figure 7As shown, the cross-network structure includes a Cross-Transformer structure and a Self-Transformer structure. The Cross-Transformer structure is used to fuse data from different modalities but belonging to the same dimension, so as to learn the relationship between different modal data more effectively. Then, the mixed features are input into the Self-Transformer module to further extract the key features in the mixed features, thereby obtaining the features of MRI data and SNP data.

[0029] As an example, before inputting the cross-network structure, the MRI data and SNP data need to be transformed in a low dimension to obtain the corresponding Key matrix, Value matrix, and Query matrix. During the fusion process, the Key matrix and Value matrix obtained from the MRI data transformation and the Query matrix obtained from the SNP data transformation are fused in the Cross-Transformer structure, and the Key matrix and Value matrix obtained from the SNP data transformation and the Query matrix obtained from the MRI data transformation are fused in the Cross-Transformer structure to obtain MRI features and SNP features. Then, the target prediction results are output through a linear layer.

[0030] This application provides a prediction method, device, equipment, and storage medium based on genomic imaging. Compared with related technologies that use correlation analysis to determine features related to Alzheimer's disease (AD) but cannot accurately predict the risk of developing the disease, this application acquires the subject's MRI data and SNP data; inputs the MRI data and SNP data into a preset prediction model; and performs prediction processing on the MRI data and SNP data based on the preset prediction model to obtain a target prediction result. The preset prediction model has a cross-network structure, and each substructure in the cross-network structure integrates modal data corresponding to the MRI data and SNP data. Each substructure integrates different modal data, and the modal data corresponding to the MRI data and SNP data are obtained by performing corresponding low-dimensional transformations on the MRI data and SNP data respectively. In this application, it is understood that by acquiring the subject's MRI data and SNP data, and inputting the MRI data and SNP data into a preset prediction model for processing, the modal data corresponding to the MRI data and the SNP data are fused through the cross-network structure in the preset prediction model, thereby increasing the correlation between the two types of data, thus determining the features related to AD, and then assessing the risk of predicting the patient's development of AD based on the target prediction results. In this application, by fusing SNP sample data and MRI sample data, the accuracy of the target prediction results is increased.

[0031] Furthermore, based on the first embodiment of this application, another embodiment of this application is provided. In this embodiment, the step of performing predictive processing on the MRI data and the SNP data based on the preset prediction model includes: Step A1: Based on the preset prediction model, extract the features of the MRI data to obtain image feature data, and extract the features of the SNP data to obtain gene feature data; As an example, MRI data is input into a three-dimensional convolutional neural network, and image features of the MRI sample data are extracted, specifically as follows: Figure 4 As shown.

[0032] As an example, SNP data is input into a Transformer-based network, and the corresponding gene features of the SNP data are extracted, specifically as follows: Figure 5 As shown.

[0033] Step A2: Perform low-dimensional transformation processing on the image feature data and the gene feature data respectively to obtain image modality data and gene modality data; As an example, before feature fusion, image features and genetic features need to be processed separately and converted into matrix data, so that feature fusion can be performed.

[0034] As an example, image modal data specifically refers to the Key matrix, Value matrix, and Query matrix derived from MRI data.

[0035] As an example, genetic modality data specifically refers to the Key matrix, Value matrix, and Query matrix derived from SNP data.

[0036] Step A3: Based on the cross-network structure of the preset prediction model, the image modal data and gene modal data are fused to obtain the target fused data, wherein different modal data are fused in each substructure of the cross-network structure; As an example, image modality data and genetic modality data are fused using a cross-network structure to obtain the target fused data, such as... Figure 7 As shown, the cross-network structure includes a two-layer Cross-Transformer structure and a Self-Transformer structure. The upper and lower parts of the cross-network structure have the same Cross-Transformer structure and Self-Transformer structure, but the output fusion features are different, outputting MRI features and SNP features respectively.

[0037] Step A4: Based on the target fusion data, perform predictive processing on the MRI data and the SNP data.

[0038] As an example, target fusion data refers to MRI features and SNP features obtained by fusing image modality data and genetic modality data.

[0039] As an example, after obtaining MRI features and SNP features, the MRI features and SNP features are concatenated and processed through a linear layer to output the corresponding target prediction results.

[0040] The step of fusing the image modality data and gene modality data based on the cross-network structure of the preset prediction model to obtain the target fused data includes: Step B1: Based on the cross-network structure of the preset prediction model, the image modality data and gene modality data are fused to obtain multiple hybrid features; As an example, based on the cross-network structure, the image modality data and gene modality data are first fused using the Cross-Transformer structure to obtain two hybrid features.

[0041] Step B2: Extract key features from the mixed features, and obtain target fusion data based on the key features.

[0042] As an example, extracting key features from mixed features and further extracting feature data from the mixed features can improve the accuracy of target prediction results.

[0043] As an example, the target fusion data is the fusion feature obtained by fusing MRI data and SNP data and further feature extraction. The MRI data and SNP data in the target fusion data are strongly correlated.

[0044] The step of extracting features from the MRI data based on the preset prediction model to obtain image feature data includes: Step C1: Based on the preset prediction model, perform convolution and pooling operations on the MRI data to obtain the first extracted features; As an example, the preset prediction model uses a convolutional neural network to extract features from MRI sample data.

[0045] As an example, basic features of MRI data are extracted by passing MRI data input to a convolutional neural network through a 3×3×3 convolutional kernel and max pooling operation.

[0046] As an example, the first extracted feature is the basic feature of the MRI data.

[0047] Step C2: Determine the residual characteristics of the MRI data; As an example, the first extracted features are sequentially input into four residual connection structures. Each residual connection structure includes two convolution operations. After each convolution operation, the data is normalized to obtain the residual features of the MRI data.

[0048] As an example, convolution and normalization operations can increase the learning speed of the model, simplify the parameter tuning process, and make the learning process of the network more stable. The ReLU activation function is used between two convolution operations to increase the non-linearity of the network. Finally, the original input of the residual connection structure and the input after two convolution operations are added together to retain more information.

[0049] Step C3: Determine the image feature data based on the first extracted features and the residual features.

[0050] As an example, the first extracted features and the residual features are added together to obtain the image feature data, and the image feature data extraction is completed.

[0051] The step of extracting features from the SNP data to obtain gene feature data includes: Step D1: Based on the preset prediction model, perform low-dimensional transformation on the SNP data to obtain transformation features; As an example, the feature extraction process for SNP data is as follows: Figure 5 As shown, the feature extraction module for SNP data consists of two Transformer modules, two linear (linear layer) modules, and one conv (convolutional) module. After the SNP data is input into the Transformer-based network, the input SNP data is first converted into one-dimensional data of length n×4. After passing through a linear layer with the same number of input and output channels, it is restored to two-dimensional data of n×4 to enhance the generalization ability of the network.

[0052] As an example, transformation features are the Query matrix, Key matrix, and Value matrix derived from low-dimensional transformation of SNP data.

[0053] As an example, a schematic diagram of the Transformer network structure is shown below. Figure 6As shown, a one-dimensional convolutional kernel of size 3 is used to convert the data to an n×32 size, which is then input into two consecutive Transformer structures. In each Transformer structure, the linear layer first converts the input into three lower-dimensional matrices: Query, Key, and Value. The Query and Key matrices are then multiplied and converted into an attention value using Softmax. This attention value is then multiplied by the Value matrix, and this process is repeated. The resulting values ​​are concatenated and restored to the same dimension as the original input data using a linear layer. This concatenated value is then added to the original input data, allowing the network to learn different patterns while retaining the original information. Layer-wise normalization is used to ensure the stability of the data feature distribution. Finally, the network converts the extracted features to a 1×512 size using a linear layer. Step D2: Determine the attention value of the transformation feature, and multiply the attention value by the transformation feature to obtain the second extracted feature; As an example, in the Transformer structure, SNP data is first transformed into three lower-dimensional matrices—Query, Key, and Value—through a linear layer. Then, the Query and Key matrices are multiplied and transformed into attention values ​​through the Softmax function. The attention values ​​are then multiplied with the Value matrix to obtain the second extracted feature.

[0054] Step D3: Based on the transformation features and the second extraction features, determine the gene feature data.

[0055] As an example, the transformed features are added to the second extracted features to obtain gene feature data.

[0056] In this embodiment, by fusing gene feature data and image feature data, the ability to extract information within and between MRI features and SNP features is enhanced. At the same time, the correlation between MRI features and SNP features is increased.

[0057] Furthermore, based on the first and second embodiments of this application, another embodiment of this application is provided. In this embodiment, before the step of inputting the MRI data and the SNP data into the preset prediction model, the following steps are included: Step E1: Preprocess the MRI data and the SNP data to obtain preprocessed MRI data and the SNP data; As an example, the acquired MRI and SNP data contain many features unrelated to AD, which require preprocessing to obtain AD-associated sample data from the patient's MRI and SNP data.

[0058] The step of inputting the MRI data and the SNP data into a preset prediction model, and performing prediction processing on the MRI data and the SNP data based on the preset prediction model to obtain the target prediction result includes: The preprocessed MRI data and SNP data are input into a preset prediction model. Based on the preset prediction model, the preprocessed MRI data and SNP data are subjected to prediction processing to obtain the target prediction result.

[0059] As an example, preprocessed MRI data and SNP data are input into a preset prediction model to perform prediction processing on the MRI data and SNP data.

[0060] The step of preprocessing the MRI data and the SNP data to obtain preprocessed MRI data and the SNP data includes: Step M1: Perform offset field correction and skull dissection on the MRI data to obtain corrected data, and register the corrected data to a preset standard template to obtain the preprocessed MRI data; As an example, during the acquisition of MRI data, there are brightness differences in MRI images, and the MRI data needs to be corrected to minimize errors.

[0061] As an example, the bias field causes inhomogeneity in the magnetic field of an MRI machine. If the bias field is not corrected, all imaging processing algorithms will output incorrect results. Therefore, MRI data needs to be preprocessed to correct the bias field before extracting features, thereby reducing misdiagnosis and improving the accuracy of disease diagnosis.

[0062] As an example, the bias field correction operation specifically corrects the bias field in an MRI image.

[0063] As an example, craniotomy specifically involves extracting brain-related tissue from images in MRI data and extracting brain tissue from inside the skull.

[0064] As an example, the corrected data specifically refers to MRI data after bias field correction and craniotomy.

[0065] As an example, the preset standard template is the Colin27 standard template. Since each subject's brain shape and size are different, the same brain structure may be in different positions in the same coordinate system. Therefore, it is necessary to transform it into the same standard space before inputting it into the network. The Colin27 standard template is a widely used standard template with high resolution. By registering different subjects with the Colin27 standard template, brains of different shapes can be transformed into the same standard space.

[0066] As an example, the image spatial resolution of all input networks is 1×1×1 mm3, and the size is 181×217×181. After reading the data, the excess edge parts are cropped, and the image size is 152×184×152.

[0067] Step M2: Fill in the SNP sites in the SNP data and extract the risk sites associated with the disease from the SNP sites to obtain the preprocessed SNP data.

[0068] As an example, SNP data is examined and quality control is performed using Plink software. SNP sites with high deletion rates and low minimum allele frequencies are deleted. The deletion rate of the deleted SNP sites can be greater than 5%, and the minimum allele frequency can be less than 0.05.

[0069] As an example, the remaining SNP loci in the SNP data are filled with genotypes using the Sanger Imputation Server platform. After filling, the SNP data is used to remove SNP loci with poor filling quality by the INFO index. Finally, disease-related risk loci are selected as needed or by using the GWAS method.

[0070] As an example, risk sites are used as inputs to the pre-defined prediction model's SNP data.

[0071] In this embodiment, by preprocessing the acquired SNP data and MRI data, disease-related features are extracted before the data is input into the model, thereby reducing the error of the obtained target prediction results.

[0072] Reference Figure 3 , Figure 3 This is a schematic diagram of the device structure of the hardware operating environment involved in the embodiments of this application.

[0073] like Figure 3 As shown, the gene imaging-based prediction device may include: a processor 1001, a memory 1005, and a communication bus 1002. The communication bus 1002 is used to realize the connection and communication between the processor 1001 and the memory 1005.

[0074] Optionally, the genomic imaging-based predictive device may also include a user interface, a network interface, a camera, RF (Radio Frequency) circuitry, sensors, a WiFi module, etc. The user interface may include a display screen and an input submodule such as a keyboard; optional user interfaces may also include standard wired or wireless interfaces. The network interface may include standard wired or wireless interfaces (such as a Wi-Fi interface).

[0075] Those skilled in the art will understand that Figure 3 The structure of the genomic imaging-based prediction device shown does not constitute a limitation on the genomic imaging-based prediction device, and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0076] like Figure 3 As shown, the memory 1005, serving as a storage medium, may include an operating system, a network communication module, and a genomic imaging-based prediction program. The operating system is a program that manages and controls the hardware and software resources of the genomic imaging-based prediction device, supporting the operation of the genomic imaging-based prediction program and other software and / or programs. The network communication module is used to enable communication between the various components within the memory 1005, as well as communication with other hardware and software in the genomic imaging-based prediction system.

[0077] exist Figure 3 In the genomic imaging-based prediction device shown, the processor 1001 is used to execute the genomic imaging-based prediction program stored in the memory 1005 to implement the steps of the genomic imaging-based prediction method described above.

[0078] The specific implementation of the prediction device based on genomic imaging in this application is basically the same as the various embodiments of the prediction method based on genomic imaging described above, and will not be repeated here.

[0079] This application also provides a prediction device based on genomic imaging, the prediction device based on genomic imaging includes: The acquisition module is used to acquire the MRI and SNP data of the subjects. The processing module is used to input the MRI data and the SNP data into a preset prediction model, and perform prediction processing on the MRI data and the SNP data based on the preset prediction model to obtain the target prediction result; In one possible implementation of this application, the processing module includes: The extraction unit is used to extract features from the MRI data based on the preset prediction model to obtain image feature data, and to extract features from the SNP data to obtain gene feature data. The conversion unit is used to perform low-dimensional conversion processing on the image feature data and the gene feature data respectively to obtain image modality data and gene modality data respectively; The fusion unit is used to fuse the image modal data and gene modal data based on the cross-network structure of the preset prediction model to obtain target fused data, wherein different modal data are fused in each substructure of the cross-network structure; A prediction unit is used to perform predictive processing on the MRI data and the SNP data based on the target fusion data.

[0080] In one possible implementation of this application, the first fusion unit includes: The first fusion subunit is used to fuse the image modality data and gene modality data based on the cross-network structure of the preset prediction model to obtain multiple hybrid features; The first extraction subunit is used to extract key features from the mixed features and obtain target fused data based on the key features.

[0081] In one possible embodiment of this application, the extraction unit includes: The first computational subunit is used to perform convolution and pooling operations on the MRI data based on the preset prediction model to obtain the first extracted features; The first determining subunit is used to determine the residual characteristics of the MRI data; The second determining subunit is used to determine the image feature data based on the first extracted features and the residual features.

[0082] In one possible embodiment of this application, the extraction unit further includes: The first transformation subunit is used to perform low-dimensional transformation on the SNP data based on the preset prediction model to obtain transformation features; The second calculation subunit is used to determine the attention value of the transformation feature and multiply the attention value by the transformation feature to obtain the second extracted feature; The third determining subunit is used to determine the gene feature data based on the transformation features and the second extraction features.

[0083] In one possible embodiment of this application, the apparatus further includes: The preprocessing module is used to preprocess the MRI data and the SNP data to obtain preprocessed MRI data and SNP data.

[0084] In one possible embodiment of this application, the processing module further includes: The processing unit is used to input the preprocessed MRI data and SNP data into a preset prediction model, and perform prediction processing on the preprocessed MRI data and SNP data based on the preset prediction model to obtain the target prediction result.

[0085] In one possible implementation of this application, the preprocessing module includes: The registration unit is used to perform offset field correction and skull dissection on the MRI data to obtain corrected data, and register the corrected data to a preset standard template to obtain the preprocessed MRI data. The extraction unit fills in the SNP sites in the SNP data and extracts the risk sites associated with the disease from the SNP sites to obtain the preprocessed SNP data.

[0086] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0087] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0088] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0089] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A prediction method based on genomic imaging, characterized in that, The method includes the following steps: Acquire MRI and SNP data from the subjects; The MRI data and the SNP data are input into a preset prediction model. Based on the preset prediction model, the MRI data and the SNP data are processed for prediction to obtain the target prediction result. The preset prediction model includes a cross-network structure. Each substructure in the cross-network structure integrates modal data corresponding to the MRI data and the SNP data, and each substructure integrates different modal data. The modal data corresponding to the MRI data and the SNP data are obtained by performing corresponding low-dimensional transformations on the MRI data and the SNP data respectively. The cross-network structure includes a two-layer Cross-Transformer structure and a two-layer Self-Transformer structure. The first substructure in the two-layer Cross-Transformer structure integrates the Query matrix of the MRI modal data with the Key matrix and Value matrix of the SNP modal data. The second substructure in the two-layer Cross-Transformer structure integrates the Query matrix of the SNP modal data with the Key matrix and Value matrix of the MRI modal data. The outputs of the first substructure and the second substructure are integrated after key features are extracted by the corresponding Self-Transformer structure. The step of performing predictive processing on the MRI data and the SNP data based on the preset prediction model includes: Based on the preset prediction model, features of the MRI data are extracted to obtain image feature data, and features of the SNP data are extracted to obtain gene feature data. The image feature data and the gene feature data are respectively subjected to low-dimensional transformation processing to obtain the key matrix, value matrix and query matrix corresponding to the image modality data, and the key matrix, value matrix and query matrix corresponding to the gene modality data. Based on the cross-network structure of the preset prediction model, the key matrix and value matrix corresponding to the image modality data and the query matrix corresponding to the gene modality data are fused, and the key matrix and value matrix corresponding to the gene modality data and the query matrix corresponding to the image modality data are fused to obtain the target fused data; Based on the target fusion data, predictive processing is performed on the MRI data and the SNP data; The step of extracting features from the MRI data based on the preset prediction model to obtain image feature data includes: Based on the preset prediction model, convolution and pooling operations are performed on the MRI data to obtain the first extracted features; Determine the residual characteristics of the MRI data; The image feature data is determined based on the first extracted features and the residual features.

2. The prediction method based on genomic imaging as described in claim 1, characterized in that, The step of fusing the image modality data and gene modality data to obtain the target fused data based on the cross-network structure of the preset prediction model includes: Based on the cross-network structure of the preset prediction model, the image modality data and gene modality data are fused to obtain multiple hybrid features; Key features are extracted from the mixed features, and target fused data is obtained based on the key features.

3. The prediction method based on genomic imaging as described in claim 1, characterized in that, The step of extracting features from the SNP data to obtain gene feature data includes: Based on the preset prediction model, the SNP data is transformed in a low dimension to obtain the transformation features; Determine the attention value of the transformation feature, and multiply the attention value by the transformation feature to obtain the second extracted feature; Based on the transformation features and the second extraction features, the gene feature data is determined.

4. The prediction method based on genomic imaging as described in claim 1, characterized in that, Before the step of inputting the MRI data and the SNP data into the preset prediction model, the following steps are included: The MRI data and the SNP data are preprocessed to obtain preprocessed MRI data and the SNP data; The step of inputting the MRI data and the SNP data into a preset prediction model, and performing prediction processing on the MRI data and the SNP data based on the preset prediction model to obtain the target prediction result includes: The preprocessed MRI data and SNP data are input into a preset prediction model. Based on the preset prediction model, the preprocessed MRI data and SNP data are subjected to prediction processing to obtain the target prediction result.

5. The prediction method based on genomic imaging as described in claim 1, characterized in that, The step of preprocessing the MRI data and the SNP data to obtain preprocessed MRI data and the SNP data includes: The MRI data is subjected to bias field correction and skull dissection to obtain corrected data, and the corrected data is registered to a preset standard template to obtain the preprocessed MRI data. The SNP sites in the SNP data are filled in, and the risk sites associated with the disease are extracted from the SNP sites to obtain the preprocessed SNP data.

6. A prediction device based on genomic imaging, characterized in that, The genomic imaging-based prediction device includes: The acquisition module is used to acquire the MRI and SNP data of the subjects. The processing module is used to input the MRI data and the SNP data into a preset prediction model, and perform prediction processing on the MRI data and the SNP data based on the preset prediction model to obtain the target prediction result; The preset prediction model includes a cross-network structure. Each substructure in the cross-network structure integrates modal data corresponding to the MRI data and the SNP data, and each substructure integrates different modal data. The modal data corresponding to the MRI data and the SNP data are obtained by performing corresponding low-dimensional transformations on the MRI data and the SNP data respectively. The cross-network structure includes a two-layer Cross-Transformer structure and a two-layer Self-Transformer structure. The first substructure in the two-layer Cross-Transformer structure integrates the Query matrix of the MRI modal data with the Key matrix and Value matrix of the SNP modal data. The second substructure in the two-layer Cross-Transformer structure integrates the Query matrix of the SNP modal data with the Key matrix and Value matrix of the MRI modal data. The outputs of the first substructure and the second substructure are integrated after key features are extracted by the corresponding Self-Transformer structure. The processing module includes: The extraction unit is used to extract features from the MRI data based on the preset prediction model to obtain image feature data, and to extract features from the SNP data to obtain gene feature data. The transformation unit is used to perform low-dimensional transformation processing on the image feature data and the gene feature data respectively, to obtain the key matrix, value matrix and query matrix corresponding to the image modality data, and the key matrix, value matrix and query matrix corresponding to the gene modality data respectively; The fusion unit is used to fuse the key matrix and value matrix corresponding to the image modality data and the query matrix corresponding to the gene modality data based on the cross-network structure of the preset prediction model, and the key matrix and value matrix corresponding to the gene modality data and the query matrix corresponding to the image modality data to obtain the target fused data. In this case, different modality data are fused in each substructure of the cross-network structure. A prediction unit is used to perform predictive processing on the MRI data and the SNP data based on the target fusion data; The extraction unit includes: The first computational subunit is used to perform convolution and pooling operations on the MRI data based on the preset prediction model to obtain the first extracted features; The first determining subunit is used to determine the residual characteristics of the MRI data; The second determining subunit is used to determine the image feature data based on the first extracted features and the residual features.

7. A predictive device based on genomic imaging, characterized in that, The device includes: a memory, a processor, and a genomic imaging-based prediction program stored in the memory and executable on the processor, the genomic imaging-based prediction program being configured to implement the steps of the genomic imaging-based prediction method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a genomic image group-based prediction program, which, when executed by a processor, implements the steps of the genomic image group-based prediction method as described in any one of claims 1 to 5.

Images