Lung cancer multi-modal data diagnosis system based on Dirichlet nonparametric hybrid model

Abstract

The invention relates to a lung cancer multi-modal data diagnosis system based on a Dirichlet nonparametric hybrid model, and belongs to the technical field of data processing. Comprising a gene data processing module used for carrying out standardization and differential expression gene screening operation on input gene data, and carrying out weighting and associated co-expression relationship analysis on screened differential expression genes to obtain gene features; the pathological image data processing module is used for extracting various image features of the CT image and performing correlation analysis and comparison on the image features to obtain the image features; the feature selection module is used for carrying out self-adaptive weighting on the gene features and the image features according to the incidence relation so as to endow different weights, and carrying out feature screening according to the different weights; and the precise diagnosis module is used for enabling the screened features to pass through a classifier to obtain a prediction result of the lung cancer subtype. The system provided by the invention realizes establishment of a potential relation model between lung cancer image features and pathogenic genes and realizes precise diagnosis of lung cancer subtypes.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to a lung cancer multimodal data diagnosis system based on a Dirichlet nonparametric mixed model. Background Art

[0002] Malignant tumors such as lung cancer are common and frequently occurring diseases that threaten human health. To achieve accurate diagnosis and treatment of patients with advanced malignant tumors, chemotherapy, targeted therapy, immunotherapy and other treatment methods need to be adopted based on data such as medical imaging, histopathological images and gene sequences. At present, there are still many problems to be solved in reducing patient harm and achieving accurate diagnosis and treatment, such as small sample size of patient data, missing data modalities, unclear relationship between modalities, which makes it impossible to accurately and quickly find the optimal treatment methods for different populations, imperfect primary medical equipment, and inability to build accurate and rapid diagnosis and prognosis prediction models. It has become extremely important and urgent to reduce patient harm, improve the level of accurate diagnosis, and strengthen the synergy of multimodal data.

[0003] Imaging genomics aims to establish the relationship between the imaging features of the disease and the genomic-related features. Clarifying the correlation of imaging genomics has positive significance for new therapeutic targets and understanding of tumor biological mechanisms. How to use deep learning methods to discover the potential association between imaging features and mutant genes has outstanding significance for tumor efficacy and prognosis evaluation.

[0004] In recent years, there have been certain studies in theory and models to explore the associations between various modal data (text, audio, images, videos, etc.). Different deep learning models have promoted multimodal medical diagnosis to a certain extent, but the unexplainability of deep learning networks still needs to be explored for accurate diagnosis. The cross-modal adversarial mechanism uses the unique adversarial training process of GAN to make the results more discriminative. Domestic and foreign researchers have conducted a series of studies on cross-modal data modeling for medical imaging genomics. However, the effectiveness and accuracy of most existing methods need to be improved. In order to meet the needs of the diversity of medical imaging data categories and the dynamic growth of quantity, many researchers have conducted research in their respective fields and proposed corresponding solutions.

[0005] In summary, many researchers have proposed a series of methods in non-medical and medical fields limited to non-lung cancer diseases to solve the modeling problem of cross-modal data, which provides a reference for the later study of imaging genomics in the middle and late stages of lung cancer. However, how to build an interpretable intelligent network model to mine the mapping relationship between image data and genetic information and process the dynamic nature of input data poses a challenge to the selection and construction of the network model. Therefore, how to solve the above problems is the difficulty and focus of the present invention. Therefore, the present invention intends to use lung cancer CT image feature data and related pathogenic gene data to explore a model that can not only achieve unsupervised feature clustering, but also reveal the multiple correlations of imaging genomics and multimodal precise diagnosis. Summary of the invention

[0006] In order to solve the above technical problems, the present invention provides a lung cancer multimodal data diagnosis system based on the Dirichlet nonparametric mixture model. The technical solution of the present invention is as follows:

[0007] A lung cancer multimodal data diagnosis system based on Dirichlet nonparametric mixture model, comprising a gene data processing module, a pathological image data processing module, a feature selection module and an accurate diagnosis module;

[0008] The gene data processing module is used to standardize the input gene data, perform a screening operation on the standardized gene data for differentially expressed genes, and perform a weighted and associated co-expression relationship analysis on the screened differentially expressed genes to obtain gene features;

[0009] The pathological image data processing module is used to extract various types of image features of CT images, and to perform correlation analysis and comparison on the extracted image features to obtain image features;

[0010] The feature selection module is used to adaptively weight the gene features and the image features according to the association relationship to assign different weights, and perform feature screening according to the different weights;

[0011] The precise diagnosis module is used to classify the screened features through a classifier to obtain a prediction result of lung cancer subtypes.

[0012] Optionally, when the gene data processing module performs a screening operation for differentially expressed genes on the standardized gene data, the following steps are performed:

[0013] S11, the matrix of the standardized gene data of each patient sample after log2 calculation is used as the expression matrix of each patient sample;

[0014] S12, FDR correction: Assume that the observed expression number corresponding to gene A is x. It is known that in all patient samples, the expression of each gene accounts for only a small part of the expression of all genes. In this case, p(x) represents the distribution of gene expression in patient samples, and its distribution follows a Poisson distribution. It is known that the total expression number uniquely matched to its genome in patient sample 1 is N1, the total expression number uniquely matched to its genome in patient sample 2 is N2, the total expression number uniquely matched to gene A in patient sample 1 is x, and the total expression number uniquely matched to gene A in patient sample 2 is y. Then the probability that the expression of gene A in the two patient samples is equal is calculated by the following formula (1-1):

[0015]

[0016] S13, after finding the largest positive integer i such that p(i) <= (i*q) / m, select genes corresponding to p(1), p(2), ..., p(i) as differentially expressed genes, where m represents the ranking of the p value of this test among all the test times, and q represents the test times;

[0017] S14, sort the p values ​​of all candidate genes according to the following formula (1-2), and determine the differentially expressed genes according to the sorting results:

[0018]

[0019] Among them, length(p) represents the length value of p, and rank(p) represents the current ranking value of p.

[0020] Optionally, when the gene data processing module performs weighted and associated co-expression relationship analysis on the screened differentially expressed genes to obtain gene features, it includes:

[0021] S1-1, calculation of the Pearson correlation coefficient between every two differentially expressed genes;

[0022] S1-2, soft threshold selection: the horizontal axis is set as the soft threshold according to the Pearson correlation coefficient between every two differentially expressed genes, the vertical axis is the evaluation parameter of the scale-free network, and the value of the soft threshold is determined to be 4;

[0023] S1-3, differentially expressed genes were classified according to their expression patterns by soft thresholding and Pearson correlation coefficient, and genes with similar expression patterns were grouped into one gene module;

[0024] S1-4, the genes with the highest Pearson correlation coefficients in each gene module were identified as core genes;

[0025] S1-5, the expression level of core genes is used as gene features.

[0026] Optionally, the image features of the CT image extracted by the pathological image data processing module include texture features and morphological features, the texture features include angular second-order moment, entropy, inverse moment, contrast, accelerated robust features, local binary pattern features and gray-level co-occurrence matrix features, and the morphological features include the number of nodules, nodule size, nodule circumference, nodule major axis, nodule minor axis, nodule roundness, nodule rectangularity, nodule compactness and chord distribution shape descriptors.

[0027] Optionally, when the feature selection module adaptively weights the gene features and the image features according to the association relationship to assign different weights, it is assumed that X (m) is an n×u m , where n is the number of patient samples, u m is a feature in M ​​modality, M∈{Image,Gene}, Image represents image modality, Gene represents gene modality, P m are the features of M modes, p1=100 means that the first mode has 100 features. The feature selection module includes:

[0028] S31, reflects the mutual relationship between gene expressions through WGCNA, and uses a graph to represent the mutual relationship between gene expressions. The nodes in the graph represent features, and the edges represent the relationship between them. The graph G(m) is used to represent the structural information in the modality, and the graph G(Image, Gene) is used to represent the relationship between the image modality and the gene modality.

[0029] S32, set virtual indicators To represent the selection state of feature j under mode m, it is divided into a set that can provide information and a set that cannot provide information. The parameter set of the set that can provide information is Where i∈{1,2,...,n} represents the patient sample, 1 represents that the feature is contributing and is selected, that is, it is a feature that can provide information, and the non-information set and parameter set Related, 0 means not selected;

[0030] S33, joint analysis is performed using the Bayesian nonparametric model, as shown in formula (6-2):

[0031]

[0032] Where m∈{1,2}, 1 represents the image modality, 2 represents the gene modality; F(·) is the patient sample x under modality M. i The conditional distribution of is expressed as the product of the distribution functions of each form, and F(M) varies with the data generating process;

[0033] S34, calculates each observation that is produced by a mixture of two components, as shown in formula (6-3):

[0034]

[0035] The joint distribution of unknown parameters obeys the function G, which is a nonparametric Dirichlet distribution with a basic measure of G0 and a concentration parameter of α0. G0 provides a priori support for the center of the patient sample, and α0 controls the strength of the concentration.

[0036] S35, rewrite formula (6-3) into a representation based on adaptive weighting, such as formula (6-4):

[0037] β k |α0,G0~Beta(1,α0)

[0038] φ k |α0,G0~G0 (6-4);

[0039] Among them, β k represents the continuous probability distribution of k, φ k represents the kth patient sample from the prior, and Beta(1,α0) is the beta distribution.

[0040] Optionally, when the feature selection module performs feature screening according to different weights, it includes:

[0041] S3-1, define random distribution as formula (6-5):

[0042]

[0043] in, is φ k is the Dirichlet function of the mass center, G is an infinite discrete distribution that can be realized with weight Next Randomly select from In this way, a discrete situation about the patient sample is obtained;

[0044] S3-2, assuming that the map of image features is ζ(1), the map of gene features is ζ(2), and the biological maps in the two forms are ζ(1,2), the image features and gene features are integrated into the joint clustering framework. The specific integration method is shown in formulas (6-6) and (6-7):

[0045]

[0046]

[0047] Among them, j~ζk indicates that features j and k are adjacent in the graph ζ, γ1 and γ2 are used to balance the correlation coefficients between different modalities and adjust the sparsity of image features and gene features, e1, e2, e' are used to control the smoothness of each modality network and the smoothness between them, and P is the Potts model, which is applicable to different modality components and is imposed with a priori indicators

[0048] Optionally, the precise diagnosis module is used to classify the screened features through a classifier to obtain a prediction result of lung cancer subtype, including:

[0049] S4-1, when training the classifier, images of different modalities are input simultaneously, and training samples of different scales are as shown in formula (5-1):

[0050]

[0051] Among them, X i is the selected feature, Y i is the label, N represents the number of training samples;

[0052] S4-2, the goal of classifier training is to learn the parameters W of the classifier. The loss function of the classifier is defined as formula (5-2):

[0053]

[0054] In formula (5-2), M is the number of outputs, M = 2, S m represents the patient sample with modality M, α m is the loss weight, training sample S m Only contributes to the loss of mode M, for α m :

[0055]

[0056] Among them, K represents the Gaussian kernel function, L m (t-1) represents the t-1th loss under the M mode;

[0057] S4-3, the loss calculation for each modality classification branch in the classifier is as shown in formulas (5-4) and (5-5):

[0058] L(X,Y|W)=L cls (p(X),y) (5-4);

[0059]

[0060] Among them, L cls (p(X),y)=-logp y(X) is the cross entropy loss. After training, the loss function is minimized and the predicted lung cancer subtype is output as Y. i Table Confidence label of sample i.

[0061] All the above optional technical solutions can be combined arbitrarily, and the present invention does not provide detailed descriptions of the structures after the combinations.

[0062] By means of the above scheme, the beneficial effects of the present invention are as follows:

[0063] The present invention proposes a lung cancer multimodal data diagnosis system based on the Dirichlet nonparametric mixed model. This system realizes the establishment of a potential relationship model between lung cancer image features and pathogenic genes and realizes accurate diagnosis of lung cancer subtypes, which can well cope with the current situation of insufficient multimodal data and unclear correlation relationships. The model is improved based on the Dirichlet nonparametric mixed model, and the model is trained using CT images and genetic data. The trained model is compatible with most of the current medical image acquisition equipment and can be flexibly embedded in various auxiliary diagnosis systems to assist physicians in accurate and rapid diagnosis of lung cancer subtypes. The results of the present invention will be applied to multiple fields. Hospitals do not need to invest a lot of money in purchasing diagnostic equipment. Existing medical equipment can complete early screening and accurate diagnosis of patients with low cost expenditure.

[0064] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention and implement it according to the contents of the specification, the following is a detailed description of the preferred embodiments of the present invention in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 The figure is a schematic diagram of the overall process of the multimodal data diagnosis system for lung cancer based on the Dirichlet nonparametric mixture model of the present invention.

[0066] Figure 2 The present invention is a flow chart of a gene data processing module in a lung cancer multimodal data diagnosis system based on the Dirichlet nonparametric mixed model.

[0067] Figure 3 The present invention is a flow chart of a CT image data processing module in a multimodal data diagnosis system for lung cancer based on the Dirichlet nonparametric mixture model.

[0068] Figure 4 Schematic diagram of feature screening and clustering results in the multimodal data diagnosis system for lung cancer based on the Dirichlet nonparametric mixed model of the present invention.

[0069] Figure 5 This is a visualization diagram of different clustering methods in the lung cancer multimodal data diagnosis system based on the Dirichlet nonparametric mixture model of the present invention.

[0070] Figure 6 This is a comparison chart of the accuracy of different methods in the lung cancer multimodal data diagnosis system based on the Dirichlet nonparametric mixture model of the present invention.

[0071] Figure 7 This is a sensitivity comparison chart of different methods in the lung cancer multimodal data diagnosis system based on the Dirichlet nonparametric mixture model of the present invention.

[0072] Figure 8 It is a specificity comparison diagram of different methods in the lung cancer multimodal data diagnosis system based on the Dirichlet nonparametric mixed model of the present invention.

[0073] Fig. 9 It is a schematic diagram of Pearson correlation coefficient association analysis in the lung cancer multimodal data diagnosis system based on the Dirichlet nonparametric mixed model of the present invention.

[0074] Fig.10 It is a visualization diagram of multimodal data association analysis in the lung cancer multimodal data diagnosis system based on the Dirichlet nonparametric mixed model of the present invention.

[0075] Fig.11 It is a ROC curve diagram of the lung cancer multimodal data diagnosis system based on the Dirichlet nonparametric mixed model of the present invention and four other methods. DETAILED DESCRIPTION

[0076] The specific implementation of the present invention is further described in detail below in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0077] like Figures 1 to 3 As shown, the lung cancer multimodal data diagnosis system based on the Dirichlet non-parametric mixed model provided by the present invention includes a gene data processing module, a pathological image data processing module, a feature selection module and an accurate diagnosis module;

[0078] The gene data processing module is used to standardize the input gene data, perform a screening operation on the standardized gene data for differentially expressed genes, and perform a weighted and associated co-expression relationship analysis on the screened differentially expressed genes to obtain gene features;

[0079] The pathological image data processing module is used to extract various types of image features of CT images, and to perform correlation analysis and comparison on the extracted image features to obtain image features;

[0080] The feature selection module is used to adaptively weight the gene features and the image features according to the association relationship to assign different weights, and perform feature screening according to the different weights;

[0081] The precise diagnosis module is used to classify the screened features through a classifier to obtain a prediction result of lung cancer subtypes.

[0082] Among them, screening of differentially expressed genes is to study the pathogenesis of the disease and conduct early diagnosis and treatment of the disease by comparing the differences in gene transcription and expression between normal and disease states.

[0083] In a specific embodiment, when the gene data processing module performs a screening operation for differentially expressed genes on the standardized gene data, the following steps are performed:

[0084] S11, the matrix obtained by log2 calculation of the standardized gene data of each patient sample is used as the expression matrix of each patient sample.

[0085] For the gene data of lung cancer patients, data standardization and data analysis are first performed. Before correction, the embodiment of the present invention first makes a frequency histogram of p-value distribution. Many corrections of p-values ​​ignore this step. Figure 2 The situation shown in the normalization of genetic data in Part A is a very ideal situation, in which most of the distribution is flat, which indicates that the null hypothesis p value is uniformly distributed between 0 and 1. According to the definition of the null hypothesis, the p value has a 5% chance of being less than 0.05, a 10% chance of being less than 0.1, and so on, so the p value null hypothesis should be uniformly distributed between 0 and 1. At the same time, a peak appears where the p value is close to 0, indicating that this peak is caused by the alternative hypothesis and a part of possible positive errors.

[0086] The method of differential analysis uses fold change (FC), which represents the fold difference of the expression of the quality inspection of the patient sample. The logarithm of the fold difference is logFC, and the positive or negative value is determined according to the control group, that is, the fold of the experimental group / control group, which indicates whether there is a difference between the two groups. Generally, when Fold change>2 is used, it is considered that there is a difference between the two groups. Usually, when limma processing in R language is used, the matrix after log2 is required as the expression matrix input. If log2FC=2 is set, the change is 4 times. Since the error control method (False Discovery Rate, FDR) correction can limit the number of false positives and reduce true positives at the same time, the embodiment of the present invention adds FDR correction after the test. FDR correction refers to the expected value of the ratio of the number of falsely rejected null hypotheses (i.e., false positives) to the total number of rejected hypotheses (i.e., the number of all results determined to be significant) in multiple hypothesis tests.

[0087] S12, FDR correction: Assume that the observed expression number corresponding to gene A is x. It is known that in all patient samples, the expression of each gene accounts for only a small part of the expression of all genes. In this case, p(x) represents the distribution of gene expression in patient samples, and its distribution follows a Poisson distribution. It is known that the total expression number uniquely matched to its genome in patient sample 1 is N1, the total expression number uniquely matched to its genome in patient sample 2 is N2, the total expression number uniquely matched to gene A in patient sample 1 is x, and the total expression number uniquely matched to gene A in patient sample 2 is y. Then the probability that the expression of gene A in the two patient samples is equal is calculated by the following formula (1-1):

[0088]

[0089] S13, after finding the largest positive integer i such that p(i) <= (i*q) / m, select the genes corresponding to p(1), p(2), ..., p(i) as differentially expressed genes, where m represents the ranking of the p value of this test among all the tests, and q represents the number of tests.

[0090] The q value is the p value corresponding to the FDR of each test, which can be understood as the maximum expected proportion of a particular finding being incorrectly identified as significant in a given hypothesis test. If you want to control the FDR not to exceed q, you only need to find the largest positive integer i such that p(i) <= (i*q) / m. Selecting genes corresponding to p(1), p(2), ..., p(i) as differentially expressed genes can statistically ensure that the FDR does not exceed q.

[0091] S14, sort the p values ​​of all candidate genes according to the following formula (1-2), and determine the differentially expressed genes according to the sorting results:

[0092]

[0093] Among them, length(p) represents the length value of p, rank(p) represents the ranking value of p, and q-value(i) represents the q value corresponding to the current maximum integer i.

[0094] Specifically, after FDR correction, differentially expressed genes can be obtained according to the size of the p value.

[0095] In a specific embodiment, when the gene data processing module performs weighted and associated co-expression relationship analysis on the screened differentially expressed genes to obtain gene features, it includes:

[0096] S1-1, calculation of the Pearson correlation coefficient between every two differentially expressed genes;

[0097] S1-2, soft threshold selection: the horizontal axis is set as the soft threshold according to the Pearson correlation coefficient between every two differentially expressed genes, the vertical axis is the evaluation parameter of the scale-free network, and the value of the soft threshold is determined to be 4;

[0098] S1-3, differentially expressed genes were classified according to their expression patterns by soft thresholding and Pearson correlation coefficient, and genes with similar expression patterns were grouped into one gene module;

[0099] S1-4, the genes with the highest Pearson correlation coefficients in each gene module were identified as core genes;

[0100] S1-5, the expression level of core genes is used as gene features.

[0101] Weighted gene co-expression network analysis is an analytical method for analyzing gene expression patterns in multiple patient samples. It can cluster genes according to the similarity of expression and analyze the association between each clustered gene and a specific trait or phenotype. Compared with focusing only on differentially expressed genes, WGCNA uses information from thousands or nearly 10,000 genes with the largest changes or all genes to identify gene sets of interest and perform significant association analysis with phenotypes. This method not only makes full use of information, but also converts the association between thousands of genes and phenotypes into the association between multiple gene sets and phenotypes, eliminating the problem of multiple hypothesis testing and correction.

[0102] After screening the differentially expressed genes, the horizontal axis is set as the soft threshold Softthreshold (power) according to the obtained relationship between the genes, and the vertical axis is the evaluation parameter of the scale-free network. The higher the value, the more the network conforms to the scale-free feature (non-scale). Among them, scale-free-network represents the number of central nodes, which represents the connectivity of the entire graph. The best threshold is to maximize the vertical coordinate values ​​on both sides. The optimal threshold in the present invention selects power = 4, and then constructs the gene co-expression network through WGCNA.

[0103] Furthermore, in the diagnosis process, texture features are used by doctors to determine the degree of cell lesions. There are significant differences in the texture features of different subtypes of lesions, so this feature plays an important role in determining whether it is a discriminatory area. The texture features extracted by the embodiment of the present invention include angular second moment (ASM), entropy (ENT), inverse difference moment (IDM), contrast (CON), accelerated robust features, local binary pattern features, and grayscale co-occurrence matrix features. ASM represents texture uniformity. The smaller the ASM value, the more uniform the staining. When the image is full of fine textures, the entropy value is larger. IDM describes the local stationarity of the texture. The larger the value, the smaller the change in different areas of the image texture, that is, the local area is very uniform. CON reflects image clarity and texture depth.

[0104] The morphological difference degree feature has an important influence on the classification of CT images at the decision boundary. The embodiment of the present invention extracts multiple morphological features, including the number of nodules, nodule size, nodule perimeter, nodule major axis, nodule minor axis, nodule roundness, nodule rectangularity, nodule compactness, nodule area, and chord distribution shape descriptors. Since nodule perimeter and nodule area are often lost due to adhesion and overlap, nodule perimeter and nodule area are considered to meet the circularity standard.

[0105] Assuming there are n patient samples from different modalities, there are M data modalities (e.g., image modality, gene modality), then the entire observed data can be expressed as:

[0106]

[0107] X (m) is an n×u m , where n is the number of patient samples, u m is a feature in M ​​modality, M∈{Image,Gene}, Image represents image modality, Gene represents gene modality, P m are the features of M modes, p1=100 means that the first mode has 100 features. The feature selection module includes:

[0108] S31, reflects the mutual relationship between gene expressions through WGCNA, and uses a graph to represent the mutual relationship between gene expressions. The nodes in the graph represent features, and the edges represent the relationship between them. The graph G(m) is used to represent the structural information in the modality, and the graph G(Image, Gene) is used to represent the relationship between the image modality and the gene modality.

[0109] Specifically, the extracted features are interdependent between and within each data pattern. WGCNA can reflect the interrelationships between gene expressions.

[0110] S32, set virtual indicators To represent the selection state of feature j under mode m, it is divided into a set that can provide information and a set that cannot provide information. The parameter set of the set that can provide information is Where i∈{1,2,...,n} represents the patient sample, 1 represents that the feature is contributing and is selected, that is, it is a feature that can provide information, and the non-information set and parameter set 0 means not selected.

[0111] Specifically, to study the data structure of high-dimensional data and the similarity of patient samples, it is necessary to select features that can represent patient samples. In order to select identifiable features of each modality and achieve clustering at the same time, the embodiment of the present invention sets a virtual indicator

[0112] S33, joint analysis is performed using the Bayesian nonparametric model, as shown in formula (6-2):

[0113]

[0114] Where m∈{1,2}, 1 represents the image modality, 2 represents the gene modality; F(·) is the patient sample x under modality M. i The conditional distribution of is expressed as the product of the distribution functions of each form, and F(M) varies with the data generating process, such as normal distribution, Bernoulli distribution, Poisson distribution, etc. In other words, each observation is composed of different distributions, as shown in formula (6-2).

[0115] S34, calculates each observation that is produced by a mixture of two components, as shown in formula (6-3):

[0116]

[0117] The joint distribution of unknown parameters obeys the function G, which is a nonparametric Dirichlet Process (DP) distribution with a basic measure of G0 and a concentration parameter of α0. G0 provides a priori support for the center of the patient sample, and α0 controls the strength of the concentration.

[0118] S35, rewrite formula (6-3) into a representation based on adaptive weighting, such as formula (6-4):

[0119] β k |α0,G0~Beta(1,α0)

[0120] φ k |α0,G0~G0 (6-4);

[0121] Among them, β k represents the continuous probability distribution of k, φ k represents the kth patient sample from the prior, and Beta(1,α0) is the beta distribution. This step is used for posterior inference.

[0122] In the collected data set, as described above, the present embodiment integrates the image features and gene expression of each patient. Therefore, for the formula It can be expressed as:

[0123]

[0124] Based on the above formula, assuming The partial basis measure of the joint distribution is normal-inverse-gamma, which is similar to u ij The distribution is Beta(α u ,β u ), thereby inducing differentiation between gene clusters.

[0125] In a specific embodiment, when the feature selection module performs feature screening according to different weights, it includes:

[0126] S3-1, define random distribution as formula (6-5):

[0127]

[0128] in, is φ k is the Dirichlet function of the mass center, G is an infinite discrete distribution that can be realized with weight Next Randomly select from In this way, a discrete situation about the patient sample is obtained.

[0129] Although each time φ is extracted k are all different, but according to the nature of DP, the φ extracted in the previous k , π k Relatively large. Assuming that this is a standard normal distribution N(0, 1), the first 100 patient samples are extracted from the standard normal distribution. Then the data near 0 is likely to appear. Among these data, the earlier the value appears, the higher its weight may be. In this way, a discretized standard normal distribution can be obtained. Then the parameters of the distribution to which each data belongs are extracted from this discretized distribution. The result is similar to that extracted from the standard normal distribution, but the same parameters may be extracted from different data. In other words, extracting a patient sample from DP not only extracts a series of parameters (that is, distribution), but also extracts the weight of each parameter (distribution). The key features that define each class will be selected, and each feature will be assigned its importance, which is extremely important for subsequent feature classification.

[0130] S3-2, based on the extracted network graph related information, assuming that the map of image features is ζ(1), the map of gene features is ζ(2), and the biological maps in the two forms are ζ(1,2), the image features and gene features are integrated into the joint clustering framework. The specific integration method is shown in formulas (6-6) and (6-7):

[0131]

[0132]

[0133] Among them, j~ζk indicates that features j and k are adjacent in the graph ζ, γ1 and γ2 are used to balance the correlation coefficients between different modalities and adjust the sparsity of image features and gene features, e1, e2, e' are used to control the smoothness of each modality network and the smoothness between them, and P is the Potts model, which is applicable to different modality components and is imposed with a priori indicators

[0134] In existing clustering analysis, potential feature dependencies and cross-dependencies are often ignored, but they provide important biological guidance for defining patient subtypes. The embodiments of the present invention can enhance grouping and feature identification under the guidance of a specific correlation network by incorporating information into a joint clustering framework, which may provide more meaningful biological explanations compared to existing solutions.

[0135] In a specific embodiment, the precise diagnosis module is used to classify the screened features through a classifier to obtain a prediction result of lung cancer subtype, including:

[0136] S4-1, when training the classifier, images of different modalities are input simultaneously, and training samples of different scales are as shown in formula (5-1):

[0137]

[0138] Among them, X i is the selected feature, Y i is the label, N represents the number of training samples;

[0139] S4-2, the goal of classifier training is to learn the parameters W of the classifier. The loss function of the classifier is defined as formula (5-2):

[0140]

[0141] In formula (5-2), M is the number of outputs (M = 2), S m represents the patient sample with modality M, α m is the loss weight, training sample S m Only contributes to the loss of mode M, for α m :

[0142]

[0143] Among them, K represents the Gaussian kernel function, L m (t-1) represents the t-1th loss under the M mode;

[0144] S4-3, the loss calculation for each modality classification branch in the classifier is as shown in formulas (5-4) and (5-5):

[0145] L(X,Y|W)=L cls (p(X),y) (5-4);

[0146]

[0147] Among them, L cls (p(X),y)=-logp y (X) is the cross entropy loss. After training, the loss function is minimized and the predicted lung cancer subtype is output as Y. i Represents the confidence label of sample i.

[0148] For a multimodal feature classifier, if a proper balance cannot be found between different modalities, it is difficult to effectively train multiple modalities. The classifier described in the embodiment of the present invention takes into account the rate of change of the loss of each modality. The classifier calculates the average weight of the task by considering the rate of change of the loss of each modality on the basis of SVM.

[0149] Figure 4 The figure is a schematic diagram of the feature screening clustering results in the lung cancer multimodal data diagnosis system based on the Dirichlet nonparametric mixture model of the present invention, which shows the results of feature clustering using the proposed method. The cluster centers are selected as C=2, 3, 4, 5, and 6 respectively. The experimental results show that when C=5, the clustering effect of the feature is the most obvious. For the above simulated data, the number of potential clusters in the feature is estimated to be 5 through the Dirichlet process infinite mixture model clustering analysis.

[0150] Figure 5 The figure is a visualization diagram of different clustering methods in the lung cancer multimodal data diagnosis system based on the Dirichlet nonparametric mixed model of the present invention. The traditional clustering method does not take into account the correlation between features, and the clustering effect is not ideal. The clustering effect boundary of the method proposed in the embodiment of the present invention is relatively clear, and the features can be effectively clustered, thereby laying the foundation for removing redundant features later.

[0151] Figure 6 , Figure 7 and Figure 8 The figures are respectively the accuracy, sensitivity and specificity comparison of different methods in the lung cancer multimodal data diagnosis system based on the Dirichlet nonparametric mixed model in the embodiment of the present invention. The final result uses the SVM classifier to evaluate the classification results. The three evaluation indicators are accuracy (ACC), true positive rate (TPR) and false positive rate (FPR), which are used to measure the classification effectiveness of all compared methods. It is worth noting that the higher the ACC and TPR, the lower the FPR, the better the effect.

[0152] Fig. 9 In order to conduct correlation analysis on the selected images and gene features, the horizontal and vertical axes are used to display some representative genes and image features, and the Pearson correlation analysis method was used for analysis.

[0153] Fig.10 This is a visualization diagram of multimodal data association analysis in the multimodal data diagnosis system for lung cancer based on the Dirichlet nonparametric mixed model in the embodiment of the present invention. The horizontal and vertical axes are the names of the extracted differential genes and image features. Studies have found that certain radiological features are related to some gene mutations. These signs may have a high correlation with combined features such as texture features extracted by the computer. It can be inferred that pleural traction signs are related to gene mutations such as EGFR. After discussion and verification with clinicians, it was found that among NSCLC patients, 78% of patients with EGFR mutations had pleural traction on chest CT, while only 51% of patients with EGFR wild-type had pleural traction. Pleural traction is a common chest CT imaging manifestation of visceral pleura involvement in NSCLC, and is also one of the important factors for predicting the prognosis of NSCLC patients undergoing surgical resection. It can be seen from the visualization results that the system proposed in the embodiment of the present invention can provide a certain theoretical basis for the subsequent mining of the association between lung cancer image features and pathogenic genes.

[0154] Fig.11 The ROC curves of the lung cancer multimodal data diagnosis system based on the Dirichlet nonparametric mixed model in the embodiment of the present invention and four other methods are shown in FIG. As can be seen from the figure, under the premise of ensuring accuracy, the method proposed in the embodiment of the present invention can effectively improve the sensitivity and specificity of the model.

[0155] Tests have shown that the AUC of the diagnosis result of the system provided by the embodiment of the present invention is 95.8%, and the response time for completing a subtype accurate diagnosis task is less than or equal to 1 second.

[0156] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. It should be pointed out that a person skilled in the art can make several improvements and modifications without departing from the technical principles of the present invention, and these improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. A lung cancer multimodal data diagnosis system based on Dirichlet nonparametric mixture model, characterized in that: It includes gene data processing module, pathological image data processing module, feature selection module and precise diagnosis module; The gene data processing module is used to standardize the input gene data, perform a screening operation on the standardized gene data for differentially expressed genes, and perform a weighted and associated co-expression relationship analysis on the screened differentially expressed genes to obtain gene features; The pathological image data processing module is used to extract various types of image features of CT images, and to perform correlation analysis and comparison on the extracted image features to obtain image features; The feature selection module is used to adaptively weight the gene features and the image features according to the association relationship to assign different weights, and perform feature screening according to the different weights; The precise diagnosis module is used to classify the screened features through a classifier to obtain a prediction result of lung cancer subtypes.

2. The lung cancer multimodal data diagnosis system based on the Dirichlet nonparametric mixture model according to claim 1, characterized in that: When the gene data processing module performs a screening operation for differentially expressed genes on the standardized gene data, the gene data processing module includes: S11, using the matrix obtained by log2 calculation of the standardized gene data of each patient sample as the expression matrix of each patient sample; S12, FDR correction: Assume that the observed expression number corresponding to gene A is x. It is known that in all patient samples, the expression of each gene accounts for only a small part of the expression of all genes. In this case, p(x) represents the distribution of gene expression in patient samples, and its distribution follows a Poisson distribution. It is known that the total expression number uniquely matched to its genome in patient sample 1 is N1, the total expression number uniquely matched to its genome in patient sample 2 is N2, the total expression number uniquely matched to gene A in patient sample 1 is x, and the total expression number uniquely matched to gene A in patient sample 2 is y. Then the probability that the expression of gene A in the two patient samples is equal is calculated by the following formula (1-1): S13, after finding the largest positive integer i such that p(i) <= (i*q) / m, select genes corresponding to p(1), p(2), ..., p(i) as differentially expressed genes, where m represents the ranking of the p value of this test among all the test times, and q represents the test times; S14, sort the p values ​​of all candidate genes according to the following formula (1-2), and determine the differentially expressed genes according to the sorting results: Among them, length(p) represents the length value of p, and rank(p) represents the current ranking value of p.

3. The lung cancer multimodal data diagnosis system based on the Dirichlet nonparametric mixture model according to claim 1 or 2, characterized in that: The gene data processing module performs weighted and associated co-expression relationship analysis on the screened differentially expressed genes to obtain gene features, including: S1-1, calculation of the Pearson correlation coefficient between every two differentially expressed genes; S1-2, soft threshold selection: the horizontal axis is set as the soft threshold according to the Pearson correlation coefficient between every two differentially expressed genes, the vertical axis is the evaluation parameter of the scale-free network, and the value of the soft threshold is determined to be 4; S1-3, differentially expressed genes were classified according to their expression patterns by soft thresholding and Pearson correlation coefficient, and genes with similar expression patterns were grouped into one gene module; S1-4, the genes with the highest Pearson correlation coefficients in each gene module were identified as core genes; S1-5, the expression level of core genes is used as gene features.

4. The lung cancer multimodal data diagnosis system based on the Dirichlet nonparametric mixture model according to claim 1, characterized in that: The image features of the CT image extracted by the pathological image data processing module include texture features and morphological features. The texture features include angular second-order moment, entropy, inverse difference moment, contrast, accelerated robust features, local binary pattern features and gray-level co-occurrence matrix features. The morphological features include the number of nodules, nodule size, nodule circumference, nodule major axis, nodule minor axis, nodule roundness, nodule rectangularity, nodule compactness and chord distribution shape descriptors.

5. The lung cancer multimodal data diagnosis system based on the Dirichlet nonparametric mixture model according to claim 1, characterized in that: When the feature selection module adaptively weights the gene features and the image features according to the association relationship to assign different weights, it is assumed that X (m) is an n×u m , where n is the number of patient samples, u m is a feature in M ​​modality, M∈{Image,Gene}, Image represents image modality, Gene represents gene modality, P m are the features of M modes, p1=100 means that the first mode has 100 features. The feature selection module includes: S31, reflects the mutual relationship between gene expressions through WGCNA, and uses a graph to represent the mutual relationship between gene expressions. The nodes in the graph represent features, and the edges represent the relationship between them. The graph G(m) is used to represent the structural information in the modality, and the graph G(Image, Gene) is used to represent the relationship between the image modality and the gene modality. S32, set virtual indicators To represent the selection state of feature j under mode m, it is divided into a set that can provide information and a set that cannot provide information. The parameter set of the set that can provide information is Where i∈{1,2,...,n} represents the patient sample, 1 represents that the feature is contributing and is selected, that is, it is a feature that can provide information, and the non-information set and parameter set Related, 0 means not selected; S33, joint analysis is performed using the Bayesian nonparametric model, as shown in formula (6-2): Where m∈{1,2}, 1 represents the image modality, 2 represents the gene modality; F(·) is the patient sample x under modality M. i The conditional distribution of is expressed as the product of the distribution functions of each form, and F(M) varies with the data generating process; S34, calculates each observation that is produced by a mixture of two components, as shown in formula (6-3): The joint distribution of unknown parameters obeys the function G, which is a nonparametric Dirichlet distribution with a basic measure of G0 and a concentration parameter of α0. G0 provides a priori support for the center of the patient sample, and α0 controls the strength of the concentration. S35, rewrite formula (6-3) into a representation based on adaptive weighting, such as formula (6-4): b k |α0,G0~Beta(1,α0) φk|α0,G0~G0 (6-4); Among them, β k represents the continuous probability distribution of k, φ k represents the kth patient sample from the prior, and Beta(1,α0) is the beta distribution.

6. The lung cancer multimodal data diagnosis system based on the Dirichlet nonparametric mixture model according to claim 5, characterized in that: When the feature selection module performs feature screening according to different weights, it includes: S3-1, define random distribution as formula (6-5): in, is φ k is the Dirichlet function of the mass center, G is an infinite discrete distribution that can be realized with weight Next Randomly select from In this way, a discrete situation about the patient sample is obtained; S3-2, assuming that the map of image features is ζ(1), the map of gene features is ζ(2), and the biological maps in the two forms are ζ(1,2), the image features and gene features are integrated into the joint clustering framework. The specific integration method is shown in formulas (6-6) and (6-7): Among them, j~ζk indicates that features j and k are adjacent in the graph ζ, γ1 and γ2 are used to balance the correlation coefficients between different modalities and adjust the sparsity of image features and gene features, e1, e2, e' are used to control the smoothness of each modality network and the smoothness between them, and P is the Potts model, which is applicable to different modality components and is imposed with a priori indicators 7. The lung cancer multimodal data diagnosis system based on the Dirichlet nonparametric mixture model according to claim 1, characterized in that: The precise diagnosis module is used to classify the screened features through a classifier to obtain a prediction result of lung cancer subtypes, including: S4-1, when training the classifier, images of different modalities are input simultaneously, and training samples of different scales are as shown in formula (5-1): Among them, X i is the selected feature, Y i is the label, N represents the number of training samples; S4-2, the goal of classifier training is to learn the parameters W of the classifier. The loss function of the classifier is defined as formula (5-2): In formula (5-2), M is the number of outputs, M = 2, S m represents the patient sample with modality M, α m is the loss weight, training sample S m Only contributes to the loss of mode M, for α m : Among them, K represents the Gaussian kernel function, L m (t-1) represents the t-1th loss under the M mode; S4-3, the loss calculation for each modality classification branch in the classifier is as shown in formulas (5-4) and (5-5): L(X,Y|W)=L cls (p(X),y) (5-4); Among them, L cls (p(X),y)=-logp y (X) is the cross entropy loss. After training, the loss function is minimized and the predicted lung cancer subtype is output as Y. i Table Confidence label of sample i.

Images