Clinical data analysis method based on causal relationship discovery
By constructing a fully connected causal relationship diagram in clinical data analysis and eliminating weak causal relationships, the problem of deviation in causal relationship analysis results in traditional methods is solved, and more accurate causal relationship discovery of disease status and risk factors is achieved, and precise medical treatment and prevention are supported. Strategy development.
Patent Information
- Application Number
- CN202411201687.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-29
- Publication Date
- 2024-11-19
AI Technical Summary
Traditional clinical data analysis methods are difficult to explore the causal relationship in depth, resulting in deviations from the actual situation, affecting the scientific nature of medical decision-making and resource allocation.
Clinical data analysis methods based on causal relationship discovery, including data cleaning, construction of fully connected causal relationship diagrams, conditional independence test, PDS tree optimization and Pearson correlation coefficient removal weak causal relationships, directional rules and external expert knowledge supplements to construct DAG Figure to determine causality.
It can more accurately explore the causal relationship between disease state and potential risk factors, improve the accuracy and effectiveness of disease diagnosis and treatment, and support the formulation of precision medicine and prevention strategies.
Smart Images

Figure CN118983109A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of clinical data analysis for causal relationship discovery, and specifically relates to a clinical data analysis method based on causal relationship discovery. Background Art
[0002] Clinical data analysis refers to the use of statistics and other data analysis techniques to organize, describe, interpret and infer data generated in clinical research in order to discover patterns and trends in the data and provide decision support for medical research and clinical practice. However, traditional clinical data analysis methods are mostly limited to revealing the correlation between variables and fail to explore causal relationships in depth. With the advent of the era of medical big data, clinical data analysis needs to gradually develop in a more intelligent and refined direction. The use of correlation limits the scientific formulation and effect evaluation of treatment strategies, which may lead to one-sided medical decision-making and irrational resource allocation.
[0003] Causal discovery aims to reveal the underlying causal structure from the observed data, that is, which variables are causes, which variables are effects, and the causal direction between them. In clinical data analysis, causal discovery can be used to mine the causal relationship between disease states and potential risk factors in clinical data. However, in the presence of weak causal relationships, if researchers mistakenly interpret weak causal relationships as strong causal relationships, misleading conclusions may be drawn, which will have an adverse impact on clinical practice. Therefore, traditional causal discovery methods face the problem of deviation between analysis results and actual conditions when processing clinical data. Summary of the invention
[0004] In view of the problem that the analysis results of traditional causal relationship discovery methods are deviated from the actual situation when processing clinical data, the present invention provides a clinical data analysis method based on causal relationship discovery.
[0005] In order to achieve the above object, the present invention adopts the following technical solutions:
[0006] A clinical data analysis method based on causal relationship discovery comprises the following steps:
[0007] Step 1: Clean clinical data and process missing data;
[0008] Step 2, using the processed clinical data to construct a fully connected causal relationship graph between variables;
[0009] Step 3, according to the conditional independence test method between variables, the fully connected causal relationship graph is deleted to obtain a causal skeleton graph;
[0010] Step 4, use the PDS tree to optimize the causal skeleton diagram and use the Pearson correlation coefficient to eliminate weak causal relationships, and use the directional rule to orient the PAG diagram;
[0011] Step 5: Use the PC algorithm to perform conditional independence test on the fully connected causal relationship graph to obtain the DAG graph and use the DAG graph and external expert knowledge to supplement the PAG graph.
[0012] Furthermore, the specific operation of step 1 is to remove duplicate, invalid and abnormal data in the clinical data, and use mean filling to fill the missing data.
[0013] Furthermore, the specific operation of step 2 is: establishing connections between all processed clinical data variables, ensuring that there is a direct causal path between any two variables in the graph, and thus forming a fully connected causal relationship graph.
[0014] Furthermore, in the conditional independence test method of step 3, when the condition set is an empty set, an unconditional independence test is performed on each pair of nodes to determine whether to delete the edge connecting the pair of nodes, and a 0-order PAG graph is obtained; when the condition set is a single variable Y, two variables X and Z are randomly selected and independence tests are performed to delete the edges; if Y is used as a condition, the originally independent X and Z will become correlated, and the orientation is a V structure, and a 1-order PAG graph is obtained.
[0015] The specific steps include:
[0016] Step 3.1, measure the dependency between nodes X and Y through MI:
[0017]
[0018] Where I(X,Y) represents the dependency between X and Y, p(x,y) is the joint probability distribution of X and Y, which is achieved by counting the number of co-occurrences of each value pair (x,y) in X and Y and dividing it by the total number of samples; p(x) and p(y) are the marginal probability distributions of X and Y, respectively, which are achieved by counting the number of occurrences of each value in each variable and dividing it by the total number of samples; x represents a value in the attribute of node X, and y represents a value in the attribute variable of node Y;
[0019] The CMI measures the conditional dependency between nodes X and Y given node Z:
[0020]
[0021] When nodes X and Y are conditionally independent given node Z, I(X,Y|Z) = 0;
[0022] First, standardize the CMI:
[0023]
[0024] Fisher z transform of CMI to test whether CMI is zero:
[0025]
[0026] Then input the sample data set DATA of clinical data, the variable set V, the number of variables is n, and the significance level α;
[0027] Step 3.2: Use the conditional set as the adjacent set for skeleton learning in the r=0 and r=1 stages
[0028] Step 3.3, perform iterations of r = 0, test the unconditional independence between each pair of variables, and when the variables are independent, remove the edges between the variables, then orient the V structure and the directions of the remaining edges to obtain a 0-order PAG graph;
[0029] In step 3.4, perform r=1 iterations, use CMI to test the independence of each pair of nodes, whose conditional set is a single node, and remove the corresponding edges; reset the direction of the directed edge in step 3.3 to an undirected edge, and then redirect the V structure and the remaining edge directions to obtain a 1st-order PAG graph.
[0030] Furthermore, the specific operation of step 4 is: when the condition set is greater than one variable, traverse all edges in the entire graph, and establish a PDS tree for the two endpoints of each edge, so that the PDS tree of each variable can be quickly found during the subsequent conditional independence test; in the subsequent PDS tree construction process, all first-order neighbor nodes of the target node will be added to the PDS tree, and the Pearson correlation coefficient will be used to calculate the correlation, and the second-order and above nodes with a correlation threshold greater than 0.2 will also be added to the PDS tree, and finally, a condition set that meets the conditions is generated from the PDS tree of the target node to test the conditional independence and delete the edges; direct the V structure and the direction of the remaining edges; terminate the conditional independence test when the condition set is the total number of variables, and obtain a global causal relationship PAG diagram.
[0031] Furthermore, the specific operations of step 5 are: using a constraint-based causal relationship discovery algorithm to construct a DAG graph; comparing the DAG graph and PAG, retaining the common causal relationships, and using the DAG graph to supplement the direction of some undirected dependencies in the PAG graph; when the direction of certain edges in the PAG graph cannot be determined, using medical expert knowledge or external medical library knowledge to assist in determination.
[0032] Compared with the prior art, the present invention has the following advantages:
[0033] In clinical data analysis, the present invention can mine the causal relationship between disease status and potential risk factors in clinical data, which is helpful for the implementation of precision medicine and the formulation of disease prevention strategies, and improves the accuracy of disease diagnosis and treatment effects. BRIEF DESCRIPTION OF THE DRAWINGS
[0034] Figure 1 The figure is a flowchart of a clinical data analysis method based on causal relationship discovery.
[0035] Figure 2 A schematic diagram of an embodiment of a clinical data analysis method based on causal relationship discovery.
[0036] Figure 3 Schematic diagram of an embodiment of the present invention using PDS to construct a 2nd-order PDS tree of cough nodes.
[0037] Figure 4 This is an example of converting a completely undirected graph into a completely directed graph according to the present invention. DETAILED DESCRIPTION
[0038] In order to gain a deeper understanding of the present invention, we will provide a comprehensive and detailed description of the present invention. However, the present invention has multiple implementations and is not limited to the specific examples listed herein. The presentation of these examples is intended to deepen the comprehensive understanding of the disclosure of the present invention.
[0039] Example 1
[0040] A clinical data analysis method based on causal relationship discovery, such as Figure 1 As shown, the following steps are included:
[0041] Step 1: Clean clinical data and process missing data;
[0042] The patient IDs of patients admitted from 2005 to 2023 were searched in the hospital system, and the basic physiological indicators of patients such as height, weight, age, gender, BMI, mental state, respiratory rate, body temperature, white blood cell count (WBC), neutrophil percentage (N%), C-reactive protein (CRP), procalcitonin (PCT), liver function, kidney function, electrolytes, blood sugar, etc. were collected from different departments. A data set with patients as rows and different indicators of patients as columns was constructed. The obtained clinical data set was removed from duplicate, invalid and abnormal data and the missing data was filled with mean filling to obtain Table 1.
[0043] Table 1
[0044]
[0045] Step 2, using the processed clinical data to construct a fully connected causal relationship graph between variables;
[0046] Connections are established between all processed clinical data variables to ensure that there is a direct causal path between any two variables in the graph, thereby forming a fully connected causal relationship graph.
[0047] Step 3, according to the conditional independence test method between variables, the fully connected causal relationship graph is deleted to obtain a causal skeleton graph;
[0048] Step 3.1, measure the dependency between nodes X and Y through MI:
[0049]
[0050] Where I(X,Y) represents the dependency between X and Y, p(x,y) is the joint probability distribution of X and Y, which is achieved by counting the number of co-occurrences of each value pair (x,y) in X and Y and dividing it by the total number of samples; p(x) and p(y) are the marginal probability distributions of X and Y, respectively, which are achieved by counting the number of occurrences of each value in each variable and dividing it by the total number of samples; x represents a value in the attribute of node X, and y represents a value in the attribute variable of node Y;
[0051] The CMI measures the conditional dependency between nodes X and Y given node Z:
[0052]
[0053] When nodes X and Y are conditionally independent given node Z, I(X,Y|Z) = 0;
[0054] First, standardize the CMI:
[0055]
[0056] Fisher z transform of CMI to test whether CMI is zero:
[0057]
[0058] Then input the sample data set DATA of clinical data, the variable set V, the number of variables is n, and the significance level α;
[0059] Step 3.2: Use the conditional set as the adjacent set for skeleton learning in the r=0 and r=1 stages
[0060] Step 3.3, perform iterations of r = 0, test the unconditional independence between each pair of variables, and when the variables are independent, remove the edges between the variables, then orient the V structure and the directions of the remaining edges to obtain a 0-order PAG graph;
[0061] In step 3.4, perform r=1 iterations, use CMI to test the independence of each pair of nodes, whose conditional set is a single node, and remove the corresponding edges; reset the direction of the directed edge in step 3.3 to an undirected edge, and then redirect the V structure and the remaining edge directions to obtain a 1st-order PAG graph.
[0062] Step 4, use the PDS tree to optimize the causal skeleton diagram and use the Pearson correlation coefficient to eliminate weak causal relationships, and use the directional rule to orient the PAG diagram;
[0063] Step 4.1, start executing iterations of r>1, traverse all edges in the entire graph, and establish a PDS tree set for the two endpoints of each edge as the Z set in the CMI formula calculation, so as to delete all edges that should be deleted as much as possible.
[0064] Step 4.2, during the construction of the PDS tree in step 4.1, all first-order neighbor nodes of the target node are added to the PDS tree, and the Pearson correlation coefficient is used to calculate the correlation. The second-order and above nodes with a correlation coefficient threshold greater than 0.2 (usually, the absolute value of the correlation coefficient between 0.3 and 0.5 is considered to be weakly correlated, so the edges with an absolute value of the correlation coefficient lower than 0.2 are allowed to be removed, while the edges with a weight greater than 0.2 are allowed to be retained) are also added to the PDS tree (the reason for adding second-order and above nodes to the PDS tree is that only using first-order neighbors will cause some redundant edges to be unable to be deleted). Finally, a subset of size r is generated to test conditional independence for edge deletion. Directed V structure and the direction of the remaining edges, an r-order PAG graph is obtained.
[0065] Step 5: Use the PC algorithm to perform conditional independence test on the fully connected causal relationship graph to obtain the DAG graph and use the DAG graph and external expert knowledge to supplement the PAG graph.
[0066] Step 5.1, use the constraint-based causal relationship discovery algorithm to construct the DAG graph. Compare the DAG graph with the PAG, retain the common causal relationships, and use the DAG graph to supplement the direction of some undirected dependencies in the PAG graph.
[0067] Step 5.2, the direction of the edge that is unique to G1 can be determined with the help of expert knowledge or external knowledge. The blacklist will store some impossible edge directions (lung cancer cannot cause age changes), and the whitelist will store some possible edge directions. Use the blacklist and whitelist to constrain the causal relationship graph G3, so the final result is as follows Figure 4 The PAG diagram shown is used to demonstrate the causal relationship between lung cancer, smoking, cough, age and increased procalcitonin.
[0068] Table 2 Comparison of causal discovery results on a simulated dataset of size 500
[0069]
[0070] The present invention adopts the FCI algorithm as the benchmark algorithm and improves it, eliminating low-correlation variables, so that researchers can focus more on those variables that have a significant impact on the results, thereby gaining a deeper understanding of the mechanism behind the phenomenon.
[0071] The above causal discovery algorithm is introduced as follows:
[0072] PC: A constraint-based causal discovery method for learning the structure of Bayesian networks from data, which cannot handle cases involving hidden variables. FCI: A constraint-based causal discovery method that can handle cases involving hidden variables. FCI+: A variant of the constraint-based causal discovery method with optimized performance. CFCI: A variant of the constraint-based causal discovery method that reduces the discovery of colliding structures. RFCI: A random forest-based causal inference method that combines the advantages of random forest models in evaluating interactions between variables to infer potential causal relationships. GFCI: An algorithm that combines a greedy strategy with fast causal inference. ICD: An iterative causal discovery algorithm that pays special attention to recovering causal graphs in the presence of potential confounders and selection bias.
[0073] The contents not described in detail in the specification of the present invention belong to the prior art known to the professional and technical personnel in the field. Although the illustrative specific embodiments of the present invention are described above to facilitate the understanding of the present invention by the technical personnel in the field, it should be clear that the present invention is not limited to the scope of the specific embodiments. For the ordinary technical personnel in the field, as long as various changes are within the spirit and scope of the present invention defined and determined by the attached claims, these changes are obvious, and all inventions and creations using the concept of the present invention are protected.
Claims
1. A clinical data analysis method based on causal relationship discovery, characterized in that: The following steps are involved: Step 1: Clean clinical data and process missing data; Step 2, using the processed clinical data to construct a fully connected causal relationship graph between variables; Step 3, according to the conditional independence test method between variables, the fully connected causal relationship graph is deleted to obtain a causal skeleton graph; Step 4, use the PDS tree to optimize the causal skeleton diagram and use the Pearson correlation coefficient to eliminate weak causal relationships, and use the directional rule to orient the PAG diagram; Step 5: Use the PC algorithm to perform conditional independence test on the fully connected causal relationship graph to obtain the DAG graph and use the DAG graph and external expert knowledge to supplement the PAG graph.
2. A clinical data analysis method based on causal relationship discovery according to claim 1, characterized in that: The specific operation of step 1 is to remove duplicate, invalid and abnormal data in the clinical data, and use mean filling to fill the missing data.
3. A clinical data analysis method based on causal relationship discovery according to claim 1, characterized in that: The specific operation of step 2 is: establish connections between all processed clinical data variables, ensure that there is a direct causal path between any two variables in the graph, and then form a fully connected causal relationship graph.
4. The clinical data analysis method based on causal relationship discovery according to claim 1, characterized in that: In the conditional independence test method of step 3, when the condition set is an empty set, an unconditional independence test is performed on each pair of nodes to determine whether to delete the edge connecting the pair of nodes, and a 0-order PAG graph is obtained; when the condition set is a single variable Y, two variables X and Z are randomly selected and an independence test is performed to delete the edge; if Y is used as a condition, the originally independent X and Z will become correlated, and the orientation is a V structure, and a 1-order PAG graph is obtained.
5. A clinical data analysis method based on causal relationship discovery according to claim 4, characterized in that: The specific steps include: Step 3.1, measure the dependency between nodes X and Y through MI: Where I(X,Y) represents the dependency between X and Y, p(x,y) is the joint probability distribution of X and Y, which is achieved by counting the number of co-occurrences of each value pair (x,y) in X and Y and dividing it by the total number of samples; p(x) and p(y) are the marginal probability distributions of X and Y, respectively, which are achieved by counting the number of occurrences of each value in each variable and dividing it by the total number of samples; x represents a value in the attribute X, and y represents a value in the attribute variable Y; The CMI measures the conditional dependency between nodes X and Y given node Z: When nodes X and Y are conditionally independent given node Z, I(X,Y|Z) = 0; First, standardize the CMI: Fisher z transform of CMI to test whether CMI is zero: Then input the sample data set DATA of clinical data, the variable set V, the number of variables is n, and the significance level α; Step 3.2: Use the conditional set as the adjacent set for skeleton learning in the r=0 and r=1 stages Step 3.3, perform iterations of r = 0, test the unconditional independence between each pair of variables, and when the variables are independent, remove the edges between the variables, then orient the V structure and the directions of the remaining edges to obtain a 0-order PAG graph; In step 3.4, perform r=1 iterations, use CMI to test the independence of each pair of nodes, whose conditional set is a single node, and remove the corresponding edges; reset the direction of the directed edge in step 3.3 to an undirected edge, and then redirect the V structure and the remaining edge directions to obtain a 1st-order PAG graph.
6. The clinical data analysis method based on causal relationship discovery according to claim 1, characterized in that: The specific operation of step 4 is: when the condition set is greater than one variable, traverse all edges in the entire graph, and establish a PDS tree for the two endpoints of each edge, so that the PDS tree of each variable can be quickly found when the conditional independence test is performed later; in the subsequent PDS tree construction process, all first-order neighbor nodes of the target node are added to the PDS tree, and the Pearson correlation coefficient is used to calculate the correlation, and the second-order and above nodes with a correlation threshold greater than 0.2 are also added to the PDS tree, and finally a condition set that meets the conditions is generated from the PDS tree of the target node to test the conditional independence and delete the edge; Directed V structure and direction of remaining edges; when the condition set is the total number of variables, the conditional independence test is terminated to obtain a global causal relationship PAG diagram.
7. The clinical data analysis method based on causal relationship discovery according to claim 1, characterized in that: The specific operations of step 5 are: using a constraint-based causal relationship discovery algorithm to construct a DAG graph; comparing the DAG graph and PAG, retaining the common causal relationships, and using the DAG graph to supplement the directions of some undirected dependencies in the PAG graph; when the directions of some edges in the PAG graph cannot be determined, using medical expert knowledge or external medical library knowledge to assist in the determination.
Citation Information
Cited By
Natural gas hydrate productivity main control factor analysis method and device, electronic equipment and storage medium
CN120492876A
Statistical analysis method and system for traditional Chinese medicine clinical research based on syndrome logic network
CN121011313A
Statistical analysis method and system for clinical research of traditional Chinese medicine based on syndrome pattern logic network
CN121011313B
