Causal discovery method and device fusing large language model field knowledge and flow model

Abstract

The application provides a causal discovery method and device fusing field knowledge of a large language model and a flow model, and relates to the technical fields of artificial intelligence and data mining. The method comprises the following steps: acquiring observation data and variable semantic information of a system to be analyzed; determining a causal topological order between variables by using a large language model; constructing a mask affine autoregressive flow model based on the causal topological order, and training the model by using the observation data; calculating a Jacobian matrix of the flow model with respect to input data, constructing a statistic based on the Jacobian matrix, and performing hypothesis testing; pruning according to a test result to obtain a candidate causal graph; inputting the candidate causal graph into the large language model, and performing secondary verification and pruning on candidate connections by using the variable semantic information to obtain a final sparse causal graph. The application introduces prior knowledge by using semantic information of the large language model, and fits data by using a more flexible function form of the flow model, thereby significantly improving the accuracy and robustness of causal discovery.

Description

Technical Field

[0001] This application relates to the fields of artificial intelligence and data mining technology, and in particular to a causal discovery method and apparatus that integrates domain knowledge from large language models and flow models. Background Technology

[0002] Causal discovery aims to infer the causal relationship structure between variables from observational data, typically represented in the form of a directed acyclic graph. This technique has significant applications in fields such as bioinformatics, medical diagnosis, economic decision-making, and root cause analysis.

[0003] Currently, causal discovery methods are mainly divided into constraint-based methods (such as the PC algorithm), fraction-based methods (such as the GES algorithm), and function-based causal model methods (such as LiNGAM and ANM). In recent years, with the development of deep learning, gradient-based neural causal discovery methods have made significant progress.

[0004] However, existing methods for causal discovery still face significant challenges in practical applications, mainly in the following aspects:

[0005] The problem of identifying causal directions is difficult to solve. Relying solely on observational data, traditional statistical methods often can only recover Markov equivalence classes, meaning they cannot distinguish causal graphs with the same conditional independence. Although additive noise models (ANMs) can be identified under certain conditions, purely data-driven methods often fail to determine the correct causal direction when faced with limited samples or Gaussian noise, resulting in incorrect arrows in the learned graph structure.

[0006] The model assumptions have limitations and are difficult to adapt to complex data distributions. Some traditional causal discovery algorithms (such as the basic LiNGAM) are only applicable to linear causal models and struggle to capture the widespread nonlinear dependencies in the real world. Furthermore, many methods based on additive noise models typically assume that noise satisfies homoscedasticity, failing to characterize heteroscedasticity, i.e., they cannot identify the characteristic that noise variance fluctuates with changes in the parent variable, leading to decreased accuracy when processing data with complex noise mechanisms.

[0007] The search space faces the challenges of combinatorial explosion and the difficulty of optimization with acyclic constraints. As the number of nodes in the causal graph increases, the number of possible DAG structures grows exponentially, making traditional discrete search strategies face the NP-hard problem of combinatorial explosion. Although methods such as NOTEARS transform the combinatorial optimization problem into a continuous optimization problem by introducing smooth acyclic constraints based on the matrix exponential trace, the computation of these constraints and their gradient solving involve expensive matrix operations (typically with cubic time complexity), resulting in enormous computational overhead on large-scale graphs. Furthermore, due to the non-convexity of the objective function, soft-constraint-based optimization processes often fail to strictly guarantee the acyclicity of the output graph, leading to residual small-weight cycles in the results and affecting the accuracy of the causal structure.

[0008] Lacking semantic-level rationality checks, existing causal discovery algorithms are not robust enough. They typically treat variables as abstract mathematical symbols (such as X1, X2), completely ignoring the physical meaning or common-sense logic behind the variable names. This can lead to algorithms outputting causal relationships that violate domain common sense (e.g., "rain causes dark clouds"). Summary of the Invention

[0009] This application provides a causal discovery method and apparatus that integrates domain knowledge from large language models and flow models. It aims to solve the problems existing in causal discovery techniques, such as difficulty in identifying causal directions, combinatorial explosion in large-scale graph search, high computational complexity and easy residual micro-loops in acyclic constraints based on continuous optimization, difficulty in effectively fitting nonlinear and heteroscedastic data distributions by traditional models, and lack of semantic rationality verification and susceptibility to statistical noise interference due to reliance on data-driven approaches. This will significantly improve the accuracy and interpretability of causal discovery results.

[0010] This application provides a causal discovery method that integrates domain knowledge from large language models with flow models, including:

[0011] Step S10: Obtain the observation data and variable semantic information of the system to be analyzed;

[0012] Step S20: Using a large language model, combined with the semantic information of the variables and their domain knowledge, determine the global causal topological ordering among the variables;

[0013] Step S30: Construct a masked affine autoregressive flow model based on the global causal topological sorting, so that the generation of any variable in the model depends only on the variable that precedes it in the sorting; train the masked affine autoregressive flow model using the observation data to fit the structural equations between the variables.

[0014] Step S40: Calculate the Jacobian matrix of the trained flow model with respect to the observed data. The Jacobian matrix represents the functional sensitivity of the variable generation mechanism to the preceding variables. Based on the Jacobian matrix, construct a test statistic and perform statistical hypothesis testing. Based on the test results, remove statistically insignificant connections to obtain a candidate causal graph.

[0015] Step S50: Input the connections in the candidate causal graph into the large language model, use the semantic information of variables to perform secondary verification on the candidate connections, and remove the connections that are judged to be unreasonable to obtain the final sparse causal graph.

[0016] This application also provides a causal discovery device that integrates domain knowledge from large language models with flow models, including:

[0017] The data acquisition module is used to acquire observation data and variable semantic information of the system to be analyzed;

[0018] The topological ordering module is used to determine the global causal topological order of variables by combining the semantic information of the variables with its own domain knowledge using a large language model.

[0019] The model building and training module is used to build a masked affine autoregressive flow model based on the global causal topological sorting, and to train the model using observation data.

[0020] The data pruning module is used to calculate the Jacobian matrix of the flow model with respect to the input data, construct test statistics based on the Jacobian matrix and perform statistical hypothesis testing, and remove statistically insignificant connections according to the test results to obtain candidate causal graphs;

[0021] The semantic pruning module is used to input candidate causal graphs into a large language model, and to perform secondary verification and pruning on candidate connections using variable semantic information, eliminating connections that are judged to be unreasonable, and obtaining the final sparse causal graph.

[0022] This application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the causal discovery method as described above, which integrates domain knowledge of a large language model and a flow model.

[0023] This application also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the causal discovery method as described above, which integrates domain knowledge of a large language model and a flow model.

[0024] This application also provides a computer program product, including a computer program that, when executed by a processor, implements any of the above-described methods for causal discovery that integrates domain knowledge from a large language model and a flow model.

[0025] Beneficial effects:

[0026] The causal discovery method, apparatus, and electronic device provided in this application, which integrates domain knowledge from a large language model with a flow model, effectively solves the problem of unidentifiable causal directions in traditional data-driven methods by introducing domain knowledge through a large language model and determining the topological order of variables, and avoids combinatorial explosion caused by large-scale graph search. Through a masked affine autoregressive flow model, it can flexibly fit the nonlinear and heteroscedastic distribution characteristics of data. Through a "dual pruning" strategy that combines statistical hypothesis testing based on the Jacobi matrix with semantic verification based on a large model, it adaptively removes weak signal noise at the statistical level and eliminates spurious connections that violate common sense at the semantic level, thereby significantly improving the accuracy, sparsity, and interpretability of causal discovery results. Attached Figure Description

[0027] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 This is a flowchart illustrating the causal discovery method that integrates domain knowledge from a large language model and a flow model, as provided in this application.

[0029] Figure 2 This is a schematic flowchart of step S20 according to the method of this application;

[0030] Figure 3 This is a schematic flowchart of step S30 of the method according to this application;

[0031] Figure 4 This is a schematic diagram of the causal discovery device that integrates domain knowledge from a large language model and a flow model, as provided in this application.

[0032] Figure 5 This is a schematic diagram of the structure of the electronic device provided in this application. Detailed Implementation

[0033] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0034] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0035] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0036] The following description, in conjunction with the accompanying drawings, describes the causal discovery method, apparatus, and electronic device that integrate large language model domain knowledge and flow model provided in the embodiments of this application.

[0037] Figure 1 The flowchart illustrates the causal discovery method that integrates large language model priors and flow models in this embodiment. Figure 1 As shown, this causal discovery method, which integrates prior knowledge from a large language model with that of a flow model, includes the following steps:

[0038] Step S10: Obtain the observation data and variable semantic information of the system to be analyzed.

[0039] Specifically, observation data is usually represented as a matrix. ,in Indicates the number of samples. This represents the dimension of the variables. The data matrix can also be represented as... Each row of this matrix represents an observation sample, and each column... It is a column vector representing the first... The observed values ​​of a system variable across all samples (e.g., rainfall, CPU utilization, etc.). Variable semantic information refers to the specific name and descriptive text of the variable, denoted as... The acquisition method can be either written by experts or generated by a large language model.

[0040] Step S20: Using a large language model combined with the semantic information of the variables and its own domain knowledge, determine the global causal topological ordering among the variables.

[0041] This step aims to address the challenge of traditional statistical methods failing to utilize semantic knowledge of variables. It involves integrating semantic information from variables... Inputting the pre-trained large language model, specific prompts are constructed to guide the model to judge the causal relationship between triple variables based on its internally stored common sense or domain knowledge (e.g., "rainfall will increase soil moisture, and vice versa"). A voting matrix is ​​constructed using the output of the large language model, and a global causal topological ranking is obtained by removing loops through a greedy strategy. ,in This indicates the position in the causal chain. The variable index of the bit. This sorting satisfies the constraint: if the variable... yes The causal parent node, then in the sorting Must be located Previously, this prior ranking not only successfully introduced prior knowledge through a large language model, but also significantly reduced the search space.

[0042] Step S30: Construct a masked affine autoregressive flow model based on the global causal topological sorting, so that the generation of any variable in the model depends only on the variable that precedes it in the sorting, and train the model using the observed data to fit the structural equation between the variables.

[0043] To capture the nonlinear dependencies and heteroscedasticity in the data, this application constructs an autoregressive flow model based on a masked autoencoder density estimation network (MADE), that is, using the masked autoencoder density estimation network (MADE) as the backbone network of the flow model. Specifically, the model uses variables... The generation process is modeled as an affine transformation:

[0044] (1)

[0045] in, Indicates topological sorting lie in The set of all previous variables. and It is a function fitted by a neural network. This is a latent noise variable.

[0046] When constructing the network, global causal topology ordering is used. A mask matrix is ​​generated and applied to the neural network to cut off all connections that violate the topological order, thus naturally guaranteeing the acyclicity of the model from a structural perspective.

[0047] Using observation data The model is trained by maximizing the log-likelihood function of the observed data, i.e. minimizing the negative log-likelihood loss, to optimize the parameters of the neural network, thereby enabling the model to accurately fit the joint probability distribution of the observed data.

[0048] Step S40: Calculate the Jacobian matrix of the trained autoregressive flow model with respect to the observed data. The Jacobian matrix represents the functional sensitivity of the variable generation mechanism to preceding variables. Based on the Jacobian matrix, construct statistics and perform hypothesis testing. Eliminate statistically insignificant connections based on the test results to obtain candidate causal graphs. Here, causal connections are specific directed edges between variables in the causal topological ordination and must follow the direction specified by the ordination.

[0049] This is the first layer of the "double pruning" strategy: statistical layer pruning. Because the model may fit weak noise in the data during training, ... All variable pairs The gradients are all non-zero. Therefore, the transformation of the autoregressive flow model is calculated. Regarding observation data Jacobian matrix ,in, It's about observation data. The partial derivative operator is used to calculate the partial derivative of the model output with respect to small changes in the input variables, transforming... This refers to the forward mapping function of a trained autoregressive flow model, which maps the input observed data to the fitted values ​​of each variable. Elements in the Jacobian matrix... Able to comprehensively reflect variables right The causal strength. Construct a global sensitivity matrix. ,in , Characteristic variables right average causal strength It is an operator for calculating the mathematical expectation over all observed samples. In actual calculations, this operator is expressed as the Jacobian matrix elements corresponding to all observed samples. The purpose of calculating the arithmetic mean is to eliminate random perturbations from individual samples, thereby obtaining a globally stable measure of causal strength across the entire data distribution. Constructing the null hypothesis. If there is no causal connection between variables, that is, the corresponding global sensitivity value is zero; set a fixed significance threshold, reject the null hypothesis corresponding to the connection less than the significance threshold, identify it as a noise connection and remove it, and retain the connection greater than the significance threshold to form a preliminary candidate causal graph.

[0050] Step S50: Input the candidate causal graph into the large language model, perform secondary verification on the candidate connections using variable semantic information, and remove connections deemed unreasonable to obtain the final sparse causal graph. This step may include:

[0051] Extract all node pairs with directed edges from the candidate causal graph and input the node pairs into the large language model;

[0052] Instructions for large language models based on nodes and Based on the semantic information of the variables and domain knowledge, determine whether the relationship between the variables is a direct causal relationship, an indirect causal relationship, or neither;

[0053] The same question was asked to the large language model multiple times, and the final result was obtained through voting.

[0054] To avoid mistakenly deleting correct edges, if the large language model determines that there is no direct or indirect causal relationship between two nodes, a pruning operation is performed: the corresponding directed edges are removed from the candidate causal graph, and the final sparse causal relationship is output.

[0055] This is the second layer of the "double pruning" strategy: semantic layer pruning. The candidate causal graph obtained in step S40 may contain spurious causal relationships that conform to statistical laws but violate common sense in physics (e.g., "lung cancer causes smoking"). This step prunes each directed edge in the candidate causal graph... The causal propositions are transformed into natural language descriptions. These propositions are then fed back to the large language model, instructing it to judge whether the causal relationship is reasonable in real-world mechanisms based on its domain knowledge. Multiple votes are used to arrive at the final result; if an edge is deemed nonexistent, it is removed from the candidate graph. After double pruning, the final output is a sparse directed acyclic graph that conforms to both statistical data characteristics and domain common-sense logic, serving as the final result of causal discovery.

[0056] Figure 2 A schematic flowchart of step S20 according to the method of this application is shown. Figure 2 As shown, step S20, which utilizes a large language model to combine the semantic information of the variables with its own domain knowledge, determines the causal topological ordering among the variables, and may include the following steps:

[0057] Step S21: Construct triple query suggestions containing variable names and semantic information, transforming the causal relationship judgment between pairs of variables into a causal relationship judgment problem within triples, in order to reduce the generation of loops.

[0058] In one specific implementation, considering that directly inputting all variables into a large language model for global sorting can easily lead to "illusions" or exceed the model's context window limitations, and that using binary queries can easily result in cycles, a method of constructing triple query suggestion words is adopted, including:

[0059] From the semantic information of the variables Extract variables from them. Given the total number of variables, construct a set of variable triples according to the principles of combinatorial mathematics. ,in These are the sets of semantic information of the variables. This involves semantic descriptions of three distinct variables selected by index. These three variables are grouped into a local analysis unit, aiming to reduce the high-dimensional global causal ranking problem to a low-dimensional local causal judgment problem, thereby effectively reducing the illusion rate and computational complexity of large language models when performing complex logical reasoning.

[0060] For each triple The query text is constructed by combining a preset prompt template. The prompt template includes, but is not limited to: Based on common sense and physical laws, please analyze the causal relationship between the following three variables: , , Please explain your reasoning.

[0061] In this way, the high-dimensional global graph structure search problem is reduced to a low-dimensional local causal judgment problem, which effectively reduces the reasoning difficulty of large language models and greatly reduces the occurrence of cycles compared with the binary questioning method.

[0062] Step S22: Input the triple query suggestion words into the large language model, fill the local causal relationship results output by the large language model into the initialized voting matrix, and accumulate the causal confidence.

[0063] In one specific implementation, to eliminate the uncertainty in the output of a large language model, this application employs an integrated voting mechanism. Specifically, constructing the voting matrix includes:

[0064] Initialize a dimension as The zero matrix as the voting matrix ,in The total number of variables, matrix elements Used to record variables For variables The cumulative confidence of the causal parent node.

[0065] Input the multiple triple query suggestions generated in step S21 into a large language model (such as GPT-4, Claude, or locally deployed qwen, etc.) in sequence, and parse the natural language response returned by the model.

[0066] If the model is for the inclusion and Determine " in the query" lead to Then, for the corresponding elements in the voting matrix... Perform an increment operation ( ).

[0067] After iterating through all query results, the final voting matrix is ​​obtained. This matrix integrates causal judgment information from all local perspectives.

[0068] Step S23: Eliminate the cyclic paths in the voting matrix based on a greedy strategy to generate a global causal topological sort.

[0069] In one specific implementation, the accumulation of local reasoning may lead to logical conflicts at the global level (e.g.) (The formed loops) may not yield a valid directed acyclic graph if the voting matrix is ​​used directly. Therefore, this step employs a greedy strategy for loop removal, specifically including:

[0070] Create an empty ordered list Used to store the final topology sort.

[0071] In the voting matrix Find the element with the largest value in the middle. This element represents the causal relationship with the highest confidence level. .

[0072] Determine the relationship Will adding a relation to an adopted set create a loop with other adopted relations?

[0073] If no loop is formed, the relationship is adopted; if a loop is formed, the high-confidence vote is considered to have a logical contradiction, and the relationship is discarded.

[0074] Repeat the above process until the voting matrix has been traversed. Extract the final global causal topological sort from all the relationships that have been adopted and fill it into an ordered list. This sorting This ensures that the flow model constructed subsequently naturally satisfies the acyclic constraint in terms of structure.

[0075] Figure 3 A schematic flowchart of step S30 of the method of this application is shown. Figure 3 As shown, step S30: Constructing a masked affine autoregressive flow model based on the causal topological sorting, such that the generation of any variable in the model depends only on the variable preceding it in the sorting, and training the model using the observed data to fit the structural equations between the variables, including the following steps:

[0076] Step S31: Rearrange the column vectors of the observation data according to the causal topological sorting so that the rearranged variable sequence satisfies the causal order constraint.

[0077] like Figure 3 As shown, the input data includes the observed sample set. and the global causal topological sorting determined in step S20 Before inputting the data into the neural network, the sample set is first reordered according to causal topological sorting.

[0078] Specifically, assume the variable order of the original data is a natural index. topological sorting This indicates the logical order in which the variables were generated. In this embodiment, the column vectors of the observation data matrix are arranged according to... Perform a permutation so that the rearranged data vector Satisfy: For any index ,variable All potential causal parent nodes are located in the index range This preprocessing step transforms complex graph structure constraints into simple sequential autoregressive constraints, greatly reducing the complexity of subsequent mask matrix design.

[0079] Step S32: Construct a multi-layer masked affine autoregressive flow model structure.

[0080] like Figure 3 As shown in the figure, the flow model constructed in this application consists of several reversible transformation blocks connected in series, labeled B1 to BN respectively. Each transformation block This represents a masked affine autoregressive transformation.

[0081] The model adopts a process-oriented generation mechanism: input data After the first transformation block Mapping to intermediate state Then, after passing through subsequent transformation blocks, it is finally mapped to a latent noise variable that follows a standard normal distribution. (In mathematics, it is usually represented as) By stacking multiple transformation blocks (e.g., 2 to 5 layers), the model can transform a simple basic distribution into an extremely complex nonlinear data distribution.

[0082] In each transform block Internally, this embodiment uses a masked autoencoder density estimation network (MADE) as the core density estimation network.

[0083] The model has autoregressive properties: that is, the transform block generates the first... When dealing with parameters of multiple variables, you can only "see" the sort order. Previous variables This naturally guarantees the acyclicity of the generated graph in terms of mathematical structure, avoiding the complex calculation of acyclic constraint terms in traditional methods.

[0084] Furthermore, each transform block The formula for the affine transformation to be performed is defined as follows:

[0085] (2)

[0086] in, These are the rearranged input variables. This refers to the transformed output variable. and These are the translation and scaling parameters output by the MADE network mentioned above.

[0087] To ensure the reversibility and numerical stability of the transformation, this embodiment modifies the scaling parameters of the network output. Perform Softplus activation and add a small positive number. (For example ),make sure . Figure 3 In This means that the feature representations obtained after the above series of affine transformations are the final ones. It should approximate the distribution of Gaussian white noise.

[0088] The model is trained using the observed data. This embodiment uses minimizing the regularized negative log-likelihood as the optimization objective. Specifically, the total loss function... It consists of two parts: a data fitting term and a parameter regularization term, and the formula is defined as follows:

[0089] (3)

[0090] in, The total number of observed samples, Indicates the first The data vector of each observation sample is input into the model. This indicates that the mask is an affine mapping of the forward mapping function of the regressive flow model. This represents the set of all learnable network weights and bias parameters in the masked affine autoregressive flow model. It is the probability density function of the standard normal distribution. The second term in the formula is the logarithm of the Jacobian determinant, and the third term is the L1 regularization term for the model parameters, used to induce sparsity in the network weights and suppress overfitting. This is the preset regularization intensity hyperparameter. The Adam optimizer minimizes this objective function, updates the network parameters, and enables the model to accurately fit the structural equations between variables.

[0091] Furthermore, embodiments of this application also provide a causal discovery apparatus that integrates domain knowledge from a large language model with a flow model, used to execute the aforementioned causal discovery method that integrates domain knowledge from a large language model with a flow model. Specifically, as... Figure 4 As shown, the causal discovery device 60, which integrates domain knowledge from a large language model and a flow model, includes: a data acquisition module 61, used to acquire observation data and variable semantic information of the system to be analyzed; a topological ordering module 62, used to use the large language model combined with the variable semantic information and its own domain knowledge to perform logical reasoning on the causal relationships between variables and determine the global causal topological order between variables; a model construction and training module 63, used to construct a masked affine autoregressive flow model based on the global causal topological order and train the model using the observation data to fit the structural equations between variables; a data pruning module 64, used to calculate the Jacobian matrix of the trained autoregressive flow model with respect to the input data, construct a test statistic based on the Jacobian matrix and perform statistical hypothesis testing, and remove statistically insignificant connections according to the test results to obtain a candidate causal graph; and a semantic pruning module 65, used to input the candidate causal graph into the large language model, use the variable semantic information to perform secondary verification on the candidate connections, and remove connections judged as unreasonable to obtain the final sparse causal graph.

[0092] In one exemplary embodiment of this application, the topological ordering module 62 specifically includes: a triplet construction unit, used to extract variable combinations from the variable semantic information, construct triplet query prompts containing variable names and semantic information, and transform the causal relationship judgment between pairs of variables into a local causal relationship judgment problem within triplets; a voting matrix generation unit, used to input the triplet query prompts into a large language model, fill the local causal relationship results output by the large language model into an initialized voting matrix, and accumulate causal confidence; and a loop elimination unit, used to search for and eliminate loop paths in the voting matrix based on a greedy strategy, and generate a globally unique causal topological order.

[0093] In one exemplary embodiment of this application, the model building and training module 63 specifically includes: a data rearrangement unit, used to rearrange the column vectors of the observed data according to the global causal topological sort, so that the rearranged variable sequence satisfies the causal order constraint; a mask setting unit, used to use a masked autoencoder density estimation network (MADE) as the backbone network, generate a mask matrix according to the global causal topological sort, and apply it to the network weights of the flow model, so that the model output depends only on the preceding variables; and a parameter optimization unit, used to construct a total loss function containing negative log-likelihood loss and parameter L1 regularization term, and use Adam to minimize the total loss function to update the model parameters.

[0094] In one exemplary embodiment of this application, the data pruning module 64 is specifically used to: calculate the Jacobian matrix of the affine transformation of the masked affine autoregressive flow model with respect to the input data, and calculate its absolute mean over all observed samples to obtain a global sensitivity matrix; set a fixed significance threshold, retain connections greater than the significance threshold, and remove connections less than the significance threshold.

[0095] In one exemplary embodiment of this application, the semantic pruning module 65 is specifically used to: extract all directed edges in the candidate causal graph, convert each directed edge into a causal proposition described in natural language, input the causal proposition into a large language model, and instruct the large language model to judge the rationality of the proposition; if the large language model determines that the directed edge does not exist after multiple votes, the corresponding directed edge is removed from the candidate causal graph.

[0096] For further implementation details regarding the causal discovery device, please refer to the causal discovery method that integrates domain knowledge from a large language model and a flow model as described above; these details will not be repeated here.

[0097] Figure 5 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 5 As shown, the electronic device may include a processor 71, a memory 72, a communication bus 73, and a communication interface 74. The processor 71, memory 72, and communication interface 74 communicate with each other via the communication bus 73. The processor 71 can call logical instructions in the memory 72 to execute a causal discovery method that integrates domain knowledge from a large language model with a flow model. This method includes: acquiring observation data and variable semantic information of the system to be analyzed; using a large language model combined with the variable semantic information and its own domain knowledge to determine a global causal topological ordering among variables; constructing a masked affine autoregressive flow model based on the global causal topological ordering, and training the model using the observation data to fit the structural equations among variables; calculating the Jacobian matrix of the trained flow model with respect to the input data, constructing a test statistic based on the Jacobian matrix and performing statistical hypothesis testing, performing preliminary pruning of causal connections based on the test results to obtain a candidate causal graph; inputting the candidate causal graph into the large language model, and using semantic information to perform secondary verification and pruning of the candidate connections to obtain the final sparse causal graph.

[0098] Furthermore, the logical instructions in the aforementioned memory 72 can be implemented as software functional units and sold or used as independent products, and can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0099] On the other hand, this application also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the causal discovery method that integrates domain knowledge of a large language model and a flow model provided by the above methods. The method includes: acquiring observation data and variable semantic information of the system to be analyzed; using a large language model combined with the variable semantic information and its own domain knowledge to determine the global causal topological ordering between variables; constructing a masked affine autoregressive flow model based on the global causal topological ordering, and training the model using the observation data to fit the structural equations between variables; calculating the Jacobian matrix of the trained flow model with respect to the input data, constructing a test statistic based on the Jacobian matrix and performing statistical hypothesis testing, performing preliminary pruning of causal connections according to the test results to obtain a candidate causal graph; inputting the candidate causal graph into the large language model, and performing secondary verification and pruning of the candidate connections using semantic information to obtain the final sparse causal graph.

[0100] In another aspect, this application also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, this computer program implements a causal discovery method that integrates domain knowledge from a large language model with a flow model. The method includes: acquiring observation data and variable semantic information of the system to be analyzed; using a large language model in combination with the variable semantic information and its own domain knowledge to determine a global causal topological ordering among variables; constructing a masked affine autoregressive flow model based on the global causal topological ordering, and training the model using the observation data to fit the structural equations among variables; calculating the Jacobian matrix of the trained flow model with respect to the input data, constructing a test statistic based on the Jacobian matrix and performing statistical hypothesis testing, performing preliminary pruning of causal connections based on the test results to obtain a candidate causal graph; inputting the candidate causal graph into the large language model, and using semantic information to perform secondary verification and pruning of the candidate connections to obtain the final sparse causal graph.

[0101] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0102] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0103] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A causal discovery method that integrates domain knowledge from large language models and flow models, characterized in that, include: Step S10: Obtain the observation data and variable semantic information of the system to be analyzed; Step S20: Using a large language model, combined with the semantic information of the variables and their domain knowledge, determine the global causal topological ordering among the variables; Step S30: Construct a masked affine autoregressive flow model based on the global causal topological sorting, so that the generation of any variable in the model depends only on the variable that precedes it in the sorting; train the masked affine autoregressive flow model using the observation data to fit the structural equations between the variables. Step S40: Calculate the Jacobian matrix of the trained streaming model with respect to the observed data, wherein the Jacobian matrix characterizes the functional sensitivity of the variable generation mechanism to the preceding variables; Based on the Jacobian matrix, a test statistic is constructed and a statistical hypothesis test is performed. Based on the test results, statistically insignificant connections are eliminated to obtain a candidate causal graph. Step S50: Input the connections in the candidate causal graph into the large language model, use the semantic information of variables to perform secondary verification on the candidate connections, and remove the connections that are judged to be unreasonable to obtain the final sparse causal graph.

2. The method according to claim 1, characterized in that, Step S20 includes: Step S21: Construct triplet query suggestions containing variable names and semantic information, transforming the causal relationship judgment between pairs of variables into the causal relationship judgment problem within triplets; Step S22: Input the triple query suggestion words into the large language model, fill the local causal relationship results output by the large language model into the initialized voting matrix, and accumulate the causal confidence. Step S23: Use a greedy strategy to eliminate cyclic paths in the voting matrix and generate a global causal topological sort.

3. The method according to claim 1, characterized in that, Step S30 includes: A masked autoencoder density estimation network is used as the backbone network of the flow model, and a mask matrix is ​​generated according to the global causal topological ordering. This matrix is ​​then applied to the neural network to cut off all connections that violate the topological ordering. Define variables The generation process is an affine transformation: ,in In topological order The set of preceding variables, For latent noise variables, and For the nonlinear function fitted to the neural network; The parameters of the neural network are optimized by minimizing the negative log-likelihood function of the observed data.

4. The method according to claim 3, characterized in that, The masked affine autoregressive flow model consists of several invertible transform blocks connected in series, each transform block This represents a masked affine autoregressive transformation; Each transform block The formula for the affine transformation to be performed is defined as follows: （2） in, These are the rearranged input variables. For sorting in Previous variables, The transformed output variable, and These are the translation and scaling parameters output by the mask autoencoder density estimation network described above.

5. The method according to claim 1, characterized in that, Step S40 includes: Calculate the autoregressive flow model transformation Regarding observation data Jacobian matrix ,in, It's about observation data. The partial derivative operator is used to calculate the partial derivative of the model output with respect to small changes in the input variables, transforming... It refers to the forward mapping function of the trained autoregressive flow model, which is the process of mapping the input observation data to the changes in the fitted values ​​of each variable; The global sensitivity matrix is ​​obtained based on the mean absolute value of the Jacobian matrix. ,in Characteristic variables right The average causal strength; Constructing the null hypothesis There is no causal connection between the variables, meaning the corresponding global sensitivity value is zero; A fixed significance threshold is set. Connections with significance values ​​less than the threshold are rejected as null hypotheses, identified as noise connections, and removed. Connections with significance values ​​greater than the threshold are retained, thus forming a preliminary candidate causal graph.

6. The method according to claim 1, characterized in that, Step S50 includes: Extract all node pairs with directed edges from the candidate causal graph and input the node pairs into the large language model; Instructions for large language models based on nodes and Based on the semantic information of the variables and domain knowledge, determine whether the relationship between the variables is a direct causal relationship, an indirect causal relationship, or neither; The same question was asked to the large language model multiple times, and the final result was obtained through voting. If the large language model determines that there is no direct or indirect causal relationship between two nodes, then a pruning operation is performed: the corresponding directed edges are removed from the candidate causal graph, and the final sparse causal graph is output.

7. A causal discovery device that integrates domain knowledge from large language models and flow models, characterized in that, include: The data acquisition module is used to acquire observation data and variable semantic information of the system to be analyzed; The topological ordering module is used to determine the global causal topological order of variables by combining the semantic information of the variables with its own domain knowledge using a large language model. The model building and training module is used to build a masked affine autoregressive flow model based on the global causal topological sorting, and to train the model using observation data. The data pruning module is used to calculate the Jacobian matrix of the flow model with respect to the input data, construct test statistics based on the Jacobian matrix and perform statistical hypothesis testing, and remove statistically insignificant connections according to the test results to obtain candidate causal graphs; The semantic pruning module is used to input candidate causal graphs into a large language model, and to perform secondary verification and pruning on candidate connections using variable semantic information, eliminating connections that are judged to be unreasonable, and obtaining the final sparse causal graph.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the causal discovery method that integrates domain knowledge of a large language model and a flow model as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the causal discovery method that integrates domain knowledge of a large language model and a flow model as described in any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the causal discovery method that integrates domain knowledge of a large language model and a flow model as described in any one of claims 1 to 6.

Images