Multi-model cross-validation iterative noise label cleaning method and system

Abstract

The application discloses a multi-model cross-validation iterative noise label cleaning method and system, and is suitable for a causal inference task, wherein the label cleaning method comprises the following steps: 1, collecting student education business index to construct an original data set; 2, dividing the original data set into K non-overlapping subsets by adopting a cross-division mode; 3, each subset is taken as a to-be-cleaned subset, and the remaining K-1 subsets are taken as training sets to perform K-group cross-validation experiments; in each experiment, a causal inference model in a model set is trained by using the training set, and the to-be-cleaned subset is subjected to causal label prediction by using the trained causal inference model; 4, the causal relationship label of the sample in the original data set is updated based on the predicted causal label under the causal consistency constraint; 5, steps 3-4 are repeatedly executed until an iteration termination condition is met, and a data set after label cleaning is obtained. The method can effectively identify and clean the multi-classification noise label in the causal inference observation sample.

Description

Technical Field

[0001] This invention belongs to the field of causal reasoning technology in smart education, specifically involving a multi-model cross-validation iterative noise label cleaning method and system. Background Technology

[0002] Causal reasoning is a technique that infers the true causal relationships between variables by analyzing their statistical characteristics and structural dependencies. Its core objective is to reveal the causal mechanisms in the data generation process, in order to determine whether a change in one variable directly leads to a change in another. Causal reasoning is widely used in smart education, such as analyzing whether the mastery of knowledge points affects the answer to questions; revealing these causal mechanisms can provide a scientific basis for personalized decision-making and intervention.

[0003] Causal relationships among multiple variables are typically modeled and represented using structural causal models (SCMs) and their corresponding causal graphs. A causal graph is a directed graph structure where nodes represent different variables, and directed edges between nodes represent direct causal relationships between variables. It can abstract the operation of complex systems as a network of causal relationships between variables. For example... Figure 1 The diagram illustrates a simple causal relationship. For a given pair of variables (A, B), the label is typically defined as a multi-class classification problem: category 0 indicates no direct causal relationship, category 1 indicates positive causality (A→B in the diagram), and category 2 indicates negative causality (A←B). The labels essentially correspond to the existence and directionality of directed edges in the causal graph.

[0004] In data-driven causal discovery tasks based on machine learning (including deep neural networks), training data needs to be constructed to enable the model to learn the true causal mechanisms between variables. In practice, the sample labels in the training data are usually constructed by combining domain expert knowledge and historical data statistics. However, due to the incompleteness of observed data and the difficulty in fully modeling potential confounding factors, the constructed causal relationship labels inevitably contain noise. When training the model with severely noisy training data, it is easy to overfit to incorrect labels, leading to incorrect causal direction judgments and distortion of the overall structure.

[0005] Currently, there is no effective method for cleaning up label noise in training data for causal inference tasks. Label noise cleaning methods are mainly concentrated in traditional supervised learning (such as image classification), typically using the prediction confidence of a single model for sample selection, or employing robust loss functions for reweighting. However, directly applying existing methods to causal discovery tasks has significant drawbacks: First, the distribution of causal features is often more complex than in ordinary classification tasks (for example, the strong correlation between A and B in the above example masks the true causal asymmetry), and a single denoising model is prone to "cognitive blind spots" and "confirmation bias," falling into a self-confirming loop by "memorizing" incorrect labels, resulting in increasingly contaminated noise. Second, existing denoising methods are mostly non-iterative, single-step filtering methods, which are ill-suited to address the cascading effects caused by directional and redundant errors in causal networks. Summary of the Invention

[0006] Purpose of the invention: The technical problem to be solved by the present invention is to address the shortcomings of the existing technology by providing a multi-model cross-validation iterative noise label cleaning method. This method can effectively identify and clean multi-class noise labels in causal inference observation samples, overcome the confirmation bias of a single model, improve label quality without relying on large-scale manual relabeling, and thus improve the accuracy and stability of causal structure learning.

[0007] To address the aforementioned technical problems, this invention discloses a multi-model cross-validation iterative noise label cleaning method, applicable to causal inference tasks, comprising the following steps:

[0008] S1. Collect student education business indicators to construct an original dataset. Each sample in the dataset includes a first variable, a second variable, a relationship feature vector between the first and second variables, and a causal relationship label. The first and second variables in a sample are different types of student education business indicators for the same student.

[0009] S2. Divide the original dataset into K non-overlapping subsets using a cross-partitioning method;

[0010] S3. Each subset is used as the subset to be cleaned, and the remaining K-1 subsets are used as the training set. K sets of cross-validation experiments are conducted. In each set of cross-validation experiments, the training set consisting of K-1 subsets is used to train multiple causal inference models in the model set M. The trained multiple causal inference models are used to predict causal labels for the samples in the subset to be cleaned. The model set M includes at least 3 causal inference models.

[0011] S4. The causal labels predicted by S3 are used to update the causal relationship labels of each sample in the original dataset under the causal consistency constraint.

[0012] S5. Repeat steps S3-S4 until the iteration termination condition is met, and obtain the labeled and cleaned dataset.

[0013] Furthermore, in step S1, the initial values ​​of the causal relationship labels of the samples in the original dataset are determined using a heuristic algorithm.

[0014] Furthermore, the relationship feature vector between the first variable and the second variable consists of statistical distribution characteristics, temporal sequence characteristics, correlation characteristics, conditional independence test characteristics, regression residual characteristics, and mutual information characteristics between the variables.

[0015] Furthermore, the multiple causal inference models in the model set M employ at least two types of modeling mechanisms.

[0016] Furthermore, any two causal inference models in the model set M satisfy: ,in The difference between the predictions of the two causal inference models on the validation subset is given. This is a preset difference threshold.

[0017] Furthermore, step S4 specifically includes:

[0018] S41. Generate candidate labels for each sample based on the causal labels predicted in S3, and the candidate label for the i-th sample. for: ;

[0019] Where L represents the number of causal inference models in the model set M. For the first The prediction of a causal inference model for the i-th sample, ; , which is a numerical representation of causal relationship labels; N is the number of samples in the original dataset;

[0020] S42. Perform causal consistency constraint verification on the candidate labels of each sample. If the causal consistency constraint verification is passed, update the causal relationship label of the sample to the candidate label; otherwise, keep the causal relationship label of the sample unchanged.

[0021] The causal consistency constraint is verified as follows:

[0022] (421) Causal Direction Consistency Verification: The causal inference model in the statistical model set M predicts the i-th sample as follows: proportion ,when The consistency of causal direction is verified at that time; among which The preset directional consistency threshold;

[0023] (422) Construct a temporary causal graph: Generate the current causal graph based on the current causal relationship labels of the samples in the dataset, map the candidate label of the i-th sample to the candidate directed edge ei and insert it into the current causal graph to obtain a temporary causal graph;

[0024] (423) Verification of the legitimacy of causal structure:

[0025] Check whether a directed cycle is formed in the temporary causal graph and whether the candidate directed edge ei conflicts with the prior forbidden edge; if there is no directed cycle and no forbidden edge conflict, then the causal structure validity is verified.

[0026] The determination of whether a candidate directed edge ei conflicts with a prior forbidden edge is as follows: a set of forbidden edges B is constructed in advance, and the elements in B are variable pairs that do not have a causal direction; if ei∈B, then the candidate directed edge is determined to conflict with the prior forbidden edge.

[0027] (424) Consistency verification of chained relationships: Compare the support of direct edges in the temporary causal graph. Support of multi-hop chain paths ;

[0028] direct edge support The calculation is based on the voting ratio of the candidate directed edge ei in the model set, specifically as follows:

[0029] ;

[0030] Where L represents the number of models. For indicator functions, ;

[0031] The support of the multi-hop chain path The path support is the maximum value of the path support among all chain paths connecting the first and second variables of the i-th sample in the temporary causal graph. The path support is the minimum or average value of the direct support of each edge in the chain path connecting the first and second variables of the i-th sample.

[0032] When a candidate directed edge ei is in the same direction as an existing chain path, it is determined that there is no chain relationship conflict; when a candidate directed edge ei is not in the same direction as an existing chain path, it is determined that there is a chain relationship conflict.

[0033] If there is no chain relationship conflict or Then the consistency verification is passed through the chain relationship;

[0034] (425) Verification of conditional independence:

[0035] Construct a set of condition variables , It consists of the adjacent variables and common cause variables of the first and second variables of the i-th sample in the current causal graph; in the set of condition variables Under the given conditions, perform a conditional independence test on the first and second variables of the i-th sample to obtain the significance probability. ;

[0036] If the candidate label indicates the existence of a direct causal edge, and The conditional independence test is passed; if the candidate label indicates that there is no direct causal edge, and The conditional independence test was passed. The significance level is determined by the following criteria: when the causal direction consistency verification, causal structure legality verification, chain relationship consistency verification, and conditional independence verification all pass, the causal consistency constraint verification is considered passed.

[0037] Furthermore, the iteration termination condition in step S5 is:

[0038] After step S4 is completed, calculate the proportion of causal relationship label changes for samples in the dataset. and based on edge set similarity and directional consistency stability Comprehensive causal structure stability The calculation steps are as follows:

[0039] Let the set of edges of the causal graph obtained after the t-th iteration be . , No. The set of edges in the causal graph obtained after round iteration is Edge set similarity Jaccard similarity calculation:

[0040] ; To calculate the number of elements in a set;

[0041] Directional consistency stability Calculated based on the proportion of the direction of the common edge in two adjacent rounds:

[0042] ;

[0043] Where e is The edges of the causal graph in the middle, It is the direction of edge e in the t-th iteration, if If it is empty, then let ;

[0044] Overall causal structure stability Calculated using a weighted average of edge set similarity and orientation consistency stability:

[0045] Where λ∈[0,1] is the weight parameter;

[0046] The iteration terminates when either condition 1 or condition 2 is met.

[0047] Condition 1 is: the label change ratio is lower than the first threshold and the overall causal structure stability is higher than the second threshold for W consecutive rounds; W is a preset structural stability judgment parameter;

[0048] Condition 2 is: reaching the maximum number of iterations.

[0049] On the other hand, this invention discloses a multi-model cross-validation iterative noise label cleaning system, which is suitable for causal inference tasks and includes:

[0050] The original dataset construction module 1 is used to collect student education business indicators to construct the original dataset. Each sample in the dataset includes a first variable, a second variable, a relationship feature vector between the first variable and the second variable, and a causal relationship label. The first variable and the second variable in a sample are different types of student education business indicators for the same student.

[0051] Cross-partitioning module 2 is used to divide the original dataset into K non-overlapping subsets using a cross-partitioning method;

[0052] Cross-validation module 3 is used to take each subset as the subset to be cleaned and the remaining K-1 subsets as the training set to conduct K sets of cross-validation experiments. In each set of cross-validation experiments, the training set consisting of K-1 subsets is used to train multiple causal inference models in the model set M, and the trained multiple causal inference models are used to predict causal labels for samples in the subset to be cleaned. The model set M includes at least 3 causal inference models.

[0053] Sample label update module 4 is used to update the causal relationship label of each sample in the original dataset based on the causal label predicted by cross-validation experiment module 3 under the constraint of causal consistency.

[0054] The iteration control module 5 is used to control the operation of the cross-validation experiment module 3 and the sample label update module 4 according to the iteration termination condition, so as to obtain the labeled cleaned dataset.

[0055] Furthermore, it also includes a model set effective difference verification module 6, which is used to calculate the prediction difference rate of any two causal inference models in the model set M. When the prediction difference rate is less than the pre-screening difference threshold, one of the two causal inference models is deleted.

[0056] Furthermore, the sample label update module 4 updates the causal relationship label of each sample in the original dataset based on the causal label predicted by the cross-validation experiment module 3 under the constraint of causal consistency, specifically including:

[0057] S41. Generate candidate labels for each sample based on the causal labels predicted in S3, and the candidate label for the i-th sample. for: ;

[0058] Where L represents the number of causal inference models in the model set M. For the first The prediction of a causal inference model for the i-th sample, ; , which is a numerical representation of causal relationship labels; N is the number of samples in the original dataset;

[0059] S42. Perform causal consistency constraint verification on the candidate labels of each sample. If the causal consistency constraint verification is passed, update the causal relationship label of the sample to the candidate label; otherwise, keep the causal relationship label of the sample unchanged.

[0060] The causal consistency constraint is verified as follows:

[0061] (421) Causal Direction Consistency Verification: The causal inference model in the statistical model set M predicts the i-th sample as follows: proportion ,when The consistency of causal direction is verified at that time; among which The preset directional consistency threshold;

[0062] (422) Construct a temporary causal graph: Generate the current causal graph based on the current causal relationship labels of the samples in the dataset, map the candidate label of the i-th sample to the candidate directed edge ei and insert it into the current causal graph to obtain a temporary causal graph;

[0063] (423) Verification of the legality of causal structure: Check whether a directed cycle is formed in the temporary causal graph and whether the candidate directed edge ei conflicts with the prior prohibited edge; if there is no directed cycle and no prohibited edge conflict, the verification of the legality of causal structure is passed.

[0064] The determination of whether a candidate directed edge ei conflicts with a prior forbidden edge is as follows: a set of forbidden edges B is constructed in advance, and the elements in B are variable pairs that do not have a causal direction; if ei∈B, then the candidate directed edge is determined to conflict with the prior forbidden edge.

[0065] (424) Consistency verification of chained relationships: Compare the support of direct edges in the temporary causal graph. Support of multi-hop chain paths ;

[0066] direct edge support The calculation is based on the voting ratio of the candidate directed edge ei in the model set, specifically as follows:

[0067]

[0068] Where L represents the number of models. For indicator functions, ;

[0069] The support of the multi-hop chain path The path support is the maximum value of the path support among all chain paths connecting the first and second variables of the i-th sample in the temporary causal graph. The path support is the minimum or average value of the direct support of each edge in the chain path connecting the first and second variables of the i-th sample.

[0070] When a candidate directed edge ei is in the same direction as an existing chain path, it is determined that there is no chain relationship conflict; when a candidate directed edge ei is not in the same direction as an existing chain path, it is determined that there is a chain relationship conflict.

[0071] If there is no chain relationship conflict or Then the consistency verification is passed through the chain relationship;

[0072] (425) Verification of conditional independence: Constructing a set of condition variables , It consists of the adjacent variables and common cause variables of the first and second variables of the i-th sample in the current causal graph; in the set of condition variables Under the given conditions, perform a conditional independence test on the first and second variables of the i-th sample to obtain the significance probability. ;

[0073] If the candidate label indicates the existence of a direct causal edge, and The conditional independence test is passed; if the candidate label indicates that there is no direct causal edge, and The conditional independence test was passed. The significance level;

[0074] When the causal direction consistency verification, causal structure legality verification, chain relationship consistency verification, and conditional independence verification all pass, the causal consistency constraint verification is deemed to have passed.

[0075] Beneficial effects: The multi-model cross-validation iterative noise label cleaning method and system disclosed in this invention have the following advantages:

[0076] 1. Breaking the single-model confirmation bias: This invention combines cross-validation with multi-model differentiation, making label updates strictly dependent on a group of models that have "never seen the sample" and have "different underlying mechanisms". This fundamentally blocks the "rote memorization" of incorrect labels by models in traditional denoising methods, which leads to a vicious cycle of self-confirmation.

[0077] 2. Introducing multi-dimensional causal logic constraints to prevent "more washing, more chaos": Existing image and text denoising techniques often disrupt the global graph structure when directly transferred to the causal domain. This invention adds four strict constraints after the voting mechanism: directional consistency, acyclic validity, chain transitivity, and conditional independence. This not only ensures the accuracy of local label correction but also safeguards the topological physical meaning of the global causal graph, preventing logical paradoxes (such as causal loops).

[0078] 3. No need for expensive manual relabeling: This invention belongs to the category of fully data-driven self-supervised cleaning. With the spiral improvement of label quality in a multi-round iterative mechanism, it can fully extract the potential logical information of the dataset itself, automatically smooth out various directional, omission, and redundant noise, and greatly reduce the data cleaning cost in causal applications. Attached Figure Description

[0079] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments, and the advantages of the present invention in the above and / or other aspects will become clearer.

[0080] Figure 1 This is a diagram illustrating a simple cause-and-effect relationship.

[0081] Figure 2 This is a flowchart of the multi-model cross-validation iterative noise label cleaning method disclosed in this invention;

[0082] Figure 3 This is a schematic diagram illustrating the accuracy variation of the method of the present invention and the prior art under different noise ratios;

[0083] Figure 4 This is a schematic diagram illustrating the accuracy changes of the method of the present invention and existing technologies during the iterative cleaning process;

[0084] Figure 5 This is a schematic diagram illustrating the changes in structural stability during the iterative process of the method of the present invention and the comparative method;

[0085] Figure 6 This is a schematic diagram of the composition of the multi-model cross-validation iterative noise label cleaning system disclosed in this invention. Detailed Implementation

[0086] This invention discloses an iterative noise label cleaning method for multi-model cross-validation, which is applicable to causal inference tasks, such as... Figure 2As shown, the steps include:

[0087] S1. Collect student education business indicators to construct an original dataset. Each sample in the dataset includes a first variable, a second variable, a relationship feature vector between the first and second variables, and a causal relationship label. The first and second variables in a sample are different types of student education business indicators for the same student.

[0088] In this embodiment, student education business metrics are derived from logs and academic records of online learning platforms or smart education systems, including but not limited to: online teaching video viewing time, in-class quiz scores, forum interaction frequency, homework completion rate, and exam scores. The original dataset is represented as follows:

[0089] ;

[0090] in, Represents the total number of samples. For the first One sample, For the input features, where As the first variable, As the second variable, for and The relational feature vector; It includes at least one or more of the following: statistical distribution characteristics between variables, temporal sequence characteristics, correlation characteristics, conditional independence test characteristics, regression residual characteristics, and mutual information characteristics. In this embodiment, the relationship feature vector... It is obtained by concatenating multiple sub-features, and is represented as:

[0091]

[0092] in, For statistical distribution characteristics, including and The mean, variance, standard deviation, skewness, kurtosis, and distributional variation indices; As a temporal sequence feature, through comparison Timestamps of corresponding educational behaviors or indicators and Timestamps of corresponding educational behaviors or indicators Obtain, for example, calculate The sign, mean, or time lag length; for example, when Indicates "video viewing duration". When representing the "homework completion rate", the timestamps from the online learning system logs indicating when students started or finished watching videos are extracted as data. and the timestamps of students submitting their homework as Calculate the order and time interval between the two; if the video viewing time is earlier than the assignment submission time, then this temporal feature supports... right The positive impact of the judgment. For relevance features, such as and The Pearson correlation coefficient, Spearman correlation coefficient, or Kendall correlation coefficient; For conditional independence tests, the characteristics include the conditional independence test under a given set of conditional variables Z. and Perform a conditional independence test and obtain the test statistic and significance probability value; To establish the characteristics of the regression residuals, separate [systems / facilities] were established. and Two regression models, Indicated by For input, with The output is the regression function; Indicated by For input, with The output is the regression function; Indicates in the assumption → In the direction, the model predicts The residuals generated during the process; Indicates in the assumption → In the direction, the model predicts The residuals generated during the process; calculate the residuals in both directions. and Composition of regression residual characteristics ; Mutual information features, including and The mutual information value.

[0093] For the first The causal relationship labels for each sample, indicating the relationship between variables. The causal relationship. In this embodiment, the label uses a three-category representation: This indicates that there is no direct causal relationship. This indicates the existence of positive causality. ); This indicates the existence of reverse causality ( ). The initial value is determined using a heuristic algorithm or prior knowledge.

[0094] The heuristic algorithm determines the causal relationship between variable pair (u, v) in the following steps: First, calculate the correlation, time sequence index and conditional independence test index between u and v; then, make a causal relationship judgment: (1) If the observation time of u is earlier than that of v, and the correlation between the two is greater than the preset correlation threshold, and they are not independent under the preset conditional variable, then it is determined that there is positive causality, and the initial causal relationship label is marked as 1, indicating u→v; (2) If the observation time of v is earlier than that of u, and the correlation between the two is greater than the correlation threshold, and they are not independent under the preset conditional variable, then it is determined that there is reverse causality, and the initial causal relationship label is marked as 2, indicating v→u; (3) If the correlation between the two is lower than the preset correlation threshold, or the conditional independence test result shows that the two are independent under the given conditional variable, then it is determined that there is no direct causal relationship, and the initial causal relationship label is marked as 0.

[0095] Due to observational biases or limitations of heuristic algorithms, such as incorrect orientation, omissions, or redundancy, The initial values ​​are noisy, such as a true causal mechanism (e.g., "viewing time" positively affects "exam score", which should be marked as 1) being incorrectly marked as unrelated (marked as 0) or reverse causality (marked as 2).

[0096] S2, Take the original dataset According to the preset cross-validation fold number Divided into Non-overlapping subsets:

[0097] , , ;

[0098] in, Indicates the first A subset This represents the cross-validation fold number. The specific partitioning process is as follows: First, the dataset... All samples were randomly rearranged using a fixed random seed to ensure reproducible partitioning results. Secondly, a stratified sampling strategy was employed, based on sample labels. The categories are evenly distributed to the samples. In each subset, the label distribution ratio is kept consistent with that of the original dataset.

[0099] Cross-validation folds Determined based on dataset size and category distribution, satisfying... , In this embodiment, the category is a set of categories. , For category The number of samples. When the number of samples in the dataset is small or unevenly distributed, take... When the dataset has a large sample size and the distribution of samples of each type is relatively uniform, take .

[0100] S3. Each subset is used as the subset to be cleaned, and the remaining K-1 subsets are used as the training set. K sets of cross-validation experiments are conducted. In each set of cross-validation experiments, the training set consisting of K-1 subsets is used to train multiple causal inference models in the model set M. The trained multiple causal inference models are used to predict causal labels for the samples in the subset to be cleaned. The model set M includes at least 3 causal inference models.

[0101] To avoid confirmation bias from homogeneous models, the causal inference models in the model set M must be selected from at least two different modeling mechanisms:

[0102] (1) Constraint-based causal discovery model: Specifically, the PC (Peter-Clark) algorithm or the FCI (Fast Causal Inference) algorithm is used to perform skeleton search and direction determination through conditional independence tests. The PC algorithm is based on the causal discovery method proposed by Spirtes, and the details can be found in the literature: Spirtes P., Glymour C., Scheines R. Causation, Prediction, and Search. MIT Press, 2000. The FCI algorithm is based on Spirtes' Fast Causal Inference method in the presence of latent variables and selection bias, and the details can be found in the literature: Spirtes P. An Anytime Algorithm for Causal Inference. Proceedings of the Eighth International Workshop on Artificial Intelligence and Statistics, 2001;

[0103] (2) Causal discovery model based on rating search: Specifically, the GES (Greedy Equivalence Search) algorithm is adopted to search for the optimal structure in Markov equivalence classes by optimizing rating functions such as BIC or AIC; referring to the greedy search structure learning method based on equivalence class space proposed by Chickering, the details can be found in the literature: Chickering DMOptimal Structure Identification With Greedy Search. Journal of Machine Learning Research, 3:507–554, 2002;

[0104] (3) Causal direction discrimination model based on function fitting, residual independence, or non-Gaussianity: Specifically, the LiNGAM (Linear Non-Gaussian Acyclic Model) algorithm or the Additive Noise Model (ANM) algorithm is adopted; among which, LiNGAM refers to the linear non-Gaussian acyclic model proposed by Shimizu et al., and the details can be found in the literature: Shimizu S., Hoyer PO, Hyvärinen A., Kerminen A. A LinearNon-Gaussian Acyclic Model for Causal Discovery. Journal of Machine Learning Research, 7:2003–2030, 2006; the additive noise model refers to the nonlinear additive noise causal discovery method proposed by Hoyer et al., and the details can be found in the literature: Hoyer PO, Janzing D., Mooij JM, Peters J., Schölkopf B. Nonlinear Causal Discovery with Additive Noise Models. Advances in Neural Information Processing Systems, 2008;

[0105] (4) Causal classification models based on neural networks or representation learning: Specifically, the Notears algorithm (based on acyclic constraint model with continuous optimization) or an end-to-end causal discovery network based on graph neural networks (GNN) is adopted. Referencing the continuous optimization acyclic constraint structure learning method proposed by Zheng et al., details can be found in the following literature: Zheng X., Aragam B., Ravikumar P., Xing EP DAGs with NO TEARS: Continuous Optimization for Structure Learning. Advances in Neural Information Processing Systems, 2018.

[0106] To ensure the diversity of elements in the model set, this embodiment introduces a diversity verification mechanism: for any two models in the model set M... and The prediction difference rate between the two is calculated on the validation subset. ;when At that time, the judgment model and If there is valid difference, they can be included in the model set; otherwise, they will be deleted. and one of the. The preset difference threshold is used; the larger the value, the more stringent the judgment of effective differences. The preferred value is [value to be filled in]. In this embodiment, the validation subset consists of multiple samples randomly selected from the original dataset.

[0107] In the In the group cross-validation experiment, the first Subset As the set to be cleaned, the rest The training set consists of subsets. :

[0108] The backslash '\' represents the subtraction operation between sets, that is, subtracting set D from set D. ;

[0109] In the training set For each causal inference model in the training model set M, the current model set is obtained: The number of models Using the current model set For the subset to be cleaned that did not participate in this round of training Make predictions. For the sample Obtain its predicted label set under L causal inference models:

[0110]

[0111] in For the first The prediction of the i-th sample by a causal inference model; ;

[0112] The prediction results effectively isolate the model from overfitting its own training data and objectively reflect the true matching degree between sample features and labels.

[0113] After the K-group cross-validation experiment, each sample in the dataset obtained a set of predicted labels output by L causal inference models.

[0114] S4. Based on the causal labels predicted in S3, update the causal relationship labels for each sample in the original dataset under the constraint of causal consistency; specifically including:

[0115] S41. Based on the causal labels predicted in S3, a majority voting strategy is used to generate candidate labels for each sample. The candidate label for the i-th sample is... for: ;

[0116] Where L represents the number of causal inference models in the model set M. For the first The prediction of a causal inference model for the i-th sample, ; , which is a numerical representation of causal relationship labels; N is the number of samples in the original dataset;

[0117] S42. Perform causal consistency constraint verification on the candidate labels of each sample. If the causal consistency constraint verification is passed, update the causal relationship label of the sample to the candidate label; otherwise, keep the causal relationship label of the sample unchanged.

[0118] The causal consistency constraint is verified as follows:

[0119] (421) Causal Direction Consistency Verification: The causal inference model in the statistical model set M predicts the i-th sample as follows: proportion ,when The consistency of causal direction is verified at that time; among which The preset directional consistency threshold;

[0120] (422) Construct a temporary causal graph: Generate the current causal graph based on the current causal relationship labels of the samples in the dataset, map the candidate label of the i-th sample to the candidate directed edge ei and insert it into the current causal graph to obtain a temporary causal graph;

[0121] (423) Verification of the legality of causal structure: Check whether a directed cycle is formed in the temporary causal graph and whether the candidate directed edge ei conflicts with the prior prohibited edge; if there is no directed cycle and no prohibited edge conflict, the verification of the legality of causal structure is passed.

[0122] The determination of whether a candidate directed edge ei conflicts with a prior forbidden edge is as follows: Based on educational psychology, temporal sequence, and domain knowledge, a set of forbidden edges B is pre-constructed, where the elements of B are variable pairs without causal direction; if ei∈B, then the candidate directed edge is determined to conflict with the prior forbidden edge; otherwise, the candidate directed edge does not conflict with the prior forbidden edge.

[0123] (424) Consistency verification of chained relationships: Compare the support of direct edges in the temporary causal graph. Support of multi-hop chain paths ;

[0124] direct edge support The calculation is based on the voting ratio of the candidate directed edge ei in the model set, specifically as follows:

[0125]

[0126] Where L represents the number of models. For indicator functions, ;

[0127] The support of the multi-hop chain path The path support is the maximum value of the path support among all chain paths connecting the first and second variables of the i-th sample in the temporary causal graph. The path support is the minimum or average value of the direct support of each edge in the chain path connecting the first and second variables of the i-th sample.

[0128] If there exists in the temporary causal graph →q1→q2⋯→ The chain path 1, and →w1→w2⋯→ Given a chain path 1, calculate the direct support of each edge in chain path 1, and take the minimum or average value as the path support r1 of chain path 1; calculate the direct support of each edge in chain path 2, and take the minimum or average value as the path support r2 of chain path 2; then... The maximum value between r1 and r2 is max(r1,r2).

[0129] When a candidate directed edge ei is in the same direction as an existing chain path, it is determined that there is no chain relationship conflict; when a candidate directed edge ei is not in the same direction as an existing chain path, it is determined that there is a chain relationship conflict.

[0130] If there is no chain relationship conflict or Then the consistency verification is passed through the chain relationship;

[0131] (425) Verification of conditional independence: Constructing a set of condition variables , It consists of the adjacent variables and common cause variables of the first and second variables of the i-th sample in the current causal graph;

[0132] In the set of condition variables Under the given conditions, perform a conditional independence test on the first and second variables of the i-th sample to obtain the significance probability. ;

[0133] If candidate tags A value of 1 or 2 indicates the existence of a direct causal edge. This indicates that the first and second variables of the i-th sample are... They are not independent under certain conditions; this is verified by conditional independence. This indicates that the first and second variables of the i-th sample are... They are independent under the given conditions, and do not pass the conditional independence test;

[0134] If candidate tags A value of 0 indicates the absence of a direct causal edge. The conditional independence test was passed. The significance level; if If it does not pass the conditional independence test, then it will not pass the conditional independence test. The significance level is indicated by .

[0135] Use respectively The above four verification results are represented by 1 indicating successful verification and 0 indicating failed verification. A comprehensive verification result is defined. If and only if When all four validation criteria—causal direction consistency, causal structure validity, chain relationship consistency, and conditional independence—pass, the causal consistency constraint validation is deemed successful, and the causal relationship label of the i-th sample is updated to a candidate label. Otherwise, leave the original label unchanged.

[0136] After step S4 is completed, a round of cross-cleaning has been performed on each sample in the dataset, resulting in the updated dataset for the current round. This serves as the input for the next round; t represents the iteration number, starting from 0. After step S4 is completed, t is incremented by one.

[0137] S5. Repeat steps S3-S4 until the iteration termination condition is met, resulting in the labeled and cleaned dataset. The iteration termination condition is:

[0138] After step S4 is completed, calculate the proportion of causal relationship label changes for samples in the dataset. and based on edge set similarity and directional consistency stability Comprehensive causal structure stability The calculation steps are as follows:

[0139] Let the set of edges of the causal graph obtained after the t-th iteration be . , No. The set of edges in the causal graph obtained after round iteration is Edge set similarity Jaccard similarity calculation:

[0140] ; To calculate the number of elements in a set;

[0141] Directional consistency stability Calculated based on the proportion of the direction of the common edge in two adjacent rounds:

[0142] ;

[0143] Where e is The edges of the causal graph in the middle, It is the direction of e in the t-th iteration, if If it is empty, then let . The calculation formula measures the proportion of causal edges that are in the same direction as those shared by round t and round t-1. The numerator is the number of causal edges e that are shared by round t and round t-1 and have the same direction, and the denominator is the number of shared causal edges.

[0144] Overall causal structure stability Calculated using a weighted average of edge set similarity and orientation consistency stability:

[0145] Where λ∈[0,1] is the weight parameter;

[0146] The iteration terminates when either condition 1 or condition 2 is met.

[0147] Condition 1 is: the label change ratio is lower than the first threshold. Furthermore, the stability of the overall causal structure is higher than the second threshold for W consecutive rounds. W is the preset structural stability judgment parameter; that is... and ;

[0148] Condition 2 is: reaching the maximum number of iterations.

[0149] Figure 3 This diagram illustrates the accuracy variations of the method of this invention and existing technologies under different noise ratios. The original noisy training data represents the initial causal relationship labels in the original dataset, exhibiting the lowest accuracy due to the highest noise level. Single-model cleaning uses a single causal inference model to clean the labels of the original dataset, resulting in improved accuracy compared to the original noisy training. Multi-model voting uses multiple causal inference models to clean the labels of the original dataset and employs a majority voting strategy to update sample labels, achieving improved accuracy compared to the single causal inference model. The method of this invention further improves the accuracy of label cleaning by employing causal consistency constraints for verification based on multiple models. Figure 4 This diagram illustrates the accuracy changes of the method of this invention and existing technologies during the iterative cleaning process. As the number of iterations increases, the accuracy improvement of the method of this invention is greater than that of single-model cleaning and multi-model voting methods, ultimately achieving the optimal accuracy. Figure 5This diagram illustrates the structural stability changes of the proposed method and the comparative methods during the iteration process. As the number of iterations increases, while the single-model cleaning and multi-model voting methods gradually stabilize, their stabilization process exhibits significant fluctuations and ultimately results in low stability. This indicates that they may still introduce directional conflicts or structural inconsistencies during the iteration process. In contrast, the proposed method, based on multi-model voting, further introduces verifications such as causal direction consistency, structural legitimacy, and chain relationship consistency. This ensures that label updates not only depend on model prediction results but are also constrained by the global causal structure, thus enabling faster attainment of a stable state and convergence at a higher stability level.

[0150] This invention also discloses a multi-model cross-validation iterative noise label cleaning system, which is suitable for causal inference tasks, such as... Figure 6 As shown, it includes:

[0151] The original dataset construction module 1 is used to collect student education business indicators to construct the original dataset. Each sample in the dataset includes a first variable, a second variable, a relationship feature vector between the first variable and the second variable, and a causal relationship label. The first variable and the second variable in a sample are different types of student education business indicators for the same student.

[0152] Cross-partitioning module 2 is used to divide the original dataset into K non-overlapping subsets using a cross-partitioning method;

[0153] Cross-validation module 3 is used to take each subset as the subset to be cleaned and the remaining K-1 subsets as the training set to conduct K sets of cross-validation experiments. In each set of cross-validation experiments, the training set consisting of K-1 subsets is used to train multiple causal inference models in the model set M, and the trained multiple causal inference models are used to predict causal labels for samples in the subset to be cleaned. The model set M includes at least 3 causal inference models.

[0154] The sample label update module 4 is used to update the causal relationship label of each sample in the original dataset under the causal consistency constraint based on the causal label predicted by the cross-validation experiment module 3 in step S4.

[0155] The iteration control module 5 is used to control the operation of the cross-validation experiment module 3 and the sample label update module 4 according to the iteration termination condition, so as to obtain the labeled cleaned dataset.

[0156] To ensure the diversity of elements in the model set, a model set effective difference verification module 6 is also included, which is used to calculate the prediction difference rate of any two causal inference models in the model set M. When the prediction difference rate is less than the pre-screening difference threshold, one of the two causal inference models is deleted.

[0157] This invention provides a multi-model cross-validation iterative noise label cleaning approach and method suitable for causal reasoning tasks. Many methods and approaches exist for implementing this technical solution; the above description is merely a preferred embodiment of the invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this invention, and these improvements and modifications should also be considered within the scope of protection of this invention. All components not explicitly stated in this embodiment can be implemented using existing technologies.

Claims

1. A multi-model cross-validation iterative noise label cleaning method, applicable to causal inference tasks, characterized in that... Including the following steps: S1. Collect student education business indicators to construct an original dataset. Each sample in the dataset includes a first variable, a second variable, a relationship feature vector between the first and second variables, and a causal relationship label. The first and second variables in a sample are different types of student education business indicators for the same student. S2. Divide the original dataset into K non-overlapping subsets using a cross-partitioning method; S3. Each subset is used as the subset to be cleaned, and the remaining K-1 subsets are used as the training set. K sets of cross-validation experiments are conducted. In each set of cross-validation experiments, the training set consisting of K-1 subsets is used to train multiple causal inference models in the model set M. The trained multiple causal inference models are used to predict causal labels for the samples in the subset to be cleaned. The model set M includes at least 3 causal inference models. S4. The causal labels predicted by S3 are used to update the causal relationship labels of each sample in the original dataset under the causal consistency constraint. S5. Repeat steps S3-S4 until the iteration termination condition is met, and obtain the labeled and cleaned dataset.

2. The multi-model cross-validation iterative noise label cleaning method according to claim 1, characterized in that, In step S1, the initial values ​​of the causal relationship labels for samples in the original dataset are determined using a heuristic algorithm.

3. The multi-model cross-validation iterative noise label cleaning method according to claim 1, characterized in that, The relationship feature vector between the first variable and the second variable consists of the statistical distribution characteristics, temporal sequence characteristics, correlation characteristics, conditional independence test characteristics, regression residual characteristics, and mutual information characteristics between the variables.

4. The multi-model cross-validation iterative noise label cleaning method according to claim 1, characterized in that, Multiple causal inference models in the model set M employ at least two types of modeling mechanisms.

5. The multi-model cross-validation iterative noise label cleaning method according to claim 1, characterized in that, Any two causal inference models in the model set M satisfy: ,in The difference between the predictions of the two causal inference models on the validation subset is given. This is a preset difference threshold.

6. The multi-model cross-validation iterative noise label cleaning method according to claim 1, characterized in that, Step S4 specifically includes: S41. Generate candidate labels for each sample based on the causal labels predicted in S3, and the candidate label for the i-th sample. for: ; Where L represents the number of causal inference models in the model set M. For the first The prediction of a causal inference model for the i-th sample, ; , which is a numerical representation of causal relationship labels; N is the number of samples in the original dataset; S42. Perform causal consistency constraint verification on the candidate labels of each sample. If the causal consistency constraint verification is passed, update the causal relationship label of the sample to the candidate label; otherwise, keep the causal relationship label of the sample unchanged. The causal consistency constraint is verified as follows: (421) Causal Direction Consistency Verification: The causal inference model in the statistical model set M predicts the i-th sample as follows: proportion ,when The consistency of causal direction is verified at that time; among which The preset directional consistency threshold; (422) Construct a temporary causal graph: Generate the current causal graph based on the current causal relationship labels of the samples in the dataset, map the candidate label of the i-th sample to the candidate directed edge ei and insert it into the current causal graph to obtain a temporary causal graph; (423) Verification of the legality of causal structure: Check whether a directed cycle is formed in the temporary causal graph and whether the candidate directed edge ei conflicts with the prior prohibited edge; if there is no directed cycle and no prohibited edge conflict, the verification of the legality of causal structure is passed. The determination of whether a candidate directed edge ei conflicts with a prior forbidden edge is as follows: a set of forbidden edges B is constructed in advance, and the elements in B are variable pairs that do not have a causal direction; if ei∈B, then the candidate directed edge is determined to conflict with the prior forbidden edge. (424) Consistency verification of chained relationships: Compare the support of direct edges in the temporary causal graph. Support of multi-hop chain paths ; direct edge support The calculation is based on the voting ratio of the candidate directed edge ei in the model set, specifically as follows: ； Where L represents the number of models. For indicator functions, ; The support of the multi-hop chain path The path support is the maximum value of the path support among all chain paths connecting the first and second variables of the i-th sample in the temporary causal graph. The path support is the minimum or average value of the direct support of each edge in the chain path connecting the first and second variables of the i-th sample. When a candidate directed edge ei is in the same direction as an existing chain path, it is determined that there is no chain relationship conflict; when a candidate directed edge ei is not in the same direction as an existing chain path, it is determined that there is a chain relationship conflict. If there is no chain relationship conflict or Then the consistency verification is passed through the chain relationship; (425) Verification of conditional independence: Constructing a set of condition variables , It consists of the adjacent variables and common cause variables of the first and second variables of the i-th sample in the current causal graph; in the set of condition variables Under the given conditions, perform a conditional independence test on the first and second variables of the i-th sample to obtain the significance probability. ; If the candidate label indicates the existence of a direct causal edge, and The conditional independence test is passed; if the candidate label indicates that there is no direct causal edge, and The conditional independence test was passed. The significance level; When the causal direction consistency verification, causal structure legality verification, chain relationship consistency verification, and conditional independence verification all pass, the causal consistency constraint verification is deemed to have passed.

7. The multi-model cross-validation iterative noise label cleaning method according to claim 1, characterized in that, The iteration termination condition in step S5 is: After step S4 is completed, calculate the proportion of causal relationship label changes for samples in the dataset. and based on edge set similarity and directional consistency stability Comprehensive causal structure stability The calculation steps are as follows: Let the set of edges of the causal graph obtained after the t-th iteration be . , No. The set of edges in the causal graph obtained after round iteration is Edge set similarity Jaccard similarity calculation: ; To calculate the number of elements in a set; Directional consistency stability Calculated based on the proportion of the direction of the common edge in two adjacent rounds: ； Where e is The edges of the causal graph in the middle, It is the direction of e in the t-th iteration; like If it is empty, then let ; Overall causal structure stability Calculated using a weighted average of edge set similarity and orientation consistency stability: Where λ∈[0,1] is the weight parameter; The iteration terminates when either condition 1 or condition 2 is met. Condition 1 is: the label change ratio is lower than the first threshold and the overall causal structure stability is... The value exceeds the second threshold for W consecutive rounds; W is a preset structural stability judgment parameter. Condition 2 is: reaching the maximum number of iterations.

8. A multi-model cross-validation iterative noise label cleaning system, the system being applicable to causal inference tasks, characterized in that, include: The original dataset construction module (1) is used to collect student education business indicators to construct the original dataset. Each sample in the dataset includes a first variable, a second variable, a relationship feature vector between the first variable and the second variable, and a causal relationship label. The first variable and the second variable in a sample are different types of student education business indicators of the same student. The cross-partitioning module (2) is used to divide the original dataset into K non-overlapping subsets using the cross-partitioning method; The cross-validation experiment module (3) is used to take each subset as the subset to be cleaned and the remaining K-1 subsets as the training set to conduct K sets of cross-validation experiments; in each set of cross-validation experiments, the training set consisting of K-1 subsets is used to train multiple causal inference models in the model set M, and the trained multiple causal inference models are used to predict causal labels for samples in the subset to be cleaned; the model set M includes at least 3 causal inference models; The sample label update module (4) is used to update the causal relationship label of each sample in the original dataset under the causal consistency constraint based on the causal label predicted by the cross-validation experiment module (3). The iteration control module (5) is used to control the operation of the cross-validation experiment module (3) and the sample label update module (4) according to the iteration termination condition, so as to obtain the labeled cleaned dataset.

9. The multi-model cross-validation iterative noise label cleaning system according to claim 8, characterized in that, It also includes a model set effective difference verification module (6), which is used to calculate the prediction difference rate of any two causal inference models in the model set M. When the prediction difference rate is less than the pre-screening difference threshold, one of the two causal inference models is deleted.

10. The multi-model cross-validation iterative noise label cleaning system according to claim 8, characterized in that, The sample label update module (4) updates the causal relationship label of each sample in the original dataset based on the causal label predicted by the cross-validation experiment module (3) under the causal consistency constraint, specifically including: S41. Generate candidate labels for each sample based on the causal labels predicted in S3, and the candidate label for the i-th sample. for: ; Where L represents the number of causal inference models in the model set M. For the first The prediction of a causal inference model for the i-th sample, ; , which is a numerical representation of causal relationship labels; N is the number of samples in the original dataset; S42. Perform causal consistency constraint verification on the candidate labels of each sample. If the causal consistency constraint verification is passed, update the causal relationship label of the sample to the candidate label; otherwise, keep the causal relationship label of the sample unchanged. The causal consistency constraint is verified as follows: (421) Causal Direction Consistency Verification: The causal inference model in the statistical model set M predicts the i-th sample as follows: proportion ,when The consistency of causal direction is verified at that time; among which The preset directional consistency threshold; (422) Construct a temporary causal graph: Generate the current causal graph based on the current causal relationship labels of the samples in the dataset, map the candidate label of the i-th sample to the candidate directed edge ei and insert it into the current causal graph to obtain a temporary causal graph; (423) Verification of the legality of causal structure: Check whether a directed cycle is formed in the temporary causal graph and whether the candidate directed edge ei conflicts with the prior prohibited edge; if there is no directed cycle and no prohibited edge conflict, the verification of the legality of causal structure is passed. The determination of whether a candidate directed edge ei conflicts with a prior forbidden edge is as follows: a set of forbidden edges B is constructed in advance, and the elements in B are variable pairs that do not have a causal direction; if ei∈B, then the candidate directed edge is determined to conflict with the prior forbidden edge. (424) Consistency verification of chained relationships: Compare the support of direct edges in the temporary causal graph. Support of multi-hop chain paths ; direct edge support The calculation is based on the voting ratio of the candidate directed edge ei in the model set, specifically as follows: ； Where L represents the number of models. For indicator functions, ; The support of the multi-hop chain path The path support is the maximum value of the path support among all chain paths connecting the first and second variables of the i-th sample in the temporary causal graph. The path support is the minimum or average value of the direct support of each edge in the chain path connecting the first and second variables of the i-th sample. When a candidate directed edge ei is in the same direction as an existing chain path, it is determined that there is no chain relationship conflict; when a candidate directed edge ei is not in the same direction as an existing chain path, it is determined that there is a chain relationship conflict. If there is no chain relationship conflict or Then the consistency verification is passed through the chain relationship; (425) Verification of conditional independence: Constructing a set of condition variables , It consists of the adjacent variables and common cause variables of the first and second variables of the i-th sample in the current causal graph; in the set of condition variables Under the given conditions, perform a conditional independence test on the first and second variables of the i-th sample to obtain the significance probability. ; If the candidate label indicates the existence of a direct causal edge, and The conditional independence test is passed; if the candidate label indicates that there is no direct causal edge, and The conditional independence test was passed. The significance level; When the causal direction consistency verification, causal structure legality verification, chain relationship consistency verification, and conditional independence verification all pass, the causal consistency constraint verification is deemed to have passed.

Images