A deep-computing-based full-link data collaboration system
By constructing a causal structure topology graph and simulating counterfactual scenarios, the marginal causal contribution of abnormal events is quantified, and data interaction rules are optimized. This solves the problem of insufficient causal dependency modeling in existing technologies and achieves precise control and resource optimization of end-to-end data collaboration.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SHANXI YUHE TRAFFIC ENGINEERING CO LTD
- Filing Date
- 2026-03-27
- Publication Date
- 2026-06-30
AI Technical Summary
Existing end-to-end data collaboration methods lack the ability to model causal dependencies, making it difficult to accurately quantify the cascading impact of abnormal events on downstream processes, failing to identify key intervention nodes, and resulting in delayed response and improper resource allocation.
By constructing a causal structure topology graph, causal relationships are mined and weighted to generate a dataset with causal weighting characteristics. This simulates virtual data sequences in counterfactual scenarios, quantifies the marginal causal contribution of abnormal events, optimizes data interaction rules and collaborative parameters, and generates end-to-end collaborative control instructions.
It enables precise quantitative assessment of abnormal events, identifies key impact nodes, optimizes resource allocation, and improves the accuracy of abnormal impact assessment and the flexibility and economy of end-to-end response.
Smart Images
Figure CN122310409A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of data processing and intelligent collaborative control technology, specifically to a full-link data collaborative system based on deep computing. Background Technology
[0002] Currently, with the rapid development of IoT and big data technologies, end-to-end data collaboration has been widely applied in supply chain management, intelligent manufacturing, smart cities, and other fields. Traditional data collaboration methods typically rely on segmented monitoring and post-event traceability, collecting real-time data from each stage and combining it with preset rules or thresholds to issue anomaly alerts and make local adjustments.
[0003] Existing end-to-end data collaboration methods, when dealing with cross-stage anomalies, lack the ability to model the causal dependencies between variables. They can only discover superficial correlations between data based on correlation analysis, and cannot accurately quantify the real cascading impact of anomalies on downstream stages. This makes it difficult to identify key intervention nodes from the source and dynamically generate globally optimal collaborative scheduling strategies, ultimately resulting in delayed end-to-end response, improper resource allocation, and amplified anomaly losses. Summary of the Invention
[0004] The purpose of this invention is to provide a deep computing-based end-to-end data collaboration system to solve the problems mentioned above.
[0005] The objective of this invention can be achieved through the following technical solutions:
[0006] A deep computing-based end-to-end data collaboration system includes:
[0007] The data fusion module collects real-time multi-source heterogeneous data from each link in the entire chain, performs time axis alignment and missing value imputation on the data, and generates an initial fused data set with a unified time sequence identifier.
[0008] The causal analysis module mines causal relationships among variables in the initial fused dataset, constructs a causal structure topology graph representing the causal dependency paths between variables, and calculates the causal influence weight of each variable based on the causal structure topology graph, generating a dataset with causal weighting features.
[0009] The counterfact generation module, when an abnormal event is detected in any link, intervenes and assigns values to the variables corresponding to the abnormal event based on the causal structure topology graph, simulates and generates virtual data sequences under at least one counterfact scenario, compares and stores the virtual data sequences with the real-time actual data sequences, and forms a counterfact comparison data set.
[0010] The quantitative analysis module inputs the counterfactual comparison data set into the causal effect quantification function. By comparing the differences between actual data and virtual data at subsequent nodes throughout the entire chain, it calculates the marginal causal contribution of abnormal events to each node and generates a cascading effect quantification matrix.
[0011] The collaborative optimization module, based on the cascaded influence quantification matrix and with the minimization of the overall loss across the entire link as a constraint, dynamically optimizes the data interaction rules and collaborative parameters of each link, and outputs the optimized collaborative control instructions for the entire link.
[0012] As a further aspect of the present invention: the generation of the data set with causal weighting features specifically includes:
[0013] For each pair of variables in the initial fused dataset, calculate the conditional dependency based on distance correlation, and construct an undirected dependency graph for variable pairs whose conditional dependency exceeds a preset threshold.
[0014] For each edge in the undirected dependency graph, the direction of the edge is determined by comparing the cross-information between the forward and backward regression residuals, thus obtaining the directed dependency graph.
[0015] Based on the topological order of the directed dependency graph, the anti-entropy value of each variable under the given parent node condition is calculated in combination with the initial fused data set. The anti-entropy value is used as the causal influence weight of the corresponding variable, and the weight is attached to the data of the corresponding variable in the initial fused data set to generate a data set with causal weighting characteristics.
[0016] As a further aspect of the present invention: the process of obtaining the directed dependency graph is as follows:
[0017] For the first and second variables connected by the current edge in the undirected dependency graph, a local weighted regression is performed with the first variable as the dependent variable and the second variable as the independent variable to obtain the first positive residual sequence;
[0018] A local weighted regression was performed with the second variable as the dependent variable and the first variable as the independent variable to obtain the second inverse residual sequence.
[0019] Calculate the cross-conditional entropy of the first positive residual sequence with respect to the second variable, and simultaneously calculate the cross-conditional entropy of the second negative residual sequence with respect to the first variable;
[0020] Determine whether the cross-conditional entropy of the first positive residual sequence with respect to the second variable is less than the cross-conditional entropy of the second negative residual sequence with respect to the first variable. If yes, the direction is determined to be from the first variable to the second variable; otherwise, the direction is determined to be from the second variable to the first variable.
[0021] As a further aspect of the present invention: the formation of the counterfactual comparison data set specifically includes:
[0022] Based on the causal structure topology graph, the target variable of the abnormal event is determined, and the set of parent node variables of the target variable and its corresponding noise terms are extracted;
[0023] An intervention operation is applied to the target variable, forcing the value of the target variable to be set to a preset intervention value. At the same time, noise items that match the parent node variable set are sampled from historical normal operation data and replaced with the original noise items.
[0024] The preset intervention value is used as the value of the target variable. The noise term is substituted into the dependency path corresponding to the descendant node of the target variable. The counterfactual value of each descendant node is calculated layer by layer to generate a complete virtual data sequence.
[0025] The virtual data sequence is merged and stored after being aligned with the real-time collected actual data sequence by timestamp, forming a counterfactual comparison data set containing factual branches and counterfactual branches.
[0026] As a further aspect of the present invention: the generation of a complete virtual data sequence specifically includes:
[0027] Based on the causal structure topology graph, determine the current descendant node to be calculated, and extract the set of parent node variables corresponding to the descendant node and the storage location of the noise term of the descendant node.
[0028] Read the counterfactual values of each parent node that have been calculated from the set of parent node variables, and substitute the counterfactual values of each parent node into the dependency function corresponding to the descendant node to calculate the baseline value of the descendant node.
[0029] Extract noise sample values that match the current counterfactual scenario from the noise item storage location of the descendant node, and accumulate and fuse the noise sample values with the reference value to obtain the counterfactual value of the descendant node;
[0030] The counterfactual values of descendant nodes are used as input values for subsequent levels. The calculation process is repeated until all descendant nodes have been calculated, generating a complete virtual data sequence.
[0031] As a further aspect of the present invention: the generation of the cascaded influence quantization matrix specifically includes:
[0032] Extract factual branch data and counterfactual branch data from the counterfactual comparison dataset, and determine the target node to be calculated according to the topological order of the causal structure topology graph;
[0033] The actual value of the target node in the fact branch is compared with the virtual value in the counterfactual branch element by element, and the deviation between the two is calculated as the initial effect value of the target node.
[0034] Tracing back along the causal structure topology graph, we find all the upstream ancestor nodes of the target node, extract the dependency strength coefficient of each ancestor node to the target node, and then weight and fuse the dependency strength coefficient with the initial effect value. After removing upstream interference, we obtain the marginal causal contribution of the target node.
[0035] The marginal causal contribution of each target node is arranged according to its hierarchical position in the causal structure topology graph, and a cascaded influence quantification matrix is constructed with target nodes as rows and hierarchical levels as columns.
[0036] As a further aspect of the present invention: the calculation process of the initial effect value is as follows:
[0037] The actual numerical sequence of the target node in the fact branch and the virtual numerical sequence in the counterfactual branch are segmented according to the same time window to obtain multiple fact subsequences and virtual subsequences within the time segment.
[0038] For each time segment, calculate the local fluctuation characteristic value of the fact subsequence and the local fluctuation characteristic value of the virtual subsequence, and use the difference between the two local fluctuation characteristic values as the time contribution value of the corresponding time segment;
[0039] The initial effect value of the target node is obtained by weighting the contribution values of all time segments by using the duration proportion of each time segment as the weight.
[0040] As a further aspect of the present invention: the optimized end-to-end cooperative control command specifically includes:
[0041] Extract the marginal causal contribution of each node from the cascaded influence quantification matrix, and mark the nodes with marginal causal contribution exceeding the preset threshold as key intervention nodes;
[0042] Based on the causal structure topology graph, the upstream propagation path to which each key intervention node belongs is determined, and a dynamic priority coefficient is assigned to each upstream propagation path. The dynamic priority coefficient is positively correlated with the sum of the marginal causal contributions of all nodes on the path.
[0043] According to the dynamic priority coefficient from high to low, the current data sampling frequency and data forwarding delay of each node in the corresponding upstream propagation path are adjusted in turn to generate the local coordination parameters of each node.
[0044] The local coordination parameters of each node are arranged according to their execution sequence in the whole link to generate a global coordination control instruction sequence containing timestamps and execution actions.
[0045] The beneficial effects of this invention are:
[0046] (1) This invention achieves a leap from traditional correlation analysis to causal inference by constructing a causal structure topology graph and combining it with counterfactual reasoning. When an abnormal event occurs, it can accurately quantify the marginal causal contribution of the event to each link in the entire chain, effectively eliminate the interference of confounding factors, and solve the problem of "not being able to see clearly and calculate accurately" in the existing technology. This enables the system to identify the real key influencing nodes and propagation paths, providing a scientific and rigorous decision-making basis for subsequent collaborative optimization, and improving the accuracy and interpretability of abnormal impact assessment.
[0047] (2) This invention is based on a cascaded impact quantification matrix and dynamically optimizes collaborative parameters with the goal of minimizing the overall loss across the entire link. By prioritizing the allocation of limited computing and communication resources to the critical path with the highest marginal contribution, the optimal balance between resource investment and anomaly blocking effect is achieved. Compared with the traditional coarse-grained response that relies on fixed thresholds, this invention can effectively reduce the overall cargo loss or delay loss across the entire link while ensuring the stability of core business, and improve the flexibility and economy of multi-link collaborative scheduling in complex dynamic environments. Attached Figure Description
[0048] The invention will now be further described with reference to the accompanying drawings.
[0049] Figure 1 This is a system block diagram of the present invention. Detailed Implementation
[0050] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0051] Please see Figure 1 As shown, this invention is a full-link data collaboration system based on deep computing, comprising:
[0052] The data fusion module collects real-time multi-source heterogeneous data from each link in the entire chain, performs time axis alignment and missing value imputation on the data, and generates an initial fused data set with a unified time sequence identifier.
[0053] The causal analysis module mines causal relationships among variables in the initial fused dataset, constructs a causal structure topology graph representing the causal dependency paths between variables, and calculates the causal influence weight of each variable based on the causal structure topology graph, generating a dataset with causal weighting features.
[0054] The counterfact generation module, when an abnormal event is detected in any link, intervenes and assigns values to the variables corresponding to the abnormal event based on the causal structure topology graph, simulates and generates virtual data sequences under at least one counterfact scenario, compares and stores the virtual data sequences with the real-time actual data sequences, and forms a counterfact comparison data set.
[0055] The quantitative analysis module inputs the counterfactual comparison data set into the causal effect quantification function. By comparing the differences between actual data and virtual data at subsequent nodes throughout the entire chain, it calculates the marginal causal contribution of abnormal events to each node and generates a cascading effect quantification matrix.
[0056] The collaborative optimization module, based on the cascaded influence quantification matrix and with the minimization of the overall loss across the entire link as a constraint, dynamically optimizes the data interaction rules and collaborative parameters of each link, and outputs the optimized collaborative control instructions for the entire link.
[0057] In the data fusion module, real-time multi-source heterogeneous data from each stage of the entire chain are collected. The data undergoes time-axis alignment and missing value imputation to generate an initial fused data set with a unified time-series identifier. Specifically, this includes:
[0058] In a full-link collaborative control scenario, the first step is to acquire the raw data from each business link. The specific data acquisition method involves using temperature and humidity sensors deployed in the warehousing environment, satellite positioning terminals attached to transport containers, and programmable logic controllers integrated into refrigeration equipment to collect real-time environmental temperature and humidity values, latitude and longitude coordinates, and equipment operating status parameters. Simultaneously, by connecting to the port customs' electronic data exchange interface and the Automatic Identification System (AIS) base station, real-time customs clearance progress messages and vessel dynamic information are obtained. This data acquisition process continues at a rate of seconds, generating a multi-source heterogeneous initial data stream containing structured numerical data and semi-structured text.
[0059] After data acquisition, the initial data stream is preprocessed. First, using Coordinated Universal Time (UTC) as the reference, the timestamp fields of all data sources are uniformly converted to a standard time format, and data within the same second-level window are merged to eliminate time asynchrony caused by network latency. Then, for null values in the data stream, linear interpolation based on observations from previous and subsequent times is used to fill in missing temperature and humidity values. For discrete state variables, the values from the previous valid time are used to fill in the missing values, ensuring the integrity of each data record.
[0060] After the above processing, the originally scattered and inconsistent data are integrated into a continuous data sequence with a unified temporal identifier. Each data point in this sequence is accompanied by a standardized time label, and all fields have been completed, forming an initial fused data set that can be used for subsequent causal analysis. This set is stored using timestamps as the primary key, fully preserving the original characteristics and temporal relationships of each link in the entire chain.
[0061] In the causal analysis module, causal relationships are mined from the variables in the initial fused dataset. A causal structure topology graph representing the causal dependency paths between variables is constructed. Based on the causal structure topology graph, the causal influence weight of each variable is calculated, generating a dataset with causal weighting features. Specifically, this includes:
[0062] After obtaining the initial fused dataset, causal relationship mining is first performed to construct an undirected dependency graph. Specifically, for any two variables in the initial fused dataset, a distance-based correlation coefficient is calculated between them. This coefficient is calculated as follows: for each numerical sequence of the two variables, the Euclidean distance between each point within the sequence is calculated, forming two distance matrices; then, the centered inner product of these two distance matrices is calculated, and the inner product value is divided by the product of the standard deviations of the two distance matrices to obtain a dependency value between 0 and 1. The above calculation is performed on all pairs of variables in the dataset to obtain the dependency of each pair of variables. The dependency value is compared with a preset threshold, which is set to 0.6. If the dependency exceeds 0.6, a potential dependency relationship is considered to exist between the two variables, and an undirected edge connects the nodes corresponding to these two variables. After traversing all variable pairs, the nodes and undirected edges together form an undirected dependency graph.
[0063] After constructing the undirected dependency graph, the direction of each edge in the graph needs to be determined to obtain the directed dependency graph. For the two variables connected by a given edge in the undirected dependency graph, let's denote them as the first variable and the second variable. First, using the first variable as the dependent variable and the second variable as the independent variable, perform a locally weighted regression. In this regression process, for each sample point, different weights are assigned based on its Euclidean distance to its neighboring sample points; the closer the sample point, the higher the weight. The weighted least squares method is used to fit the regression coefficients, and the difference between the actual value and the fitted value for each sample point is calculated to obtain the first positive residual sequence. Subsequently, using the second variable as the dependent variable and the first variable as the independent variable, the same locally weighted regression method is used to calculate the second inverse residual sequence.
[0064] After obtaining two residual sequences, their cross-conditional entropies with their corresponding variables are calculated. When calculating the cross-conditional entropy of the first positive residual sequence with respect to the second variable, the numerical sequences of the first positive residual sequence and the second variable are first discretized using equal-frequency methods, dividing them into several intervals. The joint distribution probability and marginal distribution probability of the two sequences in different intervals are statistically analyzed. The conditional entropy of the second variable is then calculated using the information entropy formula, given the values of the first positive residual sequence. The same method is used to calculate the cross-conditional entropy of the second negative residual sequence with respect to the first variable. The values of these two cross-conditional entropies are compared. If the cross-conditional entropy of the first positive residual sequence with respect to the second variable is less than that of the second negative residual sequence with respect to the first variable, the direction of the edge is determined to be from the first variable to the second variable; otherwise, the direction is determined to be from the second variable to the first variable. This direction determination process is performed on each edge in the undirected dependency graph, ultimately resulting in a directed dependency graph where all edges have definite directions.
[0065] The causal influence weight of each variable is calculated based on the directed dependency graph. First, the parent node of each variable is determined according to the directed dependency graph; a parent node is an upstream variable to which there is a directed edge. For any variable, the numerical sequence of the variable and the numerical sequences of all its parent nodes are extracted from the initial fused dataset, and these parent node numerical sequences are used as condition variables. The numerical sequence of the variable is discretized into several intervals with equal frequency, and the numerical sequences of all parent nodes are combined into a multidimensional condition vector, which is also discretized. The joint probability distribution of the variable's value interval and the value interval of the parent node's condition vector, as well as the marginal probability distribution of the parent node's condition vector value, are calculated. Using the conditional entropy calculation formula, the conditional entropy of the variable under given parent node conditions is obtained. This conditional entropy reflects the remaining uncertainty of the variable after knowing the parent node information. The negative of this conditional entropy is defined as the inverse entropy value of the variable, i.e., 1 minus the normalized conditional entropy, resulting in a value between 0 and 1. The larger this value, the stronger the explanatory power of the parent node for the variable, and this value is used as the causal influence weight of the variable.
[0066] Finally, the calculated causal influence weights are appended to the corresponding variables in the initial fused dataset. Specifically, a weight field is added to each variable in the initial fused dataset; the value of this field is the inverse entropy value of that variable, and this weight value remains unchanged across all data records corresponding to the same variable. After this processing, each variable in the original initial fused dataset carries a weight coefficient reflecting its causal importance, thus generating a dataset with causal weighting characteristics for use in subsequent steps.
[0067] In the counterfact generation module, when an abnormal event is detected in any stage, the variables corresponding to the abnormal event are intervened and assigned values based on the causal structure topology graph. This simulates and generates a virtual data sequence under at least one counterfactual scenario. The virtual data sequence is then compared and stored with the real-time actual data sequence to form a counterfactual comparison data set, specifically including:
[0068] First, the variables corresponding to the abnormal events are identified based on the causal structure topology graph and designated as the target variables. All upstream variables pointing to this target variable are extracted from the causal structure topology graph, forming a set of parent node variables. Simultaneously, a sequence of noise terms corresponding to the target variable is obtained from the initial fused dataset. The noise terms are calculated as follows: based on historical normal operation data, a locally weighted regression is performed with the parent node variables as independent variables and the target variable as the dependent variable to obtain fitted values. The difference between the actual values and the fitted values is taken as the noise terms and stored in chronological order as a noise term sequence.
[0069] Secondly, an intervention operation is applied to the target variable. The specific method of intervention is as follows: the actual value of the target variable at the current moment is forcibly replaced with a preset intervention value. This intervention value is set according to the simulation requirements of the abnormal scenario. For example, if simulating a "no delay" scenario, the value of the shipping delay variable is set to 0 (indicating normality); if simulating a "port change" scenario, the value of the port identifier variable is set to the code value of the alternative port. After intervention, the value of the target variable is no longer affected by its parent node but becomes a fixed constant.
[0070] Then, to calculate the counterfactual values of descendant nodes, noise terms need to be regenerated to match the post-intervention scenario. Specifically, samples with values similar to the parent node variable set at the current intervention time are selected from historical normal operation data, and weighted sampling is performed based on the noise terms of these samples. The sampling weights are calculated using a Gaussian kernel function, the mathematical expression of which is as follows:
[0071] ;
[0072] in, Index representing historical samples, For the first Sampling weights for each historical sample; The number of parent node variables; Indicates the first In the historical sample, the first The values of each parent node; Indicates the current intervention time. The actual values of each parent node; The bandwidth parameter is set to the median of the Euclidean distances between all historical samples and the current parent node. This is based on the calculated weights. We perform weighted random sampling on the corresponding target variable noise terms in the historical samples, and select a noise value as the new noise term in the current counterfactual scenario to replace the original noise term.
[0073] Next, based on the causal structure topology graph, the counterfactual values of all descendant nodes are calculated layer by layer downstream from the target variable. For a given descendant node to be calculated, its set of parent node variables is first determined from the causal structure topology graph. These parent nodes may be the target variable or other already calculated descendant nodes. The counterfactual values of these parent nodes are then read from the already calculated data. Simultaneously, the storage location of the noise term corresponding to this descendant node is extracted. This noise term also comes from historical data and must belong to the same historical sample as the previously sampled noise term to ensure that the correlation structure between noise terms is not disrupted.
[0074] The value of a descendant node is determined by both its parent node value and the noise term. This dependency is described using a linear additive function, expressed mathematically as follows:
[0075] ;
[0076] in, This is the counterfactual value of the descendant node to be calculated; Represents a node The set consisting of all parent nodes; From the parent node To the node The dependency coefficient is estimated using a linear regression method based on historical data during the offline phase, and is a fixed constant for each node; parent node The counterfactual value; For the corresponding node sampled from historical data The noise term is then added. Based on the above formula, the counterfactual value of each parent node is multiplied by its corresponding dependency coefficient, summed, and then the noise term is added to obtain the counterfactual value of that descendant node.
[0077] The counterfactual value of this descendant node is used as a known quantity and propagated downstream, repeating the above calculation process until all descendant nodes reachable from the target variable in the causal structure topology graph have been calculated. This generates a complete virtual data sequence containing simulated values of variables at each stage of the entire chain under intervention conditions, starting from the moment the anomalous event occurs.
[0078] Finally, the generated virtual data sequence is aligned with the real-time collected actual data sequence by timestamp. The actual data sequence reflects the actual link status, while the virtual data sequence reflects the counterfactual status after the hypothetical intervention. Both are merged and stored in the same data table. Each record contains both an actual value field and a virtual value field at the same timestamp, with an additional branch identifier field to distinguish whether the record belongs to a factual branch or a counterfactual branch. This forms a counterfactual comparison data set containing factual and counterfactual comparison information for subsequent quantitative analysis.
[0079] In the quantitative analysis module, the counterfactual comparison data set is input into the causal effect quantification function. By comparing the differences between actual and virtual data at subsequent nodes throughout the entire chain, the marginal causal contribution of the abnormal event to each node is calculated, generating a cascading impact quantification matrix, which specifically includes:
[0080] After obtaining the counterfactual comparison dataset, the factual branch data and counterfactual branch data are first extracted from it. Factual branch data refers to the actual data sequence collected in real time, while counterfactual branch data refers to the virtual data sequence generated through intervention simulation. Based on the node hierarchy order determined by the causal structure topology graph, starting from the upstream node, the target nodes to be calculated are determined sequentially, ensuring that the calculation process proceeds layer by layer from the source to the end according to the causal relationship.
[0081] For the currently determined target node to be calculated, its deviation between the fact branch and the counterfactual branch needs to be calculated as the initial effect value. The specific calculation method is as follows: First, the actual numerical sequence of the target node in the fact branch and the virtual numerical sequence in the counterfactual branch are segmented according to the same time window. The duration of each time window is set to 5 minutes, dividing the entire impact period of the abnormal event into several continuous and non-overlapping time segments. Each segment contains the values of all sampling times within that time period, resulting in multiple fact subsequences and virtual subsequences corresponding to multiple time segments. For each time segment, the local fluctuation characteristic value of the fact subsequence and the local fluctuation characteristic value of the virtual subsequence are calculated respectively. The calculation method of the local fluctuation characteristic value is as follows: the arithmetic mean of all values in the subsequence is taken to obtain the mean of the subsequence. Then, the absolute value of the difference between each value in the subsequence and the mean is calculated. These absolute values are then arithmetically averaged to obtain the mean absolute deviation of the subsequence. This mean absolute deviation is taken as the local fluctuation characteristic value. The local volatility eigenvalue of the fact subsequence within the same time segment is subtracted from the local volatility eigenvalue of the virtual subsequence to obtain the time-segment contribution value. This time-segment contribution value reflects the degree of influence of the anomalous event on the volatility of the target node within that time segment. After obtaining the time-segment contribution values for all time segments, the proportion of each time segment's duration to the total duration of the anomalous event's influence period is used as a weight. The time-segment contribution values of all time segments are then weighted and summed. The final result is used as the initial effect value of the target node.
[0082] After obtaining the initial effect value of the target node, it is necessary to remove the interference from upstream ancestor nodes to obtain the marginal causal contribution of the node itself. Specifically, based on the causal structure topology graph, trace back upstream from the target node to find all ancestor nodes that can reach the target node via directed paths, forming an upstream ancestor node set. For each ancestor node in this set, extract the dependency strength coefficient of that ancestor node to the target node from a pre-stored dependency parameter table. This dependency strength coefficient is estimated offline using a linear regression method, specifically by regressing the ancestor node's value as the independent variable and the target node's value as the dependent variable. The resulting regression coefficient is the dependency strength coefficient, reflecting the degree to which a unit change in the ancestor node causes a change in the target node. The dependency strength coefficients of all ancestor nodes are summed to obtain the total upstream dependency strength. The total upstream dependency strength is multiplied by the initial effect value to obtain the total upstream interference. The initial effect value is subtracted from the total upstream interference; the difference is the marginal causal contribution of the target node, representing the net effect of the anomalous event directly acting on the target node after removing the influence of all upstream ancestor nodes.
[0083] The above calculation process is performed on each node in the causal structure topology graph to obtain the marginal causal contribution of each node. Finally, these marginal causal contributions are arranged according to the node's hierarchical position in the causal structure topology graph. Specifically, the arrangement is as follows: the nodes are the rows of the matrix, and the hierarchical index of the causal structure topology graph is the column of the matrix. The marginal causal contribution of each node is filled into the intersection of the node's row and the column of its hierarchy. For nodes that do not belong to the hierarchy, the corresponding position is filled with 0. This results in a two-dimensional matrix, where the rows represent the nodes in the entire chain, the columns represent the hierarchical levels of causal relationship transmission, and each element in the matrix represents the marginal causal contribution of a specific node at a specific level. This matrix is defined as the cascading influence quantification matrix, which fully characterizes the quantified influence distribution of abnormal events propagating layer by layer in the entire chain.
[0084] In the collaborative optimization module, based on the cascaded influence quantification matrix and with the minimization of the overall loss across the entire link as a constraint, the data interaction rules and collaborative parameters of each link are dynamically optimized, and the optimized overall collaborative control instructions are output, specifically including:
[0085] First, the marginal causal contribution value corresponding to each node is extracted from the cascaded influence quantification matrix. The rows of this matrix correspond to the nodes in the entire chain, and the columns correspond to the levels of causal relationship transmission. Each element represents the marginal causal contribution of a specific node at a specific level. For each node, its marginal causal contributions at all levels are summed to obtain the node's total marginal causal contribution. The total marginal causal contributions of all nodes are sorted, and the arithmetic mean of the total marginal causal contributions of all nodes is calculated. A preset threshold is set to 1.5 times this arithmetic mean, and nodes with a total marginal causal contribution exceeding this threshold are marked as key intervention nodes. These nodes are the links in the anomaly propagation path that contribute the most to the overall chain loss.
[0086] Secondly, the upstream propagation path to which each key intervention node belongs is determined based on the causal structure topology graph. Specifically, for each key intervention node, the path is traced upstream along the directed edges of the causal structure topology graph until the source node of the abnormal event is reached. The sequence of nodes traversed constitutes a complete upstream propagation path. For each upstream propagation path, the sum of the marginal causal contributions of all nodes on the path (including the source node and the key intervention node itself) is calculated, and this sum is used as the initial priority score for the path. The initial priority scores of all paths are normalized by dividing the initial priority score of each path by the sum of the initial priority scores of all paths, resulting in a dynamic priority coefficient for each path. This coefficient is between 0 and 1, and the higher the sum of the contributions of the nodes on the path, the higher the dynamic priority coefficient.
[0087] Then, each upstream propagation path is processed sequentially according to its dynamic priority coefficient, from highest to lowest. For the currently processed upstream propagation path, the current data sampling frequency and data forwarding delay of each node on that path are adjusted sequentially. The data sampling frequency is adjusted by multiplying the node's current data sampling frequency by 1 and the sum of the dynamic priority coefficients to obtain a new sampling frequency, but the new frequency must not exceed the maximum sampling frequency supported by the device hardware. The data forwarding delay is adjusted by dividing the node's current data forwarding delay by 1 and the sum of the dynamic priority coefficients to obtain a new forwarding delay, but the new delay must not be lower than the minimum delay allowed by network transmission. After the above adjustments, a new set of local coordination parameters is generated for each node on the path, containing the adjusted sampling frequency and forwarding delay values.
[0088] Finally, all generated local coordination parameters are orchestrated according to the actual execution sequence of each node in the entire link. Specifically, the orchestration method is as follows: based on the hierarchical order of nodes in the causal structure topology graph and the business processing flow of each link, the time sequence of each node's coordinated control actions is determined. Each node is assigned an execution timestamp, calculated based on the node's current time plus the estimated processing delay. Each node's node identifier, execution timestamp, and corresponding local coordination parameters (sampling frequency and forwarding delay) are combined into a control command. All node control commands are arranged in chronological order according to their timestamps, forming a complete global coordinated control command sequence. This command sequence is sent to the execution terminals of each link in the entire link to guide subsequent data acquisition and forwarding operations.
[0089] The working principle of this invention is as follows: First, real-time multi-source heterogeneous data from each stage of the entire chain are collected, and time-axis alignment and missing value imputation are performed to generate an initial fused data set with a unified time sequence identifier. Then, causal relationship mining is performed on the variables in the initial fused data set to construct a causal structure topology graph representing the causal dependency paths between variables. Based on this topology graph, the causal influence weight of each variable is calculated to generate a data set with causal weighting features. When an abnormal event is detected in any stage, intervention and value assignment are performed on the variables corresponding to the abnormal event based on the causal structure topology graph to simulate and generate virtual data sequences under at least one counterfactual scenario. The data is then compared and stored with real-time actual data sequences to form a counterfactual comparison data set. This counterfactual comparison data set is then input into a causal effect quantification function. By comparing the differences between actual and virtual data at subsequent nodes throughout the entire chain, the marginal causal contribution of abnormal events to each node is calculated, generating a cascaded impact quantification matrix. Finally, based on the cascaded impact quantification matrix and with the minimization of the overall loss throughout the chain as a constraint, the data interaction rules and collaborative parameters of each link are dynamically optimized, and optimized end-to-end collaborative control instructions are output. This achieves end-to-end closed-loop control, from the quantitative analysis of abnormal events to the autonomous generation of collaborative strategies.
[0090] The foregoing has provided a detailed description of one embodiment of the present invention, but this description is merely a preferred embodiment and should not be construed as limiting the scope of the invention. All equivalent variations and modifications made within the scope of the claims of this invention should still fall within the patent coverage of this invention.
Claims
1. A full-link data collaboration system based on deep computing, characterized in that, include: The data fusion module collects real-time multi-source heterogeneous data from each link in the entire chain, performs time axis alignment and missing value imputation on the data, and generates an initial fused data set with a unified time sequence identifier. The causal analysis module mines causal relationships among variables in the initial fused dataset, constructs a causal structure topology graph representing the causal dependency paths between variables, and calculates the causal influence weight of each variable based on the causal structure topology graph, generating a dataset with causal weighting features. The counterfact generation module, when an abnormal event is detected in any link, intervenes and assigns values to the variables corresponding to the abnormal event based on the causal structure topology graph, simulates and generates virtual data sequences under at least one counterfact scenario, compares and stores the virtual data sequences with the real-time actual data sequences, and forms a counterfact comparison data set. The quantitative analysis module inputs the counterfactual comparison data set into the causal effect quantification function. By comparing the differences between actual data and virtual data at subsequent nodes throughout the entire chain, it calculates the marginal causal contribution of abnormal events to each node and generates a cascading effect quantification matrix. The collaborative optimization module, based on the cascaded influence quantification matrix and with the minimization of the overall loss across the entire link as a constraint, dynamically optimizes the data interaction rules and collaborative parameters of each link, and outputs the optimized collaborative control instructions for the entire link.
2. The end-to-end data collaboration system based on deep computing according to claim 1, characterized in that, The generation of the data set with causal weighting features specifically includes: For each pair of variables in the initial fused dataset, calculate the conditional dependency based on distance correlation, and construct an undirected dependency graph for variable pairs whose conditional dependency exceeds a preset threshold. For each edge in the undirected dependency graph, the direction of the edge is determined by comparing the cross-information between the forward and backward regression residuals, thus obtaining the directed dependency graph. Based on the topological order of the directed dependency graph, the anti-entropy value of each variable under the given parent node condition is calculated in combination with the initial fused data set. The anti-entropy value is used as the causal influence weight of the corresponding variable, and the causal influence weight is attached to the data of the corresponding variable in the initial fused data set to generate a data set with causal weighting characteristics.
3. The end-to-end data collaboration system based on deep computing according to claim 2, characterized in that, The process of obtaining the directed dependency graph is as follows: For the first and second variables connected by the current edge in the undirected dependency graph, a local weighted regression is performed with the first variable as the dependent variable and the second variable as the independent variable to obtain the first positive residual sequence; A local weighted regression was performed with the second variable as the dependent variable and the first variable as the independent variable to obtain the second inverse residual sequence. Calculate the cross-conditional entropy of the first positive residual sequence with respect to the second variable, and simultaneously calculate the cross-conditional entropy of the second negative residual sequence with respect to the first variable; Determine whether the cross-conditional entropy of the first positive residual sequence with respect to the second variable is less than the cross-conditional entropy of the second negative residual sequence with respect to the first variable. If yes, the direction is determined to be from the first variable to the second variable; otherwise, the direction is determined to be from the second variable to the first variable.
4. The end-to-end data collaboration system based on deep computing according to claim 1, characterized in that, The formation of the counterfactual comparison data set specifically includes: Based on the causal structure topology graph, the target variable of the abnormal event is determined, and the set of parent node variables of the target variable and its corresponding noise terms are extracted; An intervention operation is applied to the target variable, forcing the value of the target variable to be set to a preset intervention value. At the same time, noise items that match the parent node variable set are sampled from historical normal operation data and replaced with the original noise items. The preset intervention value is used as the value of the target variable. The noise term is substituted into the dependency path corresponding to the descendant node of the target variable. The counterfactual value of each descendant node is calculated layer by layer to generate a complete virtual data sequence. The virtual data sequence is merged and stored after being aligned with the real-time collected actual data sequence by timestamp, forming a counterfactual comparison data set containing factual branches and counterfactual branches.
5. A deep computing-based end-to-end data collaboration system according to claim 4, characterized in that, The generation of a complete virtual data sequence specifically includes: Based on the causal structure topology graph, determine the current descendant node to be calculated, and extract the set of parent node variables corresponding to the descendant node and the storage location of the noise term of the descendant node. Read the counterfactual values of each parent node that have been calculated from the set of parent node variables, and substitute the counterfactual values of each parent node into the dependency function corresponding to the descendant node to calculate the baseline value of the descendant node. Extract noise sample values that match the current counterfactual scenario from the noise item storage location of the descendant node, and accumulate and fuse the noise sample values with the reference value to obtain the counterfactual value of the descendant node; The counterfactual values of descendant nodes are used as input values for subsequent levels. The calculation process is repeated until all descendant nodes have been calculated, generating a complete virtual data sequence.
6. The end-to-end data collaboration system based on deep computing according to claim 1, characterized in that, The generation of the cascaded influence quantization matrix specifically includes: Extract factual branch data and counterfactual branch data from the counterfactual comparison dataset, and determine the target node to be calculated according to the topological order of the causal structure topology graph; The actual value of the target node in the fact branch is compared with the virtual value in the counterfactual branch element by element, and the deviation between the two is used as the initial effect value of the target node. Tracing back along the causal structure topology graph, we find all the upstream ancestor nodes of the target node, extract the dependency strength coefficient of each ancestor node to the target node, and then weight and fuse the dependency strength coefficient with the initial effect value. After removing upstream interference, we obtain the marginal causal contribution of the target node. The marginal causal contribution of each target node is arranged according to its hierarchical position in the causal structure topology graph, and a cascaded influence quantification matrix is constructed with target nodes as rows and hierarchical levels as columns.
7. A deep computing-based end-to-end data collaboration system according to claim 6, characterized in that, The calculation process for the initial effect value is as follows: The actual numerical sequence of the target node in the fact branch and the virtual numerical sequence in the counterfactual branch are segmented according to the same time window to obtain multiple fact subsequences and virtual subsequences within the time segment. For each time segment, calculate the local fluctuation characteristic value of the fact subsequence and the local fluctuation characteristic value of the virtual subsequence, and use the difference between the two local fluctuation characteristic values as the time contribution value of the corresponding time segment; The initial effect value of the target node is obtained by weighting the contribution values of all time segments by using the duration proportion of each time segment as the weight.
8. The end-to-end data collaboration system based on deep computing according to claim 1, characterized in that, The optimized end-to-end collaborative control commands specifically include: Extract the marginal causal contribution of each node from the cascaded influence quantification matrix, and mark the nodes with marginal causal contribution exceeding the preset threshold as key intervention nodes; Based on the causal structure topology diagram, the upstream propagation path to which each key intervention node belongs is determined, and a dynamic priority coefficient is assigned to each upstream propagation path. The dynamic priority coefficient is positively correlated with the sum of the marginal causal contributions of all nodes on the upstream propagation path. According to the dynamic priority coefficient from high to low, the current data sampling frequency and data forwarding delay of each node in the corresponding upstream propagation path are adjusted in turn to generate the local coordination parameters of each node. The local coordination parameters of each node are arranged according to their execution sequence in the whole link to generate a global coordination control instruction sequence containing timestamps and execution actions.