Data node adaptive optimization method based on causal reasoning and prototype learning

Abstract

The invention discloses a data node adaptive optimization method based on causal reasoning and prototype learning. The method comprises the following steps: generating a dynamic causal mapping network according to a real-time network state; according to the method, data sets of different data streams are clustered, representative prototypes are extracted, and a prototype library suitable for different network scenes is constructed; through a prototype symbiosis mechanism driven by causal mapping, a causal mapping network and prototype generation in a prototype library are bound, and real-time interaction is realized. Dynamically adjusting a layout optimization result through a self-learning feedback mechanism; the scheme has remarkable performance advantages in a high-load and large-scale data circulation network environment, and can be widely applied to a complex network environment requiring efficient and stable transmission.

Description

Technical Field

[0001] The present invention relates to the field of network data node layout optimization, and in particular to a data node adaptive optimization method based on causal reasoning and prototype learning. Background Art

[0002] With the rapid development of information technology, data circulation has become one of the important driving forces for promoting economic and social development. The rise of technologies such as big data, the Internet of Things, and artificial intelligence has led to a growing demand for the generation, transmission, and application of various types of data. Data circulation nodes are key components of the entire data circulation network, and their layout directly affects the data transmission efficiency, processing performance, and the reliability and stability of the overall system. Therefore, optimizing the layout of data circulation nodes to improve data circulation efficiency has become the focus of current research. In the data circulation system, the reasonable layout of nodes can effectively reduce the delay, bandwidth consumption, and network congestion problems during data transmission. At the same time, security, privacy protection, and data integrity during data transmission also require that the node layout has a high degree of flexibility and adaptability. Therefore, how to use intelligent methods to optimize the node layout to adapt to the complex and dynamically changing network environment is a research topic with important practical significance.

[0003] At present, the optimization methods for data flow node layout are mainly concentrated in the following categories: optimization methods based on static models, node layout optimization methods based on machine learning, and node layout methods based on distributed computing. Among them, the optimization methods based on static models assume that the topological structure of the data flow network is fixed, and use traditional theories such as graph theory and network flow models to analyze the node layout optimization problem through mathematical modeling. Common methods include minimum spanning tree, maximum flow minimum cut model, etc. This type of method performs better in scenarios with simpler structures and less changes. With the development of machine learning technology, many studies have begun to try to use machine learning algorithms to optimize node layout. Node layout optimization methods based on machine learning usually collect historical data, model the topological structure and traffic pattern of the network, and use algorithms such as supervised learning or reinforcement learning to optimize the node layout of data flow. In addition, in order to cope with the complexity of large-scale distributed networks, some studies have proposed node layout optimization methods based on distributed computing and edge computing. This type of method uses edge nodes in the network to distribute computing and decision-making, aiming to improve the efficiency of data processing and transmission and reduce the computing burden of central nodes.

[0004] Although existing optimization methods have achieved certain results in specific scenarios, there are still some significant shortcomings. First, static models lack flexibility. Optimization methods based on static models usually rely on assumptions about network structure, while in actual applications, data circulation networks are often in dynamic change. Network topology, data flow, transmission requirements and other factors may change, and static models are difficult to adapt to such complexity and dynamics, resulting in limited optimization effects. Second, although machine learning-based methods have achieved good optimization results in certain specific scenarios, this method usually relies on a large amount of historical data and manual annotation. In some scenarios where data is scarce or the network changes frequently, the generalization ability of the model is insufficient, and it is easy to overfit or fail to effectively adapt to the new environment. In addition, many existing methods do not fully consider the real-time needs of data circulation networks. In the layout optimization of data circulation nodes, real-time performance is a crucial factor. Traditional optimization methods usually require long-term calculations and cannot respond to network changes in a timely manner, affecting the efficiency of the entire system. Finally, most of the existing optimization methods are based on correlation analysis, ignoring the causal relationship between nodes and data transmission paths. In actual applications, the optimization of node layout often involves complex causal relationships. For example, the layout adjustment of one node may affect the transmission efficiency of other nodes. Existing methods that fail to fully utilize causal reasoning find it difficult to fully consider these complex factors, resulting in suboptimal layout optimization results. Summary of the invention

[0005] To this end, it is necessary to provide a data node adaptive optimization method that combines causal reasoning and prototype learning, which can respond to dynamically changing network environments more flexibly and accurately, reduce dependence on large amounts of historical data, and handle complex causal relationships while maintaining real-time performance.

[0006] To achieve the above object, the inventor provides a data node adaptive optimization method based on causal reasoning and prototype learning, comprising the steps of: S1, generates a dynamic causal mapping network based on the real-time network status; S2, by clustering different data circulation datasets, extracting representative prototypes, and building a prototype library that adapts to different network scenarios; S3, through the prototype symbiosis mechanism driven by causal mapping, binds the causal mapping network and the prototype generation in the prototype library to achieve real-time interaction; S4, dynamically adjust the layout optimization results through a self-learning feedback mechanism.

[0007] As a preferred embodiment of the present invention, step S1 further includes: S101, defines changes or adjustments in the network as causal events. At time t, the change in the network represents a causal event. , where i represents the i-th event, and the expression of the causal event is: ; in, represents the state of the nodes in the network at time t, Indicates the bandwidth usage of the network at time t, express The network performance parameters at each moment, F represents the network state change function; If the event Will cause events after , then through the causal chain function definition , the expression is: ; in, represents the random noise term, Indicates the delay of event propagation; S102, at any time t, the dynamic causal mapping model automatically generates a graph-structured causal mapping network based on each change , the expression is: ; in, It represents the causal event nodes in the network generated by the state space model. represents the similarity function, represents the causal relationship edge between causal event nodes, Represents a graph structure generating network, N represents a total of N causal events; S103, at any time t, for the generated causal mapping network , and recursively connect it to new causal events In-depth analysis of the interaction of various factors in the network and generation of dynamic causal mapping network , the expression is: ; in, represents a graph neural network, k represents a total of k new causal events, represents the Markov model, e is the natural base, is the vector squared norm operation, Indicates the delay of event propagation.

[0008] As a preferred embodiment of the present invention, step S2 includes: S201, obtain the data flow information in the network from T large-scale network flow data sets, the expression is: ; in, Represents different node states, n∈1,2,3,…,n; Clustering algorithm based on mutual information calculation Generate W cluster centers , the expression is: ; in, It means finding the minimum subscript function. and represents two different Gaussian noise terms, and They represent the corresponding weight functions, and represents the conditional probability function, represents the marginal probability distribution function, represents the mutual information calculation, which is used to measure the dependence between parameters; S202, analyzing the clustering results, extracting representative cluster centers as prototypes, and ,prototype The extraction expression is: ; in, It means there are H prototypes in total. represents the set of all cluster center points, A represents the initialization prototype vector, and e represents the natural base; As a preferred embodiment of the present invention, step S2 further comprises the following steps: S203, continue to collect new network circulation data sets , and compare these new data with the prototypes in the existing prototype library For comparison, if the distance between a data point and the existing prototype exceeds the threshold, the prototype is updated. The expression is: ; in, represents the updated prototype, Represents the cluster center Relative to the new network circulation dataset The mathematical expectation between Represents the learning rate parameter.

[0009] As a preferred embodiment of the present invention, step S3 includes the following steps: S301, introducing adaptive learning rate And the causal event feedback at the current time t, adjust the prototype parameters, and update the prototype The evolution of is defined as a game process, and the optimization goal is to minimize the error function of the current causal event, expressed as: ; in, represents the prototype after parameter adjustment, It means that the loss function measures the degree to which the updated prototype matches the causal network. represents the gradient descent function used to adjust the loss; S302, reversely optimize the causal mapping network through a multi-level feedback structure, the expression is: ; in, Indicates that after Dynamic Causal Mapping Network after Time Delay Prototype with adjusted parameters The change of mutual information between them, I represents the mutual information contrast function; S303, when the network environment changes, the network is mapped according to the dynamic causal , from the prototype after parameter adjustment Select the prototype that suits your situation , to drive layout optimization decisions, the expression is: ; in, Represents the prototype after parameter adjustment In dynamic causal mapping networks The delay below, Represents the prototype after parameter adjustment In dynamic causal mapping networks The congestion level, Indicates that each prototype in the prototype library is compared.

[0010] As a preferred embodiment of the present invention, step S4 comprises the following steps: S401, after the introduction of the self-learning feedback mechanism, the objective function of layout optimization It will be adjusted dynamically to adapt to the ever-changing network environment. For each feedback, the weight parameter in the optimization formula is adjusted according to the real-time data. The expression is: , in, and Indicates that the weight is updated dynamically and adjusted in a timely manner according to the current network status changes. Represents a prototype that fits the current situation In dynamic causal mapping networks The congestion level, Represents a prototype that fits the current situation In dynamic causal mapping networks The next delay.

[0011] Different from the prior art, the above technical solution has the following beneficial effects: (1) This method uses causal reasoning to conduct in-depth analysis of the interaction behavior, transmission paths, and data flow changes of each node in the network, identify key causal factors, and establish a causal model to clarify the causal relationship in node layout optimization instead of relying solely on historical data or correlation. This can better predict the impact of different layout schemes on network performance, make more forward-looking and accurate decisions, and improve the overall efficiency of data circulation; (2) This method introduces a prototype learning method and constructs multiple prototypes to represent different network topologies and data traffic patterns. This method can effectively deal with insufficient data and reduce the sensitivity of the model to environmental changes. Prototype learning can not only capture different types of typical patterns in the network, but also adapt to new scenarios and requirements by continuously updating the prototypes, thereby improving the adaptability of the layout optimization model in complex dynamic environments. (3) This method combines causal reasoning with prototype learning. Causal reasoning provides guidance for prototype learning, helps identify the most influential causal factors, and combines prototype learning to perform layout optimization in a variety of complex scenarios to improve the applicability and effectiveness of the optimization strategy. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 It is a framework diagram of the method described in the specific implementation method. DETAILED DESCRIPTION

[0013] In order to explain the technical content, structural features, achieved objectives and effects of the technical solution in detail, the following is a detailed description in conjunction with specific embodiments and accompanying drawings.

[0014] like Figure 1 As shown, this embodiment provides a data node adaptive optimization method based on causal reasoning and prototype learning, including the steps of: S1, generates a dynamic causal mapping network based on the real-time network status; S2, by clustering different data circulation datasets, extracting representative prototypes, and building a prototype library that adapts to different network scenarios; S3, through the prototype symbiosis mechanism driven by causal mapping, binds the causal mapping network and the prototype generation in the prototype library to achieve real-time interaction; S4, dynamically adjust the layout optimization results through a self-learning feedback mechanism.

[0015] In step S1 of the above embodiment, a dynamic causal mapping network is proposed. Different from the traditional causal reasoning method of only performing a single causal chain analysis, this embodiment regards each change in the network environment as a potential causal event. In each layout optimization decision, it does not rely on a static causal graph, but automatically generates a "causal mapping network" based on the real-time network status. The network dynamically captures the complex relationship between nodes, traffic, bandwidth, delay and other factors through multiple recursive reasoning, forming a constantly updated dynamic causal mapping network. In step S2 of the above embodiment, prototype learning clusters different data flow data sets, extracts representative prototypes, and constructs a prototype library that adapts to different network scenarios. The prototypes in the prototype library are used to describe and capture typical network topology structures and data flow patterns, thereby reducing dependence on large-scale historical data. By continuously updating and learning new prototype patterns, the system can dynamically adapt to changes in the network environment and improve the generalization ability of layout optimization, even in scenarios where data is scarce or the network changes rapidly, it can maintain a high optimization effect; In step S3 of the above embodiment, the causal mapping and prototype generation are tightly bound through the prototype symbiosis mechanism driven by causal mapping to achieve real-time interaction between them. When the causal mapping network changes, it triggers the automatic generation and evolution of the corresponding prototype layer; and conversely, the generation of new prototypes will also reversely affect the causal mapping, further improving the causal chain of the system. Through this symbiotic mechanism, the deep integration of causal reasoning and prototype learning is achieved, ensuring that the decision of layout optimization has both a scientific basis for the causal chain and sufficient flexibility and foresight; In step S4 of the above embodiment, the result of each layout optimization is re-input through the self-learning feedback mechanism to form a closed-loop feedback. The prototype library structure is continuously updated according to the optimization results and actual network performance, ultimately improving the system's adaptive ability and optimization effect. Through this closed loop, the self-learning mechanism can continuously improve the layout optimization solution, so that the network can always maintain efficient operation in a complex environment.

[0016] In the specific implementation process of the above embodiment, step S1 also includes: S101, define each change or adjustment in the network as a causal event. Assume that at a certain time t, the change in the network, such as: node load increase, bandwidth fluctuation, etc., can be represented as an event , where i represents the i-th event, and the expression of the causal event is: ; in, Indicates the status of nodes in the network at time t, such as load, number of connections, etc. Indicates the bandwidth usage of the network at time t, express Other network performance parameters such as network delay at the moment, F represents the network state change function, which is used to capture the comprehensive impact of these changes on network performance; further, if the event Will cause events after , then we can use the causal chain function definition , the expression is: ; in, Represents the random noise term, which is used to describe the influence of other unconsidered factors. Indicates the delay of event propagation; S102, at any time t, the state of the network environment will continue to change. Traditional causal reasoning is often based on fixed causal chain analysis, while the dynamic causal mapping model in this embodiment can automatically generate a graph-structured causal mapping network according to each change. , the expression is: ; in, It represents the causal event nodes in the network generated by the state space model. represents the similarity function, Represents the causal relationship edge between causal event nodes in the graph, Represents a graph structure generating network, which generates a causal mapping network by associating causal event nodes and causal relationship edges. N represents a total of N causal events.

[0017] S103, at any time t, for the generated causal mapping network , and recursively connect it to new causal events In-depth analysis of the interaction of various factors in the network and generation of dynamic causal mapping network , this process can capture the changes in network status in real time and build complex associations between causal events, expressed as: ; in, represents a graph neural network, k represents a total of k new causal events, represents the Markov model, e represents the natural base, represents the vector squared norm operation, Indicates the delay of event propagation.

[0018] In the specific implementation process of the above embodiment, step S2 also includes: S201, cluster analysis is performed on network patterns with different topological structures. First, data flow information in the network is obtained from T large-scale network flow data sets. The expression is: ; in, Indicates different node states; Then, the clustering algorithm based on mutual information calculation is used to Generate W cluster centers , the expression is: ; in, It means finding the minimum subscript function. and represents two different Gaussian noise terms, and They represent the corresponding weight functions, and represents the conditional probability function, represents the marginal probability distribution function, represents the mutual information calculation, which is used to measure the dependence between parameters; S202: Analyze the clustering results and extract representative cluster centers as prototypes. The prototypes are used to describe different network topologies and traffic patterns. ,prototype The extraction expression is: ; in, It means there are H prototypes in total. represents the set of all cluster center points, A is the initialization prototype vector, e is the natural base, It is the vector squared norm operation.

[0019] S203, in addition, continue to collect new network circulation data sets , and compare these new data with the prototypes in the existing prototype library After comparison, if the network environment is found to have changed significantly, that is, the distance between some data points and the existing prototype exceeds the threshold, the prototype is updated. The expression is: ; in, represents the updated prototype, Represents the cluster center Relative to the new network circulation dataset The mathematical expectation between is the learning rate parameter.

[0020] In the specific implementation process of the above embodiment, step S3 includes the steps of: S301, after the updated prototype is generated, the system does not directly apply the updated prototype , but by introducing an adaptive learning rate And the causal event feedback at the current time t adjusts the prototype parameters. Assume that the updated prototype The evolution of is defined as a game process, and the optimization goal is to minimize the error function of the current causal event, expressed as: ; in, represents the prototype after parameter adjustment, Represents the updated version after gradient descent optimization, which represents the causal mapping network at the current moment After the interaction of prototype learning, the optimal prototype is updated. The loss function measures the degree of match between the prototype and the causal mapping network. When the loss is large, it means that the current prototype cannot describe the relationship of the causal mapping network well. represents the gradient descent function used to decide to adjust the loss to generate a more accurate prototype, Represents the causal mapping network generated at time t.

[0021] S302, the evolution of the prototype not only affects the layout optimization, but also reversely optimizes the causal mapping network through a multi-level feedback structure. Some key causal edges in generate feedback, thereby modifying the direction and weight of the causal edges, as follows: ; in, Indicates that after Dynamic Causal Mapping Network after Time Delay Prototype with adjusted parameters The change in mutual information between them indicates how the adjustment of the prototype affects the information transmission ability in the causal mapping network during the evolution of the causal mapping network. I represents the mutual information comparison function.

[0022] S303, using prototype library driven layout optimization, when the network environment changes, according to the dynamic causal mapping network , from the prototype after parameter adjustment Select the prototype that best suits your current situation , which is used to drive layout optimization decisions. The optimization goal is to minimize congestion and latency in the network, expressed as: ; in, Represents the prototype after parameter adjustment In dynamic causal mapping networks The delay below, Represents the prototype after parameter adjustment In dynamic causal mapping networks The congestion level, Indicates that each prototype in the prototype library is compared.

[0023] In the specific implementation process of the above embodiment, step S4 includes the steps of: S401, after the introduction of the self-learning feedback mechanism, the objective function of layout optimization It will be adjusted dynamically to adapt to the ever-changing network environment. For each feedback, the system will adjust the weight parameters in the optimization formula according to the real-time data. The expression is: ; in, is the prototype that best fits the current situation. and The science department dynamically adjusts the weights according to the current network status changes, Indicates the prototype that best fits the current situation In dynamic causal mapping networks The congestion level, Indicates the prototype that best fits the current situation In dynamic causal mapping networks The next delay.

[0024] In order to verify the effectiveness of the above embodiments, a public data set was used for verification. Specifically, the General Network Traffic data set contains a wealth of real network traffic data, which is suitable for studying traffic patterns, node layout optimization, and performance analysis in high-bandwidth and complex networks. This data set provides multi-dimensional traffic parameters, including transmission rate, latency, bandwidth utilization, etc., and is suitable for research such as network traffic modeling, congestion control, and dynamic network optimization, especially in large-scale academic and scientific research network environments. It has important application value.

[0025] As shown in the experimental results in Table 1, this method performs best in network performance optimization, especially in improving throughput, reducing latency, reducing packet loss rate and jitter, and improving bandwidth utilization. Specifically, this method increases the throughput to 1300 Mbps, far exceeding the 700Mbps of the K-nearest neighbor algorithm and the 510Mbps of the naive Bayes, and significantly reduces the latency from the original 120 ms to 35 ms, showing excellent real-time performance.

[0026] Table 1: Performance comparison experimental results of different network optimization methods

[0027] In terms of packet loss rate, this method reduces it to 0.2%, while the K-nearest neighbor algorithm and naive Bayes only reduce it to 0.9% and 1.5% respectively, indicating its outstanding performance in data transmission reliability. In addition, this method is also particularly good at jitter control, reducing jitter from 30ms to 5ms, significantly improving the stability of transmission. In terms of bandwidth utilization, this method also reaches 93%, which is much higher than other methods, showing efficient use of network resources. At the same time, this method also performs well in network congestion and path optimization, successfully controlling network congestion to a "low" level and achieving "efficient" path optimization, significantly improving the efficiency and reliability of data transmission. In contrast, the K-nearest neighbor algorithm performs moderately in most indicators, while the optimization effect of the naive Bayes algorithm is poor, and it is more suitable for scenarios with low performance requirements. Overall, this method shows significant performance advantages in a network environment with high load and large-scale data flow, and is suitable for complex network environments that require efficient and stable transmission.

[0028] It should be noted that, although the above embodiments have been described in this article, the patent protection scope of the present invention is not limited thereby. Therefore, based on the innovative concept of the present invention, changes and modifications made to the embodiments described herein, or equivalent structures or equivalent process changes made using the contents of the present invention specification and drawings, directly or indirectly applying the above technical solutions to other related technical fields, are all included in the patent protection scope of the present invention.

Claims

1. A data node adaptive optimization method based on causal reasoning and prototype learning, characterized in that: Includes steps: S1, generates a dynamic causal mapping network based on the real-time network status; S2, by clustering different data circulation data sets, extracting representative prototypes, and building a prototype library that adapts to different network scenarios; S3, through the prototype symbiosis mechanism driven by causal mapping, binds the causal mapping network and the prototype generation in the prototype library to achieve real-time interaction; S4, dynamically adjust the layout optimization results through a self-learning feedback mechanism.

2. The data node adaptive optimization method based on causal reasoning and prototype learning according to claim 1 is characterized in that: Step S1 also includes: S101, defines changes or adjustments in the network as causal events. At time t, the change in the network represents a causal event. , where i represents the i-th event, and the expression of the causal event is: ； in, represents the state of the nodes in the network at time t, Indicates the bandwidth usage of the network at time t, express The network performance parameters at each moment, F represents the network state change function; If the event Will cause events after , then through the causal chain function definition , the expression is: ； in, represents the random noise term, Indicates the delay of event propagation; S102, at any time t, the dynamic causal mapping model automatically generates a graph-structured causal mapping network based on each change , the expression is: ； in, It represents the causal event nodes in the network generated by the state space model. represents the similarity function, represents the causal relationship edge between causal event nodes, Represents a graph structure generating network, N represents a total of N causal events; S103, at any time t, for the generated causal mapping network , and recursively connect it to new causal events In-depth analysis of the interaction of various factors in the network and generation of dynamic causal mapping network , the expression is: ； in, represents a graph neural network, k represents a total of k new causal events, represents the Markov model, e is the natural base, is the vector squared norm operation, Indicates the delay of event propagation.

3. The data node adaptive optimization method based on causal reasoning and prototype learning according to claim 2 is characterized in that: Step S2 includes: S201, obtain network data flow information from T large-scale network flow data sets, expressed as: ； in, Represents different node states, n∈1,2,3,…,n; Clustering algorithm based on mutual information calculation Generate W cluster centers , the expression is: ； in, It means finding the minimum subscript function. and represents two different Gaussian noise terms, and They represent the corresponding weight functions, and represents the conditional probability function, represents the marginal probability distribution function, represents the mutual information calculation, which is used to measure the dependence between parameters; S202, analyzing the clustering results, extracting representative cluster centers as prototypes, and ,prototype The extraction expression is: ； in, It means there are H prototypes in total. Represents the set of all cluster center points, A represents the initialization prototype vector, and e represents the natural base.

4. The data node adaptive optimization method based on causal reasoning and prototype learning according to claim 3 is characterized in that: Step S2 also includes the steps of: S203, continue to collect new network circulation data sets , and compare these new data with the prototypes in the existing prototype library For comparison, if the distance between a data point and the existing prototype exceeds the threshold, the prototype is updated. The expression is: ； in, represents the updated prototype, Represents the cluster center Relative to the new network circulation dataset The mathematical expectation between Represents the learning rate parameter.

5. The data node adaptive optimization method based on causal reasoning and prototype learning according to claim 3 is characterized in that: Step S3 includes the steps of: S301, introducing adaptive learning rate And the causal event feedback at the current time t, adjust the prototype parameters, and update the prototype The evolution of is defined as a game process, and the optimization goal is to minimize the error function of the current causal event, expressed as: ； in, represents the prototype after parameter adjustment, It means that the loss function measures the degree to which the updated prototype matches the causal network. represents the gradient descent function used to adjust the loss; S302, reversely optimize the causal mapping network through a multi-level feedback structure, the expression is: ； in, Indicates that after Dynamic Causal Mapping Network after Time Delay Prototype with adjusted parameters The change of mutual information between them, I represents the mutual information contrast function; S303, when the network environment changes, the network is mapped according to the dynamic causal , from the prototype after parameter adjustment Select the prototype that suits your situation , to drive layout optimization decisions, the expression is: ； in, Represents the prototype after parameter adjustment In dynamic causal mapping networks The delay below, Represents the prototype after parameter adjustment In dynamic causal mapping networks The congestion level, Indicates that each prototype in the prototype library is compared.

6. The data node adaptive optimization method based on causal reasoning and prototype learning according to claim 5 is characterized in that: Step S4 comprises the steps of: S401, after the introduction of the self-learning feedback mechanism, the objective function of layout optimization It will be adjusted dynamically to adapt to the ever-changing network environment. For each feedback, the weight parameter in the optimization formula is adjusted according to the real-time data. The expression is: in, and Indicates that the weights are updated and adjusted dynamically in a timely manner according to the current network status changes. Represents a prototype that fits the current situation In dynamic causal mapping networks The congestion level, Represents a prototype that fits the current situation In dynamic causal mapping networks The next delay.

Images