A data planning system based on a multi-node distributed real-time topology algorithm

Abstract

The application discloses a kind of data planning systems based on multi-node distributed real-time topology algorithm, it is related to data processing technical field, including node management module, data distribution module, real-time topology algorithm module, data processing module and result summary module;The node management module is used to manage the registration of all nodes, state monitoring and fault detection and recovery;The data distribution module is used to distribute data to corresponding processing node according to optimal distribution strategy provided by real-time topology algorithm module;The real-time topology algorithm module is used to calculate and maintain the best processing path of data, and dynamically adjust data flow direction according to data characteristics and node capacity;The data processing module is used to execute data processing task;The result summary module is used to obtain the processing result of each node and is summarized.The application provides a time staggered topology update mechanism, reduces the resource conflict caused by simultaneous update.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, in particular to a data planning system based on a multi-node distributed real-time topology algorithm. Background Art

[0002] In the current era of big data and cloud computing, the high efficiency and low latency of data transmission systems have become crucial factors in various application scenarios, especially in systems that need to process large amounts of sensor data, real-time information flows, and multiple tasks concurrently. How to optimize data transmission and routing decisions has become a core issue for improving the overall performance of the system. Traditional data planning systems usually adopt a centralized architecture, relying on one or several central nodes to make routing and scheduling decisions for all data. This architecture can initially ensure system consistency and centralized management, but as the system scale increases, the computing and scheduling pressure of a single central node gradually increases, and the efficiency and stability of the system are seriously affected. To address this problem, in recent years, researchers have proposed some new distributed system architectures, such as drone path planning systems based on neural networks and deep learning. These systems optimize the drone's task allocation and path planning through the collaboration of a main update node and multiple sub-computing nodes, thereby improving the system's flexibility and robustness.

[0003] However, existing data planning systems still have limitations. Especially as the number of nodes continues to increase, the scheduling pressure on the central controller will increase rapidly, leading to bottlenecks in the system's performance. Although existing technologies optimize data transmission and routing decisions through the collaborative work of the main update node and multiple sub-computing nodes, as the number of sub-nodes increases, the gradient update information and network parameters that the central node needs to process will increase exponentially, causing bottlenecks in computing and communication. Specifically, the central control node needs to frequently receive gradient information from each sub-node and update network parameters, which may cause network communication delays in large-scale nodes and complex task scenarios, thereby affecting data planning efficiency and system stability. In addition, the connection relationship and data transmission path between distributed computing nodes may be constantly adjusted as the system changes dynamically. This real-time optimization may, in some cases, lead to data transmission delays and unbalanced resource allocation, thus affecting the overall performance of the system. Summary of the Invention

[0004] In view of the above-mentioned problems, the present invention is proposed.

[0005] Therefore, the problem to be solved by the present invention is how to provide a data planning system based on a multi-node distributed real-time topology algorithm to solve the problem in the prior art that when the number of nodes increases, the scheduling pressure of the central controller increases, which may lead to a decline in system performance and affect the performance and stability of the system.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0007] In the first aspect, an embodiment of the present invention provides a data planning system based on a multi-node distributed real-time topology algorithm, which includes a node management module, a data distribution module, a real-time topology algorithm module, a data processing module and a result summary module; the node management module is used to manage the registration, status monitoring, fault detection and recovery of all nodes; the data distribution module is used to distribute data to the corresponding processing nodes according to the optimal distribution strategy provided by the real-time topology algorithm module; the real-time topology algorithm module is used to calculate and maintain the optimal processing path of the data, and dynamically adjust the data flow direction according to the data characteristics and node capabilities; the data processing module is used to execute data processing tasks; the result summary module is used to obtain the processing results of each node and summarize them.

[0008] As a preferred solution of the data planning system based on the multi-node distributed real-time topology algorithm described in the present invention, the optimal distribution strategy refers to dynamically distributing data on different nodes based on the access frequency, size, processing capacity and storage capacity of the nodes.

[0009] As a preferred solution of the data planning system based on the multi-node distributed real-time topology algorithm of the present invention, wherein: distributing data to corresponding processing nodes refers to distributing data to corresponding processing nodes based on the processing capacity and current load of the nodes according to the optimal distribution strategy provided by the real-time topology algorithm module, including the following steps: calculating the weight of each node based on the node processing capacity and current load, the specific formula is as follows:

[0010] W i =f(C i ,L i )

[0011]

[0012] Among them, W i is the weight of each node; f is a linear function; a and b are weight coefficients; C i is the processing capacity of node i; L i is the current load of node i; according to the weight W of each node i Allocate data to each node. The data allocation ratio is as follows:

[0013]

[0014] Among them, D i is the amount of data allocated to node i; D totalis the total amount of data to be allocated; n is the total number of nodes; the weight is dynamically adjusted based on the system operation status. The specific formula for dynamic adjustment is as follows:

[0015] W i (t+1)=W i (t)+ΔW i

[0016] ΔW i =g(P i ,L i ,T)

[0017] Where ΔW i is the weight adjustment amount; g is the adjustment function; P i is the real-time performance index of node i; T is time; the specific formula of the adjustment rule is as follows:

[0018] ΔW i =k·(P target -P i )-λ·(L i -L avg )

[0019] Among them, k and λ are adjustment coefficients; P target is the target performance indicator; L avg is the average load.

[0020] As a preferred solution of the data planning system based on the multi-node distributed real-time topology algorithm of the present invention, wherein: when the system load increases instantaneously due to simultaneous updates, the method of the real-time topology algorithm module includes the following steps: decentralized planning of the update time of all nodes so that only some nodes perform topology updates at any time. The specific formula is as follows:

[0021]

[0022] Among them, T update is the update cycle; N is the total number of nodes; t i Assign an update time point to each node i; dynamically adjust the update time of each node based on the importance of the node and the load situation. The specific formula is as follows:

[0023] α i =h(I i ,L i )

[0024]

[0025] Among them, α i is the adjustment factor; h is the function; I i is the importance of node i; Li is the load condition of node i; ∈ is a small positive number; the specific formula of the adjusted update time is as follows:

[0026] t i =t i +α i ·Δt

[0027] Among them, t' i is the adjusted update time; Δt is the time adjustment step; through the inter-node communication protocol, the topology information during the update period is correctly transmitted to avoid communication errors caused by staggered update times. The specific formula for defining the communication protocol P is as follows:

[0028] R confirm (i) = (t′ i ,status)

[0029] S j (t′ i )=P(R confirm (i),S j (t))

[0030] Among them, R confirm (i) Node i sends a confirmation request before updating; S j (t' i ) is the node j at time t' i Synchronous state; S j (t) is the state of node j at time t.

[0031] As a preferred solution of the data planning system based on the multi-node distributed real-time topology algorithm described in the present invention, the inter-node communication protocol adopts a link encryption method, and encryption devices are installed at the nodes at both ends of the link. The encryption devices encrypt the transmitted data and decrypt it at the receiving end.

[0032] As a preferred solution of the data planning system based on the multi-node distributed real-time topology algorithm described in the present invention, wherein: for the peak load of network bandwidth and computing resources caused by synchronous updates, the method of the real-time topology algorithm module includes the following steps: dividing the entire network into multiple subnets, each subnet containing a certain number of nodes; independently performing topology updates within each subnet, and the update operation is limited to the nodes within the subnet; topology information updates between subnets are performed asynchronously to avoid synchronous updates of the entire network; and designing an inter-subnet communication protocol to ensure consistent topology information between subnets.

[0033] As a preferred solution of the data planning system based on the multi-node distributed real-time topology algorithm described in the present invention, the inter-subnet communication protocol adopts a symmetric encryption method, and the same key is used for encryption and decryption.

[0034] Secondly, in order to further solve the security problems existing in data processing, the present invention provides a data planning method based on a multi-node distributed real-time topology algorithm in an embodiment, which includes: using a node management module to register and monitor the initial status of all nodes; when data is input into the system, the data distribution module distributes the data to the corresponding processing node according to the optimal distribution strategy provided by the real-time topology algorithm module; using the data processing module of the target node to process the distributed data; sending the processing results to the result summary module, and the result summary module summarizes all results; using the node management module to monitor the system status, and if a fault or load abnormality is found, adjusting the topology structure or restarting the task of the faulty node.

[0035] In a third aspect, an embodiment of the present invention provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: when the computer program is executed by the processor, any step of the data planning system based on a multi-node distributed real-time topology algorithm as described in the first aspect of the present invention is implemented.

[0036] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium on which a computer program is stored, wherein: when the computer program is executed by a processor, it implements any step of the data planning system based on a multi-node distributed real-time topology algorithm as described in the first aspect of the present invention.

[0037] Beneficial effects of the present invention: The present invention provides a time-staggered topology update mechanism, where different nodes are updated at different times, thereby reducing resource conflicts caused by simultaneous updates. In addition, by dividing the nodes into multiple subnets, topology updates are performed independently within each subnet, thereby reducing synchronization overhead across the entire network, thereby improving system scalability while maintaining the real-time performance of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] To more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. Those skilled in the art can also derive other drawings based on these drawings without inventive effort. Among them:

[0039] Figure 1 This is a system block diagram of the data planning system of the present invention in Example 1.

[0040] Figure 2 This is a flow chart of the data planning system based on the multi-node distributed real-time topology algorithm of the present invention in Example 1.

[0041] Figure 3Flow chart of time staggering of the multi-node distributed real-time topology algorithm of the present application in Example 1.

[0042] Figure 4 Flow chart of node partitioning of the multi-node distributed real-time topology of the present application in Example 1.

[0043] Figure 5 Structural schematic diagram of the computer device in Example 3. DETAILED DESCRIPTION

[0044] In order to make the above objectives, features and advantages of the present application more apparent, a detailed description of the specific embodiments of the present application will be given below with reference to the accompanying drawings.

[0045] In the following description, a large number of specific details are set forth in order to facilitate a thorough understanding of the present application, but the present application can also be implemented in other ways different from those described herein, and those skilled in the art can make similar generalizations without departing from the concept of the present application, so the present application is not limited to the specific embodiments disclosed below.

[0046] Secondly, the "one embodiment" or "embodiment" referred to herein means that a specific feature, structure or characteristic can be included in at least one implementation of the present application. "In one embodiment" appearing in different places in the specification does not mean the same embodiment, nor is it an embodiment that is independent of or mutually exclusive of other embodiments.

[0047] Example 1

[0048] Reference Figures 1-4 For the first embodiment of the present application, the embodiment provides a data planning system based on a multi-node distributed real-time topology algorithm, comprising:

[0049] The node management module, the data distribution module, the real-time topology algorithm module, the data processing module and the result summarizing module, wherein as shown in Figure 1 The system block diagram of the data planning system of the present application.

[0050] It should be noted that in the system initialization stage of the present application, the node management module is first started, and the registration and initial state monitoring of all nodes are completed; when data is input into the system, the data distribution module distributes the data to the corresponding processing node according to the optimal distribution strategy provided by the real-time topology algorithm module; the distributed data is processed by the data processing module of the target node; after the processing is completed, the result is sent to the result summarizing module, and the result summarizing module summarizes all the results; the node management module monitors the system state, and if a fault or abnormal load is found, the topology structure is adjusted or the task of the faulty node is restarted to ensure the stable operation of the system.

[0051] Preferably, the node management module is used to manage the registration, status monitoring, and fault detection and recovery of all nodes, and to achieve real-time updating of node information by monitoring the node status.

[0052] Preferably, Figure 2 The figure shows a flow chart of the data planning system based on the multi-node distributed real-time topology algorithm of the present invention. The data distribution module is used to distribute data to the corresponding processing nodes according to the optimal distribution strategy provided by the real-time topology algorithm module, taking into account the processing capacity and current load conditions of the nodes to achieve load balancing.

[0053] Specifically, the optimal distribution strategy refers to dynamically distributing data on different nodes based on the data's access frequency, size, node processing capabilities, and storage capabilities to ensure efficient data access and processing.

[0054] Specifically, distributing data to corresponding processing nodes means distributing data to corresponding processing nodes based on the processing capacity and current load of the nodes according to the optimal distribution strategy provided by the real-time topology algorithm module, including the following steps: calculating the weight of each node based on the processing capacity and current load of the node. The specific formula is as follows:

[0055] W i =f(C i ,L i )

[0056]

[0057] Among them, W i is the weight of each node, where the weight of each node is defined as W i is a comprehensive index; f is a linear function; a and b are weight coefficients; C i is the processing capacity of node i; L i is the current load of node i.

[0058] According to the weight W of each node i Allocate data to each node to achieve reasonable distribution of data among multiple nodes. The data distribution ratio is as follows:

[0059]

[0060] Among them, D i is the amount of data allocated to node i; D total is the total amount of data to be distributed; n is the total number of nodes.

[0061] Combined with the system operation status, the weight is dynamically adjusted to ensure stable system operation. The specific formula for dynamic adjustment is as follows:

[0062] W i(t+1)=W i (t)+ΔW i

[0063] ΔW i =g(P i ,L i ,T)

[0064] Where ΔW i is the weight adjustment amount, which is calculated according to the real-time performance and load changes of the node; g is the adjustment function; P i is the real-time performance index of node i; T is time.

[0065] The specific formula for adjusting the rules is as follows:

[0066] ΔW i =k·(P target -P i )-λ·(L i -L avg )

[0067] Among them, k and λ are adjustment coefficients; P target is the target performance indicator; L avg is the average load.

[0068] Preferably, the present invention dynamically adjusts the distribution strategy by comprehensively considering the node processing capabilities and load conditions, effectively avoiding resource waste and ensuring maximum data processing efficiency. Compared with traditional data distribution, which is difficult to balance load balancing and efficient processing, the present invention achieves efficient data distribution through weight calculation and dynamic adjustment.

[0069] Preferably, Figure 3 The figure shows a time-staggered flow chart of a multi-node distributed real-time topology algorithm of the present invention. The real-time topology algorithm module is used to calculate and maintain the optimal processing path of data and dynamically adjust the data flow according to data characteristics and node capabilities.

[0070] Specifically, such as Figure 4 The flowchart of the multi-node distributed real-time topology node partitioning of the present invention is shown. When the system load increases instantaneously due to simultaneous updates, the method of the real-time topology algorithm module includes the following steps: the update time of all nodes is decentralized and planned so that only some nodes perform topology updates at any time. The specific formula is as follows:

[0071]

[0072] Among them, T update is the update cycle; N is the total number of nodes; t i Assign an update time point to each node i to achieve decentralized update.

[0073] Based on the importance of the node and the load conditions, the update time of each node is dynamically adjusted to achieve the most optimized update processing. The specific formula is as follows:

[0074] α i =h(I i ,L i )

[0075]

[0076] Among them, α i is the adjustment factor; h is the function; I i is the importance of node i; L i is the load condition of node i; ∈ is a small positive number.

[0077] The specific formula for the adjusted update time is as follows:

[0078] t i =t i +α i ·Δt

[0079] Among them, t' i is the updated time after adjustment; Δt is the step size of time adjustment, which is used to control the amplitude of adjustment.

[0080] The inter-node communication protocol ensures that topology information is correctly transmitted during the update period, avoiding communication errors caused by staggered update times. The specific formula for defining the communication protocol P is as follows:

[0081] R confirm (i) = (t′ i ,status)

[0082] S j (t′ i )=P(R confirm (i),S j (t))

[0083] Among them, R confirm (i) Node i sends a confirmation request before updating; S j (t' i ) is the node j at time t' i The synchronization state indicates that other nodes j receive the confirmation request R confirm 9i), synchronize its status according to the communication protocol P; S j (t) is the state of node j at time t.

[0084] Preferably, independent updates within a subnet reduce the synchronization overhead of the entire network and improve system scalability. Asynchronous updates reduce communication delays and resource usage. Compared with the high cost and long latency of synchronous updates of the entire network, the present invention optimizes the update mode by introducing subnet division.

[0085] Furthermore, the inter-node communication protocol adopts a link encryption method. By installing encryption devices at the nodes at both ends of the link, the encryption devices encrypt the transmitted data and decrypt it at the receiving end to ensure the security of data transmission.

[0086] Specifically, for the peak load of network bandwidth and computing resources caused by synchronous updates, the method of the real-time topology algorithm module includes the following steps: dividing the entire network into multiple subnets, each subnet containing a certain number of nodes.

[0087] Topology updates are performed independently within each subnet, and the update operation is limited to nodes within the subnet.

[0088] Topology information updates between subnets are performed asynchronously to avoid synchronous updates of the entire network.

[0089] Design inter-subnet communication protocols to ensure consistent topology information between subnets.

[0090] Furthermore, the inter-subnet communication protocol adopts symmetric encryption, which uses the same key for encryption and decryption to ensure the security of large amounts of data during transmission.

[0091] Preferably, a data processing module is located in each node and is used to perform data processing tasks, including data cleaning, integration and analysis.

[0092] Preferably, the result summary module is used to obtain and summarize the processing results of each node.

[0093] In summary, the present invention provides a time-staggered topology update mechanism, where different nodes are updated at different times, reducing resource conflicts caused by simultaneous updates. It also divides the nodes into multiple subnets, where topology updates are performed independently within each subnet, reducing synchronization overhead across the entire network, thereby improving system scalability while maintaining the system's real-time performance.

[0094] Example 2 is an embodiment of the present invention, which provides a data planning method based on a multi-node distributed real-time topology algorithm, including: using a node management module to register and monitor the initial status of all nodes; when data is input into the system, the data distribution module distributes the data to the corresponding processing node according to the optimal distribution strategy provided by the real-time topology algorithm module; using the data processing module of the target node to process the distributed data; sending the processing results to the result summary module, and the result summary module summarizes all results; using the node management module to monitor the system status, and if a fault or load abnormality is found, adjusting the topology structure or restarting the task of the faulty node.

[0095] Example 3 is an embodiment of the present invention, which is different from the previous embodiment in that:

[0096] like Figure 5 As shown, if the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0097] The logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device). For purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0098] More specific examples (a non-exhaustive list) of the computer-readable medium include the following: an electrical connection (electronic) having one or more wires, a portable computer diskette (magnetic), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, and a portable compact disc read-only memory (CDROM). Additionally, the computer readable medium can be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, for example via optical scanning of the paper or other medium, then compiled, interpreted, or otherwise processed in a suitable manner, if necessary, and then stored in a computer memory.

[0099] It should be understood that aspects of the application can be implemented in hardware, software, firmware or combinations thereof. In the embodiments described above, various steps or methods can be implemented, in part, or in whole, in software, or firmware that is stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any of the following techniques, or combinations thereof, can be utilized to implement at least a portion of the described functionality: discrete logic circuitry having logic gates for implementing logic functions upon data signals, application specific integrated circuits having appropriate combinational logic gates, programmable gate arrays (PGA), field programmable gate arrays (FPGA), and so forth.

[0100] Embodiment 4, as an embodiment of the present application, provides a data planning system based on a multi-node distributed real-time topology algorithm. In order to verify the beneficial effects of the present application, a simulation experiment is carried out for scientific demonstration.

[0101] In this example, a distributed system containing 100 nodes is simulated. The 100 nodes are divided into 10 groups, each group containing 10 nodes. Every fixed time interval of 1 hour, a group of nodes is randomly selected for topology update, and the other groups of nodes remain normal operation. During the update process, the data of the updated nodes and other nodes are synchronized through the inter-node negotiation protocol. After the update is completed, the next group of nodes is selected for the next round of update.

[0102] By using the time staggered strategy, the system load is balanced during the update process. The update time is dynamically adjusted to adapt to the changes in the system running state. The inter-node communication protocol is optimized to ensure the correct transmission of topology information during the update period. Table 1 shows the comparison of node load and update efficiency.

[0103] Table 1 Comparison of node load and update efficiency

[0104] Experimental subjects Node 1 Node 2 Node 3 Node 4 Initial load (%) 25 30 20 35 Load after topology update (%) 22 28 18 32 Task completion time (seconds) 120 115 130 110

[0105] As can be seen from the above table, the node load of the application is uniformly distributed, and after the topology is updated, the load is reduced by about 10%, compared with the traditional system, the application significantly improves the data processing efficiency, and avoids the phenomenon of single node overload; by adopting the distributed topology updating algorithm, the updating strategy is optimized, and the system resource consumption is significantly reduced, compared with the traditional synchronous updating mechanism, the updating efficiency is effectively improved.

[0106] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present application and are not limiting. Although the present application has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solutions of the present application can be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present application, which should be covered in the scope of the claims of the present application.

Claims

1. A data planning system based on a multi-node distributed real-time topology algorithm, characterized by: include: Node management module, data distribution module, real-time topology algorithm module, data processing module and result summary module; The node management module is used to manage the registration, status monitoring, and fault detection and recovery of all nodes; The data distribution module is used to distribute data to corresponding processing nodes according to the optimal distribution strategy provided by the real-time topology algorithm module; The real-time topology algorithm module is used to calculate and maintain the optimal processing path for data, and dynamically adjust the data flow according to data characteristics and node capabilities; The data processing module is used to perform data processing tasks; The result summary module is used to obtain and summarize the processing results of each node; For simultaneous updates that cause a transient increase in system load, the method of the real-time topology algorithm module includes the following steps: The update time of all nodes is decentralized and planned so that only some nodes perform topology updates at any time. The specific formula is as follows: Among them, T update is the update cycle; N is the total number of nodes; t i Assign an update time point to each node i; The update time of each node is dynamically adjusted based on the importance and load of the node. The specific formula is as follows: a i =h(I i ,L i ) Among them, α i is the adjustment factor; h is the function; I i is the importance of node i; L i is the load condition of node i;∈ is a small positive number; The specific formula for the adjusted update time is as follows: t′ i =t i +a i ·Δt Among them, t' i is the updated time after adjustment; Δt is the step size of time adjustment; The inter-node communication protocol ensures that topology information is correctly transmitted during the update period, avoiding communication errors caused by staggered update times. The specific formula for defining the communication protocol P is as follows: R confirm (i)=(t′ i ,status) S j (t′ i )=P(R confirm (i),S j (t)) Among them, R confirm (i) Node i sends a confirmation request before updating; S j (t' i ) is the node j at time t' i Synchronous state; S j (t) is the state of node j at time t.

2. The data planning system based on a multi-node distributed real-time topology algorithm according to claim 1, characterized in that: The optimal distribution strategy is to dynamically distribute data on different nodes based on the access frequency and size of the data, the processing capability and the storage capability of the nodes.

3. The data planning system based on a multi-node distributed real-time topology algorithm according to claim 2, characterized in that: Distributing the data to the corresponding processing nodes refers to distributing the data to the corresponding processing nodes based on the processing capabilities and current load conditions of the nodes according to the optimal distribution strategy provided by the real-time topology algorithm module, including the following steps: Based on the node processing capacity and current load, the weight of each node is calculated. The specific formula is as follows: W i =f(C i ,L i ) Among them, W i is the weight of each node; f is a linear function; a and b are weight coefficients; C i is the processing capacity of node i; L i is the current load of node i; According to the weight W of each node i Allocate data to each node. The data allocation ratio is as follows: Among them, D i is the amount of data allocated to node i; D total is the total amount of data to be allocated; n is the total number of nodes; The weight is adjusted dynamically based on the system operation status. The specific formula for dynamic adjustment is as follows: W i (t+1)=W i (t)+ΔW i ΔW i =g(P i ,L i ,T) Where ΔW i is the weight adjustment amount; g is the adjustment function; P i is the real-time performance index of node i; T is time; The specific formula for adjusting the rules is as follows: ΔW i =k·(P target -P i )-λ·(L i -L avg ) Among them, k and λ are adjustment coefficients; P target is the target performance indicator; L avg is the average load.

4. The data planning system based on a multi-node distributed real-time topology algorithm according to claim 3, characterized in that: The inter-node communication protocol adopts a link encryption method. By installing encryption devices at the nodes at both ends of the link, the encryption devices encrypt the transmitted data and decrypt it at the receiving end.

5. The data planning system based on a multi-node distributed real-time topology algorithm according to claim 4, characterized in that: For synchronous updates that result in peak loads on network bandwidth and computing resources, the method of the real-time topology algorithm module includes the following steps: Divide the entire network into multiple subnets, each of which contains a certain number of nodes; Topology updates are performed independently within each subnet, and the update operation is limited to nodes within the subnet; Topology information updates between subnets are performed asynchronously to avoid synchronous updates across the entire network; Design inter-subnet communication protocols to ensure consistent topology information between subnets.

6. The data planning system based on a multi-node distributed real-time topology algorithm according to claim 5, characterized in that: The inter-subnet communication protocol adopts a symmetric encryption method, using the same key for encryption and decryption.

7. A data planning method based on a multi-node distributed real-time topology algorithm, based on the data planning system based on a multi-node distributed real-time topology algorithm according to any one of claims 1 to 6, characterized in that: include, Use the node management module to register and monitor the initial status of all nodes; When data is input into the system, the data distribution module distributes the data to the corresponding processing nodes according to the optimal distribution strategy provided by the real-time topology algorithm module; Use the data processing module of the target node to process the distributed data; The processing results are sent to the result summary module, which summarizes all the results; The node management module is used to monitor the system status. If a fault or load anomaly is found, the topology structure is adjusted or the task of the faulty node is restarted.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the data planning system based on a multi-node distributed real-time topology algorithm according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the data planning system based on a multi-node distributed real-time topology algorithm according to any one of claims 1 to 6 are implemented.

Images