A hierarchical and distributed edge data aggregation and reporting method and device

Abstract

The present invention discloses a hierarchical distributed edge data aggregation and reporting method, including dividing the data aggregation process into S stages when the edge network processes tasks, and constructing a hierarchical distributed edge data aggregation and reporting model for each stage; each stage of the data aggregation process includes multiple clusters, each cluster includes multiple cluster members, and the cluster members are edge data centers EDC. An edge data center is selected from each cluster as the cluster center of the current cluster, and the cluster members are divided according to the selected cluster center, and the cluster division of all EDCs is finally completed, and the data stored in the edge data center is transferred to the cluster center of the current stage; the hierarchical distributed edge data aggregation and reporting model includes a current stage task completion time model and a work completion total time model. The present invention also discloses a hierarchical distributed edge data aggregation and reporting device to better meet the needs of intelligent applications for low latency, high bandwidth, and efficient processing.

Description

Technical Field

[0001] The present invention relates to the technical fields of electronics, communications and information engineering, and in particular to a hierarchical and distributed edge data aggregation and reporting method and device. Background Art

[0002] With the rapid development of technologies such as artificial intelligence, the Internet of Things, and 5G, traditional centralized networks and cloud computing architectures face many challenges in dealing with the communication and computing needs of massive data. This is mainly reflected in the following aspects: First, there will be significant delays in the process of transmitting all data to the cloud, and data loss may also occur. Secondly, centralized cloud computing may lead to bandwidth bottlenecks and easily cause congestion when processing a large number of concurrent requests. In other words, the communication and computing of massive data make it impossible for networks and cloud computing to meet the high bandwidth and low latency requirements of current emerging technologies and industries. Therefore, the problems and challenges faced by traditional centralized networks and cloud computing architectures in various fields are driving the expansion of computing power from cloud computing to edge networks.

[0003] Edge network refers to an architecture that distributes computing and data processing capabilities to the edge of the network to improve the speed and efficiency of data processing. Compared with traditional centralized cloud computing, edge networks deploy resources closer to users and data sources, such as on smart devices, routers, or edge servers. By computing and processing data locally on the edge network, not only can the data transmission time be significantly reduced, but also the amount of data transmitted to the cloud can be reduced, thereby optimizing bandwidth resources and reducing network congestion. In the edge network, data is stored in multiple different edge devices (nodes). It is usually necessary to aggregate the data of edge nodes to the edge data center first, and then upload it to the cloud through the edge data center. Therefore, how to aggregate the data stored in the edge nodes to the edge data center, minimize the data aggregation time, and better meet the needs of intelligent applications for low latency, high bandwidth, and efficient processing is one of the current challenges.

[0004] As for how edge nodes aggregate data to edge data centers, existing solutions can be divided into two types: centralized data aggregation solutions and distributed data aggregation solutions. The centralized data aggregation solution is to select only one cluster center, which collects and saves information about all nodes in the network. If any node wants to communicate with any other node, it needs to request the cluster center. The distributed data aggregation algorithm can select multiple data aggregation centers, and the remaining nodes can choose which data aggregation center to aggregate the stored or processed data resources based on indicators such as geographical location, latency, bandwidth consumption, and energy consumption. Because the processing capacity of a single edge data center is limited, and in the face of the current growing demand for data-intensive services, centralized data aggregation solutions become inefficient or infeasible. That is, it is necessary to select a distributed data aggregation solution to complete the calculation and processing of data on the edge by aggregating a large amount of data transmission to multiple edge data centers. However, the selection of a distributed data aggregation solution for data aggregation also has the following problems: On the one hand, when the selected edge data center is far away from the edge node that stores the data required for some tasks, it will increase the transmission latency. On the other hand, if the amount of data stored in the edge data center is large and the network resources between edge centers are tight, transmitting data from multiple edge centers to an edge data aggregation center (cluster center) will lead to severe competition for network resources near the edge aggregation center, thus affecting the stability of data transmission. Therefore, how to reasonably select edge data centers as cluster centers to complete edge-to-cloud data aggregation and reporting is also one of the current challenges. Summary of the invention

[0005] The technical problem to be solved by the present invention is to overcome the shortcomings of the prior art and provide a hierarchical distributed edge data aggregation reporting method and device. On the one hand, the present invention is to more efficiently handle the problem of big data transmission aggregation of multiple edge data centers in a complex environment, by combining computing and network resources, while minimizing data aggregation time while ensuring low bandwidth consumption; on the other hand, it is more reasonable to select edge data centers.

[0006] The present invention adopts the following technical solutions to solve the above technical problems:

[0007] In a first aspect, a hierarchical distributed edge data aggregation and reporting method proposed in the present invention includes:

[0008] When edge network processes tasks, the data aggregation process is divided into In each stage, a hierarchical and distributed edge data aggregation and reporting model is constructed. is an integer greater than 1;

[0009] Each stage of the data aggregation process includes multiple clusters, each cluster includes multiple cluster members, and the cluster members are edge data centers. , select an edge data center in each cluster As the cluster center of the current cluster , according to the selected cluster center, divide the cluster members, and finally complete all Cluster partitioning, stored in edge data centers The data is transferred to the cluster center of the current stage;

[0010] The hierarchical distributed edge data aggregation reporting model includes the task completion time of the current stage Model and total time to complete the work Model, where

[0011] Current stage task completion time Define the model Stage task completion time The model is as follows:

[0012] ;

[0013] Total time to complete the work The model is:

[0014] ;

[0015] in, Indicated in The time to complete the task in the stage, For the Stage, Cluster Mid-edge data center With other edge data centers The communication time between For the Stage, Cluster Mid-edge data center Data processing time, is the data transmission rate in the link, For the Phase, Edge Data Center The amount of data, For the Phase, Edge Data Center Transmit data to other edge data centers The network available bandwidth between For the Phase, Edge Data Center Transmit data to other edge data centers The link distance between For the stage, Available computing resources, For the The time taken by the last cluster to complete the convergence task in the remaining clusters except the bottleneck cluster in the stage. It is a process from the second stage to the In each stage, the and The value after the difference.

[0016] As a further optimization scheme of the hierarchical distributed edge data aggregation and reporting method described in the present invention, the bottleneck cluster refers to the last cluster to complete the data aggregation task.

[0017] As a further optimization scheme of the hierarchical distributed edge data aggregation and reporting method described in the present invention, The construction process is as follows:

[0018] definition Equal to The time consumed by the bottleneck cluster in the stage. The bottleneck cluster refers to the last cluster to complete the data aggregation task;

[0019] for:

[0020] (1)

[0021] in, For the Phase, Cluster The time to complete data aggregation;

[0022] Defined as:

[0023] (2)

[0024] (3)

[0025] in, For the Phase, Edge Data Center Transmit data to other edge data centers hour, The time required to send the data; For the Phase, Edge Data Center Transmit data to other edge data centers the time required;

[0026] (4)

[0027] in, For the stage, Available computing resources;

[0028] (5).

[0029] As a further optimization scheme of the hierarchical distributed edge data aggregation and reporting method described in the present invention,

[0030] (6)

[0031] in, For each stage and difference.

[0032] As a further optimization scheme of the hierarchical distributed edge data aggregation and reporting method described in the present invention, calculation The method is as follows:

[0033] Step 1: When When initialization ;

[0034] Step 2: If The edge data center in the bottleneck cluster is not the first If the cluster center of the stage is not reached, directly execute step 3;

[0035] If The edge data center in the bottleneck cluster is still the first The cluster center of the stage is determined as follows:

[0036] If the cluster center is The bottleneck cluster center of the stage, and , update stage ;

[0037] If the cluster center is The bottleneck cluster center of the stage, and ,but remain unchanged;

[0038] If the cluster center is not The bottleneck cluster center of the stage, and , then update the current stage , ;

[0039] If the cluster center is not The bottleneck cluster center of the stage, and , then update the current stage ;

[0040] Step 3: Update the data aggregation time of the edge data center in the cluster to , and judge:

[0041] if ,renew ;

[0042] if ,renew ;

[0043] if , , All remain unchanged;

[0044] Step 4: According to step 2 or step 3, and value, update the current entry into the next stage ;

[0045] Step 5 ,if , return to step 2; otherwise the loop ends, .

[0046] As a further optimization scheme of the hierarchical distributed edge data aggregation and reporting method described in the present invention, an edge data center is selected in each cluster. As the cluster center of the current cluster , according to the selected cluster center, divide the cluster members, and finally complete all The cluster division is as follows:

[0047] Step ①, select the initial cluster center;

[0048] consider The shortest path P between min and available bandwidth in network resources As a decision metric, determine Selection of initial cluster centers in the phase;

[0049] No. Phase indivual The decision metric The definition is as follows:

[0050] (7)

[0051] in, Indicates The weight value of the average shortest path distance between an edge data center and other edge data centers; Indicates The maximum average available bandwidth between an edge data center and other edge data centers;

[0052] right Define two constraints:

[0053] Constraint 1: ;

[0054] Constraint 2: ;

[0055] in, For the current stage the number of

[0056] According to constraints 1 and 2, let , then define :

[0057] (8)

[0058] in, For the The sum of the average shortest path distances between an edge data center and other edge data centers, For the The average shortest path distance between an edge data center and other edge data centers;

[0059] According to formula (8), the final decision metric is obtained The formula is:

[0060] (9)

[0061] According to formula (9), choose The largest value Edge data centers are currently The initial cluster center of the stage;

[0062] Step ②: Select the remaining cluster centers;

[0063] Defining cluster centers , let the number of clusters to be divided be , ,but ,Right now For the cluster centers; select the shortest path between cluster centers as the judgment criterion to complete the selection of the remaining cluster centers; the details are as follows:

[0064] when When The largest as a cluster center;

[0065] when When of As the first cluster center , then select Leave The furthest As the second cluster center ;

[0066] when When , assuming that you have selected Cluster centers, , then in the selection When the cluster center is The cluster center farthest As the cluster centers;

[0067] Step 3: Divide clusters;

[0068] when , then there is only one cluster in the current stage, and there is no need to divide the clusters. All are cluster members at the initial cluster center;

[0069] when When , the clustering method is as follows:

[0070] Step A: First, compare the initial cluster center and other remaining P min To sort in ascending order, select indivual The cluster member that serves as the initial cluster center;

[0071] Define the allocated Number of The interval is:

[0072] (10)

[0073] Among them, N+ represents a natural number except 0;

[0074] Step B, then, The remaining of To sort in ascending order, select indivual Cluster members who are cluster centers;

[0075] Step C: Repeat step B until the - 1 cluster center Finally, the remaining clusters that have not been divided Cluster Center The cluster members.

[0076] As a further optimization scheme of a hierarchical distributed edge data aggregation and reporting method described in the present invention, the maximum average available bandwidth is the average value of multiple maximum available bandwidths, and the maximum available bandwidth is the value of the link with the smallest available bandwidth in the path.

[0077] In a second aspect, a hierarchical and distributed edge data aggregation and reporting device includes a data aggregation and reporting module,

[0078] The data aggregation reporting module is used to divide the data aggregation process into In each stage, a hierarchical and distributed edge data aggregation and reporting model is constructed. is an integer greater than 1;

[0079] Each stage of the data aggregation process includes multiple clusters, each cluster includes multiple cluster members, and the cluster members are edge data centers. , select an edge data center in each cluster As the cluster center of the current cluster , according to the selected cluster center, divide the cluster members, and finally complete all Cluster partitioning, stored in edge data centers The data is transferred to the cluster center of the current stage;

[0080] The hierarchical distributed edge data aggregation reporting model includes the task completion time of the current stage Model and total time to complete the work Model, where

[0081] Current stage task completion time Define the model Stage task completion time The model is as follows:

[0082] ;

[0083] Total time to complete the work The model is:

[0084] ;

[0085] in, Indicated in The time to complete the task in the stage, For the Stage, Cluster Mid-edge data center With other edge data centers The communication time between For the Stage, Cluster Mid-edge data center Data processing time, is the data transmission rate in the link, For the Phase, Edge Data Center The amount of data, For the Phase, Edge Data Center Transmit data to other edge data centers The available network bandwidth between For the Phase, Edge Data Center Transmit data to other edge data centers The link distance between For the stage, Available computing resources, For the The time taken by the last cluster to complete the convergence task in the remaining clusters except the bottleneck cluster in the stage. It is a process from the second stage to the In each stage, the and The value after the difference.

[0086] In a third aspect, an embodiment of the present invention further provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, wherein when the processor executes the computer program, the steps of the hierarchically distributed edge data aggregation and reporting method as described in the first aspect above or any corresponding embodiment thereof are implemented.

[0087] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the steps of the hierarchical distributed edge data aggregation and reporting method as described in the first aspect above or any corresponding embodiment thereof.

[0088] Compared with the prior art, the present invention adopts the above technical solution and has the following technical effects:

[0089] (1) The present invention optimizes the data aggregation and reporting scheme. Specifically, by combining computing and network resources, the data aggregation time is minimized while ensuring low bandwidth consumption. A distributed data aggregation scheme is adopted, and the data aggregation process is divided into multiple stages. The data distributed in various locations is processed by using a parallel mode in each stage.

[0090] (2) The present invention also optimizes the selection of cluster centers and the division of clusters. Specifically, by formulating a determination metric as the selection criterion for the initial cluster center, a selection scheme for the remaining cluster centers is designed based on the comparison of the distances between cluster centers. Finally, a cluster division method is designed and proposed to divide cluster members for each cluster center.

[0091] (3) When processing data aggregation and reporting, the present invention has better performance in terms of time and bandwidth consumption, and can better meet the needs of intelligent applications for low latency, high bandwidth, and efficient processing. In the face of the communication and computing needs of processing massive data, the method proposed in the present invention can be well applied to the current edge network environment. Because the method proposed in the present invention comprehensively considers factors such as data aggregation time and available bandwidth of node links in the selection of cluster centers, the division of cluster members, and the construction of data aggregation models, and finally forms a systematic edge data aggregation and reporting method, it can more efficiently process and aggregate data on the edge, further complete edge-to-cloud data aggregation and reporting, and improve the efficiency of edge data reporting. BRIEF DESCRIPTION OF THE DRAWINGS

[0092] Figure 1 It is a construction diagram of the hierarchical and distributed edge data aggregation and reporting model;

[0093] Figure 2 1 is an example diagram to prove the effectiveness of the TT optimization model proposed in the present invention; wherein (a) is an example of the completion time of the work plan before optimization, (b) is an example of the completion time of the work plan after optimization, (c) is an example of the optimal completion time of the work plan after optimization, and (d) is an example of the worst completion time of the work plan after optimization;

[0094] Figure 3 It is a flow chart of the optimal selection method of cluster centers;

[0095] Figure 4 is the task completion time of the current stage of the two algorithms in a 25-node edge network Comparison chart;

[0096] Figure 5 is the task completion time of the two algorithms in the current stage under the 38-node edge network Comparison chart;

[0097] Figure 6is the task completion time of the current stage of the two algorithms in a 50-node edge network Comparison chart;

[0098] Figure 7 is the total time to complete the work of the two methods The comparison result diagram of

[0099] Figure 8 is the total time to complete the work of different algorithms The comparison result diagram of

[0100] Fig. 9 This is a comparison chart of bandwidth consumption of different algorithms. DETAILED DESCRIPTION

[0101] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0102] Figure 1 This is a diagram of building a hierarchical distributed data edge data aggregation and reporting model. Figure 1 As shown in Figure 1, a hierarchical distributed edge data aggregation and reporting model is constructed. Each stage is defined to include multiple clusters. In each cluster, data is stored in multiple edge data centers (Edge data center). )) Each cluster needs to select one As the cluster center of the current cluster ), the data of the edge data center must be transferred to the cluster center of the current stage. In addition, the time for the current cluster to complete data aggregation is defined as CT, and the task completion time of the current stage is defined as The total time to complete the work is , is equal to the sum of the completion time of multiple stages. Therefore, if you want to reduce , we must shorten .

[0103] A hierarchical distributed edge data aggregation and reporting method, comprising:

[0104] When edge network processes tasks, the data aggregation process is divided into In each stage, a hierarchical and distributed edge data aggregation and reporting model is constructed. is an integer greater than 1;

[0105] Each stage of the data aggregation process includes multiple clusters, each cluster includes multiple cluster members, and the cluster members are edge data centers. , select an edge data center in each cluster As the cluster center of the current cluster , according to the selected cluster center, divide the cluster members, and finally complete all Cluster partitioning, stored in edge data centers The data is transferred to the cluster center of the current stage;

[0106] The hierarchical distributed edge data aggregation reporting model includes the task completion time of the current stage Model and total time to complete the work Model, where

[0107] Current stage task completion time Define the model Stage task completion time The model is as follows:

[0108] ;

[0109] Total time to complete the work The model is:

[0110] ;

[0111] in, Indicated in The time to complete the task in the stage, For the Stage, Cluster Mid-edge data center With other edge data centers The communication time between For the Stage, Cluster Mid-edge data center Data processing time, is the data transmission rate in the link, For the Phase, Edge Data Center The amount of data, For the Phase, Edge Data Center Transmit data to other edge data centers The available network bandwidth between For the Phase, Edge Data Center Transmit data to other edge data centers The link distance between For the stage, Available computing resources, For the The time taken by the last cluster to complete the convergence task in the remaining clusters except the bottleneck cluster in the stage. It is a process from the second stage to the In each stage, the and The bottleneck cluster refers to the last cluster that completes the data aggregation task.

[0112] The construction process is as follows:

[0113] definition Equal to The time consumed by the bottleneck cluster in the stage. The bottleneck cluster refers to the last cluster to complete the data aggregation task;

[0114] for:

[0115] (1)

[0116] in, For the Phase, Cluster The time to complete data aggregation;

[0117] Defined as:

[0118] (2)

[0119] (3)

[0120] in, For the Phase, Edge Data Center Transmit data to other edge data centers hour, The time required to send the data; For the Phase, Edge Data Center Transmit data to other edge data centers the time required;

[0121] (4)

[0122] in, For the stage, Available computing resources;

[0123] (5)

[0124] (6)

[0125] in, For each stage and difference.

[0126] calculate The method is as follows:

[0127] Step 1: When When initialization ;

[0128] Step 2: If The edge data center in the bottleneck cluster is not the first If the cluster center of the stage is not reached, directly execute step 3;

[0129] If The edge data center in the bottleneck cluster is still the first The cluster center of the stage is determined as follows:

[0130] If the cluster center is The bottleneck cluster center of the stage, and , update stage ;

[0131] If the cluster center is The bottleneck cluster center of the stage, and ,but remain unchanged;

[0132] If the cluster center is not The bottleneck cluster center of the stage, and , then update the current stage , ;

[0133] If the cluster center is not The bottleneck cluster center of the stage, and , then update the current stage ;

[0134] Step 3: Update the data aggregation time of the edge data center in the cluster to , and judge:

[0135] if ,renew ;

[0136] if ,renew ;

[0137] if , , All remain unchanged;

[0138] Step 4: According to step 2 or step 3, and value, update the current entry into the next stage ;

[0139] Step 5 ,if , return to step 2; otherwise the loop ends, .

[0140] Select one edge data center in each cluster As the cluster center of the current cluster , according to the selected cluster center, divide the cluster members, and finally complete all The cluster division is as follows:

[0141] Step ①, select the initial cluster center;

[0142] consider The shortest path P between min and available bandwidth in network resources As a decision metric, determine Selection of initial cluster centers in the phase;

[0143] No. Phase indivual The decision metric The definition is as follows:

[0144] (7)

[0145] in, Indicates The weight value of the average shortest path distance between an edge data center and other edge data centers; Indicates The maximum average available bandwidth between an edge data center and other edge data centers;

[0146] right Define two constraints:

[0147] Constraint 1: ;

[0148] Constraint 2: ;

[0149] in, For the current stage the number of

[0150] According to constraints 1 and 2, let , then define :

[0151] (8)

[0152] in, For the The sum of the average shortest path distances between an edge data center and other edge data centers, For the The average shortest path distance between an edge data center and other edge data centers;

[0153] According to formula (8), the final decision metric is obtained The formula is:

[0154] (9)

[0155] According to formula (9), choose The largest value Edge data centers are currently The initial cluster center of the stage;

[0156] Step ②: Select the remaining cluster centers;

[0157] Defining cluster centers , let the number of clusters to be divided be , ,but ,Right now For the cluster centers; select the shortest path between cluster centers as the judgment criterion to complete the selection of the remaining cluster centers; the details are as follows:

[0158] when When The largest as a cluster center;

[0159] when When of As the first cluster center , then select Leave The furthest As the second cluster center ;

[0160] when When , assuming that you have selected Cluster centers, , then in the selection When the cluster center is The cluster center farthest As the cluster centers;

[0161] Step 3: Divide clusters;

[0162] when , then there is only one cluster in the current stage, and there is no need to divide the clusters. All are cluster members at the initial cluster center;

[0163] when When , the clustering method is as follows:

[0164] Step A: First, compare the initial cluster center and other remaining P min To sort in ascending order, select indivual The cluster member that serves as the initial cluster center;

[0165] Define the allocated Number of The interval is:

[0166] (10)

[0167] Among them, N+ represents a natural number except 0;

[0168] Step B, then, The remaining of To sort in ascending order, select indivual Cluster members who are cluster centers;

[0169] Step C: Repeat step B until the - 1 cluster center Finally, the remaining clusters that have not been divided Cluster Center The cluster members.

[0170] Figure 2 is an example diagram to prove the effectiveness of the TT optimization model proposed in the present invention; wherein, Figure 2 (a) is an example of the completion time of the work plan before optimization. Figure 2 (b) is an example of the completion time of the optimized work plan. Figure 2 (c) is an example of the optimal completion time of the optimized work plan. Figure 2 (d) is an example of the worst completion time of the optimized work plan; the work completion time of the plan before optimization is defined as , the completion time of the optimized solution is , the optimal completion time obtained by using the optimized work plan is , the worst completion time obtained by using the optimized work plan is .

[0171] The following is combined with the example diagram to prove the present invention. Optimize the effectiveness of the model.

[0172] (1) Optimization of the work completion time model before the plan

[0173] like Figure 2 As shown in (a) of Figure 1, an example of the completion time of data aggregation and reporting using the pre-optimization solution is shown. In this task, there are three stages in total, and each stage is divided into 4, 2, and 1 clusters. The completion time of data aggregation for each cluster is , the total time to complete the work is equal to the sum of the completion time of the bottleneck clusters at each stage, that is,

[0174] (11)

[0175] (2) Optimized work completion time model

[0176] like Figure 2 As shown in (b) in the figure, an example of using the optimized solution to report the completion time of data aggregation work is shown. Unlike the solution before optimization, the time model of the optimized solution no longer unilaterally depends on the completion time of the bottleneck cluster. The main idea is that when only the bottleneck cluster has not completed data aggregation in the current stage, it will no longer wait for the bottleneck cluster to complete the work, but will immediately enter the next stage. At this time, the remaining time for the bottleneck cluster to complete the work in the current stage is In summary, the total time to complete the optimization is for

[0177] (12)

[0178] in, Indicates that the cluster completion time of the current stage is only less than the bottleneck cluster.

[0179] From formulas (11) and (12), we can see that compared with , the total time to complete the work of the optimized solution No longer rely solely on the bottleneck cluster. The size depends on ,and The value range of needs to consider the relationship between the cluster centers of the previous and next stages, that is, whether the cluster center of the current stage is the cluster center of the next stage. to compare the completion time of the two solutions ( and )size.

[0180] (3) Comparison of completion time of the two solutions

[0181] (3.1) Optimal completion time of the optimized work plan: Figure 2 As shown in (c) in the figure, when the cluster centers of the current stage are all the cluster centers of the next stage, the unfinished work of the bottleneck cluster in the current stage and the data aggregation work in the next stage do not affect each other and can be carried out simultaneously. Therefore, in the current situation, there is no need to consider the impact of the remaining time of the bottleneck cluster to complete the work in the current stage on the time to complete the work in the next stage, that is, ,at this time Produces minimum value , according to formula (13), for

[0182] (13)

[0183] and Comparison:

[0184] (14)

[0185] in, , so the current situation: .

[0186] (3.2) Worst completion time of the optimized work plan: Figure 2 As shown in (d) in the figure, when the cluster centers of the bottleneck cluster in the previous stage are all in the bottleneck cluster in the current stage, and the completion time of the bottleneck cluster in the current stage is the time when the edge data center converges to the cluster center in the current stage, then the work completion time of the cluster is for

[0187] (15)

[0188] in, ,Right now ,at this time Produces the maximum value , combined with formula (15), we have

[0189] (16)

[0190] That is the current situation: .

[0191] (3.3) When the cluster center of the current stage may be the cluster center of the next stage, when entering the next stage, it is necessary to consider the completion time of this stage in combination with the above two extreme cases. According to the description, the completion time of the work in the current situation is Between and Between

[0192]

[0193] In summary, through analysis, we can conclude that

[0194]

[0195] That is, the optimized work completion time model proposed in the present invention can consume less time to complete the same work. The total time to complete the model after optimization Model, that is:

[0196]

[0197] Figure 3 This is the flow chart of the optimal selection method for cluster centers. Figure 3 As shown, the method steps are as follows:

[0198] Step ①, selection of initial cluster center;

[0199] The present invention considers The shortest path between min ) and the available bandwidth in the network resources ( ) is used as the decision metric to determine the selection of the initial cluster center at the current stage. The definition is as follows:

[0200]

[0201] in, Indicates The weight value of the average shortest path distance between an edge data center and other edge data centers; Indicates The maximum average available bandwidth between an edge data center and other edge data centers. The maximum available bandwidth is the value of the link with the smallest available bandwidth in the path. , the present invention defines EDC i With other edge data centers The average shortest path distance between )The smaller the contribution value to the total distance within the cluster, the more likely it is to become the initial cluster center, and therefore a higher weight will be given. The definition is as follows:

[0202] First of all Define two constraints:

[0203] Constraint 1:

[0204] Constraint 2:

[0205] According to constraints 1 and 2, let , then define :

[0206] (17)

[0207] in, For the The sum of the average shortest path distances between an edge data center and other edge data centers, For the The average shortest path distance between an edge data center and other edge data centers;

[0208] According to formula (18), the final decision metric is obtained The formula is:

[0209] (18)

[0210] According to formula (19), choose The largest value Edge data centers are currently The initial cluster center of the stage;

[0211] Step ②: Select the remaining cluster centers;

[0212] Defining cluster centers , assuming that the number of clusters to be divided is , ,but ,Right now For the cluster centers; select the shortest path between cluster centers as the judgment criterion to complete the selection of the remaining cluster centers; the details are as follows:

[0213] when When The largest as a cluster center;

[0214] when When of As the first cluster center , then select Leave The furthest As the second cluster center ;

[0215] when When , assuming that you have selected Cluster centers, , then in the selection When the cluster center is The cluster center farthest As the cluster centers;

[0216] Step 3: Divide clusters;

[0217] when , then there is only one cluster in the current stage, and there is no need to divide the clusters. All are cluster members at the initial cluster center;

[0218] when When , the clustering method is as follows:

[0219] Step A: First, compare the initial cluster center and other remaining of To sort in ascending order, select indivual The cluster member that serves as the initial cluster center;

[0220] Define the allocated Number of The interval is:

[0221] (19)

[0222] Among them, N+ represents a natural number;

[0223] Step B, then, The remaining of To sort in ascending order, select indivual Cluster members who are cluster centers;

[0224] Step C: Repeat step B until the - 1 cluster center Finally, the remaining clusters that have not been divided Cluster Center The cluster members.

[0225] In order to demonstrate the performance of the hierarchical distributed edge data aggregation and reporting method (HDAR-CCS) proposed in this paper, firstly, the multi-stage geographically distributed data aggregation method (MGDD-CC) combining computing and communication resources is compared to prove the effectiveness of the improved method proposed by the present invention based on the MGDD-CC method. Figure 4-Figure 6 The two algorithms were applied to tactical edge networks with 25 nodes, 38 nodes, and 50 nodes, respectively. Comparison chart. The specific experimental setting is that in each edge network, we first set different numbers of cluster members to determine the number of stages required for the two algorithms to complete data aggregation. , and use the cluster center selection algorithm provided by the present invention to determine the cluster center and the division of cluster members in each stage of the two methods. Then, by using the two algorithms, data aggregation is completed and the time required to complete the work in each stage is obtained ( ).

[0226] like Figure 4-6 As shown, represents the time required to complete the work at each stage, MGDD-CC is the method proposed by the present invention, and MGDD-CC (comparative method 1) is the experimental comparison method. It is a process from the second stage to the In each stage, the and The value after the difference.

[0227] The present invention can be observed that, first, HDAR-CCS is in the front stage are less than or equal to the corresponding HDAR-CCS in each stage , and the stage, two methods This is because the cluster center selection algorithm proposed in the present invention has determined the number of clusters and the division of cluster members in each stage, so the cluster centers and cluster member divisions in each stage of the two methods are the same. In addition, there is only one cluster in the last stage, and it is constructed by The model shows that the HDAR-CCS for , which is the same as MGDD-CC, so HDAR-CCS is stage, except , two methods Second, it can be observed that as the number of cluster members increases, the number of stages required to complete a job decreases, while The overall trend is increasing. This is because, on the one hand, the competition for network resources around the cluster center is intensifying, which leads to an increase in transmission time; on the other hand, the amount of data transmitted to a single cluster center is increasing, which leads to an increase in processing time, i.e. Finally, a closer look shows that for 25-node and 38-node networks, when the number of cluster members is 4, the performance of the two methods at each stage is tends to the maximum, and for a 50-node network, when the number of cluster members is 5, the This is because, although the overall trend is that It increases with the number of cluster members. However, as the edge network becomes more and more complex with the increase in the number of deployed edge nodes, the structure of the network itself becomes more and more complex. Therefore, specific experimental analysis is needed to select an optimal cluster member division scheme, so as to build an edge data aggregation and reporting model that is most suitable for the current edge network.

[0228] Based on the above analysis, on the one hand, the above results show that the performance of the HDAR-CCS proposed in the present invention is better than that of the MGDD-CC. On the other hand, the rationality of the optimization model proposed in the present invention is also proved through experiments.

[0229] According to the above analysis, it is necessary to select an optimal cluster member division scheme. Therefore, the present invention compares the work completion time of the two methods ( ), select the best cluster member division scheme. Figure 7 Displays two methods for different numbers of cluster members under different node networks. The specific values ​​of the experimental results are shown in Table 1. Figure 7 In the Method column, Method 1 represents MGDD-CC and Method 2 represents HDAR-CCS.

[0230] As shown in Table 1, for 25 and 38 node networks, when the number of cluster members is 5, the performance of the two algorithms is The values ​​are the smallest. At this time, the number of cluster members is 5 and the stage is 2, which is the current optimal cluster member division scheme for the edge network. Similarly, for a 50-node network, when the number of cluster members is 4, At this time, the number of cluster members is 4 and the stage is 3 as the current optimal cluster member division scheme for the edge network. In addition, through Figure 7As shown, it can be clearly found that when the number of network nodes is more and the network scale is more complex, compared with MGDD-CC, the HDAR-CCS algorithm consumes less time to complete the work and the improved performance is more obvious. Therefore, in the face of the communication and computing needs of processing massive data, the method proposed in the present invention can be well applied to the current edge network environment, and can more efficiently process and aggregate data on the edge, and further complete the edge-to-cloud data aggregation and reporting.

[0231] Table 1 Experimental results of two methods

[0232]

[0233] In order to further prove the superiority of the proposed method, the present invention selects two distributed data aggregation algorithms in recent years, such as the hierarchical data aggregation method LDA-EPP based on fog computing (comparison method 2) and the data aggregation scheduling algorithm DEDA (comparison method 3), as comparison methods, and selects the work completion time And bandwidth consumption are used as evaluation indicators to compare the performance of the method of the present invention and other methods. Figure 8 and Fig. 9 Shows the performance of several algorithms under different node networks The comparison results of bandwidth consumption. Among them, the proposed method (HDAR-CCS) selects the best cluster member division scheme under three node networks and compares it with the other two algorithms.

[0234] like Figure 8 , 9 As shown in Figure 2, it can be found that compared with the other two methods, the HDAR-CCS method proposed in the present invention has the smallest And the lowest bandwidth consumption. Because HDAR-CCS takes into account factors such as data aggregation time and available bandwidth of node links in the selection of cluster centers, the division of cluster members, and the construction of data aggregation models, and finally forms a systematic edge data aggregation and reporting method. Therefore, whether from the experimental results or the theoretical basis, the hierarchical distributed edge data aggregation and reporting method based on cluster center selection proposed in the present invention has excellent performance.

[0235] The present invention also discloses a hierarchical distributed edge data aggregation and reporting device, including a data aggregation and reporting module.

[0236] The data aggregation reporting module is used to divide the data aggregation process into In each stage, a hierarchical and distributed edge data aggregation and reporting model is constructed. is an integer greater than 1;

[0237] Each stage of the data aggregation process includes multiple clusters, each cluster includes multiple cluster members, and the cluster members are edge data centers. , select an edge data center in each cluster As the cluster center of the current cluster , according to the selected cluster center, divide the cluster members, and finally complete all Cluster partitioning, stored in edge data centers The data is transferred to the cluster center of the current stage;

[0238] The hierarchical distributed edge data aggregation reporting model includes the task completion time of the current stage Model and total time to complete the work Model, where

[0239] Current stage task completion time Define the model Stage task completion time The model is as follows:

[0240] ;

[0241] Total time to complete the work The model is:

[0242] ;

[0243] in, Indicated in The time to complete the task in the stage, For the Stage, Cluster Mid-edge data center With other edge data centers The communication time between For the Stage, Cluster Mid-edge data center Data processing time, is the data transmission rate in the link, For the Phase, Edge Data Center The amount of data, For the Phase, Edge Data Center Transmit data to other edge data centers The available network bandwidth between For the Phase, Edge Data Center Transmit data to other edge data centers The link distance between For the stage, Available computing resources, For the The time taken by the last cluster to complete the convergence task in the remaining clusters except the bottleneck cluster in the stage. It is a process from the second stage to the In each stage, the and The value after the difference.

[0244] An embodiment of the present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor. When the processor executes the computer program, the steps of the hierarchical distributed edge data aggregation and reporting method as described in the first aspect above or any corresponding embodiment thereof are implemented.

[0245] An embodiment of the present invention also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the steps of the hierarchical distributed edge data aggregation and reporting method as described in the first aspect above or any corresponding embodiment thereof.

[0246] It should be understood by those skilled in the art that the embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes. The solutions in the embodiments of the present invention may be implemented in various computer languages, for example, object-oriented programming language Java and interpreted scripting language JavaScript, etc.

[0247] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1A device that provides the functions specified in a block or multiple blocks.

[0248] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0249] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0250] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0251] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A hierarchical and distributed edge data aggregation and reporting method, characterized in that: include: When edge network processes tasks, the data aggregation process is divided into In each stage, a hierarchical and distributed edge data aggregation and reporting model is constructed. is an integer greater than 1; Each stage of the data aggregation process includes multiple clusters, each cluster includes multiple cluster members, and the cluster members are edge data centers. , select an edge data center in each cluster As the cluster center of the current cluster , according to the selected cluster center, divide the cluster members, and finally complete all Cluster partitioning, stored in edge data centers The data is transferred to the cluster center of the current stage; The hierarchical distributed edge data aggregation reporting model includes the task completion time of the current stage Model and total time to complete the work Model, where Current stage task completion time Define the model Stage task completion time The model is as follows: ； Total time to complete the work The model is: ； in, Indicated in The time to complete the task in the stage, For the Stage, Cluster Mid-edge data center With other edge data centers The communication time between For the Stage, Cluster Mid-edge data center Data processing time, is the data transmission rate in the link, For the Phase, Edge Data Center The amount of data, For the Phase, Edge Data Center Transmit data to other edge data centers The network available bandwidth between For the Phase, Edge Data Center Transmit data to other edge data centers The link distance between For the stage, Available computing resources, For the The time taken by the last cluster to complete the convergence task in the remaining clusters except the bottleneck cluster in the stage. It is a process from the second stage to the In each stage, the and The value after the difference.

2. According to the hierarchical distributed edge data aggregation and reporting method of claim 1, it is characterized in that: The bottleneck cluster refers to the last cluster to complete the data aggregation task.

3. The hierarchical distributed edge data aggregation and reporting method according to claim 1, characterized in that: The construction process is as follows: definition Equal to The time consumed by the bottleneck cluster in the stage. The bottleneck cluster refers to the last cluster to complete the data aggregation task; for: (1) in, For the Phase, Cluster The time to complete data aggregation; Defined as: (2) (3) in, For the Phase, Edge Data Center Transmit data to other edge data centers hour, The time required to send the data; For the Phase, Edge Data Center Transmit data to other edge data centers the time required; (4) in, For the stage, Available computing resources; (5)。 4. The hierarchical and distributed edge data aggregation and reporting method according to claim 1, characterized in that: (6) in, For each stage and difference.

5. A hierarchical and distributed edge data aggregation and reporting method according to claim 4, characterized in that: calculate The method is as follows: Step 1: When When initialization ; Step 2: If The edge data center in the bottleneck cluster is not the first If the cluster center of the stage is not reached, directly execute step 3; If The edge data center in the bottleneck cluster is still the first The cluster center of the stage is determined as follows: If the cluster center is The bottleneck cluster center of the stage, and , update stage ; If the cluster center is The bottleneck cluster center of the stage, and ,but remain unchanged; If the cluster center is not The bottleneck cluster center of the stage, and , then update the current stage , ; If the cluster center is not The bottleneck cluster center of the stage, and , then update the current stage ; Step 3: Update the data aggregation time of the edge data center in the cluster to , and judge: if ,renew ; if ,renew ; if , , All remain unchanged; Step 4: According to step 2 or step 3, and value, update the current entry into the next stage ; Step 5 ,if , return to step 2; otherwise the loop ends, .

6. The hierarchical and distributed edge data aggregation and reporting method according to claim 1, characterized in that: Select one edge data center in each cluster As the cluster center of the current cluster , according to the selected cluster center, divide the cluster members, and finally complete all The cluster division is as follows: Step ①, select the initial cluster center; consider The shortest path P between min and available bandwidth in network resources As a decision metric, determine Selection of initial cluster centers in the phase; No. Phase indivual The decision metric The definition is as follows: (7) in, Indicates The weight value of the average shortest path distance between an edge data center and other edge data centers; Indicates The maximum average available bandwidth between an edge data center and other edge data centers; right Define two constraints: Constraint 1: ; Constraint 2: ; in, For the current stage the number of According to constraints 1 and 2, let , then define : (8) in, For the The sum of the average shortest path distances between an edge data center and other edge data centers, For the The average shortest path distance between an edge data center and other edge data centers; According to formula (8), the final decision metric is obtained The formula is: (9) According to formula (9), choose The largest value Edge data centers are currently The initial cluster center of the stage; Step ②: Select the remaining cluster centers; Defining cluster centers , let the number of clusters to be divided be , ,but ,Right now For the cluster centers; select the shortest path between cluster centers as the judgment criterion to complete the selection of the remaining cluster centers; the details are as follows: when When The largest as a cluster center; when When of As the first cluster center , then select Leave The furthest As the second cluster center ; when When , assuming that you have selected Cluster centers, , then in the selection When the cluster center is The cluster center farthest As the cluster centers; Step 3: Divide clusters; when , then there is only one cluster in the current stage, and there is no need to divide the clusters. All are cluster members at the initial cluster center; when When , the clustering method is as follows: Step A: First, compare the initial cluster center and other remaining P min To sort in ascending order, select indivual The cluster member that serves as the initial cluster center; Define the allocated Number of The interval is: (10) Among them, N+ represents a natural number except 0; Step B, then, The remaining of To sort in ascending order, select indivual Cluster members who are cluster centers; Step C: Repeat step B until the - 1 cluster center Finally, the remaining clusters that have not been divided Cluster Center The cluster members.

7. A hierarchical and distributed edge data aggregation and reporting method according to claim 6, characterized in that: The maximum average available bandwidth is the average of multiple maximum available bandwidths, and the maximum available bandwidth is the value of the link with the smallest available bandwidth in the path.

8. A hierarchical and distributed edge data aggregation and reporting device, characterized in that: Including data aggregation and reporting module, The data aggregation reporting module is used to divide the data aggregation process into In each stage, a hierarchical and distributed edge data aggregation and reporting model is constructed. is an integer greater than 1; Each stage of the data aggregation process includes multiple clusters, each cluster includes multiple cluster members, and the cluster members are edge data centers. , select an edge data center in each cluster As the cluster center of the current cluster , according to the selected cluster center, divide the cluster members, and finally complete all Cluster partitioning, stored in edge data centers The data is transferred to the cluster center of the current stage; The hierarchical distributed edge data aggregation reporting model includes the task completion time of the current stage Model and total time to complete the work Model, where Current stage task completion time Define the model Stage task completion time The model is as follows: ； Total time to complete the work The model is: ； in, Indicated in The time to complete the task in the stage, For the Stage, Cluster Mid-edge data center With other edge data centers The communication time between For the Stage, Cluster Mid-edge data center Data processing time, is the data transmission rate in the link, For the Phase, Edge Data Center The amount of data, For the Phase, Edge Data Center Transmit data to other edge data centers The network available bandwidth between For the Phase, Edge Data Center Transmit data to other edge data centers The link distance between For the stage, Available computing resources, For the The time taken by the last cluster to complete the convergence task in the remaining clusters except the bottleneck cluster in the stage. It is a process from the second stage to the In each stage, the and The value after the difference.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, characterized in that: When the processor executes the computer program, the steps of the hierarchical distributed edge data aggregation and reporting method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the hierarchical distributed edge data aggregation and reporting method as described in any one of claims 1 to 7 are implemented.

Images