Network adaptive traffic proxy method in dynamic edge environment

Abstract

The invention discloses a network adaptive flow proxy method in a dynamic edge environment. The method specifically comprises the following steps: acquiring transmission bandwidths, transmission delays and packet loss rates between data nodes for storing task data to be processed in a cluster and each task node; calculating the transmission time of transmitting the to-be-processed task from the data node to each task node; and determining a target task node based on the link transmission time, and transmitting the to-be-processed task to the target task node. Compared with the prior art, the optimal link selection is provided for point-to-point transmission in the edge cluster; meanwhile, the execution effectiveness of the target service is ensured; furthermore, the method can dynamically adapt to the network topology change of the mobile node. And the response time delay is optimized by reducing the task data transmission time in the edge environment. And on the other hand, the kernel mode directly forwards the data flow, so that the context switching, namely the data copying process, is reduced, and the delay and throughput performance can be improved.

Description

Technical Field

[0001] The present invention belongs to the field of communication technology, and in particular relates to a network adaptive traffic proxy method in a dynamic edge environment. Background Art

[0002] Edge computing can provide more convenient data transmission services and better experience quality for users on the edge and mobile terminals. However, since the resources of edge scenarios are limited, it is impossible to deploy all service instances on each server, so these tasks can only be offloaded to edge servers that provide the required services. Complex real-world environments and scenario requirements may make the relative positions between nodes always highly dynamic. Due to the constant changes in the relative positions of nodes, it is difficult for wireless networks between nodes to maintain stable connections, and the quality of network links between nodes will also vary greatly.

[0003] At present, edge users have put forward higher requirements for the efficiency of task and request responses, but the traffic management strategies and load balancing algorithms in cloud computing scenarios cannot be directly copied to edge scenarios. Existing research lacks consideration of the network connection fluctuations caused by the dynamic geographical location changes of data nodes in mobile scenarios, which will make the network connection quality between nodes very unstable, and even make it impossible to establish a communication link. Most of them verify the effectiveness of the methods through simulation experiments, lacking consideration of engineering applications and verification of actual effects. At the same time, the existing technology has a large delay in the forwarding process, which is not conducive to task processing.

[0004] Therefore, in view of the above technical problems, it is necessary to provide an improved traffic management strategy.

[0005] The information disclosed in this background technology section is only intended to enhance the understanding of the overall background of the invention and should not be regarded as an acknowledgment or any form of suggestion that the information constitutes the prior art already known to a person skilled in the art. Summary of the invention

[0006] The object of the present invention is to provide a network adaptive traffic proxy method in a dynamic edge environment, which can dynamically adapt to the network topology changes of mobile nodes and provide optimal link selection for point-to-point transmission within an edge cluster.

[0007] In order to achieve the above object, a technical solution provided by a specific embodiment of the present invention is as follows:

[0008] In a first aspect, the present invention provides a network adaptive traffic proxy method in a dynamic edge environment, comprising:

[0009] Collect the transmission bandwidth, transmission delay and packet loss rate between the data nodes storing the task data to be processed and each task node in the cluster;

[0010] Calculate the transmission time of the task to be processed from the data node to each task node;

[0011] Based on the link transmission time, a target task node is determined, and the task to be processed is transmitted to the target task node.

[0012] In one or more embodiments of the present invention, the method further comprises:

[0013] Broadcasting a request to the local area network where the cluster is located;

[0014] If the task node in the cluster responds to the request, the node IP of the task node is saved to the neighbor node table entry.

[0015] In one or more embodiments of the present invention, the transmission bandwidth, transmission delay and packet loss rate between the data node storing the task data to be processed and each task node in the acquisition cluster include:

[0016] The data node sends multiple data packets to the node IP corresponding to the task node in the neighbor node table entry;

[0017] Record the transmission rate, response time, number of packets sent, and number of packet losses of the data packet;

[0018] The transmission bandwidth, transmission delay and packet loss rate between the data node and each task node are calculated, and the transmission bandwidth and the transmission delay are normalized.

[0019] In one or more embodiments of the present invention, the formula for calculating the transmission bandwidth, transmission delay and packet loss rate between the data node and each task node is:

[0020]

[0021]

[0022]

[0023] Where D is the transmission delay between the data node and the task node; RTT is the time from the data node sending data to the data node receiving the confirmation message from the corresponding task node; B is the transmission bandwidth between the data node and the task node; S is the size of the data packet sent by the data node; T is the time required for the data node to send; L is the packet loss rate from the data node to the task node; P total Represents the total number of data packets sent by the data node per unit time, P recieved Represents the number of data packets received by the corresponding task node per unit time.

[0024] In one or more embodiments of the present invention, the formula for normalizing the transmission bandwidth and the transmission delay is:

[0025]

[0026]

[0027] Among them, B' is the normalized transmission bandwidth, D' is the normalized transmission delay, B max is the maximum value of the transmission bandwidth, B min is the minimum value of the transmission bandwidth, D max is the maximum value of the transmission delay, D min is the minimum value of transmission delay.

[0028] In one or more embodiments of the present invention, the formula for calculating the link transmission time of the task to be forwarded from the data node to each task node is:

[0029] T i =T1+T2

[0030]

[0031]

[0032] Among them, T i is the transmission time required from data node m to task node n; is the normalized value of the transmission bandwidth from data node m to task node n; is the normalized value of the transmission delay from data node m to task node n; is the packet loss rate from data node m to task node n, S i is the size of the task to be forwarded.

[0033] In one or more embodiments of the present invention, determining the target task node based on the link transmission time includes:

[0034] Based on a preset node load threshold, each task node in the neighbor node table entry is screened, and the node load threshold includes a CPU occupancy threshold and a memory resource utilization threshold;

[0035] Calculate the corresponding link quality score based on the transmission time between each task node in the neighbor node table after screening and the data node where the task to be forwarded is located;

[0036] The task node corresponding to the path with the largest link quality score is taken as the target task node.

[0037] In one or more embodiments of the present invention, the link quality score calculation formula is:

[0038]

[0039]

[0040] in, is the link quality score from data node m to task node n, and α is the weight value.

[0041] In one or more embodiments of the present invention, transmitting the task to be processed to the target task node includes:

[0042] Based on the target task node, configure corresponding forwarding rules;

[0043] The preset filter forwards the to-be-processed task to the target task node based on the kernel state based on the forwarding rule.

[0044] In a second aspect, the present invention provides a network adaptive traffic proxy system in a dynamic edge environment, comprising:

[0045] The network resource perception module is used to collect the transmission bandwidth, transmission delay and packet loss rate between each data node and each task node in the cluster;

[0046] A link analysis and decision module, used to respond to a task forwarding request from a data node, obtain information about the task to be forwarded, and calculate a link transmission time between the task to be forwarded from the data node to each task node;

[0047] The transmission module is used to determine the optimal transmission path based on the link transmission time, and transmit the task to the corresponding task node based on the optimal transmission path.

[0048] Compared with the prior art, the network adaptive traffic proxy method in a dynamic edge environment provided by the present invention analyzes the network link status in real time based on the geographical location distribution and network topology status of dynamic data nodes, and provides the optimal link selection for point-to-point transmission within the edge cluster; at the same time, based on the computing and storage resource status of each node, the node load is analyzed in real time to ensure the execution effectiveness of the target service; further, the present invention integrates the link status and the node load status, and can dynamically adjust the traffic forwarding strategy in real time to dynamically adapt to the network topology changes of mobile nodes. The goal of optimizing task and request response delays is achieved by reducing the task data transmission time in the edge environment. On the other hand, the data traffic is forwarded directly from the kernel state, reducing the context switching, that is, the data copy process, which can improve the delay and throughput performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0050] Figure 1 It is a schematic diagram of a network adaptive traffic proxy scenario in a dynamic edge environment in one embodiment of the present invention;

[0051] Figure 2 It is a flow chart of a network adaptive traffic proxy method in a dynamic edge environment in one embodiment of the present invention;

[0052] Figure 3 It is a structural block diagram of a network adaptive traffic proxy system in a dynamic edge environment in one embodiment of the present invention;

[0053] Figure 4 It is a structural block diagram of an electronic device in one embodiment of the present invention. DETAILED DESCRIPTION

[0054] The specific implementation modes of the present invention are described in detail below in conjunction with the accompanying drawings, but it should be understood that the protection scope of the present invention is not limited by the specific implementation modes.

[0055] Unless explicitly stated otherwise, throughout the specification and claims, the term “comprise” or variations such as “include” or “comprising”, etc., will be understood to include the stated elements or components but not to exclude other elements or components.

[0056] In the existing technical solutions, there are traffic management strategies in cloud computing scenarios, but cloud computing is to upload the data of edge devices and perform centralized computing and processing in the cloud; in contrast, edge computing uses the computing power of the network edge to process data. Among them, although edge devices provide storage and computing capabilities on network endpoints, they are essentially part of a larger cloud infrastructure after being abstracted, aggregated and shared in the network. But on the other hand, edge devices can refer to any entity with computing and other functions in the path from the device that generates data to the cloud. Therefore, in edge computing scenarios, edge devices are subject to dynamic changes such as geographical location, which have a great impact on network connection fluctuations. The computing environment is complex and changeable, and it is impossible to directly apply traffic management strategies or methods developed in a cloud computing environment.

[0057] In addition, some studies have considered network traffic as an important influencing factor in edge scenarios, but they use network traffic as the basis for adjusting edge resource allocation, which ignores the premise that edge resources are limited and only focuses on optimizing task processing delays. In delay-sensitive tasks, transmission delays may even account for a large proportion of the total task response time. Therefore, ignoring data transmission delays cannot effectively guarantee task responses under network fluctuations. Different from traditional methods, such as heuristic algorithms or the use of reinforcement learning methods to improve traffic management and data offloading strategies, all use simulation experiments to verify the effectiveness of the methods, lacking consideration of engineering applications and verification of actual results.

[0058] The inventor of the present invention discovered the main shortcomings of the prior art, and based on the shortcomings of the prior art, proposed a new technical implementation idea: from the perspective of transmission delay, considering the dynamically changing data nodes in the scenario of network fluctuations, the link information between each data node and the task node is collected in real time. By combining the real-time load capacity of the node and optimizing the response time for delay-sensitive tasks and real-time tasks, the traffic forwarding strategy is dynamically adjusted. It has strong adaptability, and at the same time significantly reduces the response delay of tasks and user requests, and is very effective in delay-sensitive tasks. It basically meets the user's demand for low-latency services in dynamic edge scenarios and greatly improves the user's overall experience.

[0059] Please refer to Figure 1 , which is a schematic diagram of an application scenario of the network adaptive traffic proxy method in a dynamic edge environment provided by the present invention under one implementation mode, and the scenario specifically includes: a network resource perception module 101, a link analysis and decision module 102 and a transmission module 103.

[0060] It should be noted that communication connections are provided between the network resource perception module 101, the link analysis and decision module 102, and the transmission module 103. The communication network derived from the communication connection may include various connection types, including but not limited to: wired connection, wireless connection, or optical fiber cable connection, etc. At the same time, the communication network may be a local area network, a metropolitan area network, a wide area network, or any combination of the three.

[0061] The network resource sensing module 101 is used to broadcast requests and collect corresponding response signals sent by task nodes, so as to judge the validity of each task node in the cluster. On the other hand, the network resource sensing module 101 is also used to collect the transmission bandwidth, transmission delay and packet loss rate between each data node and each task node in the cluster.

[0062] The link analysis and decision module 102 is used to obtain the link information formed by the data node where the task to be forwarded is located and each surviving task node through the communication connection between the aforementioned network resource perception module 101, and calculate the transmission time and link quality of each link based on the link information.

[0063] The transmission module 103 is used to determine the optimal transmission path based on the link time / quality and forward the task based on the kernel state.

[0064] In the application scenario of the network adaptive traffic proxy method in a dynamic edge environment provided by the present invention, at least one user terminal is also included, and the user terminal is used to provide a configuration platform for the administrator. It is used to configure the calculation formula, threshold, data packet / request type, formula custom weight value, etc. required for system operation based on different usage environments.

[0065] It should be noted that the user terminal is installed with a computer software program that matches the network adaptive traffic proxy method in a dynamic edge environment provided by this method; the user terminal may include but is not limited to desktop computers (PCs), desktop computers, smart phones, handheld computers, tablet computers, personal digital assistants (PDAs), and other portable electronic devices or wearable electronic devices, and the embodiments of the present invention do not limit the above contents.

[0066] It should also be noted that the network adaptive traffic proxy method under the dynamic edge environment of the embodiment of the present invention can be applied to the network adaptive traffic proxy system under the dynamic edge environment of the embodiment of the present invention. The network adaptive traffic proxy system under the dynamic edge environment can be configured in a terminal. The terminal may include but is not limited to a PC (Personal Computer), a PDA (tablet computer), a smart phone, a smart wearable device, and the like.

[0067] Please refer to Figure 2 FIG. 1 is a flow chart of a method for network adaptive traffic proxy in a dynamic edge environment in one embodiment of the present invention. The method for network adaptive traffic proxy in a dynamic edge environment specifically includes the following steps:

[0068] S201: Collecting the transmission bandwidth, transmission delay and packet loss rate between the data node storing the task data to be processed and each task node in the cluster;

[0069] It can be understood that the transmission bandwidth refers to the amount of data that can be transmitted by the link per unit time, which can intuitively represent the ability of the communication line to transmit data; secondly, the transmission delay in the present invention refers to the time required for a data packet to be transmitted in the network link, which is usually related to the link length between the sending and receiving ends, and is directly positively correlated with the distance between nodes when reflected in the wireless local area network. Compared with tasks and requests with small data volumes, in a dynamic edge network environment, the transmission delay is often greater than the processing delay, and the impact on the transmission time is also greater than the processing delay. Finally, the packet loss rate refers to the ratio of the number of lost data packets to the number of data packets sent by the sender, which can be obtained by subtracting 1 from the ratio of the number of data packets received by the receiving end per unit time to the total number of data packets sent by the sender. It reflects the success rate of data packet transmission and directly reflects the reliability of the network, because data packets that are not successfully received need to undergo a retransmission process, which will greatly affect the transmission time.

[0070] In an exemplary embodiment, the transmission bandwidth, transmission delay and packet loss rate between each data node and each task node in the cluster are collected, including: sending multiple data packets to the node IP corresponding to the task node in the neighbor node table; recording the transmission rate, response time, number of packets sent and number of packet losses of the data packets; calculating the transmission bandwidth, transmission delay and packet loss rate between the data node and each task node, and normalizing the transmission bandwidth and the transmission delay.

[0071] It should be noted that, for the acquisition of the transmission delay between the data node and the task node, in one implementation, the time when the data node sends the data packet can be recorded, and the packet receiving time of the corresponding task node can be sensed. The propagation delay of the corresponding link can be obtained by subtracting the packet receiving time from the packet sending time. In this implementation, the required link transmission delay data can be obtained with relatively simple logic and data acquisition strategies. However, in the sending process of most data packets, the receiving time of the receiving end cannot be sensed. In addition, the time data of unilateral transmission may deviate in accuracy and have poor stability. The transmission of the packet receiving time will also waste part of the time, causing delays in data acquisition.

[0072] Therefore, in another embodiment of the present invention, request data is sent to the task node as the receiving end, and when the task node receives the above request data, a response data will be fed back to the sending end. The data node that sends the request records the sending time of the request data and the receiving time of the response data. The difference between the two can be considered as the round-trip time RTT (Round-Trip-Time) between the data node and the task node. In the scenario facing the present invention, the data packet transmission time is equal to the one-way transmission time, so the transmission delay can be obtained by halving the RTT. The delay data obtained by halving the round-trip time RTT is more accurate and closer to the needs of users, and will not cause meaningless data acquisition delays.

[0073] Specifically, the formula for calculating the transmission bandwidth, transmission delay and packet loss rate between the data node and each task node is:

[0074]

[0075]

[0076]

[0077] Where D is the transmission delay between the data node and the task node; RTT is the time from the data node sending data to the data node receiving the confirmation message from the corresponding task node; B is the transmission bandwidth between the data node and the task node; S is the size of the data packet sent; T is the time required for the data node to send; L is the packet loss rate from the data node to the task node; P total Represents the total number of data packets sent by the data node per unit time, P recieved Represents the number of data packets received by the corresponding task node per unit time.

[0078] For example, in a specific embodiment, each data node sends a fixed number of UDP data packets and ICMP test packets to the IP corresponding to the task node. The actual bandwidth of the link between the data node and the corresponding task node can be calculated based on the data transmission rate of the UDP data packet, and the transmission delay on the corresponding link can be calculated based on the request round trip time of the ICMP data packet. Finally, the packet loss rate is calculated by the number of data packets received by the server and the total number of data packets sent by the data node.

[0079] Among them, ICMP itself is an error reporting mechanism. When an error occurs in the data packet processing process, ICMP reports the error to the source device of the data packet, but it will not correct the error or notify the intermediate network devices. The fields in the IP data packet contain the source and the final destination, and do not record the entire path of the message in the network transmission (unless the routing record option is set in the IP data packet). Therefore, when the device detects an error, it cannot notify the intermediate network device and can only send an error report to the source. Based on the sending and receiving nature of the protocol, the ICMP echo information (response information) can be used to confirm the round-trip time RTT (Round-Trip-Time) between the data node and the task node.

[0080] It should be noted that the transmission of data packets is only used to collect the transmission delay, transmission bandwidth and packet loss rate between nodes. The "data packet" described in the embodiment of the present application includes any data packet, message and other transmission media that can realize the above functions or at least realize one of the above functions. That is, the embodiment of the present invention does not limit the type of data packet. Similarly, the acquisition of transmission delay, transmission bandwidth and packet loss rate in the present invention is not limited to using only one data packet.

[0081] Furthermore, for the bandwidth and delay indicators, their values ​​usually start from 0, and there is no mathematical upper limit. There may be large dimensional differences in different links. In order to eliminate the dimensional differences between the indicators, in an exemplary embodiment of the present invention, it is preferred to normalize the transmission bandwidth and transmission delay calculated based on the above method so that the two are normalized to the interval [0,1]. Among them, the normalization method for the transmission bandwidth and transmission delay of the link between the data node and the task node in the embodiment of the present invention may include but is not limited to Min-Max normalization, Z-score normalization, trigonometric function normalization, etc.

[0082] In an exemplary embodiment, the maximum and minimum normalization is adopted to calculate the normalized B' and D'. The formula for normalizing the transmission bandwidth and the transmission delay is:

[0083]

[0084]

[0085] Among them, B' is the normalized transmission bandwidth, D' is the normalized transmission delay, B max is the maximum value of the transmission bandwidth, B min is the minimum value of the transmission bandwidth, D max is the maximum value of the transmission delay, D min is the minimum value of transmission delay.

[0086] S202: Calculating the transmission time of the task to be processed from the data node to each task node;

[0087] In order to further determine the differences between the links formed by different task nodes and the data node where the current task to be forwarded is located, if there are N task nodes in the cluster, it is necessary to establish N different links starting from the data node where the task to be forwarded is located, and calculate the corresponding link transmission time respectively.

[0088] It is understandable that different influencing factors have different effects on network links, and link quality cannot be measured based on a single value. It is necessary to combine all influencing factors to derive an algorithm that can quantitatively calculate network link quality, so as to maximize the time consumed by transmitting different data under different link conditions.

[0089] In an exemplary embodiment, the network adaptive traffic proxy method in a dynamic edge environment provided by the present invention calculates the link transmission time of the task to be forwarded from the data node to each task node as follows:

[0090] T i =T1+T2

[0091]

[0092]

[0093] Among them, T i is the transmission time required from data node m to task node n; is the normalized value of the transmission bandwidth from data node m to task node n; is the normalized value of the transmission delay from data node m to task node n; is the packet loss rate from data node m to task node n, S i is the size of the task to be forwarded.

[0094] In the transmission time calculation formula provided by the present invention, transmission delay, transmission bandwidth, packet loss rate and the size of the transmission data packet are applied. With the organic integration of the above multiple variables, the accurate transmission time on the link can be obtained even in the face of a changing edge environment.

[0095] S203: Determine a target task node based on the link transmission time, and transmit the task to be processed to the target task node.

[0096] It should be noted that not all task nodes can serve as receiving nodes for tasks to be forwarded. Whether the processing conditions for tasks to be forwarded are met mainly depends on the resource conditions of the corresponding task nodes. Specifically, the resource conditions of the node are used to define the node load, which directly affects whether the node has sufficient capacity to respond to the target task. It can be mainly reflected by the CPU occupancy rate and memory resource utilization rate, because the two are usually proportional in value. If the CPU and memory occupancy ratio of the node exceeds a certain threshold, the state close to overload will seriously affect the node performance and greatly reduce the execution time of the task.

[0097] Since more data means longer task execution, the computing and storage resources of the node are occupied for a longer time, which may cause the node to be overloaded when other tasks arrive. Therefore, in an exemplary embodiment of the present invention, different thresholds can be set for the CPU occupancy rate and memory resource utilization rate of different nodes according to the data volume of the task to be forwarded. When the task node meets the above threshold requirements, it is determined that the node can be used as the sending node of the task to be forwarded. When the task node does not meet the above threshold requirements, the node is abandoned in this forwarding process.

[0098] Furthermore, based on the link transmission time, the target task node is determined, including: based on a preset node load threshold, each task node in the neighbor node table item is screened, and the node load threshold includes a CPU occupancy threshold and a memory resource utilization threshold; based on the transmission time between each task node in the neighbor node table item after screening and the data node where the task to be forwarded is located, the corresponding link quality score is calculated; and the path with the largest link quality score is taken as the optimal path.

[0099] It should be noted that the link transmission time can reflect the quality of the corresponding link to a certain extent. In one embodiment of the present invention, the link with the shortest transmission time among the links formed by the data node where the task to be forwarded is located and each task node in the cluster can be used as the optimal link, and the task to be forwarded is forwarded based on this link.

[0100] However, it should be noted that in the embodiments of the present invention, the data node, that is, the data source node, does not have computing and data processing capabilities. Since drones or unmanned vehicles in dynamic edge scenarios often have limited energy and computing power, in order to ensure their own endurance, they can only offload data to nodes that can process such data for execution. It is only used to collect raw data from input devices and transmit the data to the target task node. Task nodes usually have strong computing power and corresponding service instances are deployed in advance to cope with the processing and computing requirements of different tasks.

[0101] In the existing technology, the few traffic scheduling strategies that take network traffic as an important influencing factor also use network traffic as the basis for adjusting edge-side resource allocation. This ignores the premise that edge-side resources are limited and only focuses on optimizing task processing delays. In delay-sensitive tasks, transmission delays may even account for a large proportion of the total task response time. Therefore, ignoring data transmission delays cannot effectively guarantee task responses under network fluctuations.

[0102] For example, there are two links to choose from, of which the effective bandwidth of link A is 160Mbps and the delay is 100ms; the effective bandwidth of link B is 80Mbps and the delay is 10ms. Considering that the effective bandwidth of the network determines the data transmission rate, this means that for tasks with large data volumes, the effective bandwidth will greatly affect the overall transmission time. Assuming that a task data of size 1GB is transmitted on link A, one byte occupies 8 bits, and the transmission will consume 50s when other factors are not considered, while it only takes 100s to transmit on link B. At this time, the link delay is almost negligible during the transmission process, and the effective bandwidth should be used as the main influencing factor to measure the link quality. If only 1MB of data needs to be transmitted, then under links A and B, the transmission time is 50ms and 100ms respectively when other factors are not considered. However, in actual situations, due to the different distances of links AB and different link delays, the final transmission time is 150ms and 110ms. At this time, the link delays of A and B have a greater impact on data transmission. Therefore, the link quality should be dynamically judged according to the task type and data size, and cannot be fixedly quantified.

[0103] To this end, the present invention calculates the transmission time that may be required on each link based on the task data size combined with the current network status, and adds a custom weight value to the transmission delay part of the components of the transmission time, so that the link quality evaluation value algorithm can be adapted to various edge environments and the process of transmitting different tasks. Furthermore, the longer the transmission time means the worse the link quality, so it is necessary to take the inverse of the transmission time of this additional weight value. On this basis, the entire value is multiplied by a coefficient of 1000 to amplify the dimension of the quality value. At the same time, taking into account the impact of the packet loss rate on the link transmission success rate, we use the success rate as a coefficient to quantify the data transmission time, and dynamically evaluate the score of the target link according to the data size and the link status of each node. The link scoring formula between the data source node m and the task node n is as follows:

[0104]

[0105] in, is the link quality score from data node m to task node n, α and (1-α) are weight values.

[0106] Based on the above formula, compared with the previous embodiment, the quality of the currently available link can be more intuitively and clearly reflected. Based on the setting of the weight value, the impact of the transmission delay when transmitting different tasks is quantified, making the adaptability of this solution stronger and the accuracy higher.

[0107] Through experimental measurements, it can be found that when the bandwidth decreases from high to low to the order of 10Mbps, the impact of link delay on response delay is gradually increasing, and the impact is more significant when it is less than 10Mbps. At the same time, in the file transfer experiment of different sizes from 1MB to 225MB, it can also be observed that when the bandwidth is fixed, the impact of link delay will increase with the decrease of data size, especially when the file size is reduced to below 10MB. The impact is more significant. Therefore, the value of α determines the weight division standard for large-volume data and small-volume data in the link score. If the actual bandwidth value is significantly larger than the data size to be transmitted, so that its transmission time on the link reaches the millisecond level, then the difference in bandwidth values ​​within the same order of magnitude will no longer be used as a basis for measuring transmission time. At this time, link delay will become the main factor affecting task response delay. It can be obtained from this:

[0108]

[0109]

[0110] Considering that the value of link delay does not have a very large span, and there is a dimensional difference between bandwidth and transmitted data size, it is necessary to ensure that when the bandwidth is very small or the amount of transmitted data is extremely small, the value range of α is always in [0,1]. It can be calculated by the following formula:

[0111]

[0112] Therefore, the adaptive decision-making of the present invention is mainly divided into two aspects: first, by real-time monitoring of network link conditions and the load of each task node, the status of available nodes is dynamically sensed to determine which nodes are available for data transmission. Second, by combining the actual bandwidth and the specific data size required for transmission of different tasks, the link scoring criteria are dynamically adjusted to achieve automatic adaptation to the degree of response delay impact of different network conditions on different transmission data sizes, thereby making more accurate judgments.

[0113] It should also be noted that the task nodes in the cluster may be in an abnormal state and do not support the sending and receiving of tasks. At this time, even if they meet the threshold limit of node resources, they still cannot be used as the target node for forwarding. In order to avoid forwarding to such nodes, resulting in packet loss and the like. At the same time, the workload of early data collection is reduced. Before collecting the transmission bandwidth, transmission delay and packet loss rate between each data node and each task node in the cluster, the network adaptive traffic proxy method under the dynamic edge environment provided by the present invention also includes: broadcasting a preset request to the local area network where the cluster is located; if the task node in the cluster responds to the preset request, the node IP of the task node is saved to the neighbor node table item.

[0114] For example, by broadcasting ARP requests to the local area network where the edge cluster is located, the surviving nodes will respond after receiving the ARP requests to obtain the IP addresses of all surviving nodes in the local area network. Each node will regularly detect the surviving nodes around it and save their IP addresses to the neighbor node list to cope with the changing node distribution and network disconnection in dynamic edge scenarios.

[0115] The embodiment of the present invention does not limit the type of the above-mentioned preset request. At the same time, in the subsequent step of collecting the transmission bandwidth, transmission delay and packet loss rate between each data node and each task node in the cluster, the neighbor node list can be traversed and data can only be collected for the inventory node. In addition, in order to ensure the validity of the data in the neighbor node list, it should be periodically detected whether the node is alive based on the above method, and then the neighbor node table entry should be updated.

[0116] Furthermore, in modern network systems, the processing efficiency of data traffic has become a key factor affecting system performance and user experience. Traditionally, many network applications intercept and process network data packets through user-mode programs to achieve other functions besides transmission. However, when processing network data packets in user mode, each data packet needs to be transferred from the network interface through kernel mode to the user-mode application for processing, and then returned to kernel mode after processing. This process involves multiple context switches and data copies, which often brings significant processing delays and system overhead.

[0117] Based on this, in an exemplary embodiment of the present invention, the task to be processed is transmitted to the target task node, including: configuring corresponding forwarding rules based on the target task node; and presetting a filter based on the forwarding rules to forward the task to be processed to the target task node based on the kernel state.

[0118] Among them, the preset filter is used to directly forward and process the network data packet in the kernel state without being intercepted by the user-state related filter. The above preset filter may include but is not limited to: iptables, ipvs, etc., and the embodiment of the present invention does not limit this. During deployment, the rule chain priority of the preset filter needs to be placed before the user-state filter to ensure that the preset filter intercepts the traffic for processing and forwarding before the original rules of the user-state filter at the kernel level. It avoids the traditional process of the proxy intercepting traffic in the user state, calculating and analyzing the traffic, and then regenerating the redirection request for the kernel to send. This saves the overhead of switching all traffic from user space to kernel space, and further reduces the corresponding time.

[0119] Please refer to Figure 3 As shown, based on the same inventive concept as the aforementioned network adaptive traffic proxy method in a dynamic edge environment, an embodiment of the present invention provides a network adaptive traffic proxy system 300 in a dynamic edge environment, which includes: a network resource perception module 301, a link analysis and decision module 302, and a transmission module 303.

[0120] Specifically, the network resource perception module 301 is used to collect the transmission bandwidth, transmission delay and packet loss rate between each data node and each task node in the cluster; the link analysis and decision module 302 is used to respond to the task forwarding request of the data node, obtain the information of the task to be forwarded and calculate the link transmission time of the task to be forwarded from the data node to each task node; the transmission module 303 is used to determine the optimal transmission path based on the link transmission time, and transmit the task to the corresponding task node based on the optimal transmission path.

[0121] It should be noted that the network adaptive traffic proxy system 300 in the dynamic edge environment provided by the present invention also includes a preprocessing module, which is used to broadcast requests to the local area network where the cluster is located; if the task node in the cluster responds to the request, the node IP of the task node is saved to the neighbor node table entry.

[0122] It should also be noted that the network resource perception module 301 is also used by the data node to send multiple data packets to the node IP corresponding to the task node in the neighbor node table; record the transmission rate, response time, number of packets sent and number of packet losses of the data packets; calculate the transmission bandwidth, transmission delay and packet loss rate between the data node and each task node, and normalize the transmission bandwidth and the transmission delay.

[0123] The link analysis decision module 302 is also used to screen each task node in the neighbor node table item based on a preset node load threshold, and the node load threshold includes a CPU occupancy threshold and a memory resource utilization threshold; based on the transmission time between each task node in the neighbor node table item after screening and the data node where the task to be forwarded is located, calculate the corresponding link quality score; take the task node corresponding to the path with the largest link quality score as the target task node.

[0124] The transmission module 303 is also used to configure corresponding forwarding rules based on the target task node; the preset filter is based on the forwarding rules, and forwards the to-be-processed task to the target task node based on the kernel state.

[0125] Please refer to Figure 4 As shown, an embodiment of the present invention further provides an electronic device 400, which includes at least one processor 401, a memory 402 (e.g., a non-volatile memory), a memory 403, and a communication interface 404, and the at least one processor 401, the memory 402, the memory 403, and the communication interface 404 are connected together via a bus 405. At least one processor 401 is used to call at least one program instruction stored or encoded in the memory 402, so that the at least one processor 401 performs various operations and functions of the network adaptive traffic proxy method in the dynamic edge environment described in various embodiments of this specification.

[0126] In an embodiment of the present specification, the electronic device 400 may include, but is not limited to, a personal computer, a server computer, a workstation, a desktop computer, a laptop computer, a notebook computer, a mobile electronic device, a smart phone, a tablet computer, a cellular phone, a personal digital assistant (PDA), a handheld device, a messaging device, a wearable electronic device, a consumer electronic device, and the like.

[0127] An embodiment of the present invention also provides a computer-readable medium, which carries computer execution instructions. When the computer execution instructions are executed by a processor, they can be used to implement various operations and functions of the network adaptive traffic proxy method in a dynamic edge environment described in various embodiments of this specification.

[0128] The computer-readable medium in the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device.

[0129] In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, device, or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0130] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0131] The present invention is described with reference to flowcharts and / or block diagrams of methods, apparatuses, systems, and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as a combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the process in the flowchart. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0132] The foregoing description of specific exemplary embodiments of the present invention is for the purpose of illustration and demonstration. These descriptions are not intended to limit the present invention to the precise form disclosed, and it is clear that many changes and variations can be made based on the above teachings. The purpose of selecting and describing the exemplary embodiments is to explain the specific principles of the present invention and its practical application, so that those skilled in the art can realize and utilize various different exemplary embodiments of the present invention and various different selections and changes. The scope of the present invention is intended to be limited by the claims and their equivalents.

Claims

1. A network adaptive traffic proxy method in a dynamic edge environment, characterized in that: include: Collect the transmission bandwidth, transmission delay and packet loss rate between the data nodes storing the task data to be processed and each task node in the cluster; Calculate the transmission time of the task to be processed from the data node to each task node; Based on the link transmission time, a target task node is determined, and the task to be processed is transmitted to the target task node.

2. The method for network adaptive traffic proxy in a dynamic edge environment according to claim 1, characterized in that: The method further comprises: Broadcasting a request to the local area network where the cluster is located; If the task node in the cluster responds to the request, the node IP of the task node is saved to the neighbor node table entry.

3. The network adaptive traffic proxy method under a dynamic edge environment according to claim 2 is characterized in that: The transmission bandwidth, transmission delay and packet loss rate between the data node storing the task data to be processed and each task node in the acquisition cluster include: The data node sends multiple data packets to the node IP corresponding to the task node in the neighbor node table entry; Record the transmission rate, response time, number of packets sent, and number of packet losses of the data packet; The transmission bandwidth, transmission delay and packet loss rate between the data node and each task node are calculated, and the transmission bandwidth and the transmission delay are normalized.

4. The network adaptive traffic proxy method in a dynamic edge environment according to claim 3 is characterized in that: The formula for calculating the transmission bandwidth, transmission delay and packet loss rate between the data node and each task node is: Where D is the transmission delay between the data node and the task node; RTT is the time from the data node sending data to the data node receiving the confirmation message from the corresponding task node; B is the transmission bandwidth between the data node and the task node; S is the size of the data packet sent by the data node; T is the time required for the data node to send; L is the packet loss rate from the data node to the task node; P total Represents the total number of data packets sent by the data node per unit time, P recieved Represents the number of data packets received by the corresponding task node per unit time.

5. The method for network adaptive traffic proxy in a dynamic edge environment according to claim 4 is characterized in that: The formula for normalizing the transmission bandwidth and the transmission delay is: Among them, B' is the normalized transmission bandwidth, D' is the normalized transmission delay, B max is the maximum value of the transmission bandwidth, B min is the minimum value of the transmission bandwidth, D max is the maximum value of the transmission delay, D min is the minimum value of transmission delay.

6. The network adaptive traffic proxy method in a dynamic edge environment according to claim 5 is characterized in that: The formula for calculating the link transmission time of the task to be forwarded from the data node to each task node is: T i =T1+T2 Among them, T i is the transmission time required from data node m to task node n; is the normalized value of the transmission bandwidth from data node m to task node n; is the normalized value of the transmission delay from data node m to task node n; is the packet loss rate from data node m to task node n, S i is the size of the task to be forwarded.

7. The network adaptive traffic proxy method in a dynamic edge environment according to claim 6 is characterized in that: The determining of the target task node based on the link transmission time includes: Based on a preset node load threshold, each task node in the neighbor node table entry is screened, and the node load threshold includes a CPU occupancy threshold and a memory resource utilization threshold; Calculate the corresponding link quality score based on the transmission time between each task node in the neighbor node table after screening and the data node where the task to be forwarded is located; The task node corresponding to the path with the largest link quality score is taken as the target task node.

8. The network adaptive traffic proxy method in a dynamic edge environment according to claim 7 is characterized in that: The link quality score calculation formula is: in, is the link quality score from data node m to task node n, and α is the weight value.

9. The network adaptive traffic proxy method in a dynamic edge environment according to claim 1, characterized in that: Transmitting the task to be processed to the target task node includes: Based on the target task node, configure corresponding forwarding rules; The preset filter forwards the to-be-processed task to the target task node based on the kernel state based on the forwarding rule.

10. A network adaptive traffic proxy system in a dynamic edge environment, characterized in that: include: The network resource perception module is used to collect the transmission bandwidth, transmission delay and packet loss rate between each data node and each task node in the cluster; A link analysis and decision module, used to respond to a task forwarding request from a data node, obtain information about the task to be forwarded, and calculate a link transmission time between the task to be forwarded from the data node to each task node; The transmission module is used to determine the optimal transmission path based on the link transmission time, and transmit the task to the corresponding task node based on the optimal transmission path.

Images