Edge application scheduling method and device, equipment, medium and product

Abstract

The invention provides an edge application scheduling method and device, equipment, a medium and a product, and relates to the technical field of edge computing, and the method comprises the steps: obtaining resource state data of a plurality of edge nodes; dividing the edge nodes into a plurality of groups according to the resource state data; wherein at least one of the groups is configured as a point-to-point network group for mirror distribution; allocating a corresponding priority to each group based on the resource state data and the group type; wherein the priority is used for determining a distribution sequence of the edge applications; and scheduling the edge applications to each group in batches according to the priority sequence of the priorities. Therefore, efficient distribution of edge applications is realized by performing information acquisition and dynamic grouping on the edge nodes and combining a point-to-point networking technology and a priority scheduling strategy.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of edge computing, in particular to an edge application scheduling method and device, equipment, medium and product. BACKGROUND

[0002] With the rapid development of edge computing, the number of edge applications is showing explosive growth, and the unified distribution of massive edge applications has become a problem to be solved. In the process of edge application distribution, the image center plays an important role, however, when facing high concurrency of image pulling requests, the image center is prone to bottleneck problems, resulting in low distribution efficiency. In the prior art, in order to solve the high concurrency pulling bottleneck, a batch distribution method is often used, but too many batches will result in slow distribution speed, which is difficult to meet the demand of fast distribution of massive edge applications. The current scheduling strategy lacks the joint perception ability of the real-time state (network quality, computing load) of the edge node and the performance bottleneck of the image service, and cannot dynamically adjust the distribution strategy. How to improve the distribution speed without appearing bottleneck has become a key challenge in the field of edge application distribution. SUMMARY

[0003] The purpose of the embodiments of the present application is to provide an edge application scheduling method, device, equipment, medium and product, which can solve the problems of distribution speed and efficiency in the process of fast distribution of massive edge applications.

[0004] In a first aspect, the embodiments of the present application provide an edge application scheduling method, comprising: obtaining resource state data of a plurality of edge nodes; dividing the edge nodes into a plurality of groups according to the resource state data; wherein at least one group in each group is configured as a point-to-point network group for image distribution; allocating a corresponding priority to each group based on the resource state data and the group type; wherein the priority is used to determine the distribution order of the edge applications; batch scheduling edge applications to each group according to the priority order of the priority.

[0005] Optionally, obtaining the resource state data of the plurality of edge nodes comprises: collecting the computing load, network bandwidth, image cache list and historical pulling failure record of each edge node based on a periodic collection method or an event triggered collection method to obtain the resource state data of each edge node.

[0006] Optionally, the edge application scheduling method further comprises: real-time monitoring the performance indicators of the image warehouse; when the performance indicators exceed the preset threshold, adjusting the distribution flow of the edge applications scheduled to each group.

[0007] Optionally, the edge application scheduling method further comprises: predicting a target edge application based on historical pull data, and pre-distributing the target edge application to the edge nodes during a low load period of the mirror repository; wherein the target edge application is a popular edge application whose historical pull frequency exceeds the historical average pull frequency of all edge applications.

[0008] Optionally, when the performance indicator exceeds a preset threshold, the distribution traffic of the edge application to each group is adjusted, comprising: if the network input / output usage or the disk input / output usage in the performance indicator exceeds a first usage threshold, the export bandwidth of the edge application from the mirror repository is reduced; if the central processing unit usage or the memory usage in the performance indicator exceeds a second usage threshold, the distribution to groups that have not started pulling and the distribution of low-priority tasks are stopped.

[0009] Optionally, the edge nodes are divided into multiple groups according to the resource state data, comprising: determining adjacent edge nodes based on network quality data and hardware architecture data in the resource state data, and through packet detection; dividing adjacent edge nodes with the same hardware architecture and whose network quality meets a preset network quality condition into the same point-to-point network group.

[0010] Optionally, after dividing adjacent edge nodes with the same hardware architecture and whose network quality meets a preset network quality condition into the same point-to-point network group, the method further comprises: if the number of edge nodes in the point-to-point network group is lower than a preset number threshold, or if the bandwidth between the edge nodes in the point-to-point network group is lower than the bandwidth between the corresponding edge nodes and the mirror repository, it is determined that the point-to-point network group fails to be constructed.

[0011] Optionally, the edge nodes are divided into multiple groups according to the resource state data, comprising: other edge nodes located outside the point-to-point network group are taken as first candidate edge nodes; performing weighted calculation on the resource state data of each first candidate edge node by using a preset weighted calculation equation to obtain a resource state score of each first candidate edge node; dividing each first candidate edge node into different resource state groups according to the score in descending order and according to the resource state score.

[0012] Optionally, the preset weighted calculation equation is: ; wherein, a resource status score, a central processing unit usage rate, a first weight parameter of the central processing unit usage rate, a memory usage rate, a second weight parameter of the memory usage rate, a disk input / output usage rate, a third weight parameter of the disk input / output usage rate, a network input / output usage rate, a fourth weight parameter of the network input / output usage rate, a network bandwidth, a fifth weight parameter of the network bandwidth, a historical pulling failure rate, a sixth weight parameter of the historical pulling failure rate.

[0013] Optionally, the edge application scheduling method, each weight parameter is adjusted based on a change result of a historical pulling time consumption and a historical pulling frequency.

[0014] Optionally, the edge nodes are divided into a plurality of groups according to the resource status data, including: other edge nodes located outside the resource status groups are taken as second candidate edge nodes; each second candidate edge node is divided into a different cache group based on a matching relationship between a container image already cached in the second candidate edge node and a container image set required by the edge application to be scheduled, so as to obtain an image cache group.

[0015] Optionally, each second candidate edge node is divided into a different cache group based on a matching relationship between a container image already cached in the second candidate edge node and a container image set required by the edge application to be scheduled, so as to obtain an image cache group, including: a container image set relied on by the edge application to be scheduled is determined; the container image set is compared with an image cache list in the resource status data of each second candidate edge node; if the comparison result is that the container image set is completely consistent with the image cache list of the current second candidate edge node, the current second candidate edge node is divided into a complete image cache group; if the comparison result is that the container image set is not completely consistent with the image cache list of the current second candidate edge node, the current second candidate edge node is divided into a partial image cache group.

[0016] Optionally, the edge nodes are divided into a plurality of groups according to the resource status data, including: other edge nodes outside the mirror cache group are taken as third candidate edge nodes; temporarily group the third candidate edge nodes according to the historical pull failure times to obtain a historical pull failure times group.

[0017] Optionally, based on the resource state data and the group type, corresponding priorities are assigned to each group, including: the mirror cache group is assigned a preset highest priority; the peer-to-peer network group is assigned a preset high priority; according to the score of the resource state score, a preset medium priority or a preset low priority is assigned to the resource state group; the edge nodes with a high historical pull failure rate in the historical pull failure times group are assigned a preset lowest priority.

[0018] Optionally, after the edge nodes are divided into multiple groups according to the resource state data, the method further includes: real-time monitoring of resource state data of each edge node; when it is monitored that the resource state data of the edge node changes to meet a preset regrouping condition, a regrouping operation is performed on the edge node whose resource state data has changed.

[0019] Optionally, the edge nodes are divided into multiple groups according to the resource state data, including: based on historical load data of the mirror repository and group distribution history, an upper threshold of the number of edge nodes contained in each group is adjusted.

[0020] In a second aspect, an embodiment of the present application provides an edge application scheduling device, including: a data acquisition module for acquiring resource state data of multiple edge nodes; a grouping module for dividing the edge nodes into multiple groups according to the resource state data; wherein at least one group in each group is configured as a peer-to-peer network group for mirror distribution; a priority assignment module for assigning corresponding priorities to each group based on the resource state data and the group type; wherein the priority is used to determine the distribution order of the edge application; an application scheduling module for batch scheduling edge applications to each group according to the priority order of the priority.

[0021] In a third aspect, an embodiment of the present application provides an electronic device, including: a memory for storing a computer program; a processor for executing the computer program to realize the steps of the edge application scheduling method disclosed above.

[0022] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, and the computer readable storage medium stores a computer program. The computer program is executed by a processor to implement the steps of the edge application scheduling method disclosed above.

[0023] In a fifth aspect, an embodiment of the present application provides a computer program product, which comprises computer programs / instructions. The computer programs / instructions are executed by a processor to implement the steps of the edge application scheduling method disclosed above.

[0024] To solve the above technical problems, an embodiment of the present application provides an edge application scheduling method, which comprises: acquiring resource state data of a plurality of edge nodes; dividing the edge nodes into a plurality of groups according to the resource state data; wherein at least one group in each group is configured as a point-to-point network group for mirror distribution; allocating a corresponding priority to each group based on the resource state data and the group type; wherein the priority is used to determine the distribution order of the edge application; and batch scheduling the edge application to each group according to the priority order of the priority.

[0025] As can be seen from the above technical solution, the present application realizes efficient distribution of applications by information collection and dynamic grouping of edge nodes, in combination with point-to-point networking technology and priority scheduling strategy. At the same time, by monitoring system performance bottlenecks in real time and taking corresponding adjustment strategies, the bottleneck problem of the mirror center in high concurrency pulling is avoided. Compared with the prior art, the distribution efficiency is improved, the mirror image is preferentially distributed to the point-to-point group that successfully operates, the advantages of point-to-point network technology are fully utilized, and the distribution speed is accelerated; the system performance is monitored in real time, adjustment strategies are taken according to different bottleneck types, and the stable operation of the system is ensured; the dynamic grouping and priority adjustment mechanism enables the system to adapt to changes in edge node information, and always maintains an efficient distribution state. The present application can effectively improve the distribution efficiency of massive edge application mirrors, while ensuring the stability of the system, and has important practical application value. BRIEF DESCRIPTION OF DRAWINGS

[0026] In order to more clearly illustrate the embodiments of the present application, the drawings needed in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0027] Figure 1 A flow chart of an edge application scheduling method provided by an embodiment of the present application is provided. Figure 2 A structure schematic diagram of an edge application scheduling system provided by an embodiment of the present application is provided. Figure 3A structure schematic diagram of an edge application scheduling device provided by an embodiment of the present application is provided. Figure 4 An electronic device diagram provided by an embodiment of the present application. DETAILED DESCRIPTION

[0028] The technical solutions in the embodiments of the present application will be clearly and completely described with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0029] The terms “include” and “have” and any variations in the specification and above drawings of the present application, as well as any variations related to “include” and “have”, are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device including a series of steps or units is not limited to the listed steps or units, but can include steps or units not listed.

[0030] In order for those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the drawings and specific embodiments.

[0031] With the rapid development of edge computing, the number of edge applications presents an explosive growth, and the unified distribution of massive edge applications becomes a problem to be solved. In the edge application distribution process, the mirror center plays an important role, however, when facing high-concurrency mirror pulling requests, the mirror center is prone to bottleneck problems, resulting in low distribution efficiency. In the prior art, in order to solve the high-concurrency pulling bottleneck, a batch distribution method is often used, but too many batches will result in slow distribution speed, which is difficult to meet the demand of rapid distribution of massive edge applications. The current scheduling strategy lacks the joint perception ability of the real-time state (network quality, computing load) of the edge node and the performance bottleneck of the mirror service, and cannot dynamically adjust the distribution strategy. How to improve the distribution speed without appearing bottleneck has become a key challenge in the field of edge application distribution.

[0032] Therefore, the present application provides an edge application scheduling scheme, which can improve the distribution speed without appearing bottleneck in the unified distribution process of massive edge applications.

[0033] Reference Figure 1 As shown in the figure, the present application provides an edge application scheduling method, comprising: Step S11: Obtain resource state data of a plurality of edge nodes.

[0034] In this embodiment, the computing load, network bandwidth, mirror cache list and historical pull failure record of each edge node are collected based on a periodic collection mode or an event triggered collection mode to obtain resource state data of each edge node. It can be understood that the periodic collection and real-time event triggered collection are combined to collect the resource state of the edge node, and the mirror cache list, cloud edge network bandwidth, current resource usage (CPU (Central Processing Unit), memory, disk IO (Input / Output), etc.), i.e. computing load, and historical pull failure times of each edge node are obtained to obtain the resource state data. In this way, through the above-mentioned periodic collection and real-time event triggered collection data collection mode, the timeliness of the information can be ensured, and the resource consumption caused by excessive collection can be avoided, wherein the periodic collection can be set at an appropriate period, such as collecting once every 5 minutes; when the edge node restarts or other abnormalities occur or the network environment changes greatly, real-time collection is triggered once.

[0035] Step S12: dividing the edge nodes into multiple groups according to the resource state data; wherein at least one group in each group is configured as a point-to-point network group for image distribution.

[0036] In this embodiment, the network quality data and hardware architecture data in the resource state data are used to determine adjacent edge nodes through packet detection, and the adjacent edge nodes with the same hardware architecture and network quality meeting the preset network quality condition are divided into the same point-to-point network group. It can be understood that the point-to-point networking distribution is first performed, specifically, the adjacent edge nodes are automatically discovered through packet detection, and the adjacent visible nodes are automatically grouped according to multiple factors such as network QoS (Quality of Service) and CPU architecture, and the edge nodes in the group are point-to-point networked. Specifically, the packet detection is performed through the ping protocol, and if some edge nodes disable ping, the health check of the REST API is used, for example, an edge node exposes an access interface / info to the outside, and the point-to-point networking in the group is the networking between the adjacent edge nodes in the group through the standard TCP / IP protocol (Transmission Control Protocol / Internet Protocol).

[0037] It should be noted that through the point-to-point network packet, the edge nodes in the same point-to-point network packet are in the edge application distribution process, if there is a container resource required by another edge node in a certain edge node in the group, the container resource distribution is completed through the point-to-point service directly, without the need to pull again from the cloud, to speed up the edge application distribution process.

[0038] In this embodiment, if the number of edge nodes in the point-to-point network packet is less than the preset number threshold, or if the bandwidth between the edge nodes in the point-to-point network packet is less than the bandwidth between the corresponding edge node and the mirror repository, it is determined that the point-to-point network packet construction fails. It can be understood that when the point-to-point network is successfully grouped, but the number of nodes in the group is less than the preset number threshold, it is considered as a grouping failure, for example: the number of nodes in the group is 3, but the preset number threshold is 10, it is considered as a grouping failure, and if the network is successfully grouped but the bandwidth between the edges is less than the bandwidth between the cloud and the edge, it is also considered as a point-to-point network grouping failure.

[0039] In this embodiment, other edge nodes located outside the point-to-point network packet are used as first candidate edge nodes; the resource state data of each first candidate edge node is weighted and calculated by using a preset weighting calculation equation to obtain the resource state score of each first candidate edge node; and each first candidate edge node is divided into different resource state groups in the order of the score height and according to the resource state score. The preset weighting calculation method for calculating the resource state score in the process of dividing the resource state group is: ; wherein, represents the resource state score, represents the central processor usage rate, represents the first weight parameter of the central processor usage rate, represents the memory usage rate, represents the second weight parameter of the memory usage rate, represents the disk input / output usage rate, represents the third weight parameter of the disk input / output usage rate, represents the network input / output usage rate, represents the fourth weight parameter of the network input / output usage rate, represents the network bandwidth, represents the fifth weight parameter of the network bandwidth, represents the historical pull failure rate, represents the sixth weight parameter of the historical pull failure rate.

[0040] It can be understood that other edge nodes (first candidate edge nodes) after being networked through a point-to-point network are grouped according to the resource usage of the edge nodes, and the cloud edge bandwidth and CPU, memory, disk IO, and network IO usage rates are weighted and summed, and the grouping is performed from high to low according to the score. It should be noted that when grouping according to the score, a minimum score threshold needs to be set, that is, only the resource state score greater than the minimum score threshold participates in the grouping step of grouping the resource state of the first candidate edge node according to the high-low order of the score. The calculation formula is the above-mentioned preset weighting calculation equation, and the initial value is pre-set according to experience. The experience process is: adjust each parameter to pull the container image, view the change of the pulling time, and increase the weight if it has a large impact. In the preset weighting calculation equation, the initial settings of the weight parameters are as follows: all weight parameters are initially set to 1 except that the fourth weight parameter is initially set to 2. The network bandwidth between the cloud and the edge Generally, the bandwidth size is 10 MiB level, so divide the value by 10 to avoid a too large value; the historical pulling failure rate is represented as, for example, 8 times out of 10 times of pulling, that is, 0.8.

[0041] It should be noted that each weight parameter is adjusted based on the change result of the historical pulling time consumption and the historical pulling frequency. It can be understood that after setting the initial weight value corresponding to each weight parameter, the adjustment of each weight parameter is realized based on dynamic adjustment. Specifically, the time taken by each edge node to pull the image in the group is counted, and the nodes with large time differences are queried to find out which factors have large differences. The corresponding authority is then increased, and the adjusted weight takes effect when a new edge node or a re-grouping trigger occurs.

[0042] In this embodiment, other edge nodes located outside the resource state group are taken as second candidate edge nodes; based on the matching relationship between the container images already cached in each second candidate edge node and the container image set required by the edge application to be scheduled, each second candidate edge node is divided into different cache groups to obtain an image cache group. Specifically, the container image set relied on by the edge application to be scheduled is determined; the container image set is compared with the image cache list in the resource state data of each second candidate edge node; if the comparison result is that the container image set is completely consistent with the image cache list of the current second candidate edge node, the current second candidate edge node is divided into a complete image cache group; if the comparison result is that the container image set is not completely consistent with the image cache list of the current second candidate edge node, the current second candidate edge node is divided into a partial image cache group.

[0043] It can be understood that after the resource state grouping is completed, the remaining edge nodes (second candidate edge nodes) are compared with the container image included in the edge application scheduled this time according to the cache list in the edge node reporting information, the edge nodes that have cached the application image are divided into the same group or multiple temporary groups, and are marked to avoid repeated scheduling; for example, before formal distribution, the cloud management system checks the image cache list reported by each edge node. The inventory of node A already has "v1.2 version of A container image" and "v2.0 version of B container image", and only lacks "C image", the inventory of node B has everything, and the inventory of node C has nothing, at this time, the image cache grouping described in the document is performed, node B is divided into a "complete cache group" because it has cached all the container images required by the application scheduled this time. This group has the highest priority and does not need to pull anything from the cloud, and can be directly marked as a "complete image cache group". Node A is divided into a "partial image cache group" because it only caches part of the required image. The system only needs to distribute the missing "C image" to it during distribution, avoiding repeated pulling of existing images and greatly saving bandwidth and time; node C is taken as a third candidate edge node.

[0044] In the embodiment, other edge nodes located outside the image cache grouping are taken as third candidate edge nodes; the third candidate edge nodes are temporarily grouped according to the number of historical pulling failures to obtain a historical pulling failure number grouping. It can be understood that after the image cache grouping, the edge nodes with a large number of historical pulling failures are temporarily grouped again, and are marked in the already grouped groups to avoid repeated scheduling. When the number of recent historical pulling image failures increases, the edge nodes are automatically kicked out of the grouping and the marking is removed in the existing fixed grouping.

[0045] In the embodiment, after the edge nodes are divided into multiple groups according to the resource state data, the method further includes: monitoring the resource state data of each edge node in real time; when it is monitored that the resource state data of the edge node changes to meet a preset regrouping condition, performing a regrouping operation on the edge node whose resource state data changes. It can be understood that after being divided into multiple groups, the state of the edge node is monitored in real time to facilitate inter-node regrouping. For example, if a point-to-point network of a certain node fails or the bandwidth or network quality changes, the node is kicked out of the group for redistribution. As for the size of each group, if the group is too large, the elasticity control is weak, and if the group is too small, the distribution efficiency is reduced. Therefore, the present scheme adopts an adaptive group size, dynamically adjusts the upper threshold of each type of group through historical analysis, and the upper threshold is guaranteed to be less than 10% of the bandwidth, CPU, memory and disk IO of the image warehouse. The size of the initial grouping is set to be the ratio of the image center outlet bandwidth to the maximum bandwidth of the edge node in the cloud edge, such as 100MiB / 10MiB=10.

[0046] Step S13: Assign corresponding priority to each group based on the resource state data and the group type; wherein the priority is used to determine the distribution order of the edge application.

[0047] In this embodiment, the preset highest priority is assigned to the mirror cache group, the preset high priority is assigned to the point-to-point network group, the preset medium priority or the preset low priority is assigned to the resource state group according to the score of the resource state score, and the preset lowest priority is assigned to the edge node with a high historical pull failure rate in the historical pull failure number group. It can be understood that after the edge nodes are grouped, the priority of different groups is set, the node group with mirror cache has extremely high priority, the group that can successfully form a point-to-point network has high priority, the edge node group with high bandwidth has medium priority, other groups are set to low, and the node with frequent network interruption or pull failure in history is set to extremely low. The edge application is distributed according to the priority. If the point-to-point networking group is successful but the number of nodes in the group is too small, the distribution effect will be reduced, and the group will be considered as a failure.

[0048] Step S14: Batch scheduling of the edge application to each group according to the priority order of the priority.

[0049] In this embodiment, the edge application is batch distributed according to the order of the above-mentioned priority from high to low priority. Specifically, when it is monitored that the high-priority group completes the edge application pull, it is dynamically registered as a secondary distribution source, and the subsequent low-priority group is guided to pull the edge application from the mirror repository and the ready secondary distribution source. In the process of distribution, after the high-priority group completes its own pull, it assists the low-priority group in mirror distribution as a secondary distribution node. Specifically, when a high-priority group (point-to-point network group) quickly pulls a complete mirror from the cloud, it is promoted to a secondary source. Subsequently, when the low-priority group pulls, its source address not only includes the cloud mirror repository, but also automatically adds these ready secondary source nodes. This essentially extends the cooperation of the point-to-point network from one group to cross-group cooperation. In this way, the uplink bandwidth of the edge node that has successfully distributed is fully utilized, forming multiple concurrent distribution pipelines, relieving the export pressure of the cloud mirror repository, and especially accelerating the distribution process of low-priority, resource-poor nodes. Even if the cloud mirror repository is temporarily unreachable due to some reason, the existing secondary source nodes can still continue to provide services for subsequent nodes, improving the fault tolerance and availability of the entire distribution system. This makes the system take a step towards a truly decentralized architecture, solving the single-point bottleneck and failure risk of complete dependence on the central source.

[0050] In addition, during the batch distribution scheduling process, the performance indicators of the image warehouse are monitored in real time; when the performance indicators exceed the preset threshold, the distribution flow of the edge application for each group is adjusted. Specifically, if the network input / output usage or the disk input / output usage in the performance indicators exceeds the first usage threshold, the export bandwidth of the edge application distributed from the image warehouse is reduced; if the central processing unit usage or the memory usage exceeds the second usage threshold, the distribution to the groups that have not started pulling is initiated and the low-priority task distribution is stopped. It can be understood that the performance indicators of the image warehouse, including network IO, CPU usage, memory usage and disk IO, are monitored in real time, and when any performance indicator exceeds the preset threshold, the elastic flow control strategy is executed. Specifically, when the network IO usage or the disk IO usage exceeds the first usage threshold (such as 70%), the export bandwidth of the edge application distributed from the image warehouse is reduced; if the CPU usage or the memory usage exceeds the second usage threshold, the subsequent application distribution action is suspended, and in addition, when the pressure of the image warehouse exceeds the preset pressure warning threshold, some edge groups that are pulling images need to continue to pull images. The first usage threshold, the second usage threshold and the preset pressure warning threshold can be set artificially according to historical experience, and the reliability is predictable.

[0051] It should be noted that in addition to the above threshold value mode for edge application distribution control, a mode adaptive mode can also be used, which limits the flow when the number of image pulling failures exceeds the threshold, such as 3 times, to prevent avalanches.

[0052] In this embodiment, the target edge application is predicted based on historical pulling data, and the target edge application is pre-distributed to the edge node during the low load period of the image warehouse; wherein the target edge application is a popular edge application whose historical pulling frequency exceeds the historical average pulling frequency of all edge applications. It can be understood that according to the historical pulling image frequency and the image update time, the popular image is preheated in batches to the edge node at night and other times when the pressure of the image warehouse is small, that is, it is pulled in advance to the edge node for caching. It should be noted that the above prediction method includes a grouping strategy based on historical pulling frequency and image update time during image preheating. During image preheating, images that have been updated in the recent period and have a high historical pulling frequency are preheated, and the time period is selected to support a fixed time period (such as one day) and an update image cumulative threshold (such as reaching 10 images). The specific time period or threshold is set according to the specific use scenario, and if the image updates frequently, the time one day can be reduced to half a day, and the cumulative threshold can be increased to 30 or more.

[0053] From the above technical scheme can be seen, the application realizes efficient distribution of application by information collection and dynamic grouping of edge nodes, combined with point-to-point networking technology and priority scheduling strategy. At the same time, by monitoring system performance bottleneck in real time and taking corresponding adjustment strategy, the bottleneck problem of mirror center in high concurrency pulling is avoided. Compared with the prior art, the distribution efficiency is improved, the mirror image is preferentially distributed to the point-to-point group successfully running group, the advantages of point-to-point network technology are fully utilized, and the distribution speed is accelerated; the system performance is monitored in real time, and adjustment strategy is taken according to different bottleneck types, so that the stable operation of the system is guaranteed; the dynamic grouping and priority adjustment mechanism enables the system to adapt to the change of edge node information, and always maintains efficient distribution state. The application can effectively improve the distribution efficiency of massive edge application mirror image, while guaranteeing the stability of the system, and has important practical application value.

[0054] As shown in Figure 2 The application also provides an edge application scheduling system for intelligent scheduling and distribution of massive edge application mirror image, which specifically comprises an edge node access module, an edge node management module, an application distribution module, an image management module, a P2P acceleration module and an edge management module, and the overall architecture is as shown in the accompanying Figure 2 The specific implementation steps are as follows: The edge node access module: this module exposes an access service to the outside for edge node access. The current mainstream scheme is to construct based on container technology, and there are two specific implementation forms: one is to deploy based on K8s pull, with the cloud as the control node and the edge side as the computing node, and the other is to deploy the K8s in a cluster form on the edge in a lightweight manner. This module uniformly accesses the edge nodes in the two forms of access mode, and selects a certain type for access when the edge node accesses.

[0055] The edge node management module: this module is responsible for unified viewing and management of edge nodes, accepts resource information reported by the edge module in the edge node and stores it, groups the edge nodes according to the edge node resource information and the corresponding strategy, and sets the priority.

[0056] The image management module: responsible for the life cycle management of container image, and exposes the image pulling service for edge node pulling, cooperates with the P2P acceleration module to realize image acceleration distribution, and the current mainstream image warehouse has Harbor, Registry, etc.

[0057] The edge management module: collects information on the current node, including cloud edge network bandwidth, CPU, memory, disk, network card usage rate and other information, and reports it to the edge node management module in the cloud.

[0058] P2P acceleration module: responsible for the current edge node and mirror management module and other edge nodes in the group to build P2P network to accelerate container image distribution, P2P networking success edge node pulls the image from this module, and the group networking fails to continue to pull the image in the image management module.

[0059] In this way, the distribution speed is ensured while avoiding the central bottleneck. Based on dynamic grouping and P2P joint acceleration, the limitations of static grouping are broken through, and an efficient distribution network is constructed based on real-time state; based on resource-aware elastic flow control, the dynamic balance of the distribution strategy and the center load is realized; based on predictive image preheating, the instantaneous data amount is reduced from the transmission source, and the high concurrency pressure source is radically solved.

[0060] Referring to Figure 3 The application provides an edge application scheduling device, comprising: A data acquisition module 11 is configured to acquire resource state data of a plurality of edge nodes. A grouping module 12 is configured to divide the edge nodes into a plurality of groups according to the resource state data; wherein at least one group in each group is configured as a point-to-point network group for image distribution. A priority allocation module 13 is configured to allocate corresponding priorities to each group based on the resource state data and the group type; wherein the priority is used to determine the distribution order of the edge application. An application scheduling module 14 is configured to batch schedule edge applications to each group according to the priority order of the priority.

[0061] Therefore, by constructing a point-to-point network group, isolated nodes are organized into an efficient distribution unit, effectively utilizing the bandwidth and storage resources of the edge side, greatly reducing the export pressure of the image center, thereby avoiding the bottleneck problem under high concurrency from the root cause; the priority allocation module and the application scheduling module jointly decide based on the resource state and the group type, so that the distribution strategy can automatically adjust with the change of the edge environment. Ensure that the device always operates in an optimal or near-optimal state, which not only accelerates the overall distribution process, but also guarantees the reliability of the service.

[0062] Further, the application embodiment further discloses an electronic device, Figure 4 is an electronic device structure diagram according to an example embodiment, Figure 4The content in the foregoing embodiments should not be considered as any limitation to the scope of use of the present application. The electronic device can specifically include at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25 and a communication bus 26. The memory 22 is configured to store a computer program, and the computer program is loaded and executed by the processor 21 to implement the related steps in the edge application scheduling method disclosed in any of the foregoing embodiments. In addition, the electronic device in the embodiment can be specifically an electronic computer.

[0063] In the embodiment, the power supply 23 is configured to provide working voltage for each hardware device on the electronic device; the communication interface 24 can create a data transmission channel between the electronic device and external devices, and the communication protocol followed by the communication interface 24 can be any communication protocol applicable to the technical solution of the present application, which is not limited specifically herein; the input / output interface 25 is configured to obtain external input data or output data to the outside, and the specific interface type can be selected according to the specific application needs, which is not limited specifically herein.

[0064] In addition, the memory 22 as a carrier for storing resources can be a read-only memory, a random access memory, a magnetic disk or an optical disk, etc., and the resources stored thereon can include an operating system 221, a computer program 222, etc., and the storage mode can be temporary storage or permanent storage.

[0065] The operating system 221 is configured to manage and control each hardware device on the electronic device and the computer program 222, and can be Windows Server, Netware, Unix, Linux, etc. In addition to the computer program capable of completing the edge application scheduling method executed by the electronic device disclosed in any of the foregoing embodiments, the computer program 222 can further include a computer program capable of completing other specific work.

[0066] Further, the present application further discloses a computer readable storage medium for storing a computer program; wherein the computer program is executed by the processor to implement the edge application scheduling method disclosed in the foregoing embodiments. The specific steps of the method can refer to the corresponding content disclosed in the foregoing embodiments, which will not be repeated here.

[0067] The embodiments in the specification are described in a progressive manner, and each embodiment focuses on the difference from other embodiments. The same or similar parts of each embodiment can be referred to each other. For the device disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple, and the related parts can refer to the method part.

[0068] Those skilled in the art will further appreciate that the various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the embodiments disclosed herein can be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality, without reference to a specific sequence of operations for implementing the functions, and / or without reference to a specific hierarchy of circuit elements. Given the teachings of the present application provided herein, one of ordinary skill in the art will be able to contemplate these and similar implementations, and applications, of the various illustrative logical blocks, modules, circuits, and algorithm steps described herein.

[0069] The steps of a method or algorithm described in connection with the embodiments disclosed herein can be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module can reside in random access memory (RAM), flash memory, read-only memory (ROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor such that the processor can read information from, and write information to, the storage medium. In the alternative, hard disk can be used as a storage medium.

[0070] Finally, it should be noted that the terms "first", "second", and the like, herein do not denote any order, quantity, combination, or importance, but rather are used to distinguish one element from another, and are more especially used for the purpose of distinction from other elements in a claim. Also, the terms "comprises", "comprising", or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises... a" does not, without more constraints, exclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0071] The above detailed description has set forth various embodiments of the methods and devices disclosed herein. The descriptions of the embodiments are not intended to be exhaustive or to be construed as limiting the application to the precise forms disclosed. While specific embodiments of, and examples for, the methods and devices are described herein for illustrative purposes, various equivalent modifications are possible within the scope of the application, as those skilled in the relevant art will recognize. The teachings of the present application should not be limited, however, to one or more particular methods or devices, but should be determined entirely by the claims that follow, considered with the full scope intended to be embraced by such claims.

Claims

1. An edge application scheduling method, characterized in that, include: Obtain resource status data from multiple edge nodes; The edge nodes are divided into multiple groups based on the resource status data; wherein, at least one group in each group is configured as a peer-to-peer network group for mirror distribution; Based on the resource status data and group type, a corresponding priority is assigned to each group; wherein, the priority is used to determine the distribution order of edge applications; Batch scheduling edge applications are applied to each group according to the priority order of the priorities.

2. The edge application scheduling method according to claim 1, characterized in that, The acquisition of resource status data from multiple edge nodes includes: The computational load, network bandwidth, image cache list, and historical fetch failure records of each edge node are collected using either periodic or event-triggered collection methods to obtain resource status data for each edge node.

3. The edge application scheduling method according to claim 1, characterized in that, Also includes: Real-time monitoring of mirror repository performance metrics; When the performance metric exceeds a preset threshold, the distribution traffic for each group scheduling edge application is adjusted.

4. The edge application scheduling method according to claim 3, characterized in that, Also includes: Based on historical pull data, target edge applications are predicted and pre-distributed to edge nodes during the low load period of the mirror repository; The target edge application is a popular edge application whose historical fetch count exceeds the historical average fetch count of all edge applications.

5. The edge application scheduling method according to claim 3, characterized in that, When the performance metric exceeds a preset threshold, the distribution traffic for each group scheduling edge application is adjusted, including: If the network I / O utilization or disk I / O utilization in the performance metrics exceeds the first utilization threshold, then reduce the egress bandwidth for distributing edge applications from the mirror repository. If the CPU utilization or memory utilization in the performance metrics exceeds the second utilization threshold, then the distribution of low-priority tasks to groups that have not yet started fetching will be stopped.

6. The edge application scheduling method according to claim 1, characterized in that, The step of dividing the edge nodes into multiple groups based on the resource status data includes: Based on the network quality data and hardware architecture data in the resource status data, adjacent edge nodes are determined by packet probing. Adjacent edge nodes that meet the preset network quality conditions and have the same hardware architecture are grouped into the same point-to-point network group.

7. The edge application scheduling method according to claim 6, characterized in that, After grouping adjacent edge nodes with the same hardware architecture that meet the preset network quality conditions into the same point-to-point network group, the process further includes: If the number of edge nodes in the peer-to-peer network group is lower than a preset threshold, or if the bandwidth between edge nodes in the peer-to-peer network group is lower than the bandwidth between the corresponding edge node and the mirror repository, then the peer-to-peer network group construction is deemed to have failed.

8. The edge application scheduling method according to claim 6, characterized in that, The step of dividing the edge nodes into multiple groups based on the resource status data includes: Other edge nodes located outside the point-to-point network group are designated as first candidate edge nodes; The resource status data of each first candidate edge node are weighted using a preset weighted calculation equation to obtain the resource status score of each first candidate edge node. Based on the height of the scores and according to the resource status scores, each of the first candidate edge nodes is divided into different resource status groups.

9. The edge application scheduling method according to claim 8, characterized in that, The preset weighted calculation equation is: ； in, Indicates the resource status score. Indicates the central processing unit (CPU) utilization. The first weighted parameter representing the utilization rate of the central processing unit. Indicates memory usage. The second weighted parameter representing memory utilization. Indicates disk I / O utilization. The third weighted parameter representing disk I / O utilization. Indicates network input / output utilization. The fourth weighted parameter represents the network input / output utilization. Indicates network bandwidth. The fifth weighted parameter representing network bandwidth. Indicates the historical fetch failure rate. The sixth weighted parameter represents the historical fetch failure rate.

10. The edge application scheduling method according to claim 9, characterized in that, Each weight parameter is adjusted based on the changes in historical fetch time and historical fetch count.

11. The edge application scheduling method according to claim 8, characterized in that, The step of dividing the edge nodes into multiple groups based on the resource status data includes: Other edge nodes located outside the resource status group are designated as second candidate edge nodes; Based on the matching relationship between the cached container images in each of the second candidate edge nodes and the set of container images required by the edge application to be scheduled, each of the second candidate edge nodes is divided into different cache groups to obtain image cache groups.

12. The edge application scheduling method according to claim 11, characterized in that, Based on the matching relationship between the cached container images in each of the second candidate edge nodes and the set of container images required by the edge application to be scheduled, each of the second candidate edge nodes is divided into different cache groups to obtain image cache groups, including: Determine the set of container images that the edge application to be scheduled depends on; The container image set is compared with the image cache list in the resource status data of each second candidate edge node; If the comparison result shows that the container image set is completely consistent with the image cache list of the current second candidate edge node, then the current second candidate edge node will be assigned to the full image cache group. If the comparison result shows that the container image set is not completely consistent with the image cache list of the current second candidate edge node, then the current second candidate edge node is assigned to a partial image cache group.

13. The edge application scheduling method according to claim 11, characterized in that, The step of dividing the edge nodes into multiple groups based on the resource status data includes: Other edge nodes located outside the image cache group are designated as third candidate edge nodes; The third candidate edge nodes are temporarily grouped according to the number of historical pull failures to obtain the historical pull failure count group.

14. The edge application scheduling method according to any one of claims 1 to 13, characterized in that, The process of assigning corresponding priorities to each group based on the resource status data and group type includes: Assign the highest preset priority to the image cache group; Pre-set high priority for point-to-point network packet assignment; Based on the resource status score, the resource status is grouped and assigned a preset medium priority or a preset low priority. Assign a preset minimum priority to edge nodes with a high historical pull failure rate in the historical pull failure count group.

15. The edge application scheduling method according to claim 1, characterized in that, After dividing the edge nodes into multiple groups based on the resource status data, the method further includes: Real-time monitoring of resource status data of each edge node; When the resource status data changes of the edge nodes are detected to meet the preset regrouping conditions, the edge nodes whose resource status data have changed are regrouped.

16. The edge application scheduling method according to claim 1, characterized in that, The step of dividing the edge nodes into multiple groups based on the resource status data includes: Based on the historical load data of the mirror repository and the group distribution history, adjust the upper limit threshold of the number of edge nodes contained in each group.

17. An edge application scheduling device, characterized in that, include: The data acquisition module is used to acquire resource status data from multiple edge nodes; A grouping module is used to divide the edge nodes into multiple groups based on the resource status data; wherein, at least one group in each group is configured as a peer-to-peer network group for mirror distribution; The priority allocation module is used to assign a corresponding priority to each group based on the resource status data and group type; wherein the priority is used to determine the distribution order of edge applications; The application scheduling module is used to batch schedule edge applications to each group according to the priority order of the priorities.

18. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the steps of the edge application scheduling method as described in any one of claims 1 to 16.

19. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the edge application scheduling method as described in any one of claims 1 to 16.

20. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the edge application scheduling method according to any one of claims 1 to 16.

Images