Edge application scheduling method and apparatus, device, medium, and product

Abstract

The application provides an edge application scheduling method and device, equipment, medium and product, relates to the technical field of edge computing, and comprises the following steps: acquiring resource state data of a plurality of edge nodes; dividing the edge nodes into a plurality of groups according to the resource state data; wherein at least one group in each group is configured as a point-to-point network group for mirror distribution; allocating corresponding priorities to each group based on the resource state data and the group type; wherein the priority is used to determine the distribution order of the edge application; and batch scheduling the edge application to each group according to the priority order of the priority. As can be seen, through information collection and dynamic grouping of the edge nodes, in combination with the point-to-point networking technology and the priority scheduling strategy, efficient distribution of the edge application is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of edge computing, in particular to an edge application scheduling method and device, equipment, medium and product. BACKGROUND

[0002] With the rapid development of edge computing, the number of edge applications is growing explosively, and the unified distribution of massive edge applications has become a problem to be solved. In the process of edge application distribution, the image center plays an important role, however, when facing high-concurrency image pulling requests, the image center is prone to bottleneck problems, resulting in low distribution efficiency. In the prior art, in order to solve the high-concurrency pulling bottleneck, a batch distribution method is often used, but too many batches will result in slow distribution speed, which is difficult to meet the demand of fast distribution of massive edge applications. The current scheduling strategy lacks the joint perception ability of the real-time state (network quality, computing load) of the edge node and the performance bottleneck of the image service, and cannot dynamically adjust the distribution strategy. How to improve the distribution speed without appearing bottleneck has become a key challenge in the field of edge application distribution. SUMMARY

[0003] The purpose of the embodiments of the present application is to provide an edge application scheduling method, device, equipment, medium and product, which can solve the problems of distribution speed and efficiency in the process of fast distribution of massive edge applications.

[0004] In a first aspect, the embodiments of the present application provide an edge application scheduling method, comprising:

[0005] obtaining resource state data of a plurality of edge nodes;

[0006] dividing the edge nodes into a plurality of groups according to the resource state data; wherein at least one group in each group is configured as a point-to-point network group for image distribution;

[0007] allocating a corresponding priority to each group based on the resource state data and the group type; wherein the priority is used to determine the distribution order of the edge application;

[0008] batch scheduling edge applications to each group according to the priority order of the priority.

[0009] Optionally, obtaining the resource state data of the plurality of edge nodes comprises:

[0010] collecting the computing load, network bandwidth, image cache list and historical pulling failure record of each edge node based on a periodic collection method or an event-triggered collection method to obtain the resource state data of each edge node.

[0011] Optionally, the edge application scheduling method further comprises:

[0012] Real-time monitoring of mirror repository performance metrics;

[0013] When performance metrics exceed preset thresholds, the distribution traffic for each group of edge applications will be adjusted.

[0014] Optional edge application scheduling methods also include:

[0015] Predict target edge applications based on historical pull data, and pre-distribute target edge applications to edge nodes during periods of low load in the image repository;

[0016] Among them, the target edge application is a popular edge application whose historical pull count exceeds the historical average pull count of all edge applications.

[0017] Optionally, when performance metrics exceed preset thresholds, the distribution traffic for each group of edge applications is adjusted, including:

[0018] If the network I / O utilization or disk I / O utilization in the performance metrics exceeds the first utilization threshold, then reduce the egress bandwidth for distributing edge applications from the mirror repository.

[0019] If the CPU utilization or memory utilization in the performance metrics exceeds the second utilization threshold, then the distribution of low-priority tasks to groups that have not yet started fetching will be stopped.

[0020] Optionally, edge nodes can be divided into multiple groups based on resource status data, including:

[0021] Based on network quality data and hardware architecture data in resource status data, adjacent edge nodes are determined through packet probing.

[0022] Adjacent edge nodes that meet the preset network quality conditions and have the same hardware architecture are grouped into the same point-to-point network group.

[0023] Optionally, after grouping adjacent edge nodes with the same network quality that meet preset network quality conditions and have the same hardware architecture into the same point-to-point network group, the method further includes:

[0024] If the number of edge nodes within a peer-to-peer network group is lower than a preset threshold, or if the bandwidth between edge nodes within a peer-to-peer network group is lower than the bandwidth between the corresponding edge node and the mirror repository, then the peer-to-peer network group construction is deemed to have failed.

[0025] Optionally, edge nodes can be divided into multiple groups based on resource status data, including:

[0026] Other edge nodes located outside the point-to-point network packets are selected as the first candidate edge nodes;

[0027] The resource state data of each first candidate edge node is weighted and calculated by using a preset weighted calculation equation to obtain a resource state score of each first candidate edge node.

[0028] According to the high order of the score and according to the resource state score, each first candidate edge node is divided into different resource state groups.

[0029] Optionally, the preset weighted calculation equation is:

[0030] ;

[0031] wherein, represents the resource state score, represents the central processor usage rate, represents a first weight parameter of the central processor usage rate, represents the memory usage rate, represents a second weight parameter of the memory usage rate, represents the disk input / output usage rate, represents a third weight parameter of the disk input / output usage rate, represents the network input / output usage rate, represents a fourth weight parameter of the network input / output usage rate, represents the network bandwidth, represents a fifth weight parameter of the network bandwidth, represents the historical pulling failure rate, represents a sixth weight parameter of the historical pulling failure rate.

[0032] Optionally, in the edge application scheduling method, each weight parameter is adjusted based on a change result of the historical pulling time consumption and the historical pulling times.

[0033] Optionally, the edge nodes are divided into a plurality of groups according to the resource state data, including:

[0034] Other edge nodes located outside the resource state groups are taken as second candidate edge nodes;

[0035] Based on a matching relationship between the container images cached in each second candidate edge node and a container image set required by the edge application to be scheduled, each second candidate edge node is divided into different cache groups to obtain image cache groups.

[0036] Optionally, based on a matching relationship between the container images cached in each second candidate edge node and a container image set required by the edge application to be scheduled, each second candidate edge node is divided into different cache groups to obtain image cache groups, including:

[0037] determining a container image set relied on by the edge application to be scheduled;

[0038] comparing the container image set with the image cache list in the resource state data of each second candidate edge node;

[0039] if the comparison result is that the container image set is completely consistent with the image cache list of the current second candidate edge node, the current second candidate edge node is divided into a complete image cache group;

[0040] if the comparison result is that the container image set is not completely consistent with the image cache list of the current second candidate edge node, the current second candidate edge node is divided into a partial image cache group.

[0041] Optionally, the edge nodes are divided into a plurality of groups according to the resource state data, including:

[0042] other edge nodes located outside the image cache group are taken as third candidate edge nodes;

[0043] the third candidate edge nodes are temporarily grouped according to the historical number of pulling failures to obtain a historical pulling failure number group.

[0044] Optionally, corresponding priorities are assigned to each group based on the resource state data and the group type, including:

[0045] the image cache group is assigned a preset highest priority;

[0046] the point-to-point network group is assigned a preset high priority;

[0047] the resource state group is assigned a preset medium priority or a preset low priority according to the score of the resource state score;

[0048] the edge nodes with a high historical pulling failure rate in the historical pulling failure number group are assigned a preset lowest priority.

[0049] Optionally, after the edge nodes are divided into a plurality of groups according to the resource state data, the method further includes:

[0050] real-time monitoring of resource state data of each edge node;

[0051] when it is monitored that the resource state data of the edge node changes to meet a preset regrouping condition, performing a regrouping operation on the edge node with the changed resource state data.

[0052] Optionally, the edge nodes are divided into a plurality of groups according to the resource state data, including:

[0053] based on historical load data of the image repository and group distribution history, adjusting an upper threshold of the number of edge nodes contained in each group.

[0054] Secondly, embodiments of the present invention provide an edge application scheduling device, comprising:

[0055] The data acquisition module is used to acquire resource status data from multiple edge nodes;

[0056] The grouping module is used to divide edge nodes into multiple groups based on resource status data; among them, at least one group in each group is configured as a peer-to-peer network group for mirror distribution;

[0057] The priority allocation module is used to assign corresponding priorities to each group based on resource status data and group type; the priority is used to determine the distribution order of edge applications.

[0058] The application scheduling module is used to schedule edge applications to each group in batches according to their priority order.

[0059] Thirdly, embodiments of the present invention provide an electronic device, including:

[0060] Memory, used to store computer programs;

[0061] A processor for executing computer programs to implement the steps of the aforementioned disclosed edge application scheduling method.

[0062] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the aforementioned disclosed edge application scheduling method.

[0063] Fifthly, embodiments of the present invention provide a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the aforementioned disclosed edge application scheduling method.

[0064] To address the aforementioned technical problems, embodiments of the present invention provide an edge application scheduling method, comprising: acquiring resource status data of multiple edge nodes; dividing the edge nodes into multiple groups based on the resource status data; wherein at least one group in each group is configured as a peer-to-peer network group for mirror distribution; assigning a corresponding priority to each group based on the resource status data and the group type; wherein the priority is used to determine the distribution order of edge applications; and batch scheduling edge applications to each group according to the priority order of the priorities.

[0065] From the above technical scheme can be seen, the application realizes efficient distribution of application by information collection and dynamic grouping of edge nodes, combining point-to-point networking technology and priority scheduling strategy. At the same time, by real-time monitoring of system performance bottlenecks and taking corresponding adjustment strategies, the bottleneck problem of mirror center in high concurrency pulling is avoided. Compared with the prior art, the distribution efficiency is improved, the mirror image is preferentially distributed to the point-to-point group successfully running group, the advantages of point-to-point network technology are fully utilized, and the distribution speed is accelerated; the system performance is monitored in real time, and adjustment strategies are taken according to different bottleneck types, so that the stable operation of the system is guaranteed; the dynamic grouping and priority adjustment mechanism makes the system adapt to the change of edge node information, and always maintains efficient distribution state. The application can effectively improve the distribution efficiency of massive edge application mirror image, while guaranteeing the stability of the system, and has important practical application value. BRIEF DESCRIPTION OF DRAWINGS

[0066] In order to more clearly illustrate the embodiments of the present application, the drawings needed in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0067] Figure 1 A flow chart of an edge application scheduling method is provided for the embodiments of the present application.

[0068] Figure 2 A structure schematic diagram of an edge application scheduling system is provided for the embodiments of the present application.

[0069] Figure 3 A structure schematic diagram of an edge application scheduling device is provided for the embodiments of the present application.

[0070] Figure 4 An electronic device diagram is provided for the embodiments of the present application. DETAILED DESCRIPTION

[0071] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the protection scope of the present application.

[0072] The terms "including", "containing", "having" and their conjugates, as used throughout the present specification and in the preceding Claims, are meant to encompass the inclusion of one or more elements, features, integers, steps, processes, acts and / or groups thereof; but do not specify the existence of any limitation to the number of such elements, features, integers, steps, processes, acts, regarding the possession of any of these by the subject matter disclosed.

[0073] In order to make the person skilled in the art better understand the present application, the present application will be further described in detail below in combination with the drawings and specific embodiments.

[0074] With the rapid development of edge computing, the number of edge applications presents an explosive growth, and the unified distribution of massive edge applications becomes a problem to be solved. In the process of edge application distribution, the mirror center plays an important role, however, when facing high-concurrency mirror pulling requests, the mirror center is easy to have bottleneck problem, resulting in low distribution efficiency. In the prior art, in order to solve the high-concurrency pulling bottleneck, the batch distribution method is often used, but too many batches will lead to slow distribution speed, which is difficult to meet the demand of rapid distribution of massive edge applications. The current scheduling strategy lacks the joint perception ability of the real-time state (network quality, computing load) of the edge node and the performance bottleneck of the mirror service, and cannot dynamically adjust the distribution strategy. How to improve the distribution speed without bottleneck has become a key challenge in the field of edge application distribution.

[0075] Therefore, the present application provides an edge application scheduling scheme, which can improve the distribution speed without bottleneck in the process of unified distribution of massive edge applications.

[0076] With reference to Figure 1 The present application provides an edge application scheduling method, comprising:

[0077] Step S11: Obtain resource state data of a plurality of edge nodes.

[0078] In this embodiment, the computing load, network bandwidth, mirror cache list and historical pull failure record of each edge node are collected based on a periodic collection mode or an event triggered collection mode to obtain resource state data of each edge node. It can be understood that the periodic collection and real-time event triggered collection are combined to collect the resource state of the edge node, and the mirror cache list, cloud edge network bandwidth, current resource usage (CPU (Central Processing Unit), memory, disk IO (Input / Output), etc.), i.e. computing load, and historical pull failure times of each edge node are obtained to obtain the resource state data. In this way, through the above-mentioned periodic collection and real-time event triggered collection data collection mode, the timeliness of the information can be ensured, and the resource consumption caused by excessive collection can be avoided, wherein the periodic collection can be set at an appropriate period, such as collecting once every 5 minutes; when the edge node restarts or other abnormalities occur or the network environment changes greatly, real-time collection is triggered once.

[0079] Step S12: dividing the edge nodes into multiple groups according to the resource state data; wherein at least one group in each group is configured as a point-to-point network group for image distribution.

[0080] In this embodiment, the network quality data and hardware architecture data in the resource state data are used to determine adjacent edge nodes through packet detection, and the adjacent edge nodes with the same hardware architecture and network quality meeting the preset network quality condition are divided into the same point-to-point network group. It can be understood that the point-to-point networking distribution is first performed, specifically, the adjacent edge nodes are automatically discovered through packet detection, and the adjacent visible nodes are automatically grouped according to multiple factors such as network QoS (Quality of Service) and CPU architecture, and the edge nodes in the group are point-to-point networked. Specifically, the packet detection is performed through the ping protocol, and if some edge nodes disable ping, the health check of the REST API is used, for example, an edge node exposes an access interface / info to the outside, and the point-to-point networking in the group is the networking between the adjacent edge nodes in the group through the standard TCP / IP protocol (Transmission Control Protocol / Internet Protocol).

[0081] It should be noted that through the point-to-point network packet, the edge nodes in the same point-to-point network packet perform edge application distribution process, if there is a container resource required by another edge node in a certain edge node in the group, the container resource is directly distributed and obtained through the point-to-point service, without the need to pull again from the cloud, to speed up the edge application distribution process.

[0082] In this embodiment, if the number of edge nodes in the point-to-point network packet is less than the preset number threshold, or if the bandwidth between the edge nodes in the point-to-point network packet is less than the bandwidth between the corresponding edge node and the mirror repository, it is determined that the point-to-point network packet construction fails. It can be understood that when the point-to-point network is successfully grouped, but the number of nodes in the group is less than the preset number threshold, it is considered as a grouping failure, for example: the number of nodes in the group is 3, but the preset number threshold is 10, it is considered as a grouping failure. At the same time, if the networking is successful but the bandwidth between the edges is lower than the bandwidth between the cloud and the edge, it is also considered that the point-to-point network grouping fails.

[0083] In this embodiment, other edge nodes located outside the point-to-point network packet are used as first candidate edge nodes; the resource state data of each first candidate edge node is weighted and calculated by using a preset weighting calculation equation to obtain the resource state score of each first candidate edge node; and each first candidate edge node is divided into different resource state groups in the order of the score height and according to the resource state score. The preset weighting calculation method for calculating the resource state score in the process of dividing the resource state group is: ; wherein, represents the resource state score, represents the central processor usage rate, represents the first weight parameter of the central processor usage rate, represents the memory usage rate, represents the second weight parameter of the memory usage rate, represents the disk input / output usage rate, represents the third weight parameter of the disk input / output usage rate, represents the network input / output usage rate, represents the fourth weight parameter of the network input / output usage rate, represents the network bandwidth, represents the fifth weight parameter of the network bandwidth, represents the historical pull failure rate, represents the sixth weight parameter of the historical pull failure rate.

[0084] It can be understood that other edge nodes (first candidate edge nodes) after being networked through a point-to-point network are grouped according to the resource usage of the edge nodes, and the cloud edge bandwidth and CPU, memory, disk IO, and network IO usage rates are weighted and summed, and the grouping is performed from high to low according to the score. It should be noted that when grouping according to the score, a minimum score threshold needs to be set, that is, only the resource state score greater than the minimum score threshold participates in the grouping step of grouping the resource state of the first candidate edge node according to the high-low order of the score. The calculation formula is the above-mentioned preset weighting calculation equation, and the initial value is pre-set according to experience. The experience process is: adjust each parameter to pull the container image, view the change of the pulling time, and increase the weight if it has a large impact. In the preset weighting calculation equation, the initial settings of the weight parameters are as follows: all weight parameters are initially set to 1 except that the fourth weight parameter is initially set to 2. The network bandwidth between the cloud and the edge Generally, the bandwidth size is 10 MiB level, so divide the value by 10 to avoid a too large value; the historical pulling failure rate is represented as, for example, 8 times out of 10 times of pulling, that is, 0.8.

[0085] It should be noted that each weight parameter is adjusted based on the change result of the historical pulling time consumption and the historical pulling frequency. It can be understood that after setting the initial weight value corresponding to each weight parameter, the adjustment of each weight parameter is realized based on dynamic adjustment. Specifically, the time taken by each edge node to pull the image in the group is counted, and the nodes with large time differences are queried to find out which factors have large differences. The corresponding authority is then increased, and the adjusted weight takes effect when a new edge node or a re-grouping trigger occurs.

[0086] In this embodiment, other edge nodes located outside the resource state group are taken as second candidate edge nodes; based on the matching relationship between the container images already cached in each second candidate edge node and the container image set required by the edge application to be scheduled, each second candidate edge node is divided into different cache groups to obtain an image cache group. Specifically, the container image set relied on by the edge application to be scheduled is determined; the container image set is compared with the image cache list in the resource state data of each second candidate edge node; if the comparison result is that the container image set is completely consistent with the image cache list of the current second candidate edge node, the current second candidate edge node is divided into a complete image cache group; if the comparison result is that the container image set is not completely consistent with the image cache list of the current second candidate edge node, the current second candidate edge node is divided into a partial image cache group.

[0087] It can be understood that after the resource state grouping is completed, the remaining edge nodes (second candidate edge nodes) are compared with the container image included in the edge application scheduled this time according to the cache list in the edge node reporting information, the edge nodes that have cached the application image are divided into the same group or multiple temporary groups, and are marked to avoid repeated scheduling; for example, before formal distribution, the cloud management system checks the image cache list reported by each edge node. The inventory of node A already has "v1.2 version of A container image" and "v2.0 version of B container image", and only lacks "C image", the inventory of node B has everything, and the inventory of node C has nothing, at this time, the image cache grouping described in the document is performed, node B is divided into a "complete cache group" because it has cached all the container images required by the application scheduled this time. This group has the highest priority and does not need to pull anything from the cloud, and can be directly marked as a "complete image cache group". Node A is divided into a "partial image cache group" because it only caches part of the required image. The system only needs to distribute the missing "C image" to it during distribution, avoiding repeated pulling of existing images and greatly saving bandwidth and time; node C is taken as a third candidate edge node.

[0088] In the embodiment, other edge nodes located outside the image cache grouping are taken as third candidate edge nodes; the third candidate edge nodes are temporarily grouped according to the number of historical pulling failures to obtain a historical pulling failure number grouping. It can be understood that after the image cache grouping, the edge nodes with a large number of historical pulling failures are temporarily grouped again, and are marked in the already grouped groups to avoid repeated scheduling. When the number of recent historical pulling image failures increases, the edge nodes are automatically kicked out of the grouping and the marking is removed in the existing fixed grouping.

[0089] In the embodiment, after the edge nodes are divided into multiple groups according to the resource state data, the method further includes: monitoring the resource state data of each edge node in real time; when it is monitored that the resource state data of the edge node changes to meet a preset regrouping condition, performing a regrouping operation on the edge node whose resource state data changes. It can be understood that after being divided into multiple groups, the state of the edge node is monitored in real time to facilitate inter-node regrouping. For example, if a point-to-point network of a certain node fails or the bandwidth or network quality changes, the node is kicked out of the group for redistribution. As for the size of each group, if the group is too large, the elasticity control is weak, and if the group is too small, the distribution efficiency is reduced. Therefore, the present scheme adopts an adaptive group size, dynamically adjusts the upper threshold of each type of group through historical analysis, and the upper threshold is guaranteed to be less than 10% of the bandwidth, CPU, memory and disk IO of the image warehouse. The size of the initial grouping is set to be the ratio of the image center outlet bandwidth to the maximum bandwidth of the edge node in the cloud edge, such as 100MiB / 10MiB=10.

[0090] Step S13: Assign a corresponding priority to each group based on resource status data and group type; where the priority is used to determine the distribution order of edge applications.

[0091] In this embodiment, the mirror cache group is assigned the highest preset priority; the point-to-point network group is assigned a high preset priority; the resource status group is assigned a medium or low preset priority based on its resource status score; and the edge nodes with a high historical pull failure rate in the historical pull failure count group are assigned the lowest preset priority. It can be understood that after grouping the edge nodes, different priorities are set for different groups. Nodes with mirror caches have extremely high priority, groups that can successfully establish point-to-point networks have high priority, edge nodes with high bandwidth have medium priority, other groups have low priority, and nodes that frequently experience network outages or pull failures have extremely low priority. Edge applications are distributed according to these priorities. If a point-to-point network group succeeds but has too few nodes, the distribution effect will be reduced, and the group will be considered a failure.

[0092] Step S14: Dispatch edge applications to each group in batches according to their priority order.

[0093] In this embodiment, edge applications are distributed in batches according to the priority order described above, from high to low priority. Specifically, when a high-priority group completes the fetching of an edge application, it is dynamically registered as a secondary distribution source, guiding subsequent low-priority groups to simultaneously fetch edge applications from the image repository and the ready secondary distribution sources. During the distribution process, after a high-priority group completes its own fetch, it assists low-priority groups in image distribution as a secondary distribution node. Specifically, when a high-priority group (peer-to-peer network group) quickly fetches a complete image from the cloud, it is promoted to a secondary source. Subsequently, when low-priority groups fetch, their source address includes not only the cloud image repository but also these ready secondary source nodes. This essentially extends peer-to-peer network collaboration from within a group to across groups. In this way, the uplink bandwidth of successfully distributed edge nodes is fully utilized, forming multiple concurrent distribution pipelines, alleviating the outbound pressure on the cloud image repository, and especially accelerating the distribution process for low-priority, resource-poor nodes. Even if the cloud-based mirror repository becomes temporarily unavailable, existing secondary source nodes can still continue to provide services to subsequent nodes, improving the fault tolerance and availability of the entire distribution system. This represents a step towards a truly decentralized architecture, resolving the single point of failure and risk of relying entirely on a centralized source.

[0094] In addition, during the batch distribution scheduling process, the performance indicators of the image warehouse are monitored in real time; when the performance indicators exceed the preset threshold, the distribution flow of the edge application for each group is adjusted. Specifically, if the network input / output usage or the disk input / output usage in the performance indicators exceeds the first usage threshold, the export bandwidth of the edge application distributed from the image warehouse is reduced; if the central processing unit usage or the memory usage exceeds the second usage threshold, the distribution to the groups that have not started pulling is initiated and the low-priority task distribution is stopped. It can be understood that the performance indicators of the image warehouse, including network IO, CPU usage, memory usage and disk IO, are monitored in real time, and when any performance indicator exceeds the preset threshold, the elastic flow control strategy is executed. Specifically, when the network IO usage or the disk IO usage exceeds the first usage threshold (such as 70%), the export bandwidth of the edge application distributed from the image warehouse is reduced; if the CPU usage or the memory usage exceeds the second usage threshold, the subsequent application distribution action is suspended, and in addition, when the pressure of the image warehouse exceeds the preset pressure warning threshold, some edge groups that are pulling images need to continue to pull images. The first usage threshold, the second usage threshold and the preset pressure warning threshold can be set artificially according to historical experience, and this reliability is predictable.

[0095] It should be noted that in addition to the above threshold value mode for edge application distribution control, a mode adaptive mode can also be used, which limits the flow when the number of image pulling failures exceeds the threshold, such as 3 times, to prevent avalanches.

[0096] In this embodiment, the target edge application is predicted based on historical pulling data, and the target edge application is pre-distributed to the edge node during the low load period of the image warehouse; wherein the target edge application is a popular edge application whose historical pulling frequency exceeds the historical average pulling frequency of all edge applications. It can be understood that according to the historical pulling image frequency and the image update time, the popular image is preheated in batches to the edge node at night and other times when the pressure of the image warehouse is small, that is, it is pulled in advance to the edge node for caching. It should be noted that the above prediction method includes a grouping strategy based on historical pulling frequency and image update time during image preheating. During image preheating, images that have been updated in the recent period and have a high historical pulling frequency are preheated, and the time period is selected to support a fixed time period (such as one day) and an update image cumulative threshold (such as reaching 10 images). The specific time period or threshold is set according to the specific use scenario, and if the image updates frequently, the time one day can be reduced to half a day, and the cumulative threshold can be increased to 30 or more.

[0097] From the above technical scheme can be seen, the application realizes efficient distribution of application by information collection and dynamic grouping of edge nodes, combined with point-to-point networking technology and priority scheduling strategy. At the same time, by monitoring system performance bottleneck in real time and taking corresponding adjustment strategy, the bottleneck problem of mirror center in high concurrency pulling is avoided. Compared with the prior art, the distribution efficiency is improved, the mirror image is preferentially distributed to the successfully running group of point-to-point group, the advantages of point-to-point network technology are fully utilized, and the distribution speed is accelerated; the system performance is monitored in real time, and adjustment strategy is taken according to different bottleneck types, so that the stable operation of the system is ensured; the dynamic grouping and priority adjustment mechanism enables the system to adapt to the change of edge node information, and always maintains efficient distribution state. The application can effectively improve the distribution efficiency of massive edge application mirror image, while ensuring the stability of the system, and has important practical application value.

[0098] As shown in Figure 2 The application also provides an edge application scheduling system for intelligent scheduling and distribution of massive edge application mirror image, which specifically comprises an edge node access module, an edge node management module, an application distribution module, an image management module, a P2P acceleration module and an edge management module, and the overall architecture is as shown in the accompanying Figure 2 The specific implementation steps are as follows:

[0099] The edge node access module: this module exposes an access service to the outside for edge node access. The current mainstream scheme is to construct based on container technology, and there are two specific implementation forms: one is to deploy based on K8s pull, with the cloud as the control node and the edge side as the computing node, and the other is to deploy the K8s in a cluster form on the edge in a lightweight manner. This module uniformly accesses the edge nodes in the two forms of access mode, and selects a certain type for access when the edge node accesses.

[0100] The edge node management module: this module is responsible for unified viewing and management of edge nodes, accepts resource information reported by the edge module in the edge node and stores it, groups the edge nodes according to the edge node resource information and the corresponding strategy, and sets the priority.

[0101] The image management module: responsible for the life cycle management of container image, and exposes the image pulling service for edge node pulling, cooperates with the P2P acceleration module to realize image acceleration distribution, and the current mainstream image warehouse has Harbor, Registry, etc.

[0102] The edge management module: collects the information on the current node, including cloud edge network bandwidth, CPU, memory, disk, network card usage rate and other information, and reports it to the edge node management module in the cloud.

[0103] P2P acceleration module: responsible for the current edge node and mirror management module and other edge nodes in the group to build P2P network to accelerate container image distribution, P2P networking success edge node pulls the image from this module, and the group networking fails to continue to pull the image in the image management module.

[0104] In this way, the distribution speed is ensured while avoiding the central bottleneck. Based on dynamic grouping and P2P joint acceleration, the limitations of static grouping are broken through, and an efficient distribution network is constructed based on real-time state; based on resource-aware elastic flow control, the dynamic balance of distribution strategy and central load is realized; based on predictive image preheating, the instantaneous data amount is reduced from the transmission source, and the high concurrency pressure source is radically solved.

[0105] Referring to Figure 3 The application provides an edge application scheduling device, comprising:

[0106] A data acquisition module 11 is configured to acquire resource state data of a plurality of edge nodes.

[0107] A grouping module 12 is configured to divide the edge nodes into a plurality of groups according to the resource state data; wherein at least one group in each group is configured as a point-to-point network group for image distribution.

[0108] A priority allocation module 13 is configured to allocate corresponding priorities to each group based on the resource state data and the group type; wherein the priority is used to determine the distribution order of the edge application.

[0109] An application scheduling module 14 is configured to batch schedule edge applications to each group according to the priority order of the priorities.

[0110] Therefore, by constructing a point-to-point network group, isolated nodes are organized into an efficient distribution unit, effectively utilizing the bandwidth and storage resources of the edge side, greatly reducing the export pressure of the image center, thereby avoiding the bottleneck problem under high concurrency from the root cause; the priority allocation module and the application scheduling module jointly decide based on the resource state and the group type, so that the distribution strategy can automatically adjust with the change of the edge environment. Ensure that the device always operates in an optimal or near-optimal state, which not only accelerates the overall distribution process, but also guarantees the reliability of the service.

[0111] Further, the application embodiment further discloses an electronic device, Figure 4 is an electronic device structure diagram according to an example embodiment, Figure 4The content in the foregoing embodiments should not be considered as any limitation to the scope of use of the present application. The electronic device can specifically include at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25 and a communication bus 26. The memory 22 is configured to store a computer program, and the computer program is loaded and executed by the processor 21 to implement the related steps in the edge application scheduling method disclosed in any of the foregoing embodiments. In addition, the electronic device in the embodiment can be specifically an electronic computer.

[0112] In the embodiment, the power supply 23 is configured to provide working voltage for each hardware device on the electronic device; the communication interface 24 can create a data transmission channel between the electronic device and external devices, and the communication protocol followed by the communication interface 24 can be any communication protocol applicable to the technical solution of the present application, which is not limited specifically herein; the input / output interface 25 is configured to obtain external input data or output data to the outside, and the specific interface type can be selected according to the specific application needs, which is not limited specifically herein.

[0113] In addition, the memory 22 as a carrier for storing resources can be a read-only memory, a random access memory, a magnetic disk or an optical disk, etc., and the resources stored thereon can include an operating system 221, a computer program 222, etc., and the storage mode can be temporary storage or permanent storage.

[0114] The operating system 221 is configured to manage and control each hardware device on the electronic device and the computer program 222, and can be Windows Server, Netware, Unix, Linux, etc. In addition to the computer program capable of completing the edge application scheduling method executed by the electronic device disclosed in any of the foregoing embodiments, the computer program 222 can further include a computer program capable of completing other specific work.

[0115] Further, the present application further discloses a computer readable storage medium for storing a computer program; wherein the computer program is executed by the processor to implement the edge application scheduling method disclosed in the foregoing embodiments. For the specific steps of the method, refer to the corresponding content disclosed in the foregoing embodiments, which will not be repeated here.

[0116] The embodiments in the specification are described in a progressive manner, and each embodiment focuses on the difference from other embodiments. For the same or similar parts between the embodiments, refer to each other. For the device disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple, and the relevant parts refer to the method part.

[0117] Those skilled in the art will further appreciate that the various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the embodiments disclosed herein can be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality, without reference to a specific sequence of operations for implementing the functions described. Depending upon the implementation chosen, such a function can be implemented in hardware, software, or any combination thereof. Skilled artisans will appreciate that the choice of hardware or software can involve trade-offs in terms of ease of manufacture, speed of execution, accuracy of results, and the like.

[0118] The steps of a method or algorithm described in connection with the embodiments disclosed herein can be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module can reside in random access memory (RAM), flash memory, read-only memory (ROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0119] Finally, it should be noted that the terms "first", "second", and the like, herein do not denote any order, quantity, combination, or importance, but rather are used to distinguish one element from another, and are more especially used for the purpose of distinction from other elements in the specification. Also, the terms "include", "comprise", or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements does not include only those elements recited, but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus. In addition, the terms "plurality" and "a plurality" mean "two or more" of the specified element, and do not imply that the elements must be identical.

[0120] The above detailed description of the technical solutions provided by the present application has been described in detail, and the principles and implementation modes of the present application have been described by applying specific examples. The above description of the embodiments is only applicable to help understand the method and core idea of the present application; at the same time, for those skilled in the art, according to the idea of the present application, the specific implementation mode and application range will be changed, and the above description of the specification should not be understood as a limitation of the present application.

Claims

1. An edge application scheduling method, characterized in that, include: Obtain resource status data from multiple edge nodes; The edge nodes are divided into multiple groups based on the resource status data; wherein, at least one group in each group is configured as a peer-to-peer network group for mirror distribution; Based on the resource status data and group type, a corresponding priority is assigned to each group; wherein, the priority is used to determine the distribution order of edge applications; Batch scheduling is applied to each group edge according to the priority order of the priorities; The step of dividing the edge nodes into multiple groups based on the resource status data includes: Based on the network quality data and hardware architecture data in the resource status data, and by using packet probing to determine adjacent edge nodes, adjacent edge nodes whose network quality meets the preset network quality conditions and whose hardware architecture is the same are grouped into the same point-to-point network group. Other edge nodes located outside the point-to-point network group are designated as first candidate edge nodes; the resource status data of each first candidate edge node are weighted using a preset weighted calculation equation to obtain a resource status score for each first candidate edge node; and each first candidate edge node is assigned to a different resource status group according to the order of the scores. Other edge nodes located outside the resource status group are designated as second candidate edge nodes. Based on the matching relationship between the cached container images in each second candidate edge node and the set of container images required by the edge application to be scheduled, each second candidate edge node is divided into different cache groups to obtain image cache groups. Other edge nodes located outside the image cache group are designated as third candidate edge nodes; the third candidate edge nodes are temporarily grouped according to the number of historical pull failures to obtain historical pull failure count groups; The process of assigning corresponding priorities to each group based on the resource status data and group type includes: Assign the highest preset priority to the image cache group; Assign a preset high priority to the point-to-point network packets; Based on the resource status score, the resource status is grouped and assigned a preset medium priority or a preset low priority. Assign a preset minimum priority to edge nodes with high historical pull failure rates in the historical pull failure count group.

2. The edge application scheduling method according to claim 1, characterized in that, The acquisition of resource status data from multiple edge nodes includes: The computational load, network bandwidth, image cache list, and historical fetch failure records of each edge node are collected using either periodic or event-triggered collection methods to obtain resource status data for each edge node.

3. The edge application scheduling method according to claim 1, characterized in that, Also includes: Real-time monitoring of mirror repository performance metrics; When the performance metric exceeds a preset threshold, the distribution traffic for each group scheduling edge application is adjusted.

4. The edge application scheduling method according to claim 3, characterized in that, Also includes: Based on historical pull data, target edge applications are predicted and pre-distributed to edge nodes during the low load period of the mirror repository; The target edge application is a popular edge application whose historical fetch count exceeds the historical average fetch count of all edge applications.

5. The edge application scheduling method according to claim 3, characterized in that, When the performance metric exceeds a preset threshold, the distribution traffic for each group scheduling edge application is adjusted, including: If the network I / O utilization or disk I / O utilization in the performance metrics exceeds the first utilization threshold, then reduce the egress bandwidth for distributing edge applications from the mirror repository. If the CPU utilization or memory utilization in the performance metrics exceeds the second utilization threshold, then the distribution of low-priority tasks to groups that have not yet started fetching will be stopped.

6. The edge application scheduling method according to claim 1, characterized in that, After grouping adjacent edge nodes with the same hardware architecture that meet the preset network quality conditions into the same point-to-point network group, the process further includes: If the number of edge nodes in the peer-to-peer network group is lower than a preset threshold, or if the bandwidth between edge nodes in the peer-to-peer network group is lower than the bandwidth between the corresponding edge node and the mirror repository, then the peer-to-peer network group construction is deemed to have failed.

7. The edge application scheduling method according to claim 1, characterized in that, The preset weighted calculation equation is: ； in, Indicates the resource status score. Indicates the central processing unit (CPU) utilization. The first weighted parameter representing the utilization rate of the central processing unit. Indicates memory usage. The second weighted parameter representing memory utilization. Indicates disk I / O utilization. The third weighted parameter representing disk I / O utilization. Indicates network input / output utilization. The fourth weighted parameter represents the network input / output utilization. Indicates network bandwidth. The fifth weighted parameter representing network bandwidth. Indicates the historical fetch failure rate. The sixth weighted parameter represents the historical fetch failure rate.

8. The edge application scheduling method according to claim 7, characterized in that, Each weight parameter is adjusted based on the changes in historical fetch time and historical fetch count.

9. The edge application scheduling method according to claim 1, characterized in that, Based on the matching relationship between the cached container images in each of the second candidate edge nodes and the set of container images required by the edge application to be scheduled, each of the second candidate edge nodes is divided into different cache groups to obtain image cache groups, including: Determine the set of container images that the edge application to be scheduled depends on; The container image set is compared with the image cache list in the resource status data of each second candidate edge node; If the comparison result shows that the container image set is completely consistent with the image cache list of the current second candidate edge node, then the current second candidate edge node will be assigned to the full image cache group. If the comparison result shows that the container image set is not completely consistent with the image cache list of the current second candidate edge node, then the current second candidate edge node is assigned to a partial image cache group.

10. The edge application scheduling method according to claim 1, characterized in that, After dividing the edge nodes into multiple groups based on the resource status data, the method further includes: Real-time monitoring of resource status data of each edge node; When the resource status data changes of the edge nodes are detected to meet the preset regrouping conditions, the edge nodes whose resource status data have changed are regrouped.

11. The edge application scheduling method according to claim 1, characterized in that, The step of dividing the edge nodes into multiple groups based on the resource status data includes: Based on the historical load data of the mirror repository and the group distribution history, adjust the upper limit threshold of the number of edge nodes contained in each group.

12. An edge application scheduling device, characterized in that, include: The data acquisition module is used to acquire resource status data from multiple edge nodes; A grouping module is used to divide the edge nodes into multiple groups based on the resource status data; wherein, at least one group in each group is configured as a peer-to-peer network group for mirror distribution; The priority allocation module is used to assign a corresponding priority to each group based on the resource status data and group type; wherein the priority is used to determine the distribution order of edge applications; The application scheduling module is used to batch schedule edge applications to each group according to the priority order of the priorities; The grouping module is specifically used to determine adjacent edge nodes based on network quality data and hardware architecture data in the resource status data and through packet probing; and to divide adjacent edge nodes whose network quality meets the preset network quality conditions and whose hardware architecture is the same into the same point-to-point network group. Other edge nodes located outside the point-to-point network group are designated as first candidate edge nodes; the resource status data of each first candidate edge node are weighted using a preset weighted calculation equation to obtain a resource status score for each first candidate edge node; and each first candidate edge node is assigned to a different resource status group according to the order of the scores. Other edge nodes located outside the resource status group are designated as second candidate edge nodes. Based on the matching relationship between the cached container images in each second candidate edge node and the set of container images required by the edge application to be scheduled, each second candidate edge node is divided into different cache groups to obtain image cache groups. Other edge nodes located outside the image cache group are designated as third candidate edge nodes; the third candidate edge nodes are temporarily grouped according to the number of historical pull failures to obtain historical pull failure count groups; The priority allocation module is specifically used to allocate a preset highest priority to the image cache group; allocate a preset high priority to the point-to-point network group; allocate a preset medium priority or a preset low priority to the resource status group according to the resource status score; and allocate a preset lowest priority to the edge nodes with a high historical pull failure rate in the historical pull failure count group.

13. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the steps of the edge application scheduling method as described in any one of claims 1 to 11.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the edge application scheduling method as described in any one of claims 1 to 11.

15. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the edge application scheduling method according to any one of claims 1 to 11.

Images