High-concurrency data security capability micro-service scheduling method, device and system

Abstract

The invention provides a high-concurrency data security capability micro-service scheduling method, device and system, and is applied to the technical field of power data security. The method comprises the steps of obtaining a micro-service calling graph of a target service based on a mutual scheduling relationship of each micro-service in an operation process of the target service corresponding to a data security capability service request; performing hierarchical division on the micro-service calling graph based on a dependency relationship among micro-services in the micro-service calling graph to obtain a plurality of layers of calling task sets; performing micro-service scheduling of the calling task set of each layer in sequence according to a hierarchical sequence to complete micro-service scheduling of the target service; wherein the calling task sets of different layers have a dependency relationship, and micro-service scheduling of each sub-task in the calling task set of the same layer is executed in parallel. According to the invention, the problem of low system processing efficiency of a load balancing strategy of an existing micro-service architecture in a high-concurrency scene is solved.

Description

Technical Field

[0001] The present invention relates to the field of power data security technology, and in particular to a high-concurrency data security capability microservice scheduling method, device and system. Background Art

[0002] At present, in the power system, data security capabilities such as sensitive data identification, data desensitization, and watermark traceability are widely dispersed in various systems or services, showing the characteristics of fragmentation and difficulty in coordination, and cannot meet the dynamic elastic protection needs of cross-domain flow scenarios. It is urgent to carry out research on data sharing and interactive security protection technology based on microservice architecture to achieve scenario-based protection, standardized integration, and data security capabilities of flexible architecture. In the microservice architecture, each microservice can be distributed on different servers, which form a cluster. However, with the increase in the number of microservices, how to effectively distribute the request load has become a key issue. When a client task needs to respond, if the selected microservice instance happens to be deployed on a server with a large load, the response time of the task will be longer, and some servers will be under too much pressure, and the processing efficiency of the entire cluster system will be low.

[0003] Currently, there are two commonly used open source microservice architectures: Dubbo and Spring Cloud, which only provide simple load balancing strategies and cannot cope with load balancing under high concurrency, resulting in low system processing efficiency in current high concurrency scenarios. Summary of the invention

[0004] In order to overcome the problem of low system processing efficiency of the load balancing strategy of the above-mentioned microservice architecture in high-concurrency scenarios, the present invention provides a high-concurrency data security capability microservice scheduling method, device and system.

[0005] On the one hand, the present invention provides a high-concurrency data security capability microservice scheduling method, comprising:

[0006] Based on the mutual scheduling relationship of each microservice in the running process of the target service corresponding to the data security capability service request, a microservice call graph of the target service is obtained;

[0007] Based on the dependency relationship between the microservices in the microservice call graph, the microservice call graph is divided into layers to obtain a plurality of layers of call task sets;

[0008] Perform microservice scheduling of the call task set at each layer in order of levels to complete the microservice scheduling of the target service;

[0009] Among them, there is a dependency relationship between the call task sets of different layers, and the microservice scheduling of each subtask in the call task set of the same layer is executed in parallel.

[0010] Optionally, the hierarchical division of the microservice call graph based on the dependency relationship between the microservices in the microservice call graph to obtain a plurality of layers of call task sets includes:

[0011] Traversing the microservice call graph from top to bottom, dividing the microservice call graph into layers according to the in-degree of each node in the microservice call graph, and obtaining a plurality of layers of call task sets;

[0012] The microservice scheduling of the calling task set of each layer is performed in order of levels, including:

[0013] Based on the dependency relationship between different levels, determine the priority order of the calling task set of each level;

[0014] The microservice scheduling of the calling task set at each layer is performed in sequence according to the priority order.

[0015] Optionally, the microservice scheduling of the calling task set at each layer includes:

[0016] For each layer of the calling task set, a task optimization model is constructed with the goal of minimizing response time and maximizing load balancing;

[0017] The task optimization model is solved based on the Kepler optimization algorithm to obtain an optimized scheduling strategy for the calling task set at each layer.

[0018] Optionally, the step of constructing a task optimization model with the goal of minimizing response time and maximizing load balancing includes:

[0019] For each layer of the calling task set, based on the resource utilization model of each hardware resource in the server cluster where each microservice in the calling task set is located, determine the load balancing model of each hardware resource in the server cluster corresponding to the calling task set;

[0020] Performing weighted fusion of the load balancing models of the hardware resources based on the entropy method to obtain the load balancing model corresponding to the calling task set;

[0021] Based on the running time of each microservice in the calling task set, construct a response time model corresponding to the calling task set;

[0022] Determining the objective function corresponding to the calling task set based on the load balancing model and the response time model corresponding to the calling task set;

[0023] Taking the total amount of each hardware resource called by the microservice not exceeding the total amount of each hardware resource of the server where the microservice is located as a resource constraint, combined with the microservice calling rule and the objective function, a task optimization model of the calling task set is constructed.

[0024] Optionally, before determining the load balancing model of each hardware resource in the server cluster corresponding to the calling task set based on the resource utilization model of each hardware resource in the server cluster where each microservice in the calling task set is located, the method further includes:

[0025] Topologically sort the nodes in the subgraph corresponding to the calling task set in the microservice calling graph according to the in-degree to obtain the microservice queue corresponding to the calling task set;

[0026] Based on the process of dispatching each microservice in the microservice queue to the server cluster, a resource utilization model of each hardware resource of each server in the server cluster is performed to obtain a resource utilization model of each hardware resource of each server;

[0027] Optionally, the resource utilization model of each of the hardware resources is expressed as:

[0028]

[0029] Among them, m * i,j is the resource utilization of the jth hardware resource of the ith server, x s,i Indicates whether an instance of the sth microservice is deployed on the i-th server. If so, x s,i The value is 1, otherwise x s,i The value is 0; n is the number of microservices on the i-th server, L s is the number of requests to the sth microservice in the calling program of the calling task set, m s,j is the resource amount of the jth hardware resource on the sth microservice required during the execution of the calling task set, B s is the number of containers of the sth microservice that needs to be deployed in the server cluster, M i,j is the total amount of the jth hardware resource of the i-th server, j is the serial number of the hardware resource, i is the serial number of the server, and s is the serial number of the microservice

[0030] Optionally, determining the load balancing model of each hardware resource in the server cluster corresponding to the calling task set based on the resource utilization model of each hardware resource in the server cluster where each microservice in the calling task set is located includes:

[0031] For each hardware resource, based on the resource utilization model of each hardware resource in the server cluster where each microservice in the calling task set is located, determine the average utilization of the hardware resource in the server cluster;

[0032] Based on the average utilization of the hardware resources in the server cluster, the standard deviation of the resource utilization of the hardware resources in the server cluster is determined to obtain a load balancing model of the hardware resources.

[0033] Optionally, the expression of the load balancing model of each hardware resource in the server cluster is:

[0034]

[0035]

[0036] Among them, H j is the load balancing model of the jth hardware resource, and p is the number of servers in the server cluster; is the average utilization of the jth hardware resource, m * i,j is the resource utilization of the jth hardware resource of the ith server, M i,j is the total amount of resources of the jth hardware resource of the ith server, j is the serial number of the hardware resource, and i is the serial number of the server.

[0037] Optionally, the weighted fusion of the load balancing models of the hardware resources based on the entropy method includes:

[0038] Determining a resource indicator corresponding to each hardware resource of each server in the server cluster based on the resource amount of each hardware resource of each server in the server cluster;

[0039] For each resource indicator, determining the proportion of the resource indicator of each server in the server cluster;

[0040] Determine the entropy value of each resource indicator based on the resource indicator ratio of each server;

[0041] Based on the entropy value of each resource indicator, determining the weight of the load indicator corresponding to the resource indicator;

[0042] The load balancing models of the hardware resources are weightedly integrated based on the weight of each load indicator.

[0043] Optionally, the expression of the weight of each load indicator is:

[0044]

[0045] Among them, α j is the load indicator weight of the jth hardware resource, d j is the information utility of the jth resource indicator corresponding to the load indicator, l is the number of hardware resource types in the server;

[0046] The expression of the entropy value of each resource indicator is:

[0047]

[0048] Among them, d j =1-e j , e j is the entropy value of the jth resource indicator, y ij is the proportion of the jth resource indicator of the ith server in the corresponding resource indicator of the server cluster, and p is the number of servers in the server cluster.

[0049] Optionally, the objective function of the task optimization model is expressed as:

[0050] minα j H j

[0051] min T

[0052]

[0053] Among them, H j is the load balancing model of the jth hardware resource, T is the response time model of the calling task set, is the start time of execution of the sth microservice on the ith server, r i s is the running time of the sth microservice on the ith server, n is the number of microservices on the ith server; i is the serial number of the server, and s is the serial number of the microservice;

[0054] The resource constraint expression is:

[0055]

[0056] Among them, x s,i Indicates whether an instance of the sth microservice is deployed on the i-th server. If so, x s,i The value is 1, otherwise x s,i The value is 0; L s is the number of requests to the sth microservice in the calling program of the calling task set, m s,j is the resource amount of the jth hardware resource on the sth microservice required during the execution of the calling task set, B sis the number of containers of the sth microservice that needs to be deployed in the server cluster, M i,j is the total amount of the jth hardware resource of the i-th server, where j is the serial number of the hardware resource;

[0057] The constraints corresponding to the microservice call rules are:

[0058]

[0059]

[0060] Optionally, solving the task optimization model based on the Kepler optimization algorithm to obtain an optimization scheduling strategy for the calling task set at each layer includes:

[0061] The chaos operator is used to initialize the population position, and the number of tasks in each of the calling task sets is used as the dimension of the search space. Each planet corresponds to a scheduling strategy, and the position of the planet is the serial number of the server processing the current task.

[0062] In the running stage, each of the planets moves in the search space by simulating the law of universal gravitation, and the position change of the planet during the movement is the update of the candidate solution of the planet;

[0063] Based on the fitness value corresponding to the candidate solution of each of the planets, the best planet is selected as the sun until the iteration termination condition is reached, and the obtained sun is used as the optimization scheduling strategy.

[0064] On the other hand, the present invention also provides a high-concurrency data security capability microservice scheduling device, the device comprising:

[0065] A call graph acquisition module, used to obtain a microservice call graph of the target service corresponding to the data security capability service request based on the mutual scheduling relationship of each microservice during the operation of the target service;

[0066] A hierarchical division module is used to perform hierarchical division of the microservice call graph based on the dependency relationship between the microservices in the microservice call graph to obtain a plurality of layers of call task sets;

[0067] A scheduling module is used to perform microservice scheduling of the call task set at each layer in order of levels to complete the microservice scheduling of the target service;

[0068] Among them, there is a dependency relationship between the call task sets of different layers, and the microservice scheduling of each subtask in the call task set of the same layer is executed in parallel.

[0069] On the other hand, the present invention also provides a high-concurrency data security capability microservice scheduling system, including the high-concurrency data security capability microservice scheduling device described in any of the above items.

[0070] On the other hand, the present invention also provides an electronic device, comprising: at least one processor and a memory; the memory and the processor are connected via a bus;

[0071] The memory is used to store one or more programs;

[0072] When the one or more programs are executed by the at least one processor, any one of the above-mentioned high-concurrency data security capability microservice scheduling methods is implemented.

[0073] On the other hand, the present invention also provides a readable storage medium having an execution program stored thereon, and when the execution program is executed, the high-concurrency data security capability microservice scheduling method described in any one of the above is implemented.

[0074] Compared with the prior art, the present invention has the following beneficial effects:

[0075] The present invention provides a high-concurrency data security capability microservice scheduling method, device and system. Based on the mutual scheduling relationship of each microservice in the running process of the target service corresponding to the data security capability service request, a microservice call graph of the target service is obtained; based on the dependency relationship between each microservice in the microservice call graph, the microservice call graph is hierarchically divided to obtain several layers of call task sets; microservice scheduling of each layer of the call task set is performed in sequence according to the hierarchical order. For the microservice scheduling process in a high-concurrency scenario, the dependency relationship between microservices is considered, and inter-layer microservice scheduling is performed based on the dependency relationship, thereby avoiding unnecessary waiting and resource waste and ensuring the load balancing effect; by executing the microservice scheduling of each subtask in the call task set of the same layer in parallel, the parallelism of the microservice is maximized, and the processing efficiency and system throughput of the microservice are improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] Figure 1 A flowchart of a high-concurrency data security capability microservice scheduling method of the present invention;

[0077] Figure 2 A comparison chart of resource balance between the method of the present invention and the greedy algorithm, the prediction algorithm, and the traditional ant colony algorithm under different concurrency numbers;

[0078] Figure 3 This is a comparison chart of the average response time of tasks under different concurrency numbers between the method of the present invention and the greedy algorithm, the prediction algorithm, and the traditional ant colony algorithm;

[0079] Figure 4Comparison chart of system throughput between the method of the present invention and the greedy algorithm, prediction algorithm, and traditional ant colony algorithm at different concurrency numbers;

[0080] Figure 5 A system architecture diagram of a high-concurrency data security capability microservice scheduling system of the present invention;

[0081] Figure 6 It is a schematic structural diagram of the electronic device of the present invention. DETAILED DESCRIPTION

[0082] The specific implementation modes of the present invention are further described in detail below with reference to the accompanying drawings.

[0083] Embodiment 1:

[0084] The present invention provides a high-concurrency data security capability microservice scheduling method, as shown in the schematic diagram Figure 1 As shown, including:

[0085] Step S110, obtaining a microservice call graph of the target service based on the mutual scheduling relationship of each microservice during the operation of the target service corresponding to the data security capability service request;

[0086] Step S120, dividing the microservice call graph into layers based on the dependency relationship between the microservices in the microservice call graph to obtain a plurality of layers of call task sets;

[0087] Step S130, performing microservice scheduling of the calling task set at each layer in order of levels to complete the microservice scheduling of the target service;

[0088] Among them, there is a dependency relationship between the call task sets of different layers, and the microservice scheduling of each subtask in the call task set of the same layer is executed in parallel.

[0089] In this example implementation, the data security capability service request may be a service request related to data security capability sent by the client, or may be a service request related to data security capability sent by the client and forwarded by the service gateway. The target service may be various business services related to data service capability. The execution subject of this method may be the load balancing scheduling side in the microservice architecture. Microservice refers to splitting a large distributed system into several independent microservices according to different businesses and functions, each microservice focusing on providing specific business processing functions, which is equivalent to an independent small service system. Since each microservice has a single function, flow restriction technology and load balancing algorithms can be applied more flexibly, so that the system can better cope with high concurrency scenarios. When the target service is running, the mutual calls of microservices will form a directed acyclic graph, called a microservice call graph, and the microservices in the target service will be hierarchically divided according to the dependency relationship between the microservices to form several call task sets or microservice sets, and the dependency relationship is the call relationship between the microservices; and then the microservice scheduling process of each layer of the call task set is carried out in sequence according to the hierarchical order. This example can better understand and utilize the dependencies between microservices through hierarchical division operations. The scheduling order between different layers is determined based on the dependencies to avoid unnecessary waiting and waste of resources. At the same time, it ensures that the microservices in the task set of the same layer have no dependency constraints and can be executed in parallel, thereby maximizing the parallelism of microservices and improving the processing efficiency of microservices.

[0090] In an example implementation, the hierarchical division of the microservice call graph based on the dependency relationship between the microservices in the microservice call graph in step S120 to obtain a plurality of layers of call task sets includes the following process:

[0091] Traversing the microservice call graph from top to bottom, dividing the microservice call graph into layers according to the in-degree of each node in the microservice call graph, and obtaining a plurality of layers of call task sets;

[0092] The microservice scheduling of the calling task set of each layer in step S130 is performed in order of levels, including the following process:

[0093] Based on the dependency relationship between different levels, determine the priority order of the calling task set of each level;

[0094] The microservice scheduling of the calling task set at each layer is performed in sequence according to the priority order.

[0095] In this example implementation, the nodes of the microservice call graph are the microservices, and task stratification can be performed first and then task scheduling can be performed at each layer. Task stratification traverses the microservice call graph in a top-down manner, and divides the microservice call graph into layers according to the in-degree. Through the stratification operation, the entire microservice call graph will be divided into many call task sets. There is no precedence constraint between tasks in the same call task set and they can be executed in parallel. After performing the stratification operation, the call task sets are prioritized according to the hierarchical division, and then the optimal scheduling scheme is found for each call task set in order of priority. A call task set is used as a basic unit of scheduling. Only after all tasks in the call task set of the previous layer are executed and the data transmission is completed, can the tasks of the next layer begin to be scheduled. For each layer of tasks, its load balancing degree is similar to the definition of task completion time.

[0096] In an example implementation, the microservice scheduling of the call task set at each layer includes the following process:

[0097] For each layer of the calling task set, a task optimization model is constructed with the goal of minimizing response time and maximizing load balancing;

[0098] The task optimization model is solved based on the Kepler optimization algorithm to obtain an optimized scheduling strategy for the calling task set at each layer.

[0099] In this example implementation, the call task sets of different levels are taken as the basic units, and the Kepler Optimization Algorithm (KOA) is used to find a suitable scheduling strategy for the call task sets of each layer in order of priority, so that the response time of the call task sets of each layer is minimized and the load balance is maximized. Data security capability microservice scheduling is a time-sensitive issue that requires fast and effective allocation decisions. To solve this multi-objective optimization problem, this example uses a model solution process based on the Kepler optimization algorithm. The KOA algorithm can more effectively explore and utilize the search space, is not easy to fall into the local optimal solution, and can ensure the load balancing of the scheduling process.

[0100] Exemplarily, the task optimization model is constructed with the goal of minimizing response time and maximizing load balancing, including the following process:

[0101] For each layer of the calling task set, based on the resource utilization model of each hardware resource in the server cluster where each microservice in the calling task set is located, determine the balance model of each hardware resource in the server cluster corresponding to the calling task set;

[0102] Performing weighted fusion of the balance models of the hardware resources based on the entropy method to obtain a load balance model corresponding to the calling task set;

[0103] Based on the running time of each microservice in the calling task set, construct a response time model corresponding to the calling task set;

[0104] Determine the objective function corresponding to the calling task set based on the load balancing model and the response time model corresponding to the calling task set;

[0105] Taking the total amount of hardware resources called by the microservice as less than the total amount of hardware resources of the server where the microservice is located as a resource constraint, combined with the microservice calling rule and the objective function, a task optimization model of the calling task set is constructed.

[0106] In this example implementation, the hardware resources of the server or server cluster may include central processing unit (CPU) resources, memory resources, and hard disk resources (such as hard disk IO), etc. The standard deviation of the resource utilization of each hardware resource can be used as the load balance of each hardware resource. For each layer of calling task set, a resource utilization model of each hardware resource is first constructed, and then a load balance model is constructed based on the resource utilization model. Finally, the load balance model of each hardware resource is weighted and fused to obtain the load balance model corresponding to the calling task set.

[0107] Exemplarily, the process of constructing the resource utilization model of each hardware resource in the server cluster where each microservice in the calling task set is located is as follows:

[0108] Topologically sort the nodes in the subgraph corresponding to the calling task set in the microservice calling graph according to the in-degree to obtain the microservice queue corresponding to the calling task set;

[0109] Based on the process of scheduling each microservice in the microservice queue to the server cluster, resource utilization modeling of each hardware resource of each server in the server cluster is performed to obtain a resource utilization model of each hardware resource of each server.

[0110] In this example implementation, the subgraph corresponding to the currently processed call task set is first determined in the microservice call graph, and then topological sorting is performed according to the in-degree of each node in the subgraph to obtain the microservice queue corresponding to the current task set. Assume that the microservice corresponding to the call task set is deployed in a server cluster consisting of p servers. The performance of these p servers may be the same or different. For each server node n i We can use the vector n i =(C i ,Mi ,O i ) indicates the upper limit of the total amount of CPU resources, total amount of memory resources, and hard disk resources in this server.

[0111] When a complete service is running, the mutual calls of microservices will form a directed acyclic graph, called the microservice call graph. The microservice queue D is also the workload queue, which is expressed as D = {d1, d2, ..., d |N|}, d1 is the first microservice, d2 is the second microservice, d |N| is the |N|th microservice, |N| is the number of microservices. Each microservice d s ∈D, the microservice relationship vector is represented by d s ={E s ,LT s} means that E s Represents the set of all subtasks that the task needs to call, LT s Represents microservices s The threshold for the number of requests.

[0112] Define the microservice d s to d z The number of requests is LK(d s ,d z ) indicates that the microservice program of the current task set has a microservice d z The number of requests L z It can be expressed as:

[0113]

[0114] Among them, d z ∈E s A microservice can have multiple microservice instances, depending on the microservice d z Request count threshold LT z , which needs to be deployed in the server cluster z The number of containers is:

[0115]

[0116] Among them, B z For the d that needs to be deployed in the server cluster z Number of containers, LT z For microservices z The threshold for the number of requests.

[0117] When hardware resources include CPU resources, memory resources, and hard disk resources, each microservice d s The performance can be represented by the triple d s =(c s ,ms ,o s ) indicates that, where c s ,m s ,o s They respectively represent the amount of CPU resources, memory resources, and hard disk resources needed to meet microservice requests.

[0118] When the microservice is running, all microservices in the microservice queue D need to be scheduled to the microservice cluster. When a service request arrives, the resource utilization of the hardware resources of a server node is modeled as follows:

[0119]

[0120] Among them, m * i,j is the resource utilization of the jth hardware resource of the ith server, x s,i Indicates whether an instance of the sth microservice is deployed on the i-th server. If so, x s,i The value is 1, otherwise x s,i The value is 0, that is N i is the i-th server; n is the number of microservices on the i-th server, L s is the number of requests to the sth microservice in the calling program of the calling task set, m s,j is the resource amount of the jth hardware resource on the sth microservice required during the execution of the calling task set, B s is the number of containers of the sth microservice that needs to be deployed in the server cluster, M i,j is the total amount of resources of the jth hardware resource of the i-th server, j is the serial number of the hardware resource, i is the serial number of the server, and s is the serial number of the microservice;

[0121] Since the resources of different server nodes in the microservice cluster are different, for each hardware resource, the average utilization of the hardware resource in the server cluster is determined based on the resource utilization model of each hardware resource in the server cluster where each microservice in the calling task set is located; the average utilization of each hardware resource in the server cluster is calculated by the following formula:

[0122]

[0123] In particular, when the hardware resources include CPU resources, memory resources, and hard disk resources, the average resource utilization of CPU resources, memory resources, and hard disk resources can be calculated separately. On this basis, based on the average utilization of the hardware resources in the server cluster, the standard deviation of the resource utilization of the hardware resources in the server cluster is determined to obtain the load balancing model of the hardware resources. Specifically, by calculating the standard deviation of the resource utilization of each hardware resource, the expression of the load balancing model of each hardware resource in the server cluster corresponding to the calling task set is obtained:

[0124]

[0125] Among them, H j is the load balancing model of the jth hardware resource, and p is the number of servers in the server cluster; is the average utilization of the jth hardware resource. So far, the load balancing model of each calling task set is obtained.

[0126] Furthermore, the weighted fusion of the load balancing models of the hardware resources is performed based on the entropy method, including the following process:

[0127] Determining a resource indicator corresponding to each hardware resource of each server in the server cluster based on the resource amount of each hardware resource of each server in the server cluster;

[0128] For each resource indicator, determining the proportion of the resource indicator of each server in the server cluster;

[0129] Determine the entropy value of each resource indicator based on the resource indicator ratio of each server;

[0130] Based on the entropy value of each resource indicator, determining the weight of the load indicator corresponding to the resource indicator;

[0131] The load balancing models of the hardware resources are weightedly integrated based on the weight of each load indicator.

[0132] In this example implementation, the entropy method is used to determine the load index weight of the load balancing model of each hardware resource, and then the load index weights are used to perform weighted fusion on each load balancing model. First, the calculation process of the load index weight of each load balancing model is as follows:

[0133] (1) Establish a standardized matrix

[0134] Construct node index matrix R = (r ij ) p×3 , r ijrepresents the resource size of the jth hardware resource of the i-th server in the microservice cluster, j = 1, 2, 3 represent memory resources, CPU resources and hard disk resources respectively. The node indicator matrix is ​​normalized to obtain the corresponding normalized matrix, which is recorded as Z = (z ij ) p×3 , that is, the resource indicator matrix. The elements of the resource indicator matrix are the resource indicators corresponding to the hardware resources of each server in the server cluster. For example, z ij Represents the jth resource indicator of the i-th server in the microservice cluster. Each element in the standardized matrix can be expressed as:

[0135]

[0136] In the formula, min(r ij ) represents the minimum element in the node index matrix, max(r ij ) represents the maximum element in the node index matrix, z ij For r ij The results after standardization.

[0137] (2) Calculate the server node N under the jth resource index respectively i Ratio to the total of this indicator:

[0138]

[0139] In the formula, y ij is the proportion of the j-th resource indicator of the ith server in the corresponding resource indicators of the server cluster.

[0140] (3) Calculate the entropy value of each resource indicator:

[0141]

[0142] In the formula, e j is the entropy value of the j-th resource indicator.

[0143] (4) Calculate the information utility of the j-th resource indicator corresponding to the load indicator:

[0144] d j =1-e j (9)

[0145] Where, d j is the information utility of the j-th resource indicator corresponding to the load indicator.

[0146] (5) Calculate the weight of the jth load index:

[0147]

[0148] Among them, αj is the load indicator weight of the jth hardware resource, d j is the information utility of the jth resource index corresponding to the load index, and l is the number of hardware resource types in the server. So far, the load index weights of each load balancing model are obtained.

[0149] Furthermore, the load balancing model of each of the hardware resources is weighted and integrated to obtain the load balancing model corresponding to the calling task set. The specific formula is as follows:

[0150]

[0151] Where U is the load balancing model corresponding to the calling task set.

[0152] For a microservice s , its execution completion time is defined as the sum of the start execution time and the running time, that is:

[0153]

[0154] In the formula, is the start time of execution of the sth microservice on the ith server, r i s is the running time of the sth microservice on the ith server, is the execution end time of the sth microservice on the ith server.

[0155] For a set of calling tasks, its global response time can be expressed as:

[0156]

[0157] Wherein, T is the response time model of the calling task set.

[0158] Then, based on the load balancing model and the response time model corresponding to the calling task set, the objective function corresponding to the calling task set is determined, and its specific expression is:

[0159] min U (14)

[0160] min T (15)

[0161] Formulas (14) and (15) are the objective functions corresponding to the calling task set.

[0162] The resource constraint expression is:

[0163]

[0164] Among them, x s,iIndicates whether an instance of the sth microservice is deployed on the i-th server. If so, x s,i The value is 1, otherwise x s,i The value is 0; L s is the number of requests to the sth microservice in the calling program of the calling task set, m s,j is the resource amount of the jth hardware resource on the sth microservice required during the execution of the calling task set, B s is the number of containers of the sth microservice that needs to be deployed in the server cluster, M i,j is the total amount of the jth hardware resource of the i-th server, where j is the serial number of the hardware resource;

[0165] The constraints corresponding to the microservice call rules are:

[0166]

[0167]

[0168] Formula (16) indicates that the total amount of hardware resources called by the microservice does not exceed the total amount of resources of the server where the microservice is located. For example, when j=1, formula (16) indicates that the total amount of memory resources called by the microservice cannot exceed the total amount of memory resources of the server where the microservice is located; when j=2, formula (16) indicates that the total amount of CPU resources called by the microservice cannot exceed the total amount of CPU resources of the server where the microservice is located; when j=3, formula (16) indicates that the total amount of IO resources called by the microservice cannot exceed the total amount of hard disk or IO resources of the server where the microservice is located; formula (17) indicates that the number of containers of any server node is equal to the number of microservices called to this service node, and formula (18) indicates that the same microservice instance can only be called to one server node.

[0169] The above equations (14) to (18) are a task optimization model for a calling task set.

[0170] The above implementation method proposes an operation model for data capability security microservices, takes into account the dependencies between microservices, improves scheduling efficiency and reduces resource competition by dividing the microservice call graph into many call task sets and then scheduling them; models the running time of microservices and the CPU, memory, hard disk and other operating indicators consumed by microservices, and uses the entropy weight method to determine the weights of the corresponding indicators, converting the goal into a single-objective optimization problem, accelerating the model solving process and improving processing efficiency.

[0171] In other examples, the KOA algorithm is used in the task allocation stage to allocate the best instance for each task in the microservice set for task processing. The task optimization model is solved based on the Kepler optimization algorithm to obtain the optimized scheduling strategy for the calling task set at each layer, which specifically includes the following process:

[0172] The chaos operator is used to initialize the population position, and the number of tasks in each of the calling task sets is used as the dimension of the search space. Each planet corresponds to a scheduling strategy, and the position of the planet is the serial number of the server processing the current task.

[0173] In the running stage, each of the planets moves in the search space by simulating the law of universal gravitation, and the position change of the planet during the movement is the update of the candidate solution of the planet;

[0174] Based on the fitness value corresponding to the candidate solution of each of the planets, the best planet is selected as the sun until the iteration termination condition is reached, and the obtained sun is used as the optimization scheduling strategy.

[0175] In this example implementation, since the Kepler optimization algorithm may have a slow convergence speed when solving non-convex optimization problems and high-dimensional problems, in order to speed up the convergence speed of the Kepler algorithm and make it more suitable for the high-concurrency scenario of data security capability microservice scheduling, this example uses a chaotic operator to initialize the population to improve the convergence speed. Specifically, the number of subtasks in each call task set is used as the dimension of the search space, each planet corresponds to a scheduling strategy, the position of the planet represents the node number that processes this task, and the maximum value of the node number n represents the number of candidate nodes. At the beginning, the chaotic operator is used to initialize the population, that is, the planet generates a position as a feasible solution, and randomly selects a planet as the sun, which is the initial optimal solution. In the planetary operation stage, the planet moves in the search space by simulating the law of universal gravitation to find a better solution. Each planet moves on a fixed orbit according to Kepler's law. The change in the position of the planet during the movement is the update of the candidate solution. The orbit and movement of the planet are completely determined by universal gravitation. The gravitational force between a planet (candidate solution) and the sun (optimal solution) is determined based on the Euclidean distance between the planet and the sun, the maximum and minimum fitness values ​​of all current candidate solutions, and the fitness value of the planet. Specifically, the following process may be involved:

[0176] (1) Population initialization

[0177] In the initialization phase, if a set of feasible solutions is randomly assigned to the set of pending call tasks, it may result in low diversity of the population and slow convergence speed for optimizing the problem. In order to improve the global search capability of the planet at the beginning of the algorithm, the positions of the population need to be evenly distributed in the solution space of the entire problem, so the chaos operator is used to initialize the population. Specifically, the initial position of the population is obtained by the Tent chaos map, and the specific expression is as follows:

[0178]

[0179] X k,u =[x k,u ×p+rand [0,1] ] (20)

[0180] In formula (19), x k,u represents the u-th discrete position / coordinate of planet k, x k+1,u represents the uth discrete position / coordinate of planet k+1, p is the number of servers in the server cluster, that is, the number of nodes that can be allocated, and rand [0,1] represents a random number of 0 or 1. Since the node coordinates represent the serial number of the node assigned to this task, the node coordinates should be integer values. The value in equation (19) is converted into an integer value through the rounding function of equation (20). The complete coordinates of planet k are X k =(X k,1 ,X k,2 ,...,X k,|N| ), |N| is the number of microservices in the calling task set.

[0181] (2) Update the position of celestial bodies

[0182] The development phase explores solutions close to the sun to improve the accuracy of the solution, and the exploration phase explores positions far from the sun to find new solutions to avoid falling into local optimality. The positions far from the sun in the exploration phase are updated by the following formula:

[0183]

[0184] Among them, X k (t+1) is the position / coordinate of planet k in iteration t+1, X k (t) is the position / coordinate of planet k in t rounds of iterations, Γ is a random number used to indicate the direction of revolution; V k (t) is the speed required for planet k to reach its new position, determined by gravity, X s (t) is the best position of the sun found so far, F gk(t) is the gravitational force of planet k and the sun, |r| is a random value generated according to the normal distribution, and U1 is a random value in the exploration phase, which takes values ​​of 0 and 1.

[0185] During the development phase, the distance between the sun and the planet is updated using the following formula:

[0186]

[0187] Among them, U2 is a random value in the development stage, which can be 0 or 1; X a (t) and X b (t) represent two solutions randomly selected from the entire solution space, and h is the adaptive factor of the distance between the sun and the current planet k at the control moment of the t-th iteration. The exploration phase and the development phase are performed randomly.

[0188] (3) Fitness function

[0189] After the celestial body position is updated, the best planet is selected according to the fitness function and used as the new optimal solution for a new round of iteration. The purpose of the fitness function is to evaluate the pros and cons of individuals. The optimization goal is to maximize the load balance and minimize the global response time. The fitness function used is as follows:

[0190]

[0191] Among them, F(X) represents the fitness value of the scheduling strategy corresponding to the planet at coordinate X, makespan(X) represents the global response time of the scheduling strategy corresponding to the planet at coordinate X, U(X) represents the load balance of the scheduling strategy corresponding to the planet at coordinate X, T max , T min Respectively represent the maximum and minimum global response time of all planets in the current search space. Similarly, U max , U min They represent the maximum and minimum values ​​of the load balance of all planets in the current search space respectively. α and β are the evaluation coefficients of response time and load balance respectively. For example, to ensure the fairness of the two optimization objectives, α=β=0.5 can be set.

[0192] The above implementation method improves the initialization process of the Kepler algorithm by introducing a chaotic operator, so that it converges faster and adapts to the data security capability microservice scheduling, thereby shortening the microservice operation response time, improving the cluster load balancing, and improving the data processing capability of the data security capability microservice architecture in high concurrency scenarios.

[0193] Since microservices have a single function, they can more flexibly apply traffic limiting technology and load balancing algorithms, so that the system can better cope with high-concurrency scenarios. Therefore, most of them are based on microservices for business processing. At present, in order to solve the problem of uneven load distribution of server clusters, it is necessary to rely on effective load balancing algorithms to reasonably allocate tasks to servers in the cluster, reduce the pressure on the back-end servers, and improve the processing efficiency of the entire cluster system. However, the current load balancing strategies are relatively simple and cannot cope with load balancing under high concurrency. For example, the Chinese patent application number CN202311409413.0 discloses a microservice scheduling method based on a queuing network in a cloud-edge environment, where a terminal device obtains a scheduling task to form a task queue, wherein the scheduling task parameters used to characterize the scheduling task include the task arrival rate, the link transmission capacity, and the service rate of the cloud server; a microservice scheduling model based on a queuing network is constructed according to the scheduling task parameters to obtain the average response time of the task; the average response time of the task is minimized by adjusting the ratio of the edge server and the cloud server to obtain the optimal configuration of the server. The scheme calculates the average response time of this type of work by analyzing the arrival rate of different types of work, the average number of node visits, etc., so as to obtain the optimal scheduling strategy. It can be seen that the scheme adopts the traditional global queue scheduling and only optimizes from the response time, and the optimization effect is limited. The Chinese patent with application number CN202311196586.9 discloses a microservice resource intelligent scheduling strategy based on the container cloud platform. Specifically, the microservice network topology of the container cloud platform is first obtained and the request volume, CPU, memory and request response time of each container are collected; according to the collected request volume and network topology, a training set is constructed using a sliding window, and a request volume prediction model is constructed through a graph neural network to predict the future microservice node request volume; according to the collected microservice related features, the message passing mechanism is used to transfer features to the microservice network, and a microservice response time evaluation model is constructed and trained by combining graph embedding and deep neural network algorithms; the predicted request volume is used as one of the microservice node features, and the request volume prediction model is used as the objective function. Combined with the heuristic algorithm, the optimal solution for the microservice node resources in the next time period is obtained under the QoS (Quality of Service) constraint and resources are allocated. It can be seen that this solution uses machine learning models for resource optimization, which requires a large amount of data training, and the large amount of calculations requires high hardware conditions, and it is difficult to guarantee response speed in the face of high concurrency. The dynamic load balancing algorithm related to the above method considers fewer load parameters and does not consider the impact of the dependency relationship between microservices on load balancing, resulting in the above algorithm being unable to guarantee load balancing effects and system processing efficiency in the face of high concurrency.Aiming at the problem of unbalanced server cluster load under high-concurrency data security capability microservice architecture, the present invention proposes a data security capability microservice scheduling method, which can balance server load, improve cluster system processing efficiency and reduce task response time, thereby improving system concurrency performance, compared with traditional microservice scheduling methods. Specifically, the present invention models microservices and server clusters running microservices according to the microservice operation characteristics and constructs a scheduling model, and schedules data security capability microservices to appropriate server nodes through a scheduling algorithm.

[0194] Experimental verification

[0195] By using Python 3.7 to simulate the process of scheduling data security capability microservices to server clusters, the system performance of the KOA algorithm of the present invention is compared with the greedy algorithm, the prediction algorithm, and the traditional ant colony algorithm. Figure 2-Figure 4 Specifically, a simulated server cluster was built and different scheduling algorithms were set in the cluster to perform stress testing on the cluster. The test plan was that the initial concurrency was 100, and each experiment increased by 50, and stopped when the concurrency reached 400. The cluster resource balance, average response time, and throughput were tested when different scheduling algorithms were used.

[0196] Figure 2 This is a comparison chart of resource balance of four algorithms under different concurrency numbers. The present invention adopts the profit equation proposed in the above formula (11) as the resource balance index, that is, the weighted average of the variance of each resource index value. The closer it is to 0, the more balanced the resources are. Figure 2 It can be seen that when the number of concurrent requests increases from 50 to 400, the resource balance degree of the algorithm proposed in the present invention is always maintained at a low level, and its performance under high concurrency conditions is far better than the other three algorithms.

[0197] Figure 3 This is a comparison chart of the average response time of tasks of four algorithms under different concurrency numbers. Figure 3 It can be seen that when the number of concurrency is low, there is no difference in response time among the four algorithms. However, as the number of concurrency gradually increases, the scheduling of the KOA algorithm makes cluster resources more balanced, task allocation more reasonable, and the waiting time before task execution is shorter. The average response time of all tasks is lower than that of the other three algorithms. Under high concurrency conditions, the average response time is 16.38%, 18.69% and 12.95% lower than that of the prediction algorithm, greedy algorithm and ant colony algorithm, respectively.

[0198] Figure 4 This is a comparison chart of the system throughput of the four algorithms under different concurrency numbers. Figure 4It can be seen that before the cluster reaches a high load state, the throughput gradually increases under the scheduling of the four algorithms. After reaching a high load state, since the other three algorithms cannot dynamically adjust the weights of each backend server for load distribution, some nodes fall into performance bottlenecks, resulting in the loss of concurrent connections, which leads to a significant reduction in throughput. In a high concurrency scenario, the throughput of the algorithm of the present invention is improved by 18.13%, 19.48% and 9.04% respectively compared with the prediction algorithm, greedy algorithm and ant colony algorithm.

[0199] The above experimental results verify that the present invention has obvious advantages over the other three algorithms in terms of system resource balance, average response time and throughput. The above simulation experiments show that the method proposed by the present invention improves the resource load balance of the data security capability microservice corresponding cluster, shortens the response time of the data security capability microservice, and improves the throughput of the data security capability microservice corresponding cluster in high concurrency scenarios.

[0200] Embodiment 2:

[0201] Based on the same inventive concept, the present invention also provides a high-concurrency data security capability microservice scheduling device, including: a call graph acquisition module, which is used to obtain a microservice call graph of the target service based on the mutual scheduling relationship of each microservice in the running process of the target service corresponding to the data security capability service request;

[0202] A hierarchical division module is used to perform hierarchical division of the microservice call graph based on the dependency relationship between the microservices in the microservice call graph to obtain a plurality of layers of call task sets;

[0203] A scheduling module is used to perform microservice scheduling of the call task set at each layer in order of levels to complete the microservice scheduling of the target service;

[0204] Among them, there is a dependency relationship between the calling task sets at different layers, and the microservice scheduling of each subtask in the calling task set at the same layer is executed in parallel.

[0205] In a possible implementation manner, the hierarchical division module is specifically used to:

[0206] Traversing the microservice call graph from top to bottom, dividing the microservice call graph into layers according to the in-degree of each node in the microservice call graph, and obtaining a plurality of layers of call task sets;

[0207] The scheduling module includes:

[0208] A priority determination submodule, used to determine the priority order of the call task set of each layer based on the dependency relationship between different layers;

[0209] The scheduling submodule is used to schedule the microservices of the calling task set at each layer in sequence according to the priority order.

[0210] In a possible implementation, the scheduling submodule includes:

[0211] A model building subunit is used to build a task optimization model for the call task set of each layer with the goal of minimizing response time and maximizing load balance;

[0212] The model solving subunit is used to solve the task optimization model based on the Kepler optimization algorithm to obtain the optimization scheduling strategy of the calling task set at each layer.

[0213] In a possible implementation manner, the model building subunit is specifically used for:

[0214] For each layer of the calling task set, based on the resource utilization model of each hardware resource in the server cluster where each microservice in the calling task set is located, determine the load balancing model of each hardware resource in the server cluster corresponding to the calling task set;

[0215] Performing weighted fusion of the load balancing models of the hardware resources based on the entropy method to obtain the load balancing model corresponding to the calling task set;

[0216] Based on the running time of each microservice in the calling task set, construct a response time model corresponding to the calling task set;

[0217] Determine the objective function corresponding to the calling task set based on the load balancing model and the response time model corresponding to the calling task set;

[0218] Taking the total amount of hardware resources called by the microservice as less than the total amount of hardware resources of the server where the microservice is located as a resource constraint, combined with the microservice calling rule and the objective function, a task optimization model of the calling task set is constructed.

[0219] In a possible implementation manner, the model building subunit is further specifically used for:

[0220] Topologically sort the nodes in the subgraph corresponding to the calling task set in the microservice calling graph according to the in-degree to obtain the microservice queue corresponding to the calling task set;

[0221] Based on the process of scheduling each microservice in the microservice queue to the server cluster, resource utilization modeling of each hardware resource of each server in the server cluster is performed to obtain a resource utilization model of each hardware resource of each server.

[0222] In a possible implementation manner, the resource utilization model of each of the hardware resources is expressed as:

[0223]

[0224] Among them, m * i,j is the resource utilization of the jth hardware resource of the ith server, x s,i Indicates whether an instance of the sth microservice is deployed on the i-th server. If so, x s,i The value is 1, otherwise x s,i The value is 0; n is the number of microservices on the i-th server, L s is the number of requests to the sth microservice in the calling program of the calling task set, m s,j is the resource amount of the jth hardware resource on the sth microservice required during the execution of the calling task set, B s is the number of containers of the sth microservice that needs to be deployed in the server cluster, M i,j is the total amount of resources of the jth hardware resource of the ith server, j is the serial number of the hardware resource, i is the serial number of the server, and s is the serial number of the microservice.

[0225] In a possible implementation manner, the model building subunit is further specifically used for:

[0226] For each hardware resource, based on the resource utilization model of each hardware resource in the server cluster where each microservice in the calling task set is located, determine the average utilization of the hardware resource in the server cluster;

[0227] Based on the average utilization of the hardware resources in the server cluster, the standard deviation of the resource utilization of the hardware resources in the server cluster is determined to obtain a load balancing model of the hardware resources.

[0228] In a possible implementation, the load balancing model of each hardware resource in the server cluster is expressed as:

[0229]

[0230]

[0231] Among them, H j is the load balancing model of the jth hardware resource, and p is the number of servers in the server cluster; is the average utilization of the jth hardware resource.

[0232] In a possible implementation manner, the model building subunit is specifically used for:

[0233] Determining a resource indicator corresponding to each hardware resource of each server in the server cluster based on the resource amount of each hardware resource of each server in the server cluster;

[0234] For each resource indicator, determining the proportion of the resource indicator of each server in the server cluster;

[0235] Determine the entropy value of each resource indicator based on the resource indicator ratio of each server;

[0236] Based on the entropy value of each resource indicator, determining the indicator weight of the hardware resource corresponding to the resource indicator;

[0237] The balance degrees of the hardware resources are weightedly integrated based on the indicator weight of each of the hardware resources.

[0238] In a possible implementation manner, the indicator weight of each of the hardware resources is expressed as:

[0239]

[0240] Among them, α j is the indicator weight of the jth hardware resource, d j is the information utility of the jth resource indicator corresponding to the load indicator, l is the number of hardware resource types in the server;

[0241] The expression of the entropy value of each resource indicator is:

[0242]

[0243] Among them, d j =1-e j , e j is the entropy value of the jth resource indicator, y ij is the proportion of the jth resource indicator of the ith server in the corresponding resource indicator of the server cluster, and p is the number of servers in the server cluster.

[0244] In a possible implementation, the objective function of the task optimization model is expressed as:

[0245] minα j H j

[0246] min T

[0247]

[0248] Among them, H jis the load balancing model of the jth hardware resource, T is the response time model of the calling task set, is the start execution time of the sth microservice on the ith server, r i s is the running time of the sth microservice on the ith server, n is the number of microservices on the ith server; i is the serial number of the server, and s is the serial number of the microservice;

[0249] The resource constraint expression is:

[0250]

[0251] Among them, x s,i Indicates whether an instance of the sth microservice is deployed on the i-th server. If so, x s,i The value is 1, otherwise x s,i The value is 0; L s is the number of requests to the sth microservice in the calling program of the calling task set, m s,j is the resource amount of the jth hardware resource on the sth microservice required during the execution of the calling task set, B s is the number of containers of the sth microservice that needs to be deployed in the server cluster, M i,j is the total amount of the jth hardware resource of the i-th server, where j is the serial number of the hardware resource;

[0252] The constraints corresponding to the microservice call rules are:

[0253]

[0254]

[0255] In a possible implementation manner, the model solving subunit is specifically used for:

[0256] The chaos operator is used to initialize the population position, and the number of tasks in each of the calling task sets is used as the dimension of the search space. Each planet corresponds to a scheduling strategy, and the position of the planet is the serial number of the server processing the current task.

[0257] In the running stage, each of the planets moves in the search space by simulating the law of universal gravitation, and the position change of the planet during the movement is the update of the candidate solution of the planet;

[0258] Based on the fitness value corresponding to the candidate solution of each of the planets, the best planet is selected as the sun until the iteration termination condition is reached, and the obtained sun is used as the optimization scheduling strategy.

[0259] Embodiment 3:

[0260] Based on the same inventive concept, the present invention also provides a high-concurrency data security capability microservice scheduling system, the system comprising: the data security capability microservice scheduling device in the aforementioned embodiment 2.

[0261] In this example implementation, load balancing scheduling can be performed by the aforementioned data security capability microservice scheduling device. Figure 5 As shown, it is an example of a high-concurrency data security capability microservice system architecture diagram. The system may include: a service gateway, a load balancing scheduling module, a registration center, a load information storage module and a business processing module. The service gateway receives the service request sent by the client, which is loaded on the terminal device. The client can be a browser, a mobile application or other client application. The client sends a request to the service gateway and waits for a response. The service gateway acts as an intermediary between the client and the server cluster of the business processing module, and is used to receive client requests and pass the requests to the load balancing scheduling module. The load balancing scheduling module first obtains the available nodes from the registration center according to the current load information and the load balancing scheduling algorithm when the request arrives, and then obtains the latest node information from the load information storage module (i.e., collects load information), and then selects the most suitable server for each microservice to process the request. The server cluster consists of many server nodes, and each server can run multiple microservice instances. The load information collection module is responsible for regularly collecting load information such as CPU usage and memory usage from the server cluster, and then sends it to the load information storage module for storage. The registration center is responsible for managing and maintaining the available microservice instances of each server node to provide service registration and service discovery functions for the load balancing scheduling module. The device of the present invention can be used as a load balancing scheduling module. The processing flow of the data security capability microservice scheduling system is as follows: the client sends a request to the service gateway, and the service gateway selects an appropriate server to process the request through the load balancing scheduling module. The load balancing scheduling module selects the server based on the current load information and the load balancing algorithm. The selected server receives the request, and the business logic is processed by the relevant microservice instance. After the processing is completed, the server returns the response result to the service gateway, and finally the service gateway transmits the result back to the client.

[0262] The present invention designs a high-concurrency data security capability microservice system architecture, collects load information through a load balancing collection module, obtains available node information through a registration center, and then schedules data security capability microservices through a load balancing scheduling module. A new load balancing algorithm is adopted in the load balancing scheduling module, and the proposed microservice scheduling model and algorithm are used to solve and allocate microservices to the optimal nodes.

[0263] Example 4

[0264] like Figure 6 As shown, the present invention also provides an electronic device, which may be a computer device, a single-chip device, an intelligent mobile device, etc. The electronic device in this embodiment may include a processor, a memory, a transceiver component, etc. The memory, the processor, and the transceiver component are connected via a bus; the memory may be used to store an execution program, and an exemplary execution program may include instructions; the processor is used to execute the instructions stored in the memory. The memory may also be used to store data, which may be called and / or modified when the instructions are executed.

[0265] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. It is the computing core and control core of the terminal, which is suitable for implementing one or more instructions, specifically suitable for loading and executing one or more instructions in a storage medium to implement the corresponding method flow or corresponding functions, so as to implement the steps of a high-concurrency data security capability microservice scheduling method in the above embodiment.

[0266] Example 5

[0267] Based on the same inventive concept, the present invention also provides a readable storage medium, specifically an electronic device readable storage medium (Memory), which is a memory device in an electronic device for storing programs and data. It can be understood that the storage medium here can include both built-in storage media in electronic devices and, of course, extended storage media supported by electronic devices. The storage medium provides a storage space, which stores the operating system of the terminal. In addition, one or more instructions suitable for being loaded and executed by a processor are also stored in the storage space, and these instructions can be one or more execution programs (including program codes). It should be noted that the storage medium here can be a high-speed RAM memory or a non-volatile memory, such as at least one disk memory. The processor loads and executes one or more instructions stored in the storage medium, which can implement the steps of a high-concurrency data security capability microservice scheduling method in the above embodiment.

[0268] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0269] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0270] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0271] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0272] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit its protection scope. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the field should understand that after reading the present invention, those skilled in the art can still make various changes, modifications or equivalent substitutions to the specific implementation methods of the application, but these changes, modifications or equivalent substitutions are all within the protection scope of the claims to be approved.

Claims

1. A high-concurrency data security capability microservice scheduling method, characterized in that: The method comprises: Based on the mutual scheduling relationship of each microservice in the running process of the target service corresponding to the data security capability service request, a microservice call graph of the target service is obtained; Based on the dependency relationship between the microservices in the microservice call graph, the microservice call graph is divided into layers to obtain a plurality of layers of call task sets; Perform microservice scheduling of the call task set at each layer in order of levels to complete the microservice scheduling of the target service; Among them, there is a dependency relationship between the call task sets of different layers, and the microservice scheduling of each subtask in the call task set of the same layer is executed in parallel.

2. The method according to claim 1, characterized in that The hierarchical division of the microservice call graph is performed based on the dependency relationship between the microservices in the microservice call graph to obtain a plurality of layers of call task sets, including: Traversing the microservice call graph from top to bottom, dividing the microservice call graph into layers according to the in-degree of each node in the microservice call graph, and obtaining a plurality of layers of call task sets; The microservice scheduling of the calling task set of each layer is performed in order of levels, including: Based on the dependency relationship between different levels, determine the priority order of the calling task set of each level; The microservice scheduling of the calling task set at each layer is performed in sequence according to the priority order.

3. The method according to claim 2, characterized in that The microservice scheduling of the call task set at each layer includes: For each layer of the calling task set, a task optimization model is constructed with the goal of minimizing response time and maximizing load balancing; The task optimization model is solved based on the Kepler optimization algorithm to obtain an optimized scheduling strategy for the calling task set at each layer.

4. The method according to claim 3, characterized in that The task optimization model is constructed with the goal of minimizing response time and maximizing load balance, including: For each layer of the calling task set, based on the resource utilization model of each hardware resource in the server cluster where each microservice in the calling task set is located, determine the load balancing model of each hardware resource in the server cluster corresponding to the calling task set; Performing weighted fusion of the load balancing models of the hardware resources based on the entropy method to obtain the load balancing model corresponding to the calling task set; Based on the running time of each microservice in the calling task set, construct a response time model corresponding to the calling task set; Determining the objective function corresponding to the calling task set based on the load balancing model corresponding to the calling task set and the response time model; Taking the total amount of each hardware resource called by the microservice not exceeding the total amount of each hardware resource of the server where the microservice is located as a resource constraint, combined with the microservice calling rule and the objective function, a task optimization model of the calling task set is constructed.

5. The method according to claim 4, characterized in that Before determining the load balancing model of each hardware resource in the server cluster corresponding to the calling task set based on the resource utilization model of each hardware resource in the server cluster where each microservice in the calling task set is located, the method further includes: Topologically sort the nodes in the subgraph corresponding to the calling task set in the microservice calling graph according to the in-degree to obtain the microservice queue corresponding to the calling task set; Based on the process of scheduling each microservice in the microservice queue to the server cluster, resource utilization modeling of each hardware resource of each server in the server cluster is performed to obtain a resource utilization model of each hardware resource of each server.

6. The method according to claim 5, characterized in that The expression of the resource utilization model of each of the hardware resources is: Among them, m * i,j is the resource utilization of the jth hardware resource of the ith server, x s,i Indicates whether an instance of the sth microservice is deployed on the i-th server. If so, x s,i The value is 1, otherwise x s,i The value is 0; n is the number of microservices on the i-th server, L s is the number of requests to the sth microservice in the calling program of the calling task set, m s,j is the resource amount of the jth hardware resource on the sth microservice required during the execution of the calling task set, B s is the number of containers of the sth microservice that needs to be deployed in the server cluster, M i,j is the total amount of resources of the jth hardware resource of the ith server, j is the serial number of the hardware resource, i is the serial number of the server, and s is the serial number of the microservice.

7. The method according to claim 4, characterized in that The step of determining the load balancing model of each hardware resource in the server cluster corresponding to the calling task set based on the resource utilization model of each hardware resource in the server cluster where each microservice in the calling task set is located comprises: For each hardware resource, based on the resource utilization model of each hardware resource in the server cluster where each microservice in the calling task set is located, determine the average utilization of the hardware resource in the server cluster; Based on the average utilization of the hardware resources in the server cluster, the standard deviation of the resource utilization of the hardware resources in the server cluster is determined to obtain a load balancing model of the hardware resources.

8. The method according to claim 7, characterized in that The expression of the load balancing model of each hardware resource in the server cluster is: Among them, H j is the load balancing model of the jth hardware resource, and p is the number of servers in the server cluster; is the average utilization of the jth hardware resource, m * i,j is the resource utilization of the jth hardware resource of the ith server, M i,j is the total resource amount of the jth hardware resource of the ith server, j is the serial number of the hardware resource, and i is the serial number of the server.

9. The method according to claim 4, characterized in that The weighted fusion of the load balancing models of the hardware resources based on the entropy method includes: Determining a resource indicator corresponding to each hardware resource of each server in the server cluster based on the resource amount of each hardware resource of each server in the server cluster; For each resource indicator, determining the proportion of the resource indicator of each server in the server cluster; Determine the entropy value of each resource indicator based on the resource indicator ratio of each server; Based on the entropy value of each resource indicator, determining the weight of the load indicator corresponding to the resource indicator; The load balancing models of the hardware resources are weightedly integrated based on the weight of each load indicator.

10. The method according to claim 9, characterized in that The expression of the weight of each load indicator is: Among them, α j is the load indicator weight of the jth hardware resource, d j is the information utility of the jth resource indicator corresponding to the load indicator, l is the number of hardware resource types in the server; The expression of the entropy value of each resource indicator is: Among them, d j =1-e j , e j is the entropy value of the jth resource indicator, y ij is the proportion of the jth resource indicator of the ith server in the corresponding resource indicator of the server cluster, and p is the number of servers in the server cluster.

11. The method according to claim 10, characterized in that The objective function of the task optimization model is expressed as: minα j H j min T Among them, H j is the load balancing model of the jth hardware resource, T is the response time model of the calling task set, is the start time of execution of the sth microservice on the ith server, r i s is the running time of the sth microservice on the ith server, n is the number of microservices on the ith server; i is the serial number of the server, and s is the serial number of the microservice; The resource constraint expression is: Among them, x s,i Indicates whether an instance of the sth microservice is deployed on the i-th server. If so, x s,i The value is 1, otherwise x s,i The value is 0; L s is the number of requests to the sth microservice in the calling program of the calling task set, m s,j is the resource amount of the jth hardware resource on the sth microservice required during the execution of the calling task set, B s is the number of containers of the sth microservice that needs to be deployed in the server cluster, M i,j is the total amount of the jth hardware resource of the i-th server, where j is the serial number of the hardware resource; The constraints corresponding to the microservice call rules are:

12. The method according to claim 3, characterized in that Solving the task optimization model based on the Kepler optimization algorithm to obtain an optimization scheduling strategy for the calling task set at each layer includes: The chaos operator is used to initialize the population position, and the number of tasks in each of the calling task sets is used as the dimension of the search space. Each planet corresponds to a scheduling strategy, and the position of the planet is the serial number of the server processing the current task. In the running stage, each of the planets moves in the search space by simulating the law of universal gravitation, and the position change of the planet during the movement is the update of the candidate solution of the planet; Based on the fitness value corresponding to the candidate solution of each of the planets, the best planet is selected as the sun until the iteration termination condition is reached, and the obtained sun is used as the optimization scheduling strategy.

13. A high-concurrency data security capability microservice scheduling device, characterized in that: The device comprises: A call graph acquisition module, used to obtain a microservice call graph of the target service corresponding to the data security capability service request based on the mutual scheduling relationship of each microservice during the operation of the target service; A hierarchical division module is used to perform hierarchical division of the microservice call graph based on the dependency relationship between the microservices in the microservice call graph to obtain a plurality of layers of call task sets; A scheduling module is used to perform microservice scheduling of the call task set at each layer in order of levels to complete the microservice scheduling of the target service; Among them, there is a dependency relationship between the call task sets of different layers, and the microservice scheduling of each subtask in the call task set of the same layer is executed in parallel.

14. The device according to claim 13, characterized in that The hierarchical division module is specifically used for: Traversing the microservice call graph from top to bottom, dividing the microservice call graph into layers according to the in-degree of each node in the microservice call graph, and obtaining a plurality of layers of call task sets; The scheduling module includes: A priority determination submodule, used to determine the priority order of the call task set of each layer based on the dependency relationship between different layers; The scheduling submodule is used to schedule the microservices of the calling task set at each layer in sequence according to the priority order.

15. The device according to claim 14, characterized in that The scheduling submodule includes: A model building subunit is used to build a task optimization model for the call task set of each layer with the goal of minimizing response time and maximizing load balance; The model solving subunit is used to solve the task optimization model based on the Kepler optimization algorithm to obtain the optimization scheduling strategy of the calling task set at each layer.

16. The device according to claim 15, characterized in that The model building subunit is specifically used for: For each layer of the calling task set, based on the resource utilization model of each hardware resource in the server cluster where each microservice in the calling task set is located, determine the load balancing model of each hardware resource in the server cluster corresponding to the calling task set; Performing weighted fusion of the load balancing models of the hardware resources based on the entropy method to obtain the load balancing model corresponding to the calling task set; Based on the running time of each microservice in the calling task set, construct a response time model corresponding to the calling task set; Determining the objective function corresponding to the calling task set based on the load balancing model corresponding to the calling task set and the response time model; Taking the total amount of hardware resources called by the microservice as less than the total amount of hardware resources of the server where the microservice is located as a resource constraint, combined with the microservice calling rule and the objective function, a task optimization model of the calling task set is constructed.

17. The device according to claim 16, characterized in that The model building subunit is also specifically used for: Topologically sort the nodes in the subgraph corresponding to the calling task set in the microservice calling graph according to the in-degree to obtain the microservice queue corresponding to the calling task set; Based on the process of scheduling each microservice in the microservice queue to the server cluster, resource utilization modeling of each hardware resource of each server in the server cluster is performed to obtain a resource utilization model of each hardware resource of each server.

18. The device according to claim 17, characterized in that The expression of the resource utilization model of each of the hardware resources is: Among them, m * i,j is the resource utilization of the jth hardware resource of the ith server, x s,i Indicates whether an instance of the sth microservice is deployed on the i-th server. If so, x s,i The value is 1, otherwise x s,i The value is 0; n is the number of microservices on the i-th server, L s is the number of requests to the sth microservice in the calling program of the calling task set, m s,j is the resource amount of the jth hardware resource on the sth microservice required during the execution of the calling task set, B s is the number of containers of the sth microservice that needs to be deployed in the server cluster, M i,j is the total amount of resources of the jth hardware resource of the ith server, j is the serial number of the hardware resource, i is the serial number of the server, and s is the serial number of the microservice.

19. The device according to claim 16, characterized in that The model building subunit is also specifically used for: For each hardware resource, based on the resource utilization model of each hardware resource in the server cluster where each microservice in the calling task set is located, determine the average utilization of the hardware resource in the server cluster; Based on the average utilization of the hardware resources in the server cluster, the standard deviation of the resource utilization of the hardware resources in the server cluster is determined to obtain a load balancing model of the hardware resources.

20. The device according to claim 19, characterized in that The expression of the load balancing model of each hardware resource in the server cluster is: Among them, H j is the load balancing model of the jth hardware resource, and p is the number of servers in the server cluster; is the average utilization of the jth hardware resource, m * i,j is the resource utilization of the jth hardware resource of the ith server, M i,j is the total resource amount of the jth hardware resource of the ith server, j is the serial number of the hardware resource, and i is the serial number of the server.

21. The device according to claim 16, characterized in that The model building subunit is specifically used for: Determining a resource indicator corresponding to each hardware resource of each server in the server cluster based on the resource amount of each hardware resource of each server in the server cluster; For each resource indicator, determining the proportion of the resource indicator of each server in the server cluster; Determine the entropy value of each resource indicator based on the resource indicator ratio of each server; Based on the entropy value of each resource indicator, determining the indicator weight of the hardware resource corresponding to the resource indicator; The balance degrees of the hardware resources are weightedly integrated based on the indicator weight of each of the hardware resources.

22. The device according to claim 21, characterized in that The expression of the indicator weight of each hardware resource is: Among them, α j is the indicator weight of the jth hardware resource, d j is the information utility of the jth resource indicator corresponding to the load indicator, l is the number of hardware resource types in the server; The expression of the entropy value of each resource indicator is: Among them, d j =1-e j , e j is the entropy value of the jth resource indicator, y ij is the proportion of the jth resource indicator of the ith server in the corresponding resource indicator of the server cluster, and p is the number of servers in the server cluster.

23. The device according to claim 22, characterized in that The objective function of the task optimization model is expressed as: minα j H j min T Among them, H j is the load balancing model of the jth hardware resource, T is the response time model of the calling task set, is the start execution time of the sth microservice on the ith server, r i s is the running time of the sth microservice on the ith server, n is the number of microservices on the ith server; i is the serial number of the server, and s is the serial number of the microservice; The resource constraint expression is: Among them, x s,i Indicates whether an instance of the sth microservice is deployed on the i-th server. If so, x s,i The value is 1, otherwise x s,i The value is 0; L s is the number of requests to the sth microservice in the calling program of the calling task set, m s,j is the resource amount of the jth hardware resource on the sth microservice required during the execution of the calling task set, B s is the number of containers of the sth microservice that needs to be deployed in the server cluster, M i,j is the total amount of the jth hardware resource of the i-th server, where j is the serial number of the hardware resource; The constraints corresponding to the microservice call rules are:

24. The device according to claim 15, characterized in that The model solving subunit is specifically used for: The chaos operator is used to initialize the population position, and the number of tasks in each of the calling task sets is used as the dimension of the search space. Each planet corresponds to a scheduling strategy, and the position of the planet is the serial number of the server processing the current task. In the running stage, each of the planets moves in the search space by simulating the law of universal gravitation, and the position change of the planet during the movement is the update of the candidate solution of the planet; Based on the fitness value corresponding to the candidate solution of each of the planets, the best planet is selected as the sun until the iteration termination condition is reached, and the obtained sun is used as the optimization scheduling strategy.

25. A high-concurrency data security capability microservice scheduling system, characterized in that: A high-concurrency data security capability microservice scheduling device comprising any one of claims 13 to 24.

26. An electronic device, characterized in that: include: at least one processor and memory; The memory and the processor are connected via a bus; The memory is used to store one or more programs; When the one or more programs are executed by the at least one processor, the high-concurrency data security capability microservice scheduling method according to any one of claims 1 to 12 is implemented.

27. A readable storage medium, characterized in that: An execution program is stored thereon, and when the execution program is executed, a high-concurrency data security capability microservice scheduling method as described in any one of claims 1 to 12 is implemented.

Images