Government affair big data platform micro-service deployment method, device and equipment and medium

Abstract

The invention discloses a government affair big data platform micro-service deployment method, device and equipment and a medium, and relates to the field of edge computing, and the method comprises the steps: obtaining the basic information of each to-be-deployed micro-service in a to-be-deployed micro-service set in a government affair big data platform; obtaining basic information of each edge server in an available edge server set in the current edge computing network architecture; obtaining basic information of each to-be-processed application in the to-be-processed application set; performing decision variable configuration based on the obtained first information, second information and third information, and constructing a target micro-service deployment model by using decision variables; and solving the target micro-service deployment model based on a genetic algorithm, the to-be-deployed micro-service set, the first information, the available edge server set, the second information, the to-be-processed application set and the third information to determine a micro-service deployment decision. According to the invention, the accuracy and reliability of deployment decision can be effectively improved, so that the limitation of the existing scheme is solved.

Description

Technical Field

[0001] The present invention relates to the field of edge computing, and particularly to a method, device, equipment and medium for microservice deployment of a government affairs big data platform. Background Art

[0002] With the rapid development of artificial intelligence technology and big data technology, data middle platforms such as big data platforms and data operation dashboards have higher and higher requirements for network response latency and data privacy.

[0003] However, due to reasons such as too long transmission links and homogeneous server resources in the cloud computing architecture of traditional deployment solutions, the network transmission latency is relatively long, and it is difficult to meet the current rapidly growing latency requirements. In addition, due to excessive resource occupation in the monolithic deployment architecture of the software platform in traditional deployment solutions, collaboration cannot be carried out, and it is also difficult to adapt to the current microservice deployment requirements. Summary of the Invention

[0004] In view of this, the purpose of the present invention is to provide a method, device, equipment and medium for microservice deployment of a government affairs big data platform, which can effectively improve the accuracy and reliability of deployment decisions, thereby solving the limitations of existing solutions and effectively improving the real-time performance of microservice responses and microservice performance. The specific solutions are as follows:

[0005] In a first aspect, the present application provides a method for microservice deployment of a government affairs big data platform, including:

[0006] Obtain a set of microservices to be deployed in the government affairs big data platform, and collect the basic information of each microservice to be deployed in the set of microservices to be deployed to obtain first information;

[0007] Obtain a set of available edge servers in the current edge computing network architecture, and collect the basic information of each edge server in the set of available edge servers to obtain second information;

[0008] Obtain a set of applications to be processed, and collect the basic information of each application to be processed in the set of applications to be processed to obtain third information; wherein, each application to be processed includes different combinations of microservices to be deployed, and the third information includes topology structure information;

[0009] Configure decision variables based on the first information, the second information and the third information, and use the obtained decision variables to construct a microservice deployment model to obtain a target microservice deployment model;

[0010] Solve the target microservice deployment model based on the genetic algorithm, the set of microservices to be deployed, the first information, the set of available edge servers, the second information, the set of applications to be processed, and the third information to determine the microservice deployment decision.

[0011] Optionally, obtaining the set of microservices to be deployed in the government affairs big data platform and collecting the basic information of each microservice to be deployed in the set of microservices to be deployed includes:

[0012] Count the microservices to be deployed in the government affairs big data platform to obtain the set of microservices to be deployed;

[0013] Obtain the expected computing speed, expected memory resources, and expected computing resources of each microservice to be deployed in the set of microservices to be deployed to obtain the first information.

[0014] Optionally, obtaining the set of available edge servers in the current edge computing network architecture and collecting the basic information of each edge server in the set of available edge servers includes:

[0015] Count the edge servers in the available state in the current edge computing network architecture to obtain the set of available edge servers;

[0016] Obtain the number of cores, total memory resources, processing speed, required computing resources of each edge server in the set of available edge servers, and the data transmission speed between each of the edge servers to obtain the second information.

[0017] Optionally, collecting the basic information of each application to be processed in the set of applications to be processed includes:

[0018] Obtain the topological information of the structure of each application to be processed in the set of applications to be processed to obtain the topological structure information; the topological structure information includes the set of microservice traversals corresponding to each single run of each application to be processed;

[0019] Obtain the average arrival rate of requests of each application to be processed to obtain the average arrival rate information of requests;

[0020] Based on the topological structure information, determine the dependency relationships between the microservices in the set of microservice traversals corresponding to each application to be processed to obtain the set of microservice dependency relationships corresponding to each application to be processed;

[0021] Based on the topological structure information and the corresponding set of microservice dependency relationships, respectively collect the microservice execution data corresponding to each vertex in the corresponding vertex set during each single run of each application to be processed to obtain the single run information corresponding to each application to be processed.

[0022] Optionally, configuring decision variables based on the first information, the second information, and the third information, and constructing a microservice deployment model by using the obtained decision variables includes:

[0023] Configuring decision variables based on the microservice set to be deployed and the available edge server set;

[0024] Determining an objective function based on the application set to be processed and the single-run delay corresponding to each of the applications to be processed in the third information;

[0025] Determining the constraint conditions of the microservice deployment model based on the microservice set to be deployed, the available edge server set, the application set to be processed, the first information, the second information, the third information, and the decision variables, and determining the target microservice deployment model based on the constraint conditions and the objective function.

[0026] Optionally, solving the target microservice deployment model based on the genetic algorithm, the microservice set to be deployed, the first information, the available edge server set, the second information, the application set to be processed, and the third information includes:

[0027] Initializing genetic parameters by using the microservice set to be deployed and a genetic algorithm with an elitist strategy to obtain the population size, crossover probability, mutation probability, and iteration number threshold;

[0028] After encoding the deployment situation of the microservices to be deployed based on the microservice set to be deployed, the available edge server set, and the population size, generating a greedy solution and an initial solution to complete the corresponding population initialization operation to obtain the initialized current population and the current population individual coding information;

[0029] Determining the fitness value corresponding to each individual in the current population based on the application set to be processed, the third information, the second information, and the objective function and constraint conditions in the target microservice deployment model;

[0030] Selecting individuals based on the fitness value and the roulette wheel selection strategy with an elitist reservation strategy to obtain a selection result;

[0031] Determining any two microservices to be deployed on different edge servers based on the crossover probability and the selection result, and triggering a deployment location exchange operation;

[0032] Based on the mutation probability, statistically determine the to-be-deployed microservice with the longest execution time among the current to-be-processed applications, randomly select a microservice from the determined target microservice set for redeployment to complete the corresponding mutation operation, and determine the current population and the individual coding information of the current population;

[0033] Update the current iteration number, and determine whether the current iteration number is less than the iteration number threshold to obtain an iteration number judgment result;

[0034] If the iteration number judgment result indicates less than, then re-jump to the step of determining the fitness value corresponding to each individual in the current population based on the to-be-processed application set, the third information, the second information, and the objective function and constraint conditions in the target microservice deployment model, until the current iteration number is equal to the iteration number threshold, and determine the microservice deployment decision based on the current population and the individual coding information of the current population, so as to trigger the corresponding microservice deployment operation according to the microservice deployment decision.

[0035] Optionally, the corresponding population initialization operation is completed by generating a greedy solution and an initial solution, including:

[0036] Based on the first expected computing resource information in the first information, perform weighted calculation on the priorities of the to-be-deployed microservices in the to-be-deployed microservice set, and sort the to-be-deployed microservices in descending order according to the obtained microservice priority information to obtain a first sorting result;

[0037] Based on the processing speed information and the second expected computing resource information in the second information, sort each edge server in the available edge server set to obtain a second sorting result;

[0038] Based on the first sorting result and the second sorting result, reallocate the configuration relationship between the microservices and the edge servers, and update the to-be-deployed microservice set and the available edge server set to complete the corresponding greedy solution generation operation, and obtain an updated first microservice set and an updated edge server set;

[0039] When the first microservice set is a non-empty set, determine the target edge server node set based on the first information and the memory constraint;

[0040] Randomly select a server node for the to-be-deployed microservice with the smallest priority in the first microservice set based on the target edge server node set, and trigger a microservice set update operation to obtain an updated second microservice set;

[0041] Determine whether the second microservice set is empty, and when it is not, complete the corresponding population initialization operation based on the second microservice set to obtain the initialized current population.

[0042] In a second aspect, the present application provides a microservice deployment device for a government affairs big data platform, including:

[0043] A first information determination module, configured to obtain a set of microservices to be deployed in the government affairs big data platform, and collect basic information of each microservice to be deployed in the set of microservices to be deployed, so as to obtain first information;

[0044] A second information determination module, configured to obtain a set of available edge servers in the current edge computing network architecture, and collect basic information of each edge server in the set of available edge servers, so as to obtain second information;

[0045] A third information determination module, configured to obtain a set of applications to be processed, and collect basic information of each application to be processed in the set of applications to be processed, so as to obtain third information; wherein, each application to be processed includes different combinations of microservices to be deployed, and the third information includes topology structure information;

[0046] A deployment model construction module, configured to perform decision variable configuration based on the first information, the second information, and the third information, and construct a microservice deployment model by using the obtained decision variables to obtain a target microservice deployment model;

[0047] A deployment decision determination module, configured to solve the target microservice deployment model based on a genetic algorithm, the set of microservices to be deployed, the first information, the set of available edge servers, the second information, the set of applications to be processed, and the third information to determine a microservice deployment decision.

[0048] In a third aspect, the present application provides an electronic device, including:

[0049] A memory, configured to store a computer program;

[0050] A processor, configured to execute the computer program to implement the steps of the foregoing microservice deployment method for the government affairs big data platform.

[0051] In a fourth aspect, the present application provides a computer-readable storage medium, configured to store a computer program, and when the computer program is executed by a processor, the steps of the foregoing microservice deployment method for the government affairs big data platform are implemented.

[0052] It can be seen that in this application, a set of microservices to be deployed in the government affairs big data platform is obtained, and the basic information of each microservice to be deployed in the set of microservices to be deployed is collected to obtain the first information; a set of available edge servers in the current edge computing network architecture is obtained, and the basic information of each edge server in the set of available edge servers is collected to obtain the second information; a set of applications to be processed is obtained, and the basic information of each application to be processed in the set of applications to be processed is collected to obtain the third information; wherein, different combinations of microservices to be deployed are included in each of the applications to be processed, and the third information includes topology information; decision variable configuration is performed based on the first information, the second information, and the third information, and a microservice deployment model is constructed using the obtained decision variables to obtain a target microservice deployment model; the target microservice deployment model is solved based on the genetic algorithm, the set of microservices to be deployed, the first information, the set of available edge servers, the second information, the set of applications to be processed, and the third information to determine a microservice deployment decision. That is to say, in this application, first, the basic information and set of microservices to be deployed, the basic information and set of available edge servers, and the basic information and set of applications to be processed are obtained, then a target microservice deployment model is constructed based on the first information, the second information, and the third information, and then the target microservice deployment model is solved based on the genetic algorithm and the information obtained from the foregoing operations to determine a microservice deployment decision. In this way, the accuracy and reliability of the deployment decision can be effectively improved, thereby solving the limitations of the existing solutions and effectively improving the real-time performance of microservice response and microservice performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained according to the provided drawings without creative efforts.

[0054] Figure 1 It is a flowchart of a method for deploying microservices in a government affairs big data platform provided by this application;

[0055] Figure 2 It is a schematic diagram of an example of a deployment solution of microservices in edge servers provided by this application;

[0056] Figure 3 It is a schematic diagram of an application topology structure provided by this application;

[0057] Figure 4 It is a schematic diagram of a solution process for a deployment model based on a genetic algorithm provided by this application;

[0058] Figure 5 Schematic diagram of a computational example application topology structure provided for this application;

[0059] Figure 6 Schematic diagram of the structure of a microservice deployment device for a government affairs big data platform provided for this application;

[0060] Figure 7 Structural diagram of an electronic device provided for this application. Detailed implementation manners

[0061] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0062] However, due to reasons such as too long transmission links and homogeneous server resources in the cloud computing architecture in the traditional deployment scheme, the network transmission delay is relatively long, and it is difficult to meet the current rapidly growing delay requirements. In addition, due to excessive resource occupation in the monolithic deployment architecture of the software platform in the traditional deployment scheme, collaboration cannot be carried out, and it is also difficult to adapt to the current microservice deployment requirements. For this reason, this application provides a microservice deployment scheme for a government affairs big data platform, which can effectively improve the accuracy and reliability of deployment decisions, thereby solving the limitations of the existing scheme and effectively improving the real-time performance of microservice responses and microservice performance.

[0063] See Figure 1 As shown, the embodiments of the present invention disclose a microservice deployment method for a government affairs big data platform, including:

[0064] Step S11, obtaining a set of microservices to be deployed in the government affairs big data platform, and collecting basic information of each microservice to be deployed in the set of microservices to obtain first information.

[0065] Specifically, in this embodiment, first, all microservices to be deployed in the current scheduling of the government affairs big data platform are obtained to form a set, and detailed information of each microservice in the set is obtained. That is, first, the microservices to be deployed in the government affairs big data platform (represented by ) are counted to obtain a set of microservices to be deployed (represented by representation); obtain the expected computing speed, expected memory resources, and expected computing resources of each to-be-deployed microservice in the set of to-be-deployed microservices to obtain first information. It should be understood that in this embodiment, a microservice consists of a business system module of a government affairs big data platform, and each microservice represents a specific data business system. Each microservice runs in one process and can be independently deployed in a Docker container. Different microservices organize and coordinate with each other through a lightweight communication protocol to jointly complete complex business logics.

[0066] Among them, the expected computing speed is the computing speed required by the microservice ∈ and is represented by ; the expected memory resources are the memory resources required by the microservice ∈ and are represented by ; the expected computing resources are the computing module resources required by the microservice ∈ and are represented by .

[0067] Step S12: Obtain the set of available edge servers in the current edge computing network architecture, and collect the basic information of each edge server in the set of available edge servers to obtain second information.

[0068] In this embodiment, in addition to the set of to-be-deployed microservices, it is also necessary to determine all available edge servers in this scheduling, form a set, and obtain the detailed information of each server in the set. That is, first count the edge servers in the available state in the current edge computing network architecture to obtain the set of available edge servers (represented by ); then obtain the number of cores (represented by ), total memory resources (represented by ), processing speed (represented by ), required computing resources (represented by ), and the data transmission speed between each edge server in the set of available edge servers (represented by ) to obtain second information. Among them, the data transmission speed represents the data transmission speed between edge server and edge server , and is represented by , where ≠ , and , ∈ .

[0069] It should be understood that in this embodiment, a core of an edge server is allocated to each microservice to be deployed. Therefore, the maximum number of microservices that can be deployed on one edge server is equal to the number of cores of the edge server. In addition, each microservice is deployed on only one edge server and only one instance is deployed. An example of the deployment scheme of microservices in the edge server is as Figure 2 shown. Microservices 1 and 2 are deployed on edge server A, microservice 3 is deployed on edge server B, and microservice 4 is deployed on edge server C. Edge servers A, B, and C are connected through links.

[0070] Step S13: Obtain a set of applications to be processed, and collect the basic information of each application to be processed in the set of applications to be processed to obtain third information; wherein, different combinations of microservices to be deployed are included in each application to be processed, and the third information includes topology structure information.

[0071] Specifically, in this embodiment, it is also necessary to confirm all applications participating in this scheduling, form a set, and obtain the topology information and other detailed information of all application structures in the set. That is, first obtain the topology information of the structure of each application to be processed (denoted by ) in the set of applications to be processed (denoted by ) to obtain topology structure information (denoted by ); the topology structure information includes a set of microservice traversals for a single run corresponding to each application to be processed (also known as the vertex set, denoted by , ); then obtain the average arrival rate of requests of each application to be processed to obtain request average arrival rate information (denoted by ); then based on the topology structure information, determine the dependency relationships between the microservices in the set of microservice traversals corresponding to each application to be processed to obtain a set of microservice dependency relationships corresponding to each application to be processed (denoted by ); then based on the topology structure information and the corresponding set of microservice dependency relationships, respectively collect the microservice execution data corresponding to each vertex in the corresponding vertex set during a single run of each application to be processed to obtain single-run information corresponding to each application to be processed. Among them, the topology structure information = ( , ), is a directed acyclic graph and ∈ . is the vertex set, which represents the set of microservices traversed during a single run of application . Each microservice in is a vertex of the directed acyclic graph; is the arc set, which represents application The set of dependencies among microservices required for a single run Represents a vertex The corresponding microservice can only run after the microservice corresponding to the vertex is completed, where 、 ∈ . The single-run information in the third piece of information is the computation volume of the microservice corresponding to vertex during a single run of the application . In addition, the third piece of information also includes the amount of data transmitted from the microservice corresponding to vertex to the microservice corresponding to vertex after the microservice corresponding to vertex completes its execution during a single run of the application . .

[0072] It should be understood that in this embodiment, each application is composed of different microservices combined with each other. Each application can provide more complete and comprehensive services compared with a single microservice to support data sharing, data governance, data aggregation, data monetization, etc. in the government affairs field, thereby helping the data management department clarify data assets and establish an integrated big data management system. In addition, requests for all applications are executed in sequence according to the rule of first come, first served. In this embodiment, an example of the topological structure of an application can be as Figure 3 shown, and applications 1, 2, 3, 4, and 5 are connected according to the Figure 3 shown topological relationship.

[0073] Step S14: Configure decision variables based on the first piece of information, the second piece of information, and the third piece of information, and construct a microservice deployment model using the obtained decision variables to obtain a target microservice deployment model.

[0074] Specifically, after this embodiment completes obtaining all the information required for this scheduling, it will set decision variables and construct a microservice deployment model based on the basic information of the obtained microservices, servers, and applications. That is, first, configure decision variables based on the set of microservices to be deployed and the set of available edge servers; then determine the objective function based on the set of applications to be processed and the single-run delay corresponding to each application to be processed in the third piece of information; then determine the constraint conditions of the microservice deployment model based on the set of microservices to be deployed, the set of available edge servers, the set of applications to be processed, the first piece of information, the second piece of information, the third piece of information, and the decision variables, and determine the target microservice deployment model based on the constraint conditions and the objective function.

[0075] It should be further understood that in this embodiment, decision variables can be specifically set and a 0-1 integer programming model for microservice deployment can be established. The decision variables have the following specific meanings:

[0076] ;

[0077] Among them, ∈ and ∈ .

[0078] The objective function of the deployment model can be as follows:

[0079] ;

[0080] Among them, is the single-run latency of application ; is the average arrival rate of requests for application , where , ∈ ;

[0081] The constraint conditions of the deployment model are as follows:

[0082] (1);

[0083] (2);

[0084] (3);

[0085] (4);

[0086] (5);

[0087] (6);

[0088] Among them, constraint (1) means that the expected computing speed of any microservice ∈ does not exceed the computing speed of its corresponding deployed edge server; constraint (2) means that the expected computing resources of any microservice ∈ do not exceed the computing module resources available on its corresponding deployed edge server; constraint (3) means that the memory resources of any edge server ∈ are not less than the sum of the expected memory resources of the microservices deployed on it; constraint (4) means that the microservices deployed on any edge server ∈ The number of microservices on it does not exceed its maximum deployable number of microservices; Constraint (5) means that each microservice ∈ is only deployed on one edge server; Constraint (6) represents the value range of the decision variables. In addition, the communication time between microservices on the same edge server is not counted.

[0089] It should be understood that regarding the single-run latency of the specific calculation process:

[0090] Define as the time required from the start of the application to the microservice corresponding to the vertex and the completion of the microservice execution. The specific meaning of

[0091] ;

[0092] Among them, is the set composed of vertices with in-degree zero in ; is the running latency of the microservice corresponding to the vertex is the data transmission latency from the vertex to the vertex ; is the time required to start from the application to all the predecessor microservices of the microservice corresponding to the vertex and transmit the data to the microservice corresponding to the vertex ; is the computational workload of the microservice corresponding to the vertex during a single run of the application . Regarding the decision variables , , , are as follows:

[0093] ;

[0094] ;

[0095] ;

[0096] Perform a topological sort on all vertices in and calculate the completion time of each vertex in the sorted order in turn , then the single-run latency of the application

[0097] ;

[0098] wherein is the set of vertices with out-degree zero in

[0099] Step S15: Solve the target microservice deployment model based on the genetic algorithm, the set of microservices to be deployed, the first information, the set of available edge servers, the second information, the set of applications to be processed, and the third information, so as to determine the microservice deployment decision.

[0100] In this embodiment, as shown in Figure 4 , after determining the target microservice deployment model, the deployment model will be solved based on the genetic algorithm, the set of microservices to be deployed, the set of available edge servers, and the set of applications to be processed. That is, first, use the set of microservices to be deployed and the genetic algorithm with elitist strategy to initialize the genetic parameters to obtain the population size, crossover probability, mutation probability, and iteration threshold; after encoding the deployment situation of the microservices to be deployed based on the set of microservices to be deployed, the set of available edge servers, and the population size, complete the corresponding population initialization operation by generating the greedy solution and the initial solution to obtain the initialized current population and the current population individual coding information; determine the fitness value corresponding to each individual in the current population based on the set of applications to be processed, the third information, the second information, and the objective function and constraint conditions in the target microservice deployment model; perform individual selection based on the fitness value and the roulette wheel selection strategy with elitist retention strategy to obtain the selection result; determine any two microservices to be deployed on different edge servers based on the crossover probability and the selection result, and trigger the deployment position exchange operation; based on the mutation probability, count the microservice to be deployed with the longest execution time among the current applications to be processed, and randomly select a microservice from the determined set of target microservices for redeployment to complete the corresponding mutation operation, and determine the current population and the current population individual coding information; update the current iteration count, and determine whether the current iteration count is less than the iteration threshold to obtain the iteration count judgment result; if the iteration count judgment result indicates less than, then jump back to the step of determining the fitness value corresponding to each individual in the current population based on the set of applications to be processed, the third information, the second information, and the objective function and constraint conditions in the target microservice deployment model until the current iteration count is equal to the iteration threshold, and determine the microservice deployment decision based on the current population and the current population individual coding information, so as to trigger the corresponding microservice deployment operation according to the microservice deployment decision.

[0101] Regarding the population initialization operation, in this embodiment, first, the priorities of each to-be-deployed microservice in the to-be-deployed microservice set are weighted based on the first expected computing resource information in the first information, and the to-be-deployed microservices are sorted in descending order according to the obtained microservice priority information to obtain the first sorting result; then, each edge server in the available edge server set is sorted based on the processing speed information in the second information and the second expected computing resource information to obtain the second sorting result; based on the first sorting result and the second sorting result, the configuration relationship between the microservices and the edge servers is reallocated, and the to-be-deployed microservice set and the available edge server set are updated to complete the corresponding greedy solution generation operation and obtain the updated first microservice set and the updated edge server set; when the first microservice set is a non-empty set, the target edge server node set is determined based on the first information and the memory constraint; based on the target edge server node set, a server node is randomly selected for the to-be-deployed microservice with the smallest priority in the first microservice set, and a microservice set update operation is triggered to obtain the updated second microservice set; it is judged whether the second microservice set is empty, and when it is not, the corresponding population initialization operation is completed based on the second microservice set to obtain the initialized current population.

[0102] It can be understood that in a specific embodiment, the specific steps for solving the deployment model based on the genetic algorithm are as follows:

[0103] 1). Initialize the genetic algorithm parameters.

[0104] The specific parameters include the population size, that is, the number of population individuals is set to 100; the crossover probability is 0.8; the mutation probability is 0.05; the maximum number of algorithm iterations, that is, the iteration threshold is 200.

[0105] 2). Population initialization.

[0106] Encoding: Use the natural number encoding method to encode the deployment of microservices in the to-be-deployed microservice set. The encoding length is the number of microservices, each position represents a microservice, and the element at each position represents the serial number of the edge server to which the corresponding microservice is deployed. The specific encoding example is as follows:

[0107] When there are 5 microservices and 4 edge servers, the deployment situation of the microservices can be encoded as:

[0108] ;

[0109] The meaning of this encoding is: Microservice 1 and microservice 3 are deployed on edge server 1, microservice 2 is deployed on edge server 2, microservice 4 is deployed on edge server 3, and microservice 5 is deployed on edge server 4.

[0110] Regarding population initialization: In the greedy solution. Given the set of microservices to be deployed and the set of available edge servers , the generation scheme of the greedy solution is as follows:

[0111] Step 1: Calculate the weighted priority of microservices based on the expected computing speed and expected computing resources of the microservices to be deployed in , and sort all the microservices to be deployed in in descending order according to the microservice priority;

[0112] Step 2: Sort the edge servers according to the computing speed and computing module resources of the edge servers in ;

[0113] Step 3: Assign the microservice with the highest priority to the server with the strongest computing power, and update and ;

[0114] Step 4: If the set is empty, the initial solution generation algorithm ends; otherwise, go to Step 3.

[0115] For the remaining chromosomes, in this embodiment, an available server is randomly selected from the allocable server nodes to generate the initial solution. The specific algorithm steps are as follows:

[0116] Step 1: Determine the set of server nodes where all the microservices to be deployed in meet the resource constraints according to the computing speed, computing module resources, and memory constraints;

[0117] Step 2: Randomly select an available server node for the microservice with the smallest number in the set , and update the set ;

[0118] Step 3: If the set is empty, output the allocation result and terminate the algorithm; otherwise, go to Step 2.

[0119] 3) Fitness value calculation. Calculate the fitness value of an individual according to the objective function of the deployment model.

[0120] 4) Selection operation. Use roulette wheel selection with elitist retention strategy for selection.

[0121] 5) Crossover operation. Select any two microservices deployed on different edge servers and exchange their deployment positions.

[0122] 6) Mutation operation. Use the selection of the current Among all the microservices in the application, randomly select a server to redeploy the microservice with the longest execution time.

[0123] 7) Increment the number of iterations and determine whether the maximum number of iterations has been reached. If not, go to step 3).

[0124] In this way, a microservice deployment decision can be obtained when the number of iterations reaches the maximum number of iterations.

[0125] In summary, this embodiment provides a method for deploying a government affairs big data platform based on a genetic algorithm to solve the problem of microservice deployment in an edge computing network architecture. By deploying server nodes on the edge side close to the terminal and splitting a single application into multiple microservices, the decision-making response ability of the terminal can be improved. The specific solution is to map the microservice deployment problem of edge computing into a mathematical model solving problem and solve this problem through a genetic algorithm to obtain a microservice deployment plan, thereby reducing the response delay of the service and improving the utilization efficiency of resources and real-time decision-making ability. The specific beneficial effects are as follows:

[0126] (1) The method provided in this embodiment can comprehensively consider the computing resource requirements of microservices and the computing resources available on edge servers, improve the real-time performance of microservice responses, the resource utilization rate of server nodes, and the degree of load balancing.

[0127] (2) The method provided in this embodiment adopts a genetic algorithm based on a greedy algorithm to generate solutions and an elite retention strategy. Utilizing the global search characteristics of the genetic algorithm, it can obtain better decision-making results.

[0128] (3) The genetic algorithm given in this embodiment comprehensively considers the transmission delay and running delay of microservices and the topological structure relationship in the application, accurately describes the matching relationship among microservices, applications, and servers, and can be effectively applied to the microservice deployment plan of the big data platform.

[0129] It can be seen that in this application, first, the basic information and set of microservices to be deployed, the basic information and set of available edge servers, and the basic information and set of applications to be processed are obtained. Then, a target microservice deployment model is constructed based on the first information, the second information, and the third information. After that, the target microservice deployment model is solved based on the genetic algorithm and the information obtained from the foregoing operations to determine the microservice deployment decision. In this way, the accuracy and reliability of the deployment decision can be effectively improved, thereby solving the limitations of the existing solutions and effectively improving the real-time performance of microservice responses and microservice performance.

[0130] The following combines Figure 5 with the schematic diagrams disclosed in

[0131] Taking the microservice deployment instance of a government affairs big data platform as an example for illustration. The government affairs big data platform consists of a directory system, a resource system, an application authorization system, a fusion service system, a gateway system, an operation management system, an operation monitoring system, a supply and demand docking system, a case system, and an objection handling system. The information of each microservice is shown in Table 1. The number of available edge servers is 5, and the computing speed, storage capacity, and number of cores of the edge servers are shown in Table 2. The data transmission rate between any two edge servers is shown in Table 3.

[0132] Table 1

[0133]

[0134] Table 2

[0135]

[0136] Table 3

[0137]

[0138] Among them, MIPS is Million Instructions Per Second, representing the average execution speed of single-word fixed-point instructions; MB is Megabyte, which is the basic unit of data storage capacity; FLOPS is Floating Point Operations Per Second, representing the number of floating-point operations per second. Table 1 introduces the specific functions and basic information of microservices A, B, C, D, E, F, and G. Table 2 introduces the basic information of edge servers 1, 2, 3, 4, and 5. Table 3 introduces the data transmission rate between edge servers 1, 2, 3, 4, and 5.

[0139] Considering Figure 5 the three application topologies shown, the average arrival rates of applications A, B, and C are set to 1, 2, and 1 respectively. The traffic on any arc in each application topology is randomly generated between 5KB and 50KB, and the computing volume of any microservice is represented by the number of million instructions required for a single run, which is randomly generated between 0.01 and 0.1. Then, the microservice deployment process described in the foregoing embodiments can be executed to construct and solve the target microservice deployment to determine the corresponding deployment decision.

[0140] Referring to Figure 6 as shown, the embodiment of the present application also correspondingly discloses a microservice deployment device for a government affairs big data platform, including:

[0141] The first information determination module 11 is configured to obtain a set of microservices to be deployed in the government affairs big data platform, and collect the basic information of each microservice to be deployed in the set of microservices to be deployed, so as to obtain the first information;

[0142] The second information determination module 12 is configured to obtain a set of available edge servers in the current edge computing network architecture, and collect the basic information of each edge server in the set of available edge servers, so as to obtain the second information;

[0143] The third information determination module 13 is configured to obtain a set of applications to be processed, and collect the basic information of each application to be processed in the set of applications to be processed, so as to obtain the third information; wherein, different combinations of microservices to be deployed are included in each application to be processed, and the third information includes topology structure information;

[0144] The deployment model construction module 14 is configured to perform decision variable configuration based on the first information, the second information, and the third information, and construct a microservice deployment model using the obtained decision variables, so as to obtain a target microservice deployment model;

[0145] The deployment decision determination module 15 is configured to solve the target microservice deployment model based on a genetic algorithm, the set of microservices to be deployed, the first information, the set of available edge servers, the second information, the set of applications to be processed, and the third information, so as to determine a microservice deployment decision.

[0146] It can be seen that in this application, first, the basic information and set of microservices to be deployed, the basic information and set of available edge servers, and the basic information and set of applications to be processed are obtained. Then, a target microservice deployment model is constructed based on the first information, the second information, and the third information. After that, the target microservice deployment model is solved based on a genetic algorithm and the information obtained from the foregoing operations to determine a microservice deployment decision. In this way, the accuracy and reliability of the deployment decision can be effectively improved, thereby solving the limitations of the existing solution and effectively improving the real-time performance of microservice response and microservice performance.

[0147] In some specific embodiments, the first information determination module 11 may specifically include:

[0148] The microservice set to be deployed determination unit is configured to count the microservices to be deployed in the government affairs big data platform, so as to obtain a set of microservices to be deployed;

[0149] The microservice information acquisition unit is configured to obtain the expected computing speed, expected memory resources, and expected computing resources of each microservice to be deployed in the set of microservices to be deployed, so as to obtain the first information.

[0150] In some specific embodiments, the second information determination module 12 may specifically include:

[0151] An available edge server set determination unit, configured to count the edge servers in the available state in the current edge computing network architecture to obtain an available edge server set;

[0152] A server information acquisition unit, configured to acquire the core count, total memory resources, processing speed, required computing resources of each edge server in the available edge server set, and the data transmission speed between each of the edge servers, to obtain second information.

[0153] In some specific embodiments, the third information determination module 13 may specifically include:

[0154] A topology structure information acquisition unit, configured to acquire the topology information of the structures of each to-be-processed application in the to-be-processed application set to obtain topology structure information; the topology structure information includes a microservice traversal set corresponding to a single run of each of the to-be-processed applications;

[0155] A request average arrival rate acquisition unit, configured to acquire the average arrival rate of requests of each of the to-be-processed applications to obtain request average arrival rate information;

[0156] A microservice dependency determination unit, configured to determine the dependency relationships between the microservices in the microservice traversal set corresponding to each of the to-be-processed applications based on the topology structure information, to obtain a microservice dependency relationship set corresponding to each of the to-be-processed applications;

[0157] A single-run information acquisition unit, configured to respectively collect the microservice execution data corresponding to each vertex in the corresponding vertex set during a single run of each of the to-be-processed applications based on the topology structure information and the corresponding microservice dependency relationship set, to obtain single-run information corresponding to each of the to-be-processed applications.

[0158] In some specific embodiments, the deployment model construction module 14 may specifically include:

[0159] A variable configuration unit, configured to configure decision variables based on the to-be-deployed microservice set and the available edge server set;

[0160] A target function determination unit, configured to determine a target function based on the to-be-processed application set and the single-run delay corresponding to each of the to-be-processed applications in the third information;

[0161] A constraint determination unit, configured to determine the constraints of the microservice deployment model based on the set of microservices to be deployed, the set of available edge servers, the set of applications to be processed, the first information, the second information, the third information, and the decision variables, and determine the target microservice deployment model based on the constraints and the objective function.

[0162] In some specific embodiments, the deployment decision determination module 15 may specifically include:

[0163] A parameter initialization unit, configured to initialize genetic parameters using the set of microservices to be deployed and a genetic algorithm with an elite strategy to obtain the population size, crossover probability, mutation probability, and iteration count threshold.

[0164] A population initialization unit, configured to encode the deployment status of the microservices to be deployed based on the set of microservices to be deployed, the set of available edge servers, and the population size, and complete the corresponding population initialization operation by generating a greedy solution and an initial solution to obtain the initialized current population and the current population individual encoding information.

[0165] An individual fitness value determination unit, configured to determine the fitness value corresponding to each individual in the current population based on the set of applications to be processed, the third information, the second information, and the objective function and constraints in the target microservice deployment model.

[0166] An individual selection unit, configured to perform individual selection based on the fitness value and a roulette wheel selection strategy with an elite retention strategy to obtain a selection result.

[0167] A position exchange unit, configured to determine any two microservices to be deployed on different edge servers based on the crossover probability and the selection result, and trigger a deployment position exchange operation.

[0168] A microservice mutation unit, configured to statistically determine the microservice to be deployed with the longest execution time among the current microservices to be processed based on the mutation probability, and randomly select a microservice from the determined target microservice set for redeployment to complete the corresponding mutation operation, and determine the current population and the current population individual encoding information.

[0169] An iteration count judgment unit, configured to update the current iteration count and judge whether the current iteration count is less than the iteration count threshold to obtain an iteration count judgment result.

[0170] A step jump unit, configured to, if the iteration number determination result indicates less than, re-jump to the step of determining the fitness value corresponding to each individual in the current population based on the set of applications to be processed, the third information, the second information, and the objective function and constraint conditions in the target microservice deployment model, until the current iteration number is equal to the iteration number threshold, and determine a microservice deployment decision based on the current population and the individual coding information of the current population, so as to trigger a corresponding microservice deployment operation according to the microservice deployment decision.

[0171] In some specific embodiments, the population initialization unit may specifically include:

[0172] A microservice sorting subunit, configured to perform weighted calculation on the priorities of the microservices to be deployed in the set of microservices to be deployed based on the first expected computing resource information in the first information, and perform descending sorting on the microservices to be deployed according to the obtained microservice priority information to obtain a first sorting result;

[0173] A server sorting subunit, configured to sort each of the edge servers in the set of available edge servers based on the processing speed information and the second expected computing resource information in the second information to obtain a second sorting result;

[0174] A greedy solution generation subunit, configured to reallocate the configuration relationship between the microservices and the edge servers based on the first sorting result and the second sorting result, and update the set of microservices to be deployed and the set of available edge servers to complete the corresponding greedy solution generation operation, and obtain an updated first microservice set and an updated edge server set;

[0175] A target server node determination subunit, configured to determine a set of target edge server nodes based on the first information and the memory constraint when the first microservice set is a non-empty set;

[0176] A microservice set update subunit, configured to randomly select a server node for the microservice to be deployed with the lowest priority in the first microservice set based on the set of target edge server nodes, and trigger a microservice set update operation to obtain an updated second microservice set;

[0177] An initialization completion subunit, configured to determine whether the second microservice set is empty, and when it is not, complete the corresponding population initialization operation based on the second microservice set to obtain an initialized current population.

[0178] Furthermore, an embodiment of the present application also discloses an electronic device, Figure 7It is a structural diagram of an electronic device 20 shown according to an exemplary embodiment. The content in the figure should not be considered as any limitation on the scope of use of this application.

[0179] Figure 7 This is a schematic structural diagram of an electronic device 20 provided by an embodiment of this application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. Among them, the memory 22 is used to store a computer program, and the computer program is loaded and executed by the processor 21 to implement the relevant steps in the method for deploying microservices of the government affairs big data platform disclosed in any of the foregoing embodiments. In addition, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0180] In this embodiment, the power supply 23 is used to provide working voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows is any communication protocol applicable to the technical solution of this application, and no specific limitation is imposed on it here; the input / output interface 25 is used to obtain external input data or output data to the outside, and its specific interface type can be selected according to specific application needs, and no specific limitation is made here.

[0181] In addition, as a carrier for resource storage, the memory 22 may be a read-only memory, a random access memory, a magnetic disk, or an optical disc, etc. The resources stored thereon may include an operating system 221, a computer program 222, etc., and the storage method may be temporary storage or permanent storage.

[0182] Among them, the operating system 221 is used to manage and control each hardware device and the computer program 222 on the electronic device 20, and it may be Windows Server, Netware, Unix, Linux, etc. In addition to the computer program that can be used to complete the method for deploying microservices of the government affairs big data platform executed by the electronic device 20 disclosed in any of the foregoing embodiments, the computer program 222 may further include a computer program that can be used to complete other specific tasks.

[0183] Furthermore, this application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the method for deploying microservices of the government affairs big data platform disclosed above. For the specific steps of this method, reference may be made to the corresponding content disclosed in the foregoing embodiments, and details will not be repeated here.

[0184] In this specification, the various embodiments are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. For the same or similar parts among the embodiments, reference can be made to each other. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple. For related parts, reference can be made to the description in the method section.

[0185] Those skilled in the art can further realize that the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the examples have been generally described according to their functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of this application.

[0186] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be directly implemented by hardware, software modules executed by a processor, or a combination of both. The software modules can be placed in a random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium well-known in the technical field.

[0187] Finally, it should also be noted that in this text, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or further includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "including a..." does not exclude the existence of additional identical elements in the process, method, article or device including the said element.

[0188] The technical solutions provided in this application have been introduced in detail above. Specific examples are used herein to elaborate on the principles and implementation manners of this application. The descriptions of the above embodiments are only used to help understand the method and its core idea of this application; at the same time, for those of ordinary skill in the art, based on the idea of this application, there will be changes in the specific implementation manners and application scopes. In summary, the content of this specification should not be construed as a limitation to this application.

Claims

1. A microservice deployment method for a government affairs big data platform, characterized in that, Including: Obtain the set of microservices to be deployed in the government affairs big data platform, and collect the basic information of each microservice to be deployed in the set of microservices to be deployed, so as to obtain the first information; Obtain the set of available edge servers in the current edge computing network architecture, and collect the basic information of each edge server in the set of available edge servers, so as to obtain the second information; Obtain the set of applications to be processed, and collect the basic information of each application to be processed in the set of applications to be processed, so as to obtain the third information; wherein, each of the applications to be processed includes different combinations of microservices to be deployed, and the third information includes topology information; Configure decision variables based on the first information, the second information, and the third information, and use the obtained decision variables to construct a microservice deployment model to obtain a target microservice deployment model; Solve the target microservice deployment model based on the genetic algorithm, the set of microservices to be deployed, the first information, the set of available edge servers, the second information, the set of applications to be processed, and the third information, so as to determine the microservice deployment decision.

2. The microservice deployment method for the government affairs big data platform according to claim 1, wherein The obtaining the set of microservices to be deployed in the government affairs big data platform and collecting the basic information of each microservice to be deployed in the set of microservices to be deployed includes: Count the microservices to be deployed in the government affairs big data platform to obtain the set of microservices to be deployed; Obtain the expected computing speed, expected memory resources, and expected computing resources of each microservice to be deployed in the set of microservices to be deployed, so as to obtain the first information.

3. The microservice deployment method for the government affairs big data platform according to claim 1, wherein, The obtaining the set of available edge servers in the current edge computing network architecture and collecting the basic information of each edge server in the set of available edge servers includes: Count the edge servers in the available state in the current edge computing network architecture to obtain the set of available edge servers; Obtain the number of cores, total memory resources, processing speed, required computing resources of each edge server in the set of available edge servers, and the data transmission speed between each of the edge servers, so as to obtain the second information.

4. The method for microservice deployment of the government affairs big data platform according to claim 1, wherein The collecting the basic information of each application to be processed in the set of applications to be processed includes: Obtain the topology information of the structure of each application to be processed in the set of applications to be processed to obtain topology information; the topology information includes the set of microservice traversals corresponding to a single run of each of the applications to be processed; Obtain the average arrival rate of requests of each of the applications to be processed to obtain the average arrival rate information of requests; Based on the topology information, determine the dependency relationship between each microservice in the set of microservice traversals corresponding to each application to be processed, so as to obtain the set of microservice dependency relationships corresponding to each application to be processed; Based on the topology information and the corresponding set of microservice dependency relationships, respectively collect the microservice execution data corresponding to each vertex in the corresponding vertex set during a single run of each of the applications to be processed, so as to obtain the single run information corresponding to each application to be processed.

5. The microservice deployment method for the e-government big data platform according to claim 1, wherein Performing decision variable configuration based on the first information, the second information, and the third information, and constructing a microservice deployment model using the obtained decision variables, including: Configuring decision variables based on the microservice set to be deployed and the available edge server set; Determining an objective function based on the application set to be processed and the single-run delay corresponding to each application in the third information; Determining the constraint conditions of the microservice deployment model based on the microservice set to be deployed, the available edge server set, the application set to be processed, the first information, the second information, the third information, and the decision variables, and determining the target microservice deployment model based on the constraint conditions and the objective function.

6. The method for deploying microservices in a government affairs big data platform according to any one of claims 1 to 4, characterized in that Solving the target microservice deployment model based on the genetic algorithm, the microservice set to be deployed, the first information, the available edge server set, the second information, the application set to be processed, and the third information, including: Initializing genetic parameters using the microservice set to be deployed and a genetic algorithm with an elitist strategy to obtain the population size, crossover probability, mutation probability, and iteration count threshold; After encoding the deployment status of the microservices to be deployed based on the microservice set to be deployed, the available edge server set, and the population size, generating a greedy solution and an initial solution to complete the corresponding population initialization operation, so as to obtain the initialized current population and the current population individual coding information; Determining the fitness value corresponding to each individual in the current population based on the application set to be processed, the third information, the second information, and the objective function and constraint conditions in the target microservice deployment model; Performing individual selection based on the fitness value and a roulette wheel selection strategy with an elitist reservation strategy to obtain a selection result; Determining any two microservices to be deployed on different edge servers based on the crossover probability and the selection result, and triggering a deployment location exchange operation; Based on the mutation probability, counting the microservice to be deployed with the longest execution time among the current applications to be processed, and randomly selecting a microservice from the determined target microservice set for redeployment to complete the corresponding mutation operation, and determining the current population and the current population individual coding information; Updating the current iteration count, and determining whether the current iteration count is less than the iteration count threshold to obtain an iteration count judgment result; If the iteration count judgment result indicates less than, then re-jump to the step of determining the fitness value corresponding to each individual in the current population based on the application set to be processed, the third information, the second information, and the objective function and constraint conditions in the target microservice deployment model, until the current iteration count is equal to the iteration count threshold, determining a microservice deployment decision based on the current population and the current population individual coding information, and triggering a corresponding microservice deployment operation according to the microservice deployment decision.

7. The method for deploying microservices in a government affairs big data platform according to claim 6, wherein, The corresponding population initialization operation is completed by generating a greedy solution and an initial solution, including: Based on the first expected computing resource information in the first information, the priorities of each of the to-be-deployed microservices in the to-be-deployed microservice set are weighted, and each of the to-be-deployed microservices is sorted in descending order according to the obtained microservice priority information to obtain a first sorting result; Based on the processing speed information and the second expected computing resource information in the second information, each of the edge servers in the available edge server set is sorted to obtain a second sorting result; Based on the first sorting result and the second sorting result, the configuration relationship between the microservices and the edge servers is reallocated, and the to-be-deployed microservice set and the available edge server set are updated to complete the corresponding greedy solution generation operation, and an updated first microservice set and an updated edge server set are obtained; When the first microservice set is a non-empty set, a target edge server node set is determined based on the first information and the memory constraint; Based on the target edge server node set, a server node is randomly selected for the to-be-deployed microservice with the smallest priority in the first microservice set, and a microservice set update operation is triggered to obtain an updated second microservice set; It is judged whether the second microservice set is empty, and when it is not empty, the corresponding population initialization operation is completed based on the second microservice set to obtain an initialized current population.

8. A microservice deployment device for a government affairs big data platform, characterized in that, Including: A first information determination module, configured to obtain a to-be-deployed microservice set in a government affairs big data platform, and collect basic information of each to-be-deployed microservice in the to-be-deployed microservice set to obtain first information; A second information determination module, configured to obtain an available edge server set in a current edge computing network architecture, and collect basic information of each edge server in the available edge server set to obtain second information; A third information determination module, configured to obtain a to-be-processed application set, and collect basic information of each to-be-processed application in the to-be-processed application set to obtain third information; wherein, each of the to-be-processed applications includes different combinations of to-be-deployed microservices, and the third information includes topology structure information; A deployment model construction module, configured to perform decision variable configuration based on the first information, the second information, and the third information, and construct a microservice deployment model by using the obtained decision variables to obtain a target microservice deployment model; A deployment decision determination module, configured to solve the target microservice deployment model based on a genetic algorithm, the to-be-deployed microservice set, the first information, the available edge server set, the second information, the to-be-processed application set, and the third information to determine a microservice deployment decision.

9. An electronic device, characterized in that, Including: A memory, configured to store a computer program; A processor, configured to execute the computer program to implement the government affairs big data platform microservice deployment method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, For storing a computer program which, when executed by a processor, implements the method for deploying microservices of a government affairs big data platform according to any one of claims 1 to 7.

Images