A processing unit deployment strategy verification method, device, electronic device and medium

Abstract

The present invention discloses a method, device, electronic device and medium for verifying a processing unit deployment strategy. The method includes: obtaining a processing unit deployment task, and determining a target processing unit diagram instance matching the task based on a processing unit diagram library; generating a processing unit deployment request based on the target processing unit diagram instance and the node status information of a node server cluster; sending the deployment request to a simulation server, generating a deployment strategy based on a reinforcement learning algorithm to be tested and a deployment request, generating and returning a deployment strategy file to be tested; converting the deployment strategy file into a target deployment strategy; converting the target deployment strategy into a deployment strategy instruction, and deploying the target deployment strategy; determining the response time corresponding to the deployed target deployment strategy, and determining a deployment scheme verification result based on the response time corresponding to the target deployment scheme, thereby verifying the deployment strategy generated using reinforcement learning and obtaining a more accurate response time.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a method, device, electronic device and medium for verifying a processing unit deployment strategy. Background Art

[0002] With the development of computer technology, it is necessary to deploy each processing unit in stream processing to a suitable network node server to reduce the response time of the longest path from the data source to the data consumer.

[0003] Currently, processing units are usually deployed in the nearest network node server using reinforcement learning in a simulation environment to shorten the response time as much as possible, thereby generating the optimal processing unit deployment strategy in the simulation environment.

[0004] However, it is difficult for a simulation environment to simulate fluctuations in a real network environment, and fluctuations in a real network environment have a certain impact on the response time. Therefore, the deployment strategy generated from the simulation environment may not be the optimal deployment strategy, and the deployment strategy of the simulation environment cannot be directly applied to the real network environment. Therefore, it is necessary to verify the deployment strategy generated by reinforcement learning in the simulation environment. Summary of the invention

[0005] The present invention provides a processing unit deployment strategy verification method, device, electronic device and medium to verify the deployment strategy generated by reinforcement learning and obtain a more accurate response time.

[0006] According to one aspect of the present invention, a processing unit deployment strategy verification method is provided, the method is applied to a verification server in a microservice system, and the method includes:

[0007] Obtaining a processing unit deployment task created by a user, and determining a target processing unit graph instance matching the processing unit deployment task based on a processing unit graph library;

[0008] Generate a processing unit deployment request based on the target processing unit graph instance and node status information of a node server cluster in the microservice system;

[0009] Sending the processing unit deployment request to the simulation server, so that the simulation server generates a deployment strategy based on the reinforcement learning algorithm to be tested and the processing unit deployment request, and generates and returns a deployment strategy file to be tested;

[0010] Converting the deployment strategy file into a target deployment strategy under a unified standard through a unified interface;

[0011] Converting the target deployment strategy into a deployment strategy instruction that can act on the node server cluster, and deploying the target deployment strategy by executing the deployment strategy instruction;

[0012] Determine the response time corresponding to the target deployment strategy after deployment, and determine the deployment solution verification result based on the response time corresponding to the target deployment solution.

[0013] According to another aspect of the present invention, a processing unit deployment strategy verification device is provided, the device comprising:

[0014] A deployment task acquisition module, used to acquire a processing unit deployment task created by a user, and determine a target processing unit graph instance matching the processing unit deployment task based on a processing unit graph library;

[0015] A deployment request generation module, used to generate a processing unit deployment request based on the target processing unit graph instance and the node status information of the node server cluster in the microservice system;

[0016] A deployment strategy file generation module, used to send the processing unit deployment request to the simulation server, so that the simulation server generates a deployment strategy based on the reinforcement learning algorithm to be tested and the processing unit deployment request, and generates and returns the deployment strategy file to be tested;

[0017] A deployment strategy file conversion module, used to convert the deployment strategy file into a target deployment strategy under a unified standard through a unified interface;

[0018] A policy deployment module, used to convert the target deployment policy into a deployment policy instruction that can act on the node server cluster, and deploy the target deployment policy by executing the deployment policy instruction;

[0019] The solution verification module is used to determine the response time corresponding to the target deployment strategy after deployment, and determine the deployment solution verification result based on the response time corresponding to the target deployment solution.

[0020] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0021] at least one processor; and

[0022] a memory communicatively connected to the at least one processor; wherein,

[0023] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the processing unit deployment strategy verification method described in any embodiment of the present invention.

[0024] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the processing unit deployment strategy verification method described in any embodiment of the present invention when executed.

[0025] The technical solution of the embodiment of the present invention obtains the processing unit deployment task created by the user, and determines the target processing unit graph instance matching the processing unit deployment task based on the processing unit graph library; generates a processing unit deployment request based on the target processing unit graph instance and the node status information of the node server cluster in the microservice system; sends the processing unit deployment request to the simulation server, so that the simulation server generates a deployment strategy based on the reinforcement learning algorithm to be tested and the processing unit deployment request, and generates and returns the deployment strategy file to be tested; converts the deployment strategy file into a target deployment strategy under a unified standard through a unified interface; converts the target deployment strategy into a deployment strategy instruction that can act on the node server cluster, and deploys the target deployment strategy by executing the deployment strategy instruction; determines the response time corresponding to the target deployment strategy after deployment, and determines the deployment scheme verification result based on the response time corresponding to the target deployment scheme, thereby realizing the verification of the deployment strategy generated by reinforcement learning and obtaining a more accurate response time.

[0026] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0028] Figure 1 is a flowchart of a processing unit deployment strategy verification method provided according to Embodiment 1 of the present invention;

[0029] Figure 2 This is an example diagram of a node server and K8S interface connection method involved in Embodiment 1 of the present invention;

[0030] Figure 3 This is an example diagram of deploying a processing unit in a simulation environment involved in Embodiment 1 of the present invention;

[0031] Figure 4 is a flowchart of another processing unit deployment strategy verification method provided according to Embodiment 2 of the present invention;

[0032] Figure 5 It is a structural diagram of a processing unit deployment strategy verification device provided according to Embodiment 3 of the present invention;

[0033] Figure 6 It is a structural diagram of an electronic device that implements the processing unit deployment strategy verification method of an embodiment of the present invention. DETAILED DESCRIPTION

[0034] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0035] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0036] Embodiment 1

[0037] Figure 1 A flowchart of a method for verifying a deployment strategy of a processing unit is provided for the first embodiment of the present invention. This embodiment is applicable to the case where a deployment strategy generated by reinforcement learning is verified. The method can be executed by a processing unit deployment strategy verification device. The processing unit deployment strategy verification device can be implemented in the form of hardware and / or software. The processing unit deployment strategy verification device can be configured in an electronic device. The method is applied to a verification server in a microservice system, such as Figure 1 As shown, the method includes:

[0038] S110, obtaining a processing unit deployment task created by a user, and determining a target processing unit graph instance matching the processing unit deployment task based on a processing unit graph library.

[0039] The processing unit may refer to a processing unit in a stream processing technology. The processing unit may process an input data stream based on a preset function and output a filtered data stream. The processing unit deployment task may refer to a task of how to deploy at least one processing unit in a container in a node server. A container is a minimum platform that can be used to deploy tasks. Exemplarily, a microservice system is a containerized management system centrally deployed in a cluster; Figure 2 This diagram shows an example of how to connect a node server to a K8S interface. Figure 2 , the node servers with processing units deployed in the node server cluster communicate data through the container management K8S interface; multiple containers in the same node server cannot communicate directly, and communication needs to be achieved through the interfaces between the node servers. For example, when processing unit 1 needs to be deployed to node server v1, processing unit 1 can be transferred to node server v0 through the K8S interface, so that node server v0 generates a container in which the processing unit can be deployed. A processing unit graph may refer to a processing unit graph in which multiple processing units are arranged in a preset arrangement. A processing unit graph instance may refer to a processing unit graph instance in which at least one processing unit graph is arranged in a preset arrangement. A processing unit graph library may refer to a candidate pool containing multiple processing unit graph instances, which is used to provide a processing unit graph instance that matches each processing unit deployment task. A target processing unit graph instance may refer to a processing unit graph instance that implements a corresponding processing unit deployment task.

[0040] Specifically, the user creates a processing unit deployment task based on a certain problem in the main server. The main server can detect and select a target processing unit diagram example matching the processing unit deployment task in the processing unit diagram library according to the processing unit deployment task created by the user.

[0041] For example, a user may raise the question of how to determine whether a car has had a car accident, and based on this question, create a processing unit deployment task in the main server. The main server may detect and select a processing unit graph instance corresponding to the car accident condition in the processing unit graph library according to the processing unit deployment task created by the user, and use the processing unit graph instance as the target processing unit graph instance that matches the processing unit deployment task.

[0042] S120: Generate a processing unit deployment request based on the target processing unit graph instance and the node status information of the node server cluster in the microservice system.

[0043] The node status information may refer to the status information of the node server, which is used to characterize the deployment resources of the corresponding node server. For example, the node status information may be the storage status of the node server, etc. The processing unit deployment request may refer to a request to deploy a target processing unit graph instance to at least one node server of the node server cluster.

[0044] Specifically, the target processing unit graph instance can be split into multiple target processing units, and the data flow direction between the original processing units is retained. Based on the multiple split target processing units, the deployment resources required for each target processing unit are determined. The deployment resources of each node server can be determined based on the node status information of the node server cluster in the microservice system. Based on the deployment resources required by the target processing unit graph instance and the deployment resources of each node server, the node server that can be used to deploy the target processing unit is determined, and a processing unit deployment request is generated.

[0045] Exemplarily, S120 may include: determining a target node server in the node server cluster where the processing unit can be deployed based on node status information of the node server cluster in the microservice system; and generating a processing unit deployment request based on the target node server and the target processing unit graph instance.

[0046] The target node server may refer to a node server that has sufficient deployment resources and can be used to deploy the target processing unit.

[0047] Specifically, the target processing unit graph instance can be split into multiple target processing units, and the data flow direction between the original processing units is retained. Based on the multiple split target processing units, the deployment resources required for each target processing unit are determined. The deployment resources of each node server can be determined based on the node status information of the node server cluster in the microservice system. Based on the deployment resources required by the target processing unit graph instance and the deployment resources of each node server, the target node server in the node server cluster where the processing unit can be deployed is determined, and a processing unit deployment request is generated based on the target node server and the target processing unit graph instance, thereby ensuring that the processing unit can be deployed in a node server with sufficient deployment resources, avoiding deployment failures due to insufficient node server deployment resources, and improving the accuracy of processing unit deployment.

[0048] S130: Send the processing unit deployment request to the simulation server, so that the simulation server generates a deployment strategy based on the reinforcement learning algorithm to be tested and the processing unit deployment request, and generates and returns a deployment strategy file to be tested.

[0049] Among them, reinforcement learning (RL), also known as reinforcement learning, evaluation learning or enhanced learning, is one of the paradigms and methodologies of machine learning. Reinforcement learning is used to describe and solve the problem of an agent learning a strategy to maximize rewards or achieve a specific goal during its interaction with the environment. The reinforcement learning algorithm to be tested may include at least one reinforcement learning algorithm. The deployment strategy may refer to a strategy for how to deploy a processing unit in a node server. Each reinforcement learning algorithm corresponds to at least one deployment strategy. A deployment strategy file may refer to a file that records deployment strategies. The deployment strategy file to be tested may refer to a file that records the shortest response time corresponding to each deployment strategy. The response time may refer to the time taken from the start of execution of a deployment strategy to the completion of execution of the strategy.

[0050] Specifically, a processing unit deployment request is sent to a simulation server so that the simulation service can select a target processing unit graph instance and a node server that can be used to deploy the target processing unit in the simulation server based on the received processing unit deployment request; based on the reinforcement learning algorithm to be tested, the target processing unit is deployed in a container in an available node server, and multiple deployment strategies are generated. Among them, a container can only store one processing unit. Each deployment strategy is executed in turn in the simulation environment in the simulation server, and each deployment strategy and the corresponding response time are recorded. A preset number of deployment strategies with a shorter response time corresponding to each reinforcement learning algorithm are used as deployment strategy files to be tested, and the determined deployment strategy files to be tested are returned to the main server.

[0051] Figure 3 An example diagram of deploying a processing unit in a simulation environment is given. Figure 3, the agent deploys all processing units to the node server according to the deployment strategy; after all processing units are deployed, the agent starts to process the response and records the environment and response time in the deployment decision maker; rewards are processed according to the response time; the principle of reward can refer to but is not limited to the shorter the response time, the more rewards are obtained. All deployment strategies are deployed and executed in sequence until the last deployment strategy is executed. The deployment strategy with the shortest response time corresponding to each algorithm, that is, the most reward, can be used as the deployment strategy file to be tested and returned. For example, the processing units can be deployed in edge servers v0, v1, and v2 respectively; based on the data flow in the processing unit graph instance, the agent can input the data to be processed from edge server v0 and edge server v2, and process the data in edge server v0 and edge server v2 respectively; after the data processing is completed, the data processed by edge server v0 is transmitted to edge server v1; after the data processing is completed, the data processed by edge server v2 is transmitted to edge server v1; wherein, each edge server can transmit the data after completing the data processing, without waiting for the other edge servers to complete the processing and transmit together. The edge server v1 processes the received data in the order in which it is received, and sends a feedback signal to the agent after all the data has been processed; the feedback signal indicates that the data has been processed. At this time, the agent can calculate the response time corresponding to the deployment strategy based on the first data input time and the received feedback signal time.

[0052] Exemplarily, "the simulation server generates a deployment strategy based on the reinforcement learning algorithm to be tested and the processing unit deployment request, and generates and returns the deployment strategy file to be tested" in S130 may include: based on the reinforcement learning algorithm to be tested and the processing unit deployment request, performing reinforcement learning training on the intelligent agent model, obtaining the deployment strategy file with the shortest response time and returning it as the deployment strategy file to be tested.

[0053] Among them, the intelligent agent model can be used for reinforcement learning training.

[0054] Specifically, the simulation server can select the target processing unit diagram instance and the node server that can be used to deploy the target processing unit in the simulation server based on the received processing unit deployment request; based on the reinforcement learning algorithm to be tested, the target processing unit is deployed in the container in the available node server to generate multiple deployment strategies. Each deployment strategy is executed in turn in the simulation environment in the simulation server, and each deployment strategy and the corresponding response time are recorded in the corresponding deployment strategy file. The deployment strategy with the shortest response time corresponding to each reinforcement learning algorithm is selected, and the deployment strategy file corresponding to the selected deployment strategy is used as the deployment strategy file to be tested; the determined deployment strategy file to be tested is returned to the main server, thereby providing a data basis for the subsequent verification of the deployment strategy with the shortest response time in the simulation environment, and the time required to determine the deployment strategy with the shortest response time can be shortened through reinforcement learning, thereby improving the efficiency of data selection.

[0055] S140. Convert the deployment strategy file into a target deployment strategy under a unified standard through a unified interface.

[0056] The target deployment strategy may refer to a standard unified deployment strategy.

[0057] Specifically, the main server converts the returned deployment strategy file through a unified interface to obtain a standard unified target deployment strategy.

[0058] S150: Convert the target deployment policy into a deployment policy instruction that can act on the node server cluster, and deploy the target deployment policy by executing the deployment policy instruction.

[0059] Among them, the deployment policy instructions may refer to instructions that can be read and executed by the microservice system.

[0060] Specifically, based on the K8S interface, the target deployment strategy can be converted into a script or program that can act on the node server cluster, and the converted script or program can be used as a deployment strategy instruction. Among them, the deployment strategy instruction can include a specific deployment method of the target deployment strategy. The target deployment strategy is deployed by executing the deployment strategy instruction, and the processing unit to be deployed is deployed in the container in the node server through the K8S interface, and a node server cluster with processing units deployed based on the target deployment strategy is obtained.

[0061] S160: Determine the response time corresponding to the target deployment strategy after deployment, and determine the deployment solution verification result based on the response time corresponding to the target deployment solution.

[0062] The verification result may refer to the result of comparing the response times corresponding to multiple target deployment solutions.

[0063] Specifically, for each target deployment strategy, the input data is packaged and processed; the packaged data to be processed is sent to the first processing unit so that the first processing unit processes it and generates processed data. The processed data is sent to the next processing unit and data processing is performed until the last processing unit is executed. The time required for all processing units to complete execution in a preset order is recorded, and the recorded required time is used as the response time of the corresponding target deployment strategy in a real network environment. After determining the response time of all target deployment strategies, all response times can be compared. Among them, the verification method can be to compare the response time of each target deployment strategy in a simulation environment with the response time in a real network environment; if the difference between the two is within a preset error range, it can be determined that the deployment strategy obtained by reinforcement learning in the simulation environment is an accurate deployment strategy.

[0064] Exemplarily, "determining the response time corresponding to the target deployment strategy after deployment" in S160 may include: obtaining a data processing request from a data source, and sending a data processing request and a flag data packet to a first node server where a first processing unit is deployed, so that the first node server responds to the data processing request, and sends the response result and the identification data packet to a second node server where a second processing unit is deployed, and processes and sends them in sequence until the last node server where the last processing unit is deployed completes the response, and then sends the final response result and the identification data packet to the consumer end; based on the first sending time of the identification data packet and the receiving time of the consumer end, determine the response time corresponding to the target deployment strategy after deployment.

[0065] The data source may refer to a database or a database server used by a database application. The first processing unit deployment may refer to a processing unit at the most upstream position in a processing unit graph instance. The second processing unit may refer to a processing unit that is downstream of the first processing unit in a processing unit graph instance and has no other processing units between the first processing unit and the first processing unit. The marker data packet may be used to distinguish different data packets to avoid using two data packets as the basis for calculating the response time.

[0066] Specifically, a data processing request is obtained from a data source, and the data generated in the data source is packaged and processed to generate a marker data packet to be marked; a data processing request and a marker data packet are sent to the first node server where the first processing unit is deployed and the first time is recorded, so that the first node server responds to the data processing request, and based on the data flow direction and the preset transmission rate in the processing unit diagram instance, the response result and the identification data packet are sent to the second node server where the second processing unit is deployed downstream through the K8S interface, and the sending is processed in sequence until the last node server where the last processing unit is deployed responds, and the final response result and identification data packet are sent to the consumer end and the second time is recorded; the first time of the identification data packet, i.e., the first sending time and the second time, i.e., the receiving time of the consumer end, are obtained, and the difference between the second time and the first time is used as the response duration corresponding to the target deployment strategy after deployment, thereby realizing the execution of the deployment strategy in a real network environment, and more accurately determining the response duration, providing an accurate verification data basis for subsequent deployment plan verification.

[0067] It should be noted that the deployment strategy deploys the corresponding processing unit in the Raspberry Pi K8S cluster. These deployment strategies are generated by different reinforcement algorithms, so it is necessary to provide algorithm configuration functions. In addition, the system can implement variable processing units and network settings.

[0068] The technical solution of the embodiment of the present invention obtains the processing unit deployment task created by the user, and determines the target processing unit graph instance matching the processing unit deployment task based on the processing unit graph library; generates a processing unit deployment request based on the target processing unit graph instance and the node status information of the node server cluster in the microservice system; sends the processing unit deployment request to the simulation server, so that the simulation server generates a deployment strategy based on the reinforcement learning algorithm to be tested and the processing unit deployment request, and generates and returns the deployment strategy file to be tested; converts the deployment strategy file into a target deployment strategy under a unified standard through a unified interface; converts the target deployment strategy into a deployment strategy instruction that can act on the node server cluster, and deploys the target deployment strategy by executing the deployment strategy instruction; determines the response time corresponding to the deployed target deployment strategy, and determines the deployment plan verification result based on the response time corresponding to the target deployment plan, thereby verifying the deployment strategy generated by reinforcement learning and obtaining a more accurate response time.

[0069] Embodiment 2

[0070] Figure 4This is a flowchart of a method for verifying a deployment strategy of a processing unit provided in the second embodiment of the present invention. This embodiment describes in detail the way to process an existing deployment strategy based on the above embodiment. The explanations of the terms that are the same or corresponding to the above disclosed embodiments are not repeated here. Figure 4 As shown, the method includes:

[0071] S210: Acquire a processing unit deployment task created by a user, and determine a target processing unit graph instance that matches the processing unit deployment task based on a processing unit graph library.

[0072] S220: Generate a processing unit deployment request based on the target processing unit graph instance and the node status information of the node server cluster in the microservice system.

[0073] S230: Send the processing unit deployment request to the simulation server, so that the simulation server generates a deployment strategy based on the reinforcement learning algorithm to be tested and the processing unit deployment request, and generates and returns a deployment strategy file to be tested.

[0074] S240. Convert the deployment strategy file into a target deployment strategy under a unified standard through a unified interface.

[0075] S250: Convert the target deployment policy into a deployment policy instruction that can act on the node server cluster, and deploy the target deployment policy by executing the deployment policy instruction.

[0076] S260: Determine the response time corresponding to the target deployment strategy after deployment.

[0077] S270: Determine an existing deployment policy file based on an existing deployment algorithm and a processing unit deployment request.

[0078] The existing deployment algorithm may refer to a preset baseline algorithm. The existing deployment algorithm is a non-reinforcement learning algorithm. For example, the existing deployment algorithm may include but is not limited to an incremental deployment algorithm and a load balancing algorithm. The existing deployment strategy file may refer to a file that records the shortest response time corresponding to each existing deployment strategy.

[0079] Specifically, the existing deployment algorithm can be pre-stored in an existing algorithm library. When a processing unit deployment request is received, the existing deployment algorithm can be selected from the existing algorithm library, and the existing deployment strategy file can be determined based on the existing deployment algorithm and the processing unit deployment request. If the selected existing deployment algorithm has a corresponding existing response time, the deployment solution verification result can be directly determined based on the existing response time and the response time corresponding to the target deployment strategy.

[0080] S280. Convert the existing deployment strategy file into an existing deployment strategy under a unified standard through a unified interface.

[0081] The existing deployment strategy may refer to a standard unified deployment strategy.

[0082] Specifically, the main server converts the existing deployment policy file through a unified interface to obtain a standard unified existing deployment policy.

[0083] S290: Convert the existing deployment policy into an existing deployment policy instruction that can act on the node server cluster, and deploy the existing deployment policy by executing the existing deployment policy instruction.

[0084] Among them, the existing deployment policy instructions may refer to instructions that can be read and executed by the microservice system.

[0085] Specifically, based on the K8S interface, the existing deployment strategy can be converted into a script or program that can act on the node server cluster, and the converted script or program can be used as an existing deployment strategy instruction. Among them, the existing deployment strategy instruction can include a specific deployment method of the existing deployment strategy. The existing deployment strategy is deployed by executing the existing deployment strategy instruction, and the processing unit to be deployed is deployed in the container in the node server through the K8S interface, and a node server cluster with processing units deployed based on the existing deployment strategy is obtained.

[0086] S291. Determine the response time corresponding to the existing deployment strategy after deployment, and store the response time.

[0087] Specifically, for each existing deployment strategy, the input data is packaged and processed; the packaged data to be processed is sent to the first processing unit so that the first processing unit processes it and generates processed data. The processed data is sent to the next processing unit and data processing is performed until the last processing unit is executed. The time required for all processing units to complete execution in a preset order is recorded, and the recorded time required is used as the response time of the corresponding existing deployment strategy in the actual network environment. After determining the response time of all existing deployment strategies, all response times can be stored for direct call next time, thereby improving the efficiency of deployment plan verification.

[0088] S292. Determine a deployment plan verification result based on the response time corresponding to the target deployment plan and the response time corresponding to the existing deployment strategy.

[0089] Specifically, the response time corresponding to the target deployment scheme and the response time corresponding to the existing deployment strategy can be compared, and the response time corresponding to the target deployment scheme that is shorter than the response time corresponding to the existing deployment strategy can be retained; the response time of each retained target deployment strategy in the simulation environment can be compared with the response time in the real network environment; if the difference between the two is within the preset error range, it can be determined that the deployment strategy obtained by reinforcement learning in the simulation environment is an accurate deployment strategy. The deployment scheme verification results can be displayed in a visual way such as a bar chart, color highlighting, and highlighting the optimal solution, so that users can see the deployment scheme verification results more subjectively.

[0090] Exemplarily, "determining the deployment plan verification result based on the response time corresponding to the target deployment plan" in S292 may include: comparing and verifying the response time corresponding to the target deployment plan with the response time corresponding to the existing deployment strategy, and outputting the deployment plan with the shortest response time and the corresponding response time; or, comparing the response time corresponding to the target deployment plan and the response time corresponding to the existing deployment strategy with a response time threshold, and outputting the deployment plan and the corresponding response time that are less than or equal to the response time threshold.

[0091] The response time threshold may refer to a response time value preset based on business requirements.

[0092] Specifically, the response time corresponding to the target deployment scheme is compared and verified with the response time corresponding to the existing deployment strategy, the deployment scheme with the shortest response time is determined, and the deployment scheme with the shortest response time and the corresponding response time are output. Optionally, the response time corresponding to the target deployment scheme and the response time corresponding to the existing deployment strategy can be compared with the response time threshold, the target deployment scheme and the existing deployment scheme that are less than or equal to the response time threshold are retained, and the deployment schemes that are less than or equal to the response time threshold and the corresponding response time are output; or the retained target deployment scheme and the existing deployment scheme are compared and verified again, the deployment scheme with the shortest response time is determined, and the deployment scheme with the shortest response time and the corresponding response time are output.

[0093] The technical solution of the embodiment of the present invention compares and verifies the response time corresponding to the existing deployment strategy with the response time corresponding to the target deployment plan, thereby more comprehensively verifying the target deployment plan and further obtaining a more accurate response time; and stores the response time corresponding to the existing deployment strategy after the determined deployment so that it can be directly called next time, thereby improving the efficiency of deployment plan verification.

[0094] The following is an embodiment of a processing unit deployment strategy verification device provided in an embodiment of the present invention. The device and the processing unit deployment strategy verification method of the above-mentioned embodiments belong to the same inventive concept. For details not described in detail in the embodiment of the processing unit deployment strategy verification device, please refer to the embodiment of the above-mentioned processing unit deployment strategy verification method.

[0095] Embodiment 3

[0096] Figure 5 This is a schematic diagram of the structure of a processing unit deployment strategy verification device provided by Embodiment 3 of the present invention. Figure 5 As shown, the device includes: a deployment task acquisition module 310, a deployment request generation module 320, a deployment policy file generation module 330, a deployment policy file conversion module 340, a policy deployment module 350 and a solution verification module 360.

[0097] Among them, the deployment task acquisition module 310 is used to obtain the processing unit deployment task created by the user, and determine the target processing unit graph instance that matches the processing unit deployment task based on the processing unit graph library; the deployment request generation module 320 is used to generate a processing unit deployment request based on the target processing unit graph instance and the node status information of the node server cluster in the microservice system; the deployment strategy file generation module 330 is used to send the processing unit deployment request to the simulation server, so that the simulation server generates a deployment strategy based on the reinforcement learning algorithm to be tested and the processing unit deployment request, and generates and returns the deployment strategy file to be tested; the deployment strategy file conversion module 340 is used to convert the deployment strategy file into a target deployment strategy under a unified standard through a unified interface; the strategy deployment module 350 is used to convert the target deployment strategy into a deployment strategy instruction that can act on the node server cluster, and deploy the target deployment strategy by executing the deployment strategy instruction; the solution verification module 360 ​​is used to determine the response time corresponding to the target deployment strategy after deployment, and determine the deployment solution verification result based on the response time corresponding to the target deployment solution.

[0098] The technical solution of the embodiment of the present invention obtains the processing unit deployment task created by the user, and determines the target processing unit graph instance matching the processing unit deployment task based on the processing unit graph library; generates a processing unit deployment request based on the target processing unit graph instance and the node status information of the node server cluster in the microservice system; sends the processing unit deployment request to the simulation server, so that the simulation server generates a deployment strategy based on the reinforcement learning algorithm to be tested and the processing unit deployment request, and generates and returns the deployment strategy file to be tested; converts the deployment strategy file into a target deployment strategy under a unified standard through a unified interface; converts the target deployment strategy into a deployment strategy instruction that can act on the node server cluster, and deploys the target deployment strategy by executing the deployment strategy instruction; determines the response time corresponding to the deployed target deployment strategy, and determines the deployment plan verification result based on the response time corresponding to the target deployment plan, thereby verifying the deployment strategy generated by reinforcement learning and obtaining a more accurate response time.

[0099] Optionally, the deployment request generation module 320 is specifically used to: determine the target node server in the node server cluster where the processing unit can be deployed based on the node status information of the node server cluster in the microservice system; and generate a processing unit deployment request based on the target node server and the target processing unit graph instance.

[0100] Optionally, the deployment strategy file generation module 330 is specifically used to: perform reinforcement learning training on the intelligent agent model based on the reinforcement learning algorithm to be tested and the processing unit deployment request, and obtain the deployment strategy file with the shortest response time as the deployment strategy file to be tested and return it.

[0101] Optionally, the solution verification module 360 ​​may include:

[0102] The data packet transmission submodule is used to obtain the data processing request from the data source, and send the data processing request and the identification data packet to the first node server where the first processing unit is deployed, so that the first node server responds to the data processing request, and sends the response result and the identification data packet to the second node server where the second processing unit is deployed, and processes and sends them in sequence until the last node server where the last processing unit is deployed completes the response, and then sends the final response result and the identification data packet to the consumer end;

[0103] The timing submodule is used to determine the response time corresponding to the target deployment strategy after deployment based on the first sending time of the identification data packet and the receiving time of the consumer end.

[0104] Optionally, the microservice system is a containerized management system centrally deployed in a cluster; data communication is performed between node servers with processing units deployed in the node server cluster through a container management K8S interface.

[0105] Optionally, the device further comprises:

[0106] An existing deployment strategy file generation module, used to determine an existing deployment strategy file based on an existing deployment algorithm and a processing unit deployment request;

[0107] An existing deployment strategy file conversion module is used to convert the existing deployment strategy file into an existing deployment strategy under a unified standard through a unified interface;

[0108] An existing policy deployment module is used to convert an existing deployment policy into an existing deployment policy instruction that can act on a node server cluster, and deploy the existing deployment policy by executing the existing deployment policy instruction;

[0109] The timing module is used to determine the response time corresponding to the existing deployment strategy after deployment and store the response time.

[0110] Optionally, the solution verification module 360 ​​is specifically used to: compare and verify the response time corresponding to the target deployment solution with the response time corresponding to the existing deployment strategy, and output the deployment solution with the shortest response time and the corresponding response time; or, compare the response time corresponding to the target deployment solution and the response time corresponding to the existing deployment strategy with a response time threshold, and output the deployment solution and the corresponding response time that are less than or equal to the response time threshold.

[0111] The processing unit deployment strategy verification device provided in the embodiment of the present invention can execute the processing unit deployment strategy verification method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0112] It is worth noting that in the embodiment of the above-mentioned processing unit deployment strategy verification device, the various modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the various functional modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.

[0113] Embodiment 4

[0114] Figure 6A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0115] like Figure 6 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0116] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0117] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as a processing unit deployment strategy verification method.

[0118] In some embodiments, the processing unit deployment strategy verification method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the processing unit deployment strategy verification method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to execute the processing unit deployment strategy verification method in any other appropriate manner (e.g., by means of firmware).

[0119] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0120] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0121] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0122] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0123] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0124] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0125] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0126] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for verifying a processing unit deployment strategy, characterized in that: The verification server used in the microservice system includes: Obtaining a processing unit deployment task created by a user, and determining a target processing unit graph instance matching the processing unit deployment task based on a processing unit graph library; Generate a processing unit deployment request based on the target processing unit graph instance and node status information of a node server cluster in the microservice system; Sending the processing unit deployment request to the simulation server, so that the simulation server generates a deployment strategy based on the reinforcement learning algorithm to be tested and the processing unit deployment request, and generates and returns a deployment strategy file to be tested; Converting the deployment strategy file into a target deployment strategy under a unified standard through a unified interface; Converting the target deployment strategy into a deployment strategy instruction that can act on the node server cluster, and deploying the target deployment strategy by executing the deployment strategy instruction; Determine the response time corresponding to the target deployment strategy after deployment, and determine the deployment solution verification result based on the response time corresponding to the target deployment solution.

2. The method according to claim 1, characterized in that The generating a processing unit deployment request based on the target processing unit graph instance and the node status information of the node server cluster in the microservice system includes: Determine, based on the node status information of the node server cluster in the microservice system, a target node server in the node server cluster where a processing unit can be deployed; The processing unit deployment request is generated based on the target node server and the target processing unit map instance.

3. The method according to claim 1, characterized in that The simulation server generates a deployment strategy based on the reinforcement learning algorithm to be tested and the processing unit deployment request, and generates and returns a deployment strategy file to be tested, including: Based on the reinforcement learning algorithm to be tested and the processing unit deployment request, reinforcement learning training is performed on the agent model, and a deployment strategy file with the shortest response time is obtained and returned as the deployment strategy file to be tested.

4. The method according to claim 1, characterized in that: Determining the response time corresponding to the target deployment strategy after deployment includes: Obtain a data processing request from a data source, and send the data processing request and an identification data packet to a first node server where a first processing unit is deployed, so that the first node server responds to the data processing request, and sends the response result and the identification data packet to a second node server where a second processing unit is deployed, and processes and sends them in sequence until the last node server where a last processing unit is deployed completes the response, and then sends the final response result and the identification data packet to a consumer end; Based on the first sending time of the identification data packet and the receiving time of the consumer end, the response duration corresponding to the target deployment strategy after deployment is determined.

5. The method according to claim 4, characterized in that The microservice system is a containerized management system centrally deployed in a cluster; the node servers with processing units deployed in the node server cluster communicate data through the container management K8S interface.

6. The method according to claim 1, characterized in that The method further comprises: Determining an existing deployment strategy file based on an existing deployment algorithm and the processing unit deployment request; Converting the existing deployment strategy file into an existing deployment strategy under a unified standard through a unified interface; Converting the existing deployment strategy into an existing deployment strategy instruction that can act on the node server cluster, and deploying the existing deployment strategy by executing the existing deployment strategy instruction; Determine the response time corresponding to the existing deployment strategy after deployment, and store the response time.

7. The method according to claim 6, characterized in that Determining a deployment solution verification result based on a response duration corresponding to the target deployment solution includes: Compare and verify the response time corresponding to the target deployment solution with the response time corresponding to the existing deployment strategy, and output the deployment solution with the shortest response time and the corresponding response time; or, The response time corresponding to the target deployment scheme and the response time corresponding to the existing deployment strategy are compared with the response time threshold, and the deployment scheme and the corresponding response time that are less than or equal to the response time threshold are output.

8. A processing unit deployment strategy verification device, characterized in that: include: A deployment task acquisition module, used to acquire a processing unit deployment task created by a user, and determine a target processing unit graph instance matching the processing unit deployment task based on a processing unit graph library; A deployment request generation module, used to generate a processing unit deployment request based on the target processing unit graph instance and the node status information of the node server cluster in the microservice system; A deployment strategy file generation module, used to send the processing unit deployment request to the simulation server, so that the simulation server generates a deployment strategy based on the reinforcement learning algorithm to be tested and the processing unit deployment request, and generates and returns the deployment strategy file to be tested; A deployment strategy file conversion module, used to convert the deployment strategy file into a target deployment strategy under a unified standard through a unified interface; A policy deployment module, used to convert the target deployment policy into a deployment policy instruction that can act on the node server cluster, and deploy the target deployment policy by executing the deployment policy instruction; The solution verification module is used to determine the response time corresponding to the target deployment strategy after deployment, and determine the deployment solution verification result based on the response time corresponding to the target deployment solution.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the processing unit deployment strategy verification method described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the processing unit deployment strategy verification method described in any one of claims 1 to 7 when executed.

Images