Computing power scheduling method, device, electronic device and storage medium

Abstract

The present application discloses a computing power scheduling method, device, electronic device and storage medium, constructs a computing network chain scheduling model, converts the computing network chain scheduling model into a Markov decision model, constructs a target state reward function based on the total weighted end-to-end delay, and then uses the training sample business to train and update the intelligent agent of the Markov decision model to obtain a trained intelligent agent; the current business is input into the trained intelligent agent to obtain a computing power scheduling decision for the current business. The present application expresses the scheduling problem as a multiple Markov decision process, realizes automatic scheduling of computing power according to the business, automatically allocates the business to the data center and the server, and deploys the computing network chain function package to the server so that the server can execute the business. In addition, since the goal of the computing network chain scheduling model is to minimize the total weighted end-to-end delay, it can reduce the end-to-end delay when processing the business.

Description

Technical Field

[0001] The present application relates to the field of computing power scheduling technology, and in particular to a computing power scheduling method, device, electronic device and storage medium. Background Art

[0002] With the development of information networks, remote communication technologies such as video, monitoring, and control have garnered widespread attention from academia and industry, making remote surgery a reality. Remote surgery can overcome spatial limitations. Leveraging information technology, it enables patients in economically underdeveloped areas to receive surgical treatment from specialized physicians in developed regions. This prevents patients from missing optimal treatment times due to transportation, and allows high-quality medical resources to be expanded from developed to remote areas at a lower cost. However, cross-domain transmission and processing of various types of traffic present challenges in meeting stringent end-to-end latency requirements, which can cause surgical delays and severely impact the effectiveness and success rate of surgery. To reduce information latency during remote surgery, it is necessary to leverage medical information networks and establish a collaborative service model served by multiple data centers in different regions. This will facilitate cross-regional information transmission and processing, providing coordination and support for surgical operations.

[0003] However, existing medical information networks can only guarantee the communication and computing delays of telemedicine services within a single region, but cannot guarantee the delays of cross-domain collaborative medical services. In particular, the delay requirements for cross-domain remote surgery are more stringent than others, and network technology must be introduced to reduce cross-domain end-to-end delays. Summary of the Invention

[0004] The present application aims to solve at least one of the technical problems existing in the prior art. To this end, the present application proposes a computing power scheduling method, device, electronic device, and storage medium that can realize automatic computing power scheduling and reduce end-to-end latency.

[0005] To achieve the above-mentioned objectives, a first embodiment of the present application provides a computing power scheduling method, which is applied to a computing network chain processing module of a multi-region business processing system, wherein the computing network chain module is provided with multiple computing network chain function packages, each of which is provided with multiple functions for executing business; the multi-region business processing system further includes multiple regions, each of the regions includes multiple data centers, and each of the data centers includes multiple servers;

[0006] The method comprises:

[0007] A computing network chain scheduling model is constructed with the goal of minimizing the total weighted end-to-end delay. The total weighted end-to-end delay is expressed as:

[0008]

[0009] Among them, T Prepresents the total weighted end-to-end delay; ω p Indicates the priority of the p-th service; D p represents the end-to-end delay of the p-th service;

[0010] Converting the computing network chain scheduling model into a Markov decision model, and constructing a target state reward function based on the total weighted end-to-end delay; the actions of the Markov decision model include computing network chain function package actions, data center actions, and server actions;

[0011] Acquire training sample services, and determine, based on the training sample services, a data center action space of the data center action, a server action space of the server action, and a computing network chain function package action space of the computing network chain function package action;

[0012] The intelligent agent of the Markov decision model selects and executes a target data center action from the data center action space, selects and executes a target server action from the server action space, and selects and executes a target computing network chain function package action from the computing network chain function package action space to complete the training sample business, thereby changing the global state of the Markov decision model;

[0013] Calculating a state reward when changing the global state through the target state reward function, and training and updating the agent based on the state reward to obtain a trained agent;

[0014] The current business is input into the trained intelligent agent to obtain a computing power scheduling decision for the current business.

[0015] According to some embodiments of the present application, the agent includes a computing network chain function package actor network, a data center actor network, and a server actor network;

[0016] The intelligent agent of the Markov decision model selects and executes a target data center action from the data center action space, selects and executes a target server action from the server action space, and selects and executes a target computing network chain function package action from the computing network chain function package action space, including:

[0017] Calculate a first probability distribution of the data center in the data center action space, calculate a second probability distribution of the server action in the server action space, and calculate a third probability distribution of the computing network chain function package action in the computing network chain package action space;

[0018] The target data center action is selected and executed from the data center action space based on the first probability distribution through the data center actor network; the target server action is selected and executed from the server action space based on the second probability distribution through the server actor network; the target computing network chain function package action is selected and executed from the computing network chain function package action space based on the third probability through the computing network chain function package actor network.

[0019] According to some embodiments of the present application, the global state includes a server state and a computing network chain function package state;

[0020] Calculating a first probability distribution of the data center in the data center action space, calculating a second probability distribution of the server action in the server action space, and calculating a third probability distribution of the computing network chain function package action in the computing network chain package action space include:

[0021] Encoding the state of the computing network chain function package using a graph isomorphism network to obtain a first encoding vector;

[0022] Encoding the server state using a fully connected layer to obtain a second encoding vector;

[0023] Based on the first encoding vector, the second encoding vector, and the value function, calculate a first action state reward for each data center action, calculate a second action reward for each server action, and calculate a third action reward for each computing network chain function package action;

[0024] The first probability distribution is calculated based on the first action reward; the second probability distribution is calculated based on the second action reward; and the third probability distribution is calculated based on the third action reward.

[0025] According to some embodiments of the present application, the calculation formula of the first probability distribution is:

[0026]

[0027] in, Characterize the probability of the yth data center action, the first action reward representing the yth data center action; The first action reward representing the f-th data center action;

[0028] The calculation formula of the second probability distribution is expressed as:

[0029]

[0030] in, Characterize the probability of the jth server action, the second action reward representing the jth server action; the second action reward representing the mth server action;

[0031] The calculation formula of the third probability distribution is:

[0032]

[0033] in, Characterizes the probability of the i-th computing network chain function package action, The third action reward representing the vth action of the computing network chain function package; The third action reward representing the i-th action of the computing network chain function package.

[0034] According to some embodiments of the present application, the agent further includes a critic network;

[0035] The step of calculating a state reward when changing the global state by using the target state reward function, and training and updating the agent based on the state reward to obtain a trained agent includes:

[0036] constructing an estimator based on the state reward and the state-value function of the critic network;

[0037] An advantage function is constructed based on the estimator, and the advantage function is:

[0038]

[0039]

[0040]

[0041] in, is the clipping objective function, is the entropy objective function, is the estimator; is the parameter probability ratio; η is the clipping parameter; is the expectation; clip is the clipping function; entropy is the entropy function; ρ c is the hyperparameter of the clipping objective function, ρ e is the hyperparameter of the entropy objective function, is the agent’s strategy, o represents the computing chain function actor network, f represents the data center actor network, and m represents the server actor network;

[0042] The critic network is updated based on the advantage function, a minimization mean square error target of the updated critic network is calculated, and the strategy of the agent is updated based on the minimization mean square error target to obtain the trained agent.

[0043] According to some embodiments of the present application, the calculation formula for minimizing the mean square error objective is:

[0044]

[0045] Among them, ξ MSE (φ) represents the minimization of mean square error; is the expectation, φ characterizes the parameters of the critic network, r t For status rewards; A state-value function that characterizes the critic network.

[0046] According to some embodiments of the present application, the estimator is expressed as:

[0047]

[0048] in, is the estimator; r t' For status rewards; A state-value function representing the critic network; γ t' Characterize the discount factor.

[0049] To achieve the above-mentioned objectives, a second embodiment of the present application provides a computing power scheduling device, which is applied to a computing network chain processing module of a multi-region business processing system, wherein the computing network chain module is provided with multiple computing network chain function packages, each of which is provided with multiple functions for executing business; the multi-region business processing system further includes multiple regions, each of which includes multiple data centers, and each of which includes multiple servers;

[0050] The device comprises:

[0051] A construction unit is used to construct a computing network chain scheduling model with the goal of minimizing the total weighted end-to-end delay, where the total weighted end-to-end delay is expressed as:

[0052]

[0053] Among them, T P represents the total weighted end-to-end delay; ω p Indicates the priority of the p-th service; D p represents the end-to-end delay of the p-th service;

[0054] A conversion unit, configured to convert the computing network chain scheduling model into a Markov decision model, and construct a target state reward function based on the total weighted end-to-end delay; the actions of the Markov decision model include computing network chain function package actions, data center actions, and server actions;

[0055] Acquire training sample services, and determine, based on the training sample services, a data center action space of the data center action, a server action space of the server action, and a computing network chain function package action space of the computing network chain function package action;

[0056] a selection and execution unit, configured for the agent of the Markov decision model to select and execute a target data center action from the data center action space, select and execute a target server action from the server action space, and select and execute a target computing network chain function package action from the computing network chain function package action space, to complete the training sample business, thereby changing the global state of the Markov decision model;

[0057] A training unit, configured to calculate a state reward when the global state is changed by using the target state reward function, and to train and update the agent based on the state reward to obtain a trained agent;

[0058] The application unit is used to input the current business into the trained intelligent agent to obtain a computing power scheduling decision for the current business.

[0059] To achieve the above-mentioned purpose, the third aspect embodiment of the present application provides an electronic device, which includes a memory and a processor, the memory stores a computer program, and the processor implements the computing power scheduling method described in any one of the first aspect embodiments when executing the computer program.

[0060] To achieve the above-mentioned objectives, the fourth embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the computing power scheduling method described in any one of the first embodiment.

[0061] The computing power scheduling method, device, electronic device and storage medium of the present application construct a computing network chain scheduling model with the goal of minimizing the total weighted end-to-end delay, and then convert the computing network chain scheduling model into a Markov decision model, construct a target state reward function based on the total weighted end-to-end delay, and then use the training sample business to train and update the intelligent agent of the Markov decision model to obtain a trained intelligent agent; the current business is input into the trained intelligent agent to obtain a computing power scheduling decision for the current business. The present application expresses the scheduling problem as a multiple Markov decision process, and performs self-learning of the optimal scheduling strategy to solve the computing network chain scheduling model. It can automatically schedule computing power according to the business, automatically allocate the business to the data center and server, and deploy the computing network chain function package to the server so that the server can execute the business. In addition, since the goal of the computing network chain scheduling model is to minimize the total weighted end-to-end delay, it can reduce the end-to-end delay when processing the business.

[0062] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become obvious from the description below, or will be learned through practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] The present application is further described below with reference to the accompanying drawings and embodiments, wherein:

[0064] Figure 1 This is a schematic diagram of the structure of a multi-region business processing system according to an embodiment of the present application;

[0065] Figure 2 This is a schematic diagram of information interaction in a wide area network according to an embodiment of the present application;

[0066] Figure 3 This is a schematic diagram of a processing flow in a multi-region business processing system;

[0067] Figure 4 This is a schematic diagram of a network function class computing network chain function package according to an embodiment of the present application;

[0068] Figure 5 A schematic diagram of a computing network chain function package of an embodiment of the present application;

[0069] Figure 6 A flowchart of a computing power scheduling method according to an embodiment of the present application;

[0070] Figure 7 yes Figure 6 The specific process of step S640 is shown;

[0071] Figure 8 for Figure 7 Detailed flow chart of step S710;

[0072] Figure 9 for Figure 1 Specific flow diagram of step S650;

[0073] Figure 10 This is a schematic diagram of the structure of the computing power scheduling device according to an embodiment of the present application;

[0074] Figure 11 This is a schematic structural diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0075] The following describes embodiments of the present application in detail. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application and are not to be construed as limiting the present application.

[0076] In the description of this application, it should be understood that descriptions involving orientations, such as up, down, front, back, left, right, etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings. They are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, they cannot be understood as limitations on this application.

[0077] In the description of this application, "several" means more than one, "plurality" means more than two, "greater than," "less than," and "exceed" are understood to exclude the number itself, while "above," "below," and "within" are understood to include the number itself. The use of "first" and "second" in the description is solely for the purpose of distinguishing technical features and should not be construed as indicating or implying relative importance, implicitly specifying the number of the indicated technical features, or implicitly specifying the order of the indicated technical features.

[0078] In the description of this application, unless otherwise clearly defined, terms such as setting, installing, and connecting should be understood in a broad sense, and technicians in the relevant technical field can reasonably determine the specific meanings of the above terms in this application based on the specific content of the technical solution.

[0079] In the description of this application, reference to the terms "one embodiment," "some embodiments," "illustrative embodiments," "examples," "specific examples," or "some examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of this application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in any appropriate manner in any one or more embodiments or examples.

[0080] First, let’s analyze some of the terms used in this application:

[0081] Multiple Markov decision processes (MMDPs): Markov decision processes are a class of optimization decisions for stochastic dynamic systems with continuous observations. At each observed state, a decision is chosen from a set of permissible decisions (controls, actions, etc.), and the rules for selecting decisions at each moment are independent of the system's previous history. Markov decision processes, proposed by Bellman, combine dynamic programming with Markov processes. They are based on the Markov property, which states that the next state of a system depends only on the current state and the chosen decision, and are independent of past states and decisions. MMDPs introduce the interaction and decision-making of multiple agents or systems based on traditional Markov decision processes (MDPs). In a Markov decision process, an action is the action or strategy that an agent or decision maker can take at each state. An agent is an entity capable of perceiving its environment, receiving information, making decisions, and taking actions to achieve its goals. Within the MDP framework, agents are often used to simulate decision-making systems or individuals that make choices in uncertain environments to maximize a cumulative reward or achieve a specific goal. An agent can be understood as a neural network.

[0082] Graph Isomorphism Network (GIN) is a graph neural network (GNN) architecture that aims to learn the representation of the entire graph through a series of graph isomorphism operations.

[0083] Proximal Policy Optimization (PPO) is a reinforcement learning algorithm proposed by John Schulman et al. of OpenAI in 2017. PPO belongs to the category of policy gradient methods, which directly optimize the policy (i.e., the model's behavior) to find the policy that maximizes the expected reward. In PPO, the actor network and the critic network are the two core components of the actor-critic architecture. The actor network learns an action policy for a given state, that is, a mapping from state to action. This network is often referred to as the policy network. The actor network interacts with the environment and outputs an action based on the current state. This action is based on the learned policy. In the PPO algorithm, the actor network continuously updates its parameters during training to optimize the policy. Because PPO is an off-policy training method, the actor network does not immediately apply the new policy when updating. Instead, it uses data collected with the old policy to update the new policy to ensure training stability and performance.

[0084] The critic network evaluates the performance of the current policy, typically using a value function to estimate the long-term value of a state or a state-action pair. This network is often referred to as the value network. The critic network does not interact directly with the environment, but instead evaluates the policy based on the trajectories generated by the actor network's interaction with the environment. By evaluating the value of the policy under different states, the critic network can provide guidance for the training of the actor network. In the PPO algorithm, the critic network continuously updates its parameters to more accurately estimate the value of the state. These value estimates are used to calculate the policy gradient, which guides the training of the actor network. The PPO algorithm also utilizes techniques such as importance sampling and KL divergence to constrain the difference between the old and new policies to ensure training stability and performance.

[0085] Based on the Internet of Things (IoT), the medical information network utilizes a variety of sensors to sense various vital sign data in real time and transmits it to medical facilities via wireless network technology. Leveraging the internet, this data can be shared and transmitted across medical facilities at all levels. Cloud computing and big data technologies provide powerful data storage, analysis, and processing capabilities for massive amounts of vital sign data. By integrating sensing, communication, storage, and computing resources and adopting a centralized resource management approach, the health and medical information network can dynamically dispatch network-wide resources based on medical urgency. By integrating the IoT, the internet, cloud computing, and big data technologies, the health and medical information network can, to a certain extent, transcend spatial limitations and support the remote collection, transmission, and processing of medical information.

[0086] However, existing medical information networks can only guarantee the communication and computing latency of telemedicine services within a single region, and cannot guarantee the latency of cross-domain collaborative medical services. In particular, the latency requirements for cross-domain remote surgery are more stringent than for other services, and network technologies must be introduced to reduce cross-domain end-to-end latency. While there has been extensive research on time-sensitive networks and deterministic networks to address cross-domain transmission latency, there is still a lack of research on computational latency. In recent years, multi-computing center collaborative computing networks have gradually emerged. By integrating computing resources from multiple data centers in multiple regions, they can alleviate computing loads, enhance computing power, improve information processing efficiency, and effectively resolve the link bottleneck problem caused by a single data center. The multi-computing center collaborative service model is introduced into medical information networks to support cross-domain remote surgery services and reduce their computing latency.

[0087] Based on this, the present application proposes a computing power scheduling method, device, electronic device and storage medium, which can realize automatic scheduling of computing power and reduce end-to-end latency.

[0088] The computing power scheduling method of the embodiments of the present application can be applied to a terminal or a server. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a router, a programmable switch, a network card, etc.; the server can be configured as an independent physical server, or as a server cluster or distributed system composed of multiple physical servers, or as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements the computing power scheduling method, etc., but is not limited to the above forms.

[0089] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multi-processor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific services or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which services are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0090] Reference Figure 1The computing power scheduling method of the embodiment of the present application can be applied to a multi-region business processing system under the collaboration of multiple computing power centers in the medical field. The multi-region business processing system includes a computing network chain processing module, which is provided with multiple computing network chain function packages, and the computing network chain function packages are provided with multiple functions for executing business; the multi-region business processing system also includes multiple regions, each region includes multiple data centers, and each data center includes multiple servers. Each region also includes related equipment, such as terminals, camera equipment, surgical robots and monitoring equipment. Some areas include medical information screens, surgical site screens and control equipment. The system also includes a supercomputing center and an intelligent computing center. The supercomputing center is mainly responsible for the advanced management and optimization of network functions, and uses advanced network function virtualization technologies such as firewalls, deep packet inspection, and intrusion detection systems to ensure the integrity and security of data flows, and effectively prevent data leakage and network attacks. Through software-defined network technology, network resources are dynamically allocated, data transmission efficiency is improved, and data flow of high-priority services is guaranteed to be unimpeded. Data loss prevention strategies are introduced to ensure the secure transmission and storage of sensitive data.

[0091] The Intelligent Computing Center focuses on efficient data processing and intelligent analysis. By analyzing patient medical text, images, and multimodal data, it generates medical data insights to support medical decision-making. Advanced machine learning models and artificial intelligence algorithms are used to build patient profiles based on historical and real-time patient data and a global medical knowledge base, enabling inference of disease conditions and forecasting of development trends. Pattern recognition techniques are used to identify abnormal patterns in patient data, proactively identifying potential health issues and supporting early intervention. The Intelligent Computing Center's computing power, powered by artificial intelligence (AI) algorithms, provides a one-stop service from data mining to intelligent decision-making. The Computing Network Chain module is deployed within the Intelligent Computing Center.

[0092] The multi-region business processing system also includes a central processor (CPU). Data centers have local controllers, which monitor and manage the operational status and resource usage of their respective data centers, ensuring the secure execution of regional services and the integrity of medical data. They implement local decisions, such as resource allocation, load balancing, and fault recovery, and coordinate with the central controller and other local controllers to optimize cross-data center business processing. The central controller is the hub of the entire network architecture, comprehensively managing information and operations across all data centers and local controllers, making global policy decisions, coordinating collaboration between the supercomputing center and the intelligent computing center, optimizing business scheduling and resource allocation, monitoring the health of the entire network, and promptly responding to potential network issues or security threats.

[0093] Each region is equipped with multiple data centers, each of which performs medical data collection and pre-processing, providing basic medical data services and caching capabilities to prepare data for subsequent advanced processing. The supercomputing center is equipped with an advanced data flow management system that can effectively reduce network latency while ensuring the security and reliability of medical data transmission. The intelligent computing center focuses on medical data analysis and complex computing services, utilizing the latest artificial intelligence technologies to optimize the medical data processing process and improve data utilization, thus forming a collaborative service architecture for multiple computing centers. This hierarchical architecture allows each data center to optimally handle corresponding business needs based on its expertise and resources, significantly improving overall processing power and efficiency while effectively reducing latency. By balancing the load across data centers, overload and link bottlenecks in any single data center are avoided, thereby improving computing resource utilization across the entire region.

[0094] For cross-regional treatment, it is necessary to achieve inter-regional interconnection through a wide area network. The wide area network is equipped with a wide area network controller, which is Figure 1 The WAN utilizes a centralized controller in the system. Software-Defined Networking (SDN) is a centralized management approach. Building on the existing network, a controller cluster and software-defined switch cluster are deployed to form a software-defined wide area network (SD-WAN). This proposed network architecture dynamically schedules network resources based on the needs of different medical services, providing secure, efficient, and flexible service assurance, thereby ensuring the capacity, security, and efficiency of medical information transmission and processing.

[0095] Reference Figure 2 and Figure 3 , Figure 2 Schematic diagram of information interaction in a wide area network. Figure 3 The figure is a schematic diagram of a processing flow in the above-mentioned multi-region business processing system. In a specific embodiment, remote surgery involves performing a surgical operation between a treatment end and a control end, and the treatment end and the control end are respectively located in different areas. The treatment end is usually located in a remote area with limited medical resources. Camera equipment is deployed to capture real-time images and videos of patients. Surgery is performed using a surgical robot. Multiple monitoring devices are set up to collect vital sign data. All real-time data captured during the operation is transmitted to the control end for further processing and analysis. The control end is usually located in an economically developed area with advanced medical facilities. Control equipment is deployed to remotely operate the surgical robot. Medical information screens are used to display patient vital sign information. The surgical scene screen is used to display images and videos of patients. The accuracy and safety of the surgical process are ensured through real-time processing and feedback of data sent by the surgical end.

[0096] The treatment end of the source hospital initiates a remote surgery request through the terminal, and the request information is forwarded to the wide area network controller; after receiving the request information, the wide area network controller analyzes the source address, destination address, collaboration time and other information in the request information, and sends the analysis results to the business arrangement module; the business arrangement module arranges the business based on the analysis results to obtain a series of businesses; the business arrangement module sends the business to the computing network chain processing module, and the computing network chain processing module executes the computing power scheduling method of the embodiment of the present application based on the business to obtain the computing power scheduling decision, and then the computing network chain processing module sends the scheduling decision to the configuration module. After receiving the policy, the configuration module generates and outputs the network configuration file, distributes it to the switch of the data plane through the interface, and sends the business-related information to the source hospital and the destination hospital. After receiving the configuration, the switch of the data plane forwards various types of traffic from the edge router to the adjacent wide area network according to the policy, and then the wide area network forwards the traffic to each data center, and finally forwards the traffic to the control end of the destination hospital. Among them, the business arrangement module, the computing network chain processing module, and the configuration module can be deployed in the central controller. The switch of the data plane is a network device mainly used to forward and exchange data in the data communication network. Data forwarding in the data center is performed through edge routers.

[0097] It should be noted that the Computing Network Chain processing module is equipped with multiple Computing Network Chain function packages. The Computing Network Chain function package (CNC) is specially designed to optimize data flow and resource allocation to ensure efficient and low-latency communication even in cross-domain operations. The Computing Network Chain function package is equipped with multiple functions for executing business. Among them, the Computing Network Chain function package includes the network function class Computing Network Chain function package and the computing class Computing Network Chain function package. CNC is assigned to a server in a data center for execution, that is, when executing CNC, CNC needs to be deployed to the server of the data center. Figure 4 The network function class computing network chain function package has multiple network function functions, and each network function has a processing order. The network function function is one of the following functions:

[0098] Firewall (FW): FW acts as a barrier to monitor and control traffic entering and leaving the network according to pre-set security rules and policies.

[0099] Deep packet inspection (DPI): DPI inspects data headers and content and further extracts data features. Based on the DPI's inspection results on the data header, only authorized traffic is allowed to pass through the firewall.

[0100] Intrusion detection system (IDS): Based on the characteristics of the content data extracted from DPI, the IDS further identifies and classifies traffic into normal traffic and malicious traffic. Normal traffic is allowed to pass through the intrusion prevention system, while malicious traffic is filtered out by the intrusion prevention system.

[0101] High Intrusion Detection System (H-IDS): Compared with the intrusion detection system, H-IDS has a stronger feature database and analysis capabilities, and can provide a higher level of security, but it takes longer to process.

[0102] Web Application Firewall (WAF): WAF is deployed at the front end of the web application server and defends against attacks and prevents SQL injection by analyzing the content in the HTTP protocol.

[0103] Bastion Host (BH): BH has a security audit function that can intercept illegal commands and monitor internal personnel's incorrect and illegal operations.

[0104] Data Loss Prevention (DLP): DLP can classify data according to its sensitivity to prevent highly sensitive information from being stolen.

[0105] Reference Figure 5 The computing class computing network chain function package has multiple computing functions, and each computing function has a processing order. The computing function is one of the following functions:

[0106] Medical Text Data Analysis (MBDA): Medical text data analysis refers to the process of extracting meaningful information and insights from text data in the medical field.

[0107] Medical Image Feature Extraction (MIFE): Medical image feature extraction refers to the process of identifying and extracting relevant visual features or patterns from medical images. It involves applying various image processing techniques and algorithms to analyze and quantify specific aspects of an image, such as texture, shape, intensity, or spatial relationships.

[0108] Medical Multimodal Data Fusion (MMDF): Multimodal medical data fusion refers to the integration of different types of medical data from multiple sources or modes, such as images, text, genomics, clinical data, and sensor data.

[0109] Intelligent Auxiliary Diagnosis (IAD): Based on the processing results obtained by MMDF, IAD refers to the process of using artificial intelligence (AI) and advanced computing technologies to support healthcare professionals in diagnosing medical conditions.

[0110] Reference Figure 6 Based on the above-mentioned multi-region business processing system, the embodiment of the present application proposes a computing power scheduling method, including but not limited to steps S610 to S660.

[0111] Step S610: Construct a computing network chain scheduling model with the goal of minimizing the total weighted end-to-end delay. The total weighted end-to-end delay is expressed as:

[0112]

[0113] Among them, T P represents the total weighted end-to-end delay; ω p Indicates the priority of the p-th service; D p represents the end-to-end delay of the p-th service;

[0114] It is worth noting that, referring to Figure 2 As shown in Table 1, when the service is a medical service, different medical information has different time sensitivities and requires the network to provide different levels of delay guarantees. Information priorities are divided and weights are set according to the delay requirements of medical information. Table 1 shows the priorities of different medical information (i.e., different services) and the weights corresponding to the priorities. As can be seen from Table 1, in this embodiment, ω p ∈[1, 2, 3, 4].

[0115] Table 1

[0116]

[0117] It is worth noting that in the computing network chain scheduling model, the number of cross-domain remote business areas is L. The number of transactions is P, The starting area and target area of ​​business P can be expressed as ds p and dd p ,ds p ∈{1,2,…L},dd p ∈{1,2,...L}, the priority weight of P is ω p , and ω p∈{1,2,3,4}. ε represents the type of P's computing network chain function package, ε∈{n,c}. Where n represents the network function type computing network chain function package, and c represents the computing type computing network chain function package. The set of CNC can be expressed as:

[0118]

[0119] This allows network function CNC and computing CNC to be formed accordingly:

[0120] and

[0121] Is a CNC collection for any business, consisting of It is composed of an ordered sequence of CNCs, expressed as:

[0122]

[0123] in and Including business network function class CNC collection and business computing CNC collection Depend on It consists of a network function class CNC arranged in sequence; Depend on All CNCs must comply with the following rules:

[0124]

[0125] Characterizing each CNC requires execution in sequence. In addition, for the security of the data center, computing CNCs must be processed after network function CNCs, namely: Execute network function CNC first, then execute calculation CNC.

[0126] The number of data centers is F, Assume that the communication time h between data center f and data center l is f,l is certain, and h f,g is the communication time between data center f and data center g, g∈{1,2,...,F}, each data center deploys Mn networking function servers for executing network function class CNC and Mc computing function servers for executing computing class CNC, and Each network function CNC can only be processed by one network function server in Mn, and each computing CNC can only be processed by one computing function server in Mc. Define 0-1 variables like Then the computing network chain function package Server m assigned to the data center ε The processing time of CNC is recorded as The start time of CNC processing is CNC processing completion time is The completion time is It can be defined as:

[0127]

[0128]

[0129]

[0130]

[0131] The end-to-end delay of the service is D p It can be defined as:

[0132]

[0133] Under the same end-to-end delay, the cost is proportional to the service priority. Therefore, high-priority services should be completed as soon as possible. Introducing the total weighted end-to-end delay T P As a measure of the cost of cross-domain remote business, it is defined as:

[0134]

[0135] Minimizing the total weighted end-to-end delay is expressed as:

[0136]

[0137] The computing network chain scheduling model in this embodiment of the application has the following constraints:

[0138]

[0139]

[0140] Among them, constraint (1) means that the processing must be carried out in the order of CNC, and the processing start time of the next CNC must be greater than the completion time of the previous CNC. In addition, if the next CNC and the previous CNC are not assigned to the same data center, the data must be transferred to the designated data center before the next CNC can be processed. Constraints (2)(3)(4) mean: if two CNCs with different services are assigned to the same server, the functional processing of one CNC cannot be started before the processing of the other CNC is completed. In order to impose the above constraints, an auxiliary binary variable is defined, For the binary variables defined,

[0141]

[0142] Specifically, KZ is a large positive number. It should be noted that for constraint (3), This constraint is non-restrictive. For constraint (4), if Then this constraint is non-restrictive. Constraint (5) indicates that a CNC can only be processed on one server. Constraint (6) indicates that the start time of the first network function class CNC must be greater than or equal to the communication time between the region that initiates the request and the data center that allocates the CNC. Constraint (7) indicates that the processing start time of the first computing class CNC of any business must be greater than or equal to the completion time of its network function class CNC. If the last function of the network function class CNC and the first function of the computing class CNC are not processed in the same data center, the first function of the computing class CNC needs to be transferred to the designated data center before it can be processed. Constraints (8) and (9) indicate that the CNC processing time must be greater than 0.

[0143] The solution of the computing network chain scheduling model can be expressed as:

[0144]

[0145] in, The i-th CNC representing business p is assigned to the m-th server in the f-th data center for execution. Indicates the start time of CNC execution.

[0146] Step S620: Convert the computing network chain scheduling model into a Markov decision model, and construct a target state reward function based on the total weighted end-to-end delay; the actions of the Markov decision model include computing network chain function package actions, data center actions, and server actions;

[0147] Step S630: Acquire training sample services, and determine the data center action space of the data center action, the server action space of the server action, and the computing network chain function package action space of the computing network chain function package action based on the training sample services;

[0148] Step S640: The agent of the Markov decision model selects and executes a target data center action from the data center action space, selects and executes a target server action from the server action space, and selects and executes a target computing network chain function package action from the computing network chain function package action space to complete the training sample business, thereby changing the global state of the Markov decision model.

[0149] It is worth noting that in the computing network chain scheduling model, a total of T consecutive decision steps need to be solved. The computing network chain scheduling model is converted into a Markov decision model. The Markov decision model can be defined by a five-tuple (S, A, P, r, γ), where S is the state, A is the action, and P is S t+1 At a given S t and action a t The probability of occurrence of the situation, r t It is for action a t The state of the Markov decision model is the global state, which includes s t include and in In CNC state, Indicates the server status. is a disjunctive graph Where V is a combination arc, which represents the precedence constraint between continuous functions from the same CNC; is the set of remaining arcs. The CNC set is expressed as:

[0150]

[0151] if Allocated, For reservation completion time, otherwise, is the estimated minimum completion time, calculated as:

[0152]

[0153] in, is assigned to execute data centers, allocated to execute The communication time of the data center. In, for Server status includes two-dimensional features Indicates that the server m ε Deployed in data center f, where For server m ε The end working time before t.

[0154] The actions of the Markov decision model include the network chain function package actions, data center actions, and server actions, specifically:

[0155] a o ∈A o , a f ∈A f , a m ∈A m ;

[0156] a o Characterizes the action of the computing network chain function package; a f Characterize data center actions; a m Represents server actions, A o Representing the action space of the computing network chain function package, the action space of the computing network chain function package is the set of computing network chain function package actions that can realize the training sample business; A f Characterize the data center action space, which is the set of data center actions that can realize the training sample business; A m Represents the server action space, and the service space is the set of server actions that can realize the training sample business. o Perform sampling and select a data center and a server to implement the training sample business. Update the direction of the extracted arc according to the network chain function package action and server action to generate a new state Then Updated to It is worth noting that an action can be understood as a response to a selection. A computing network chain function package action represents a selected computing network chain function package; a data center action represents a selected data center; and a server action represents a selected server. A complete action includes the actions of the Markov decision model, including the computing network chain function package action, the data center action, and the server action. When executing the actions of a complete Markov decision model, for example, the complete Markov action is The computing network chain function package represented by The data center is f and the server is m ε , then the process of executing the Markov action is to: enclose the algorithm chain function package Deployed to server m in data center f ε In the server m εExecute the computing network chain function package The function in .

[0157] The target reward function is:

[0158] r(s t ,a o ,a f ,a m )=-d t ;

[0159] d t =T p (s t+1 )-T p (s t );

[0160]

[0161] Among them, r represents the target state reward function, and dt represents the interval between two consecutive time steps t and t+1. The goal of the agent in the Markov decision model is to maximize the cumulative reward, which is the cumulative reward after all business assignments are completed T P The cumulative value of negative values.

[0162] Step S650, calculating the state reward when changing the global state through the target state reward function, and training and updating the agent based on the state reward to obtain a trained agent;

[0163] Step S660: Input the current business into the trained intelligent agent to obtain a computing power scheduling decision for the current business.

[0164] The computing power scheduling method of the embodiment of the present application constructs a computing network chain scheduling model with the goal of minimizing the total weighted end-to-end delay through the above steps S610 to S660, then converts the computing network chain scheduling model into a Markov decision model, constructs a target state reward function based on the total weighted end-to-end delay, and then uses the training sample business to train and update the intelligent agent of the Markov decision model to obtain a trained intelligent agent; the current business is input into the trained intelligent agent to obtain a computing power scheduling decision for the current business. The present application expresses the scheduling problem as a multiple Markov decision process, performs self-learning optimal scheduling strategy to solve the computing network chain scheduling model, and can automatically schedule computing power according to the business, automatically allocate the business to the data center and server, and deploy the computing network chain function package to the server so that the server can execute the business. In addition, since the goal of the computing network chain scheduling model is to minimize the total weighted end-to-end delay, it can reduce the end-to-end delay when processing the business, so that it can be applied to the field of telemedicine with high requirements for end-to-end delay, ensuring the rapid transmission and processing of medical information.

[0165] In one embodiment, the agent includes an actor network, which includes a computing network chain function package actor network, a data center actor network, and a server actor network. The actor network is represented as:

[0166]

[0167] in, Indicates the strategy of the actor network of the computing network chain function package, represents the policy of the datacenter actor network, represents the policy of the server actor network, θ o ,θ f ,θ m Calculate the parameters of the strategy of the actor network of the network chain function package, the parameters of the strategy of the data center actor network, and the parameters of the strategy of the server actor network respectively.

[0168] Reference Figure 7 , Figure 7 yes Figure 6 The specific process of step S640 is shown in FIG. Figure 7 The illustrated steps include but are not limited to step S710 and step S720.

[0169] Step S710, calculating a first probability distribution of a data center in a data center action space, calculating a second probability distribution of a server action in a server action space, and calculating a third probability distribution of a computing network chain function package action in a computing network chain package action space;

[0170] Step S720: select and execute the target data center action from the data center action space based on the first probability distribution through the data center actor network; select and execute the target server action from the server action space based on the second probability distribution through the server actor network; select and execute the target computing network chain function package action from the computing network chain function package action space based on the third probability through the computing network chain function package actor network.

[0171] Specifically, after step S710, the data center actor network selects and executes a target data center action from the data center action space based on the first probability distribution, and the server actor network selects and executes a target server action from the server action space based on the second probability distribution. The computing network chain function package actor network selects and executes a target computing network chain function package action from the computing network chain function package action space based on the third probability distribution. Because the selection of target data center actions, target server actions, and target computing network chain function package actions is all based on probability distribution, selecting actions with a higher probability based on probability distribution results in higher rewards compared to randomly selecting actions. This allows for faster convergence and improved training speed during subsequent training.

[0172] In one embodiment, after obtaining the first, second, and third probability distributions, a target data center action, target server action, and target computing network chain function package action are determined based on a greedy decoding strategy. A greedy decoding strategy, also known as a greedy algorithm or a greedy algorithm, is an algorithmic strategy that takes the best or optimal (i.e., most favorable) option in each step, hoping to achieve the best or optimal result.

[0173] In one embodiment, referring to Figure 8 , Figure 8 for Figure 7 Specific flow chart of step S710 in FIG. Figure 8 The illustrated process includes but is not limited to steps S810 to S840.

[0174] Step S810: Encode the state of the computing network chain function package using a graph isomorphism network to obtain a first encoding vector;

[0175] It is worth noting that using the graph isomorphism network (GIN) to map the state Encode, Defined as a disjunctive graph with time step t Each computing network chain function package is encoded through k layers of GIN. Each layer of GIN is as follows:

[0176]

[0177] is the embedding vector of node v in the graph isomorphism network from the kth layer of GIN, is the k-th layer of the multilayer perceptron (MLP) with parameters: θ k ,∈ (k) is a learnable parameter, N(v) is the neighborhood set of node v. The disjunctive graph Gt is a mixed graph of undirected arcs and directed arcs.

[0178] Step S820: Encode the server state using a fully connected layer to obtain a second encoding vector;

[0179] It is worth noting that there is no graph structure in the server status information. Each node represents the main feature of each machine, and the nodes are not connected to other nodes with directed or undirected arcs. Therefore, a fully connected layer is used to analyze the server status. At time step t, each server is in the server state The main feature is defined as a two-dimensional vector Indicates that the server m ε Deployed in data center f, where For server m εThe end working time before t. The output of each node in the fully connected layer is the embedding vector and the output pooling vector u t .

[0180] Step S830: Calculate the first action state reward for each data center action, the second action reward for each server action, and the third action reward for each computing network chain function package action based on the first encoding vector, the second encoding vector, and the value function.

[0181] It is worth noting that the first action state reward of the data center action is calculated by the following formula, expressed as:

[0182]

[0183] in, Represents the first action state reward, and MLP is a multi-layer perceptron. is the embedding vector output by the fully connected layer, is the average value of the final embedded nodes of the fully connected layer, u t is the pooling vector of the fully connected layer.

[0184] The second action reward of the server action is calculated by the following formula, expressed as:

[0185]

[0186] in, Represents the second action state reward, and MLP is a multi-layer perceptron. is the embedding vector output by the fully connected layer, is the average value of the final embedded nodes in the fully connected layer, u t is the pooling vector of the fully connected layer.

[0187] The third action reward of each computing network chain function package action is calculated by the following formula, expressed as:

[0188]

[0189] in, Represents the third action state reward, and MLP is a multi-layer perceptron. is the embedding vector of node v in the graph isomorphism network from the kth layer of GIN, is the average value of the final embedded nodes of the graph isomorphism network, u t is the pooling vector of the graph isomorphism network.

[0190] Step S840: Calculate a first probability distribution based on the first action reward; calculate a second probability distribution based on the second action reward; and calculate a third probability distribution based on the third action reward.

[0191] The calculation formula for the first probability distribution is:

[0192]

[0193] in, Characterizes the probability of the y-th data center action, Represents the first action reward of the y-th data center action; Characterizes the first action reward of the f-th data center action;

[0194] The calculation formula of the second probability distribution is expressed as:

[0195]

[0196] in, Characterizes the probability of the jth server action, The second action reward representing the jth server action; The second action reward representing the mth server action;

[0197] The calculation formula of the third probability distribution is:

[0198]

[0199] in, Represents the probability of the i-th computing network chain function package action, Represents the third action reward of the vth computing network chain function package action; Represents the third action reward of the i-th computing network chain function package action.

[0200] The embodiment of the present application calculates the first probability distribution, the second probability distribution and the third probability distribution through steps S810 to S840, thereby facilitating the data center actor network to select and execute the target data center action from the data center action space based on the first probability distribution, and the server actor network to select and execute the target server action from the server action space based on the second probability distribution; and by selecting and executing the target computing network chain function package action from the computing network chain function package action space based on the third probability through the computing network chain function package actor network, it helps to improve the training speed.

[0201] In one embodiment, the agent of the embodiment of the present application further includes a critic network. Figure 9 , Figure 9 for Figure 1 Specific flow chart of step S650 in FIG. Figure 9 The illustrated process includes but is not limited to steps S910 to S930.

[0202] Step S910, constructing an estimator based on the state reward and the state value function of the critic network;

[0203] In one embodiment, the estimator is represented as:

[0204]

[0205] in, is the estimator; r t' For status rewards; A state-value function representing the critic network; γ t' Characterize the discount factor.

[0206] Step S920: construct an advantage function based on the estimator. The advantage function is:

[0207]

[0208]

[0209]

[0210] in, is the clipping objective function, is the entropy objective function, is the estimator; is the parameter probability ratio; η is the clipping parameter; is the expectation; clip is the clipping function; entropy is the entropy function; ρ c and are the hyperparameters of the clipping objective function, ρ e is the hyperparameter of the entropy objective function, is the agent’s strategy, o represents the computing network chain function actor network, f represents the data center actor network, and m represents the server actor network;

[0211] Step S930: Update the critic network based on the advantage function, calculate the minimum mean square error target of the updated critic network, and update the agent's strategy based on the minimum mean square error target to obtain a trained agent.

[0212] In one embodiment, the calculation formula for minimizing the mean square error objective is:

[0213]

[0214] Among them, ξ MSE (φ) represents the minimization of the mean square error, is the expectation, φ represents the parameters of the critic network, and rt is the state reward; Characterizes the state value function of the critic network, and MSE is the mean square error function.

[0215] Through the above steps S910 to S930, after obtaining the optimization function, during the training process, the agent maximizes the advantage function, which takes into account the probability distribution of the reward and avoids excessive changes in the strategy through a method called clipping. At the same time, entropy is introduced to encourage exploration, that is, to try new actions that have not yet been determined to be beneficial. In this process, the agent updates the parameters of the actor network and the critic network, and then calculates the minimized mean square error target of the updated critic network. The strategy of the agent is iteratively updated based on the minimized mean square error target. When the performance of the strategy of the agent reaches the preset conditions, or the number of iterations reaches the preset number, the update is stopped to obtain a trained agent. In this way, in this application, not only the computing network chain scheduling model is converted into a Markov decision model, but also the proximal strategy optimization algorithm is combined to optimize the agent, which is conducive to the agent making better scheduling decisions.

[0216] Reference Figure 10 , Figure 10 This is a schematic diagram of the structure of the computing power scheduling device of an embodiment of the present application. The computing power scheduling device provided in the second embodiment of the present application is applied to the computing network chain processing module of the multi-region business processing system, the computing network chain module is provided with multiple computing network chain function packages, and the computing network chain function packages are provided with multiple functions for executing business; the multi-region business processing system also includes multiple regions, each region includes multiple data centers, and each data center includes multiple servers;

[0217] The device includes:

[0218] The construction unit 1010 is used to construct a computing network chain scheduling model with the goal of minimizing the total weighted end-to-end delay. The total weighted end-to-end delay is expressed as:

[0219]

[0220] Among them, T P represents the total weighted end-to-end delay; ω p Indicates the priority of the p-th service; D p represents the end-to-end delay of the p-th service;

[0221] A conversion unit 1020 is configured to convert the computing network chain scheduling model into a Markov decision model, and construct a target state reward function based on the total weighted end-to-end delay; the actions of the Markov decision model include computing network chain function package actions, data center actions, and server actions;

[0222] Acquire training sample business 1030, and determine a data center action space for data center actions, a server action space for server actions, and a computing network chain function package action space for computing network chain function package actions based on the training sample business;

[0223] The selection and execution unit 1040 is used for the intelligent agent of the Markov decision model to select and execute the target data center action from the data center action space, select and execute the target server action from the server action space, and select and execute the target computing network chain function package action from the computing network chain function package action space to complete the training sample business, thereby changing the global state of the Markov decision model;

[0224] A training unit 1050 is configured to calculate a state reward when changing the global state through a target state reward function, and to train and update the agent based on the state reward to obtain a trained agent;

[0225] The application unit 1060 is used to input the current business into the trained intelligent agent to obtain a computing power scheduling decision for the current business.

[0226] The computing power scheduling device can execute the computing power scheduling method of the embodiment of the first aspect of this application. When executing the computing power scheduling method of the embodiment of the first aspect, the computing power scheduling device constructs a computing network chain scheduling model with the goal of minimizing the total weighted end-to-end delay, and then converts the computing network chain scheduling model into a Markov decision model, constructs a target state reward function based on the total weighted end-to-end delay, and then uses the training sample business to train and update the intelligent agent of the Markov decision model to obtain a trained intelligent agent; the current business is input into the trained intelligent agent to obtain a computing power scheduling decision for the current business. This application expresses the scheduling problem as a multiple Markov decision process, and performs self-learning of the optimal scheduling strategy to solve the computing network chain scheduling model. It can realize automatic scheduling of computing power according to the business, automatically allocate the business to the data center and server, and deploy the computing network chain function package to the server so that the server can execute the business. In addition, since the goal of the computing network chain scheduling model is to minimize the total weighted end-to-end delay, it can reduce the end-to-end delay when processing the business.

[0227] In one embodiment, the agent includes a computing network chain function package actor network, a data center actor network, and a server actor network. The selection and execution unit of the computing power scheduling device includes a computing subunit and a selection and execution subunit.

[0228] The computing subunit is used to:

[0229] Calculate the first probability distribution of the data center in the data center action space, calculate the second probability distribution of the server action in the server action space, and calculate the third probability distribution of the computing network chain function package action in the computing network chain package action space;

[0230] The selection and execution subunit is used to select and execute the target data center action from the data center action space based on the first probability distribution through the data center actor network; select and execute the target server action from the server action space based on the second probability distribution through the server actor network; and select and execute the target computing network chain function package action from the computing network chain function package action space based on the third probability through the computing network chain function package actor network.

[0231] In one embodiment, the calculation subunit further includes a first encoding subunit, a second encoding subunit, a reward calculation subunit, and a probability calculation subunit.

[0232] The first encoding subunit is used to encode the state of the computing network chain function package using a graph isomorphism network to obtain a first encoding vector;

[0233] The second encoding subunit is used to encode the server state using a fully connected layer to obtain a second encoding vector;

[0234] The reward calculation subunit is used to calculate the first action state reward of each data center action, the second action reward of each server action, and the third action reward of each computing network chain function package action based on the first coding vector, the second coding vector, and the value function;

[0235] The probability calculation subunit is used to calculate a first probability distribution based on the first action reward; calculate a second probability distribution based on the second action reward; and calculate a third probability distribution based on the third action reward.

[0236] In one embodiment, the training unit includes an estimator construction subunit, an advantage function construction subunit, and an updating subunit.

[0237] The estimator construction subunit is used to construct an estimator based on the state reward and the state value function of the critic network;

[0238] The advantage function construction subunit is used to construct an advantage function based on the estimator;

[0239] The updating subunit is used to update the critic network based on the advantage function, calculate the minimum mean square error target of the updated critic network, and update the strategy of the agent based on the minimum mean square error target to obtain the trained agent.

[0240] Reference Figure 11 , Figure 11 FIG. 1 is a schematic diagram of the structure of an electronic device according to an embodiment. The electronic device includes:

[0241] The processor 1101 can be implemented as a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0242] The memory 1102 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1102 can store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1102 and is called by the processor 1101 to execute the computing power scheduling method of the embodiments of this application.

[0243] Input / output interface 1103, used to implement information input and output;

[0244] Communication interface 1104, used to implement communication interaction between this device and other devices, which can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WiFi, Bluetooth, etc.);

[0245] Bus 1105 , which transmits information between various components of the device (e.g., processor 1101 , memory 1102 , input / output interface 1103 , and communication interface 1104 );

[0246] The processor 1101 , the memory 1102 , the input / output interface 1103 and the communication interface 1104 are connected to each other in communication within the device via a bus 1105 .

[0247] The fourth aspect of the present application is a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the computing power scheduling method of any one of the first aspect embodiments.

[0248] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0249] The embodiments described in the embodiments of this application are intended to more clearly illustrate the technical solutions of the embodiments of this application and do not constitute a limitation on the technical solutions provided by the embodiments of this application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0250] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0251] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, i.e., they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of this embodiment.

[0252] Those skilled in the art will appreciate that all or some of the steps in the methods, systems, and functional modules / units in the devices disclosed above may be implemented as software, firmware, hardware, or appropriate combinations thereof.

[0253] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0254] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the mapping relationship of the mapping objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next mapping objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0255] In the several embodiments provided in this application, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For example, the division of the above units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0256] The units described above as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0257] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0258] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes multiple instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: various media that can store programs, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0259] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but are not intended to limit the scope of the present invention. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and essence of the present invention should be within the scope of the present invention.

Claims

1. A computing power scheduling method, characterized in that: A computing network chain processing module applied to a multi-region business processing system, the computing network chain processing module having a plurality of computing network chain function packages, each of which having a plurality of functions for executing business; the multi-region business processing system further comprising a plurality of regions, each of the regions comprising a plurality of data centers, each of the data centers comprising a plurality of servers; The method comprises: A computing network chain scheduling model is constructed with the goal of minimizing the total weighted end-to-end delay. The total weighted end-to-end delay is expressed as: ； in, represents the total weighted end-to-end delay; Indicates the priority of the p-th service; represents the end-to-end delay of the p-th service, where P is the number of services; The computing network chain scheduling model is converted into a Markov decision model, and a target state reward function is constructed based on the total weighted end-to-end delay; the actions of the Markov decision model include computing network chain function package actions, data center actions, and server actions; the intelligent agent includes the computing network chain function package actor network, data center actor network, and server actor network; the global state includes the server state and the computing network chain function package state; Acquire training sample services, and determine, based on the training sample services, a data center action space of the data center action, a server action space of the server action, and a computing network chain function package action space of the computing network chain function package action; A graph isomorphism network is used to encode the state of the computing network chain function package to obtain a first encoding vector; a fully connected layer is used to encode the server state to obtain a second encoding vector; based on the first encoding vector, the second encoding vector and the value function, a first action reward of each data center action, a second action reward of each server action, and a third action reward of each computing network chain function package action are calculated; a first probability distribution is calculated based on the first action reward; a second probability distribution is calculated based on the second action reward; and a third probability distribution is calculated based on the third action reward; a target data center action is selected and executed from the data center action space based on the first probability distribution by the data center actor network; a target server action is selected and executed from the server action space based on the second probability distribution by the server actor network; and a target computing network chain function package action is selected and executed from the computing network chain function package action space based on the third probability by the computing network chain function package actor network to complete the training sample business, thereby changing the global state of the Markov decision model; Calculating a state reward when changing the global state through the target state reward function, and training and updating the agent based on the state reward to obtain a trained agent; Obtain analysis results based on the request information initiated by the treatment end of the source hospital, perform business orchestration based on the analysis results to obtain the current business, input the current business into the trained intelligent agent, and obtain a computing power scheduling decision for the current business.

2. The computing power scheduling method according to claim 1, characterized in that: The agent also includes a critic network; The step of calculating a state reward when changing the global state by using the target state reward function, and training and updating the agent based on the state reward to obtain a trained agent includes: constructing an estimator based on the state reward and the state-value function of the critic network; An advantage function is constructed based on the estimator, and the advantage function is: in, is the clipping objective function, is the entropy objective function, is the estimator; is the parameter probability ratio; is the cropping parameter; For expectations; is the clipping function; is the entropy function; is the hyperparameter of the tailored objective function, is the hyperparameter of the entropy objective function, is the agent’s strategy, , o represents the computing network chain function package actor network, f represents the data center actor network, and m represents the server actor network; The critic network is updated based on the advantage function, a minimization mean square error target of the updated critic network is calculated, and the strategy of the agent is updated based on the minimization mean square error target to obtain the trained agent.

3. The computing power scheduling method according to claim 2, characterized in that: The calculation formula for minimizing the mean square error objective is: in, Characterize the minimization of mean square error; For expectations, The parameters that characterize the critic network, For status rewards; A state-value function that characterizes the critic network.

4. A computing power scheduling device, characterized in that: A computing network chain processing module applied to a multi-region business processing system, the computing network chain processing module having a plurality of computing network chain function packages, each of which having a plurality of functions for executing business; the multi-region business processing system further comprising a plurality of regions, each of the regions comprising a plurality of data centers, each of the data centers comprising a plurality of servers; The device comprises: A construction unit is used to construct a computing network chain scheduling model with the goal of minimizing the total weighted end-to-end delay, where the total weighted end-to-end delay is expressed as: ； in, represents the total weighted end-to-end delay; Indicates the priority of the p-th service; represents the end-to-end delay of the p-th service, where P is the number of services; A conversion unit, configured to convert the computing network chain scheduling model into a Markov decision model, and construct a target state reward function based on the total weighted end-to-end delay; the actions of the Markov decision model include computing network chain function package actions, data center actions, and server actions; the intelligent agent includes a computing network chain function package actor network, a data center actor network, and a server actor network; and the global state includes a server state and a computing network chain function package state; Acquire training sample services, and determine, based on the training sample services, a data center action space of the data center action, a server action space of the server action, and a computing network chain function package action space of the computing network chain function package action; a selection and execution unit, configured to encode the state of the computing network chain function package using a graph isomorphism network to obtain a first encoding vector; encode the server state using a fully connected layer to obtain a second encoding vector; calculate the first action reward of each data center action, the second action reward of each server action, and the third action reward of each computing network chain function package action based on the first encoding vector, the second encoding vector, and the value function; calculate a first probability distribution based on the first action reward; calculate a second probability distribution based on the second action reward; and calculate a third probability distribution based on the third action reward; select and execute the target data center action from the data center action space based on the first probability distribution through the data center actor network; select and execute the target server action from the server action space based on the second probability distribution through the server actor network; select and execute the target computing network chain function package action from the computing network chain function package action space based on the third probability through the computing network chain function package actor network to complete the training sample business, thereby changing the global state of the Markov decision model; A training unit, configured to calculate a state reward when the global state is changed by using the target state reward function, and to train and update the agent based on the state reward to obtain a trained agent; The application unit is used to obtain the analysis results based on the request information initiated by the treatment end of the source hospital, perform business arrangement based on the analysis results to obtain the current business, input the current business into the trained intelligent agent, and obtain a computing power scheduling decision for the current business.

5. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the computing power scheduling method described in any one of claims 1 to 3 when executing the computer program.

6. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the computing power scheduling method according to any one of claims 1 to 3 is implemented.

Images