Allocation method and device of computing power resources and nonvolatile storage medium

Abstract

The invention discloses a computing power resource allocation method and device and a nonvolatile storage medium. The method comprises the following steps: acquiring a service demand of a user; the service demand is divided into a preset number of sub-tasks, environment resource information of the computing power network is acquired, each sub-task at least comprises a sub-task data size and a sub-task computing power resource size, and the environment resource information at least comprises computing power resource information of computing power nodes in the computing power network; determining the preset number of subtasks and the environment resource information as environment state data; analyzing the environment state data by adopting a computing power resource allocation model to obtain a computing power resource allocation result; and allocating computing power resources to the computing power nodes corresponding to the preset number of sub-tasks based on the computing power resource allocation result. According to the method and the device, the technical problem of low resource utilization rate caused by unbalanced computing power resource allocation of different computing power nodes in a computing power network when computing power resource allocation is carried out on user services is solved.

Description

Technical Field

[0001] This application relates to the field of communication technologies, and in particular, to a method, an apparatus, and a non-volatile storage medium for allocating computing power resources. Background Art

[0002] With the rapid development of new network technologies such as 5G and the Internet of Things, it has brought about the rapid development of the digital economy and an explosive increase in digital application scenarios. Subsequently, it has brought about a huge increase in data and a high demand for computing power. The massive data not only requires reliable transmission but also real-time intelligent analysis, making traditional networks face challenges in terms of latency, bandwidth, reliability, etc. Facing the growing demand for big data computing power, the traditional big data architecture model can no longer solve various problems encountered currently. The computing power network can complete the efficient integration of cloud-edge-end computing power through professional, flexible, and collaborative methods. By utilizing its characteristic of flexible allocation of resources at all levels, it can solve the problems of diversified business scenarios, diverse computing power requirements, and low computing power utilization in the field of big data computing and analysis.

[0003] Under the framework of the computing power network, managing and adapting limited computing power resources has become an urgent problem to be solved. In related technologies, the allocation of computing power resources is solved by using traditional optimization methods. Usually, several edge cloud servers are deployed. When the computing power resource load is too high, it is necessary to balance the resource load between the edge clouds to improve the accuracy of the computing power allocation service. Due to the huge scale of the computing power resource data volume, in related technologies, the search time is too long, it is easy to fall into local optimality, and the solution found may not be the global optimum; when allocating computing power resources for user services, the computing power resource pools of some computing power nodes are overloaded, while the computing power resource pools of some nodes are still in an idle state, resulting in insufficient utilization of each computing power resource.

[0004] In view of the above problems, no effective solution has been proposed yet. Summary of the Invention

[0005] Embodiments of this application provide a method, an apparatus, and a non-volatile storage medium for allocating computing power resources, so as to at least solve the technical problem that when allocating computing power resources for user services, the allocation of computing power resources of different computing power nodes in the computing power network is unbalanced, resulting in low resource utilization rate.

[0006] According to one aspect of the embodiments of the present application, a method for allocating computing power resources is provided, including: obtaining the service requirements of a user, where the service requirements at least include a target service and the size of the computing power resources corresponding to the target service; dividing the service requirements into a preset number of subtasks, and obtaining the environmental resource information of the computing power network, where each subtask at least includes the subtask data volume and the size of the subtask computing power resources, and the environmental resource information at least includes the computing power resource information of the computing power nodes in the computing power network; determining the preset number of subtasks and the environmental resource information as environmental state data; analyzing the environmental state data by using a computing power resource allocation model to obtain a computing power resource allocation result, where the computing power resource allocation model is used to determine the computing power resource allocation result based on the environmental state data; and allocating computing power resources to the computing power nodes corresponding to the preset number of subtasks based on the computing power resource allocation result.

[0007] In some embodiments of the present application, the computing power resource information of the computing power node at least includes the available computing power resource size of the computing power node and the available storage resource size of the computing power node.

[0008] In some embodiments of the present application, analyzing the environmental state data by using a computing power resource allocation model to obtain a computing power resource allocation result includes: determining a first environmental state of the computing power resource allocation model based on the environmental state data; under the constraint of an objective function, determining a reward score corresponding to the computing power allocation strategy in the first environmental state through a prediction network of the computing power resource allocation model, where the objective function is used to minimize the consumption cost of all subtasks, and the consumption cost includes the user terminal processing cost corresponding to the subtask and the computing power node processing cost; and determining the computing power allocation strategy with the highest reward score as the computing power resource allocation result.

[0009] In some embodiments of the present application, when the Boolean variable corresponding to the subtask is a first identifier, the subtask is executed by the user terminal, and the user terminal processing cost is determined based on a first time delay, a first preset coefficient corresponding to the first time delay, a first energy consumption, and a second preset coefficient corresponding to the first energy consumption.

[0010] In some embodiments of the present application, when the Boolean variable corresponding to the subtask is a second identifier, the subtask is executed by the computing power node, and the computing power node processing cost is determined based on a second time delay, a third preset coefficient corresponding to the second time delay, a second energy consumption, and a fourth preset coefficient corresponding to the second energy consumption.

[0011] In some embodiments of the present application, the consumption cost is determined based on the first identifier, the second identifier, the computing power node processing cost, the user terminal processing cost, and a fifth preset coefficient.

[0012] In some embodiments of the present application, the computing power resource allocation model is trained in the following manner: determining the second environmental state of the initial computing power resource allocation model based on the historical environmental state data of the computing power network; iteratively executing the following steps until the preset number of iterations is reached, and then stopping the iteration to obtain the computing power resource allocation model: Step 1: In the second environmental state, determining the first action through the prediction network of the computing power resource allocation model, where the first action is used to indicate one or a set of computing power resource allocation strategies in the second environmental state; Step 2: Executing the first action in the computing power network to obtain the first reward score and the third environmental state, and determining the second reward score in the third environmental state through the target network of the computing power resource allocation model; Step 3: Determining the first action, the first reward score, the second environmental state, and the third environmental state as a learning experience tuple, storing the learning experience tuple in the experience pool of the computing power resource allocation model, and determining the second environmental state as the new second environmental state; Step 4: Incrementing the number of iterations by 1, where the initial value of the number of iterations is 0.

[0013] In some embodiments of the present application, the method further includes: obtaining the number of iterations; in the case where the number of iterations is a multiple of the preset update frequency, updating the parameters of the target network to the parameters of the prediction network.

[0014] In some embodiments of the present application, the method further includes: determining the loss value of the computing power resource allocation model based on the loss function; sorting the learning experience tuples in the experience pool in descending order according to the loss value to obtain a learning experience tuple sequence; determining the learning experience tuples ranked at the preset number of digits in the learning experience tuple sequence as the target learning experience tuple set; in the case where the number of iterations is equal to the preset experience replay cycle number, clearing the learning experience tuples in the experience pool except the target learning experience tuple set.

[0015] According to another aspect of the embodiments of the present application, there is also provided an apparatus for allocating computing power resources, including: an acquisition module, configured to acquire the service requirements of a user, where the service requirements at least include a target service and the size of the computing power resources corresponding to the target service; a division module, configured to divide the service requirements into a preset number of subtasks, and acquire the environmental resource information of the computing power network, where each subtask at least includes the subtask data volume and the size of the computing power resources of the subtask, and the environmental resource information at least includes the computing power resource information of the computing power nodes in the computing power network; a determination module, configured to determine the preset number of subtasks and the environmental resource information as environmental state data; an analysis module, configured to analyze the environmental state data by using a computing power resource allocation model to obtain a computing power resource allocation result, where the computing power resource allocation model is used to determine the computing power resource allocation result based on the environmental state data; an allocation module, configured to allocate computing power resources to the computing power nodes corresponding to the preset number of subtasks based on the computing power resource allocation result.

[0016] According to another aspect of the embodiments of the present application, there is also provided a non-volatile storage medium storing a program, wherein when the program runs, it controls the device where the non-volatile storage medium is located to execute the computing power resource allocation method of any one of the above.

[0017] According to another aspect of the embodiments of the present application, there is also provided an electronic device, including: a memory and a processor, the processor is used to run the program stored in the memory, wherein when the program runs, it executes the computing power resource allocation method of any one of the above.

[0018] According to another aspect of the embodiments of the present application, there is also provided a computer program product, including computer instructions, and the computer instructions are executed by the processor to execute the computing power resource allocation method of any one of the above.

[0019] In the embodiments of the present application, by obtaining the business requirements of the user, where the business requirements at least include the target business and the size of the computing power resources corresponding to the target business; dividing the business requirements into a preset number of subtasks, and obtaining the environmental resource information of the computing power network, where each subtask at least includes the subtask data volume and the size of the subtask computing power resources, and the environmental resource information at least includes the computing power resource information of the computing power nodes in the computing power network; determining the preset number of subtasks and the environmental resource information as environmental state data; using a computing power resource allocation model to analyze the environmental state data to obtain a computing power resource allocation result, where the computing power resource allocation model is used to determine the computing power resource allocation result based on the environmental state data; based on the computing power resource allocation result, allocating computing power resources to the computing power nodes corresponding to the preset number of subtasks. By dividing the business requirements of the user into multiple subtasks, then understanding and analyzing the resource usage of the current computing power network, determining the subtasks and the environmental resource information as environmental state data, using the computing power resource allocation model to analyze the environmental state data to obtain a computing power resource allocation result, and finally allocating computing power resources to the computing power nodes corresponding to the preset number of subtasks based on the computing power resource allocation result, it is ensured that the computing power nodes corresponding to the preset number of subtasks are allocated computing power resources, avoiding the situation of unbalanced computing power resource allocation, and further solving the technical problem that when allocating computing power resources for user services, the computing power resources of different computing power nodes in the computing power network are unbalanced, resulting in low resource utilization rate. Description of the Drawings

[0020] The drawings described herein are used to provide a further understanding of the present application, and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application, and do not constitute an improper limitation of the present application. In the drawings:

[0021] Figure 1It is a hardware structure block diagram of a computer terminal for implementing a method for allocating computing power resources according to an embodiment of the present application;

[0022] Figure 2 It is a flowchart of a method for allocating computing power resources according to an embodiment of the present application;

[0023] Figure 3 It is a flowchart of another method for allocating computing power resources according to an embodiment of the present application;

[0024] Figure 4 It is a training flowchart of a computing power resource allocation model according to an embodiment of the present application. Detailed implementation manners

[0025] In order to enable those skilled in the art to better understand the solutions of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0026] The information collected in the embodiments of the present application is information and data authorized by the user or fully authorized by all parties. Moreover, for the processing of relevant data such as collection, storage, use, processing, transmission, provision, disclosure, and application, all comply with the relevant laws, regulations, and standards of the relevant regions, necessary confidentiality measures are taken, it does not violate public order and good customs, and a corresponding operation entry is provided for the user to choose to authorize or reject the automated decision result; if the user chooses to reject, the expert decision-making process will be entered.

[0027] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily need to be used to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of the present application described here can be implemented in an order other than those illustrated or described here. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products, or devices.

[0028] In order to better understand the embodiments of the present application, the technical terms involved in the embodiments of the present application are explained as follows:

[0029] Computing power network: A new type of information infrastructure that allocates and flexibly schedules computing resources, storage resources, and network resources on demand among cloud, network, edge, and terminal according to business requirements. It includes resources such as computing, network, and storage of nodes, can connect and integrate distributed computing nodes, and overall schedule. Through the improvement of network architecture and protocols, it realizes the optimization and efficient utilization of network and computing resources. Through ubiquitous network connections, the computing power network integrates multi-level computing power, storage, etc., connects computing power resources that are relatively geographically dispersed, overall allocates and schedules computing tasks, and provides the best resource allocation plan for the industry according to different business requirements, thereby realizing the on-demand optimal allocation and use of the entire network resources.

[0030] Deep Q-Network (abbreviated as DQN): A deep reinforcement learning algorithm that combines traditional Q-learning algorithm with deep learning technology to handle decision-making problems in high-dimensional input states. The core idea of DQN is to use a deep neural network to approximate the Q function in Q-learning (a value iteration algorithm), so as to be able to learn the relationship between actions and values in a complex high-dimensional input space. This algorithm is particularly suitable for application scenarios such as image processing and game control because it can directly learn from pixel-level inputs and make decisions.

[0031] In the related art, when allocating computing power resources for user services, the computing power resource pools of some computing power nodes are overloaded, while the computing power resource pools of some nodes are still idle, resulting in insufficient utilization of each computing power resource. Therefore, there is a technical problem that when allocating computing power resources for user services, the allocation of computing power resources of different computing power nodes in the computing power network is unbalanced, resulting in low resource utilization. To solve this problem, relevant solutions are provided in the embodiments of this application, which are described in detail below.

[0032] According to the embodiments of this application, an embodiment of a method for allocating computing power resources is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0033] The method embodiments provided by the embodiments of this application can be executed in a computer terminal or a similar computing device. Figure 1 A hardware structure block diagram of a computer terminal for implementing a method for allocating computing power resources is shown. As Figure 1As shown, the computer terminal 10 may include one or more processors 102 (shown as 102a, 102b, ……, 102n in the figure) (the processor 102 may include, but is not limited to, a processing device such as a microprocessor MCU or a field-programmable gate array FPGA), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it may further include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the BUS bus), a network interface, a power supply, and / or a camera. Those of ordinary skill in the art can understand that Figure 1 the structure shown is only schematic and does not limit the structure of the above-mentioned electronic device. For example, the computer terminal 10 may further include more or fewer components than Figure 1 shown in Figure 1 or have a different configuration from

[0034] It should be noted that the above one or more processors 102 and / or other data processing circuits are generally referred to as "data processing circuits" herein. The data processing circuit may be embodied in software, hardware, firmware, or any combination thereof, in whole or in part. In addition, the data processing circuit may be a single independent processing module, or be incorporated in whole or in part into any one of the other elements in the computer terminal 10. As involved in the embodiments of the present application, the data processing circuit is a kind of processor control (such as the selection of a variable resistor terminal path connected to an interface).

[0035] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the computing power resource allocation method in the embodiments of the present application. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, realizes the above-mentioned computing power resource allocation. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some instances, the memory 104 may further include a memory remotely set relative to the processor 102, and these remote memories can be connected to the computer terminal 10 through a network. Examples of the above network include, but are not limited to, the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0036] The transmission device 106 is used to receive or send data via a network. Specific examples of the above-mentioned network may include a wireless network provided by the communication provider of the computer terminal 10. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a Radio Frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0037] The display can be, for example, a touch-screen liquid crystal display (LCD), which enables the user to interact with the user interface of the computer terminal 10.

[0038] Under the above operating environment, an embodiment of the present application provides an embodiment of the allocation of computing power resources. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0039] As Figure 2 shown, it is a flowchart of a method for allocating computing power resources provided by an embodiment of the present application, including:

[0040] Step S202, obtain the service requirements of the user, where the service requirements at least include the target service and the size of the computing power resources corresponding to the target service.

[0041] In the technical solution provided in step S202, the computing power network scenario consists of users, base stations, computing power nodes, and computing power network resource controllers. The base station is connected to the computing power nodes through the computing power network to establish a communication link. The computing power network resource controller makes a comprehensive computing power resource allocation strategy according to the amount of computing task data, network status, and computing capabilities at each node. Computing power usually refers to the speed and ability of a computer system to process data, which can be measured by the performance of a central processing unit (CPU), a graphics processing unit (GPU), a tensor processing unit (TPU), or other dedicated chips. A computing power node is a component in the computing power network that has independent computing power resources. These nodes can be high-performance servers with computing capabilities, supercomputers, cloud computing platforms, edge computing devices, or any hardware device with computing capabilities. Computing power nodes play the role of data processing and task execution in the computing power network.

[0042] Business requirements refer to the computing power resources required for specific operations initiated by users, which include at least the target operation and the size of the computing power resources corresponding to the target operation, and may also include the operation deployment location. The target operation refers to a specific task or service that a user expects to execute in a computing power network. This can be any form of computationally intensive task, such as the training of machine learning models, big data analysis, graphics rendering, video transcoding, the execution of complex algorithms, etc. The types and content of target operations may vary according to the specific needs and scenarios of users, but all require a certain amount of computing power resources to be completed. The size of the computing power resources corresponding to the target operation refers to the measurement of the computing resources required to complete the target operation. For example, this measurement is usually expressed in terms of indicators of hardware resources such as the number of CPU cores, computing speed (such as floating-point operations per second), the number of GPU cores, memory size, hard disk or solid-state drive storage capacity, network bandwidth, etc. In some cases, the size of the computing power resources may also include software resources, such as the support level of specific computing frameworks, libraries, or platforms. The size of the computing power resources required by business requirements may vary according to factors such as the computational complexity of the operation, the size of the data volume, and the expected execution speed. Specifically: users submit business requirements through dedicated interfaces or application programs, and these requirements may be given in the form of tables, interface call parameters, configuration files, or direct text descriptions. After receiving a user request, the computing power network resource controller will parse these requirements and extract the type of the target operation and the size of the required computing power resources.

[0043] Step S204: Divide the business requirements into a preset number of subtasks, and obtain the environmental resource information of the computing power network. Each subtask includes at least the subtask data volume and the size of the subtask computing power resources, and the environmental resource information includes at least the computing power resource information of the computing power nodes in the computing power network. The computing power resource information of the computing power nodes includes at least the size of the available computing power resources of the computing power nodes and the size of the available storage resources of the computing power nodes.

[0044] In the technical solution provided in step S202, the business requirements are analyzed to analyze the computing power requirements. Based on the characteristics and complexity of the business requirements, the business requirements are divided into a preset number of subtasks. Each subtask is a part of the business requirements and can be executed and managed independently. Each subtask corresponds to a computing power node. The subtask data volume refers to the size of the data that each subtask needs to process, including the size of the input data, intermediate data, or output data, and is an important basis for allocating storage resources. The subtask computing power resource size reflects the computing power required to complete the subtask, including the computing speed and the number of cores of the CPU, GPU, TPU, etc., as well as the size of the memory. This is a key parameter for allocating computing resources. Each subtask may also include the number of storage resources required by the subtask. The environmental resource information is data about the current state of the computing power network, which helps the system understand the available resources in the network and their status, so as to make reasonable resource allocation decisions. The environmental resource information of the computing power network includes the computing power resource information of the computing power nodes in the computing power network. The computing power resource information includes at least: the available computing power resource size of the computing power node: This refers to the current available computing power of each computing power node, including the computing speed and memory capacity of hardware such as the CPU and GPU. The available storage resource size of the computing power node: This involves the available space size of the storage device (such as a hard disk) of each computing power node, which is particularly important for large data processing tasks.

[0045] Step S206, determine the preset number of subtasks and the environmental resource information as the environmental state data.

[0046] Through the division of subtasks of the business requirements and the acquisition of environmental resource information, a more detailed "environmental state data" can be constructed, which will be used as the input of the computing power resource allocation model. The environmental state data contains the specific requirements of the subtasks and the availability of resources in the network, enabling the resource allocation model to determine the optimal computing power resource allocation strategy based on this information. This strategy will guide the system on how to allocate resources among the computing power nodes of the computing power network to ensure that each subtask is processed efficiently and in a timely manner, while avoiding excessive consumption or waste of resources.

[0047] Step S208, use the computing power resource allocation model to analyze the environmental state data to obtain the computing power resource allocation result, where the computing power resource allocation model is used to determine the computing power resource allocation result based on the environmental state data.

[0048] In the technical solution provided in step S208, there are various implementation manners for analyzing the environmental state data by using the computing power resource allocation model to obtain the computing power resource allocation result. For example: determining a first environmental state of the computing power resource allocation model based on the environmental state data; under the constraint of the objective function, determining a reward score corresponding to the computing power allocation strategy in the first environmental state through the prediction network of the computing power resource allocation model, where the objective function is used to minimize the consumption cost of all subtasks, and the consumption cost includes the user terminal processing cost corresponding to the subtask and the computing power node processing cost; determining the computing power allocation strategy with the highest reward score as the computing power resource allocation result.

[0049] In the case where the Boolean variable corresponding to the subtask is the first identifier, the subtask is executed by the user terminal, and the user terminal processing cost is determined based on the first delay, the first preset coefficient corresponding to the first delay, the first energy consumption, and the second preset coefficient corresponding to the first energy consumption.

[0050] In the case where the Boolean variable corresponding to the subtask is the second identifier, the subtask is executed by the computing power node, and the computing power node processing cost is determined based on the second delay, the third preset coefficient corresponding to the second delay, the second energy consumption, and the fourth preset coefficient corresponding to the second energy consumption. The computing power resource information can be transmitted between Border Gateway Protocol neighbors, and while transmitting the computing power resource information, the first delay and the second delay between the user and the computing power node are measured. Different user services have different requirements for delay, and the corresponding delays are also different.

[0051] The consumption cost of all subtasks is determined based on the first identifier, the second identifier, the computing power node processing cost, the user terminal processing cost, and the fifth preset coefficient.

[0052] The following are specific embodiments:

[0053] Obtain environmental status data and preprocess the environmental status data, such as data cleaning, format conversion, normalization, etc., to ensure that the model can correctly parse and use this information. Extract features for model analysis from the preprocessed environmental status data. For example, it includes the computing power resource information of computing power nodes, the computing requirements of subtasks, the latency between user terminals and computing power nodes, etc. Based on feature extraction, define a first environmental status that contains all the key information in the environmental status data. This status is usually represented by a set of numerical vectors, where each dimension corresponds to a feature. For example, one dimension may represent the CPU utilization rate of a computing power node, and another dimension may represent the data processing ability of a user terminal. These state vectors will be input into the prediction network of the computing power resource allocation model as the starting point for computing power resource allocation analysis and decision-making. The first environmental status of the computing power resource allocation model is the "status" before the model starts analysis. The computing power resource allocation model will start from this status and explore the environment through a series of actions (computing power resource allocation strategies) to find the strategy that can maximize the reward (or minimize the cost). Before analyzing the environmental status data using the computing power resource allocation model, first initialize the computing power resource allocation model, including setting network parameters, defining the reward mechanism and objective function, etc. Before the model starts running, it is necessary to set the initial parameters of the prediction network (for example, the prediction Q-network) and the target network (for example, the target Q-network), as well as the initial state of the experience pool.

[0054] The computing power resource allocation model adopts an improved DQN resource scheduling algorithm. The target network mechanism adopts a dual Q-network approach, namely a prediction network (e.g., a prediction Q-network) and a target network (e.g., a target Q-network). The prediction Q-network and the target Q-network are the same, but the parameters of the target Q-network are fixed for a period of time before being updated. The computing power resource allocation model includes a prediction network, a target network, and an experience pool. The prediction network is the main working network of the computing power resource allocation model. Its task is to predict the reward scores (i.e., Q-values, referring to the expected long-term rewards) of various possible resource allocation strategies (i.e., actions) based on the current environmental state data (i.e., the first environmental state). The output of the prediction network is the reward score for each action, and these estimates will be used to guide the model's decision-making in the current environmental state, that is, to select which resource allocation strategy to execute. The target network has exactly the same structure as the prediction network, but its parameters are fixed for a period of time and copied from the prediction network. The main role of the target network is to provide a stable and unchanging reference point for the prediction network, which is used to calculate the loss function and update the parameters of the prediction network. In traditional DQN, the parameters of the prediction network are continuously updated during the training process, which may lead to instability of the reward scores (i.e., Q-values), making it difficult for the model to learn. The experience pool is a component in the computing power resource allocation model used to store past interaction experiences, and these experiences exist in the form of learning experience tuples. The experience pool has two functions: one is to store a large amount of historical interaction information to provide learning instances for the model; the other is to implement experience replay. By randomly sampling learning experience tuples from the experience pool for learning, it can avoid overfitting caused by the correlation between data, and at the same time enable the model to learn more extensive and complex computing power resource allocation strategies from historical experiences.

[0055] When analyzing environmental state data based on the computing power resource allocation model, it is necessary to satisfy the constraints of the objective function. The objective function is used to minimize the consumption costs of all subtasks. The consumption costs include the processing costs of the user terminals corresponding to the subtasks and the processing costs of the computing power nodes. For example, the objective function can be expressed by the following formula:

[0056] Objective function = Min Cost all ,

[0057] where, Min Cost all represents minimizing the consumption costs of all subtasks, and Cost all represents the consumption costs of all subtasks.

[0058] The consumption costs of all subtasks are the sum of the consumption costs of each subtask. For example, it is expressed by the following formula:

[0059]

[0060] where, Costi Represents the total cost consumed by the subtask, where i is an integer between 1 and n, and n is an integer greater than 1.

[0061] The consumption cost of any subtask is determined based on the first identifier, the second identifier, the processing cost of the computing power node, the processing cost of the user terminal, and the fifth preset coefficient. For example, it can be expressed by the following formula:

[0062] Cost i = f(A i ) * Cost i,local + (1 - f(A i )Cost i,j ),

[0063] where Cost i is the consumption cost of the i-th subtask A i , f(A i ) represents a boolean variable. When f(A i ) is the first identifier (e.g., 1), the subtask is executed by the user terminal; when it is the second identifier (e.g., 0), the subtask is executed by the computing power node. Cost i,local represents the processing cost of the user terminal corresponding to the subtask, and Cost i,j represents the processing cost of the computing power node corresponding to the subtask.

[0064] The processing cost of the user terminal corresponding to the subtask is determined based on the first delay, the first preset coefficient corresponding to the first delay, the first energy consumption, and the second preset coefficient corresponding to the first energy consumption. For example, it can be expressed by the following formula:

[0065] Cost i,local = αT i，local + βE i，local ,

[0066] where Cost i,local represents the processing cost of the user terminal corresponding to the subtask, T i，local represents the first delay (i.e., the time required for the i-th subtask on the user terminal), α represents the first preset coefficient corresponding to the first delay, E i，local represents the first energy consumption (i.e., the energy consumption required for the i-th subtask on the user terminal), β represents the second preset coefficient corresponding to the first energy consumption, and the sum of α and β is 1.

[0067] The processing cost of the computing power node corresponding to the subtask is determined based on the second delay, the third preset coefficient corresponding to the second delay, the second energy consumption, and the fourth preset coefficient corresponding to the second energy consumption. For example, it can be expressed by the following formula:

[0068]

[0069] Among them, Cost i,j represents the processing cost of the computing power node corresponding to the subtask, j represents the computing power node, b represents the third preset coefficient corresponding to the second time delay (b and α can be the same or different), represents the second time delay (i.e., the time required for subtask i to be processed at the computing power node), represents the second energy consumption (i.e., the energy consumption required for subtask i to be processed at the computing power node), c represents the fourth preset coefficient corresponding to the second energy consumption (c and β can be the same or different), and the sum of the third preset coefficient and the fourth preset coefficient is 1.

[0070] The prediction network receives the current first environmental state as input. Given the environmental state data, through its internal neural network structure, the prediction network predicts a reward score for each possible computing power allocation strategy (i.e., action). This reward score reflects the long-term effect that is expected to be achieved after adopting the corresponding strategy, especially the degree of compliance with the objective function (cost minimization). The calculation of the reward score is based on the objective function, which measures the impact of the resource allocation strategy on the consumption cost. Specifically, a high reward score means a low consumption cost and a better strategy. The prediction network comprehensively evaluates the advantages and disadvantages of different strategies by calculating the cost when each subtask is executed at the user terminal or the computing power node, combined with factors such as the priority of the task, network latency, and energy consumption. After evaluating all possible computing power allocation strategies, the model (i.e., the computing power resource allocation model) will select the allocation strategy with the highest reward score as the computing power resource allocation result. This means that the model will adopt this strategy to guide the actual allocation of computing power resources in order to achieve the goal of minimizing the consumption cost. The computing power resource allocation result specifically specifies how to optimally allocate the resources in the computing power network to each subtask to meet the user service requirements, while ensuring the efficient use of resources and the effective control of costs. The computing power resource allocation result contains the following key information: Resource allocation strategy: This is the core part of the model output, which details which computing power node each subtask should be allocated to and the specific size of the computing power resources allocated. Each subtask corresponds to a computing power node. Allocation of computing power nodes: The allocation result will clearly indicate which computing power nodes are selected to execute the subtasks and which subtask's computing and storage requirements each computing power node will undertake. Task scheduling information: In addition to resource allocation, the allocation result may also contain detailed information about task scheduling, such as the execution order of subtasks, the possibility of parallel computing, and how to balance the task load among multiple nodes. Cost and performance metrics: The allocation result usually also comes with a cost analysis and performance prediction, such as the expected total consumption cost (including the processing cost of the user terminal and the processing cost of the computing power node), the task completion time, and the usage of network resources. For example Figure 3As shown, it is a flowchart of another method for allocating computing power resources provided according to an embodiment of the present application. S3.1 determines the task execution cost function (i.e., the consumption cost of the above-mentioned subtasks), S3.2 determines the objective function, S3.3 establishes a computing power resource allocation model, and analyzes the environmental state data based on the established computing power resource allocation model to obtain the computing power resource allocation result. The output of the entire neural network will be mainly based on the output of the target neural network, which can reduce the influence of "overestimation"; the closer to convergence, the more it will change to be mainly based on the output of the prediction network, so that the target network and the prediction network jointly determine the final network output to reduce the influence of "overestimation" and ensure the stability of the neural network.

[0071] The above computing power resource allocation model is trained in the following manner:

[0072] First, based on the historical environmental state data of the computing power network, determine the second environmental state of the initial computing power resource allocation model, set a preset number of iterations, initialize the initial computing power resource allocation model, as well as the prediction network (prediction Q network) and the target network (target Q network), and iteratively execute the following steps until the preset number of iterations is reached, then stop the iteration to obtain the computing power resource allocation model:

[0073] Step 1: In the second environmental state, determine the first action a through the prediction network of the computing power resource allocation model t , for example, the action refers to allocating a certain proportion of computing power resources to a specific task or computing power node, and determine the first action a based on the greedy strategy t :

[0074] Among them, t represents the time, ε is a constant less than 1 preset based on the greedy strategy, and with a probability (p) of 1 - ε, randomly select an action (random action) as the first action a t , this is called exploration. In order to discover new potentially valuable actions, with a probability (p = ε), the action that maximizes Q(S t , a t , θ1) will be selected as the first action a, where the Q function is a value function used to evaluate the long-term value of the first action a in the state S t , a t , θ1), and θ1 is the parameter of the prediction network. t Step 2: Execute the first action a in the computing power network t , obtain the first reward score r t and the third environmental state S

[0075] t+1 t i t+1 t+1 t+1, and determine the second reward score in the third environmental state (the next environmental state of the second environmental state) through the target network of the computing power resource allocation model; specifically:

[0076] In the last step of the last iteration, the second reward score y i (also known as the predicted reward value) is equal to the current first reward score r i . In other cases, the second reward score y i is equal to the first reward score r i plus the discount factor γ (pre-set) multiplied by the maximum Q value of the target network (target Q-network) for the next state, the third environmental state S t+1 . The maximum Q value of the target network (target Q-network) for the next state S t+1 is determined as follows: The target network evaluates the Q values of all possible actions and selects the maximum Q value as the predicted value of the future reward. The maximum Q value represents the maximum value of the expected cumulative reward that can be obtained by taking the optimal action starting from the state S t+1 .

[0077] Step 3: Determine the first action a t , the first reward score r i , the second environmental state S t and the third environmental state S t+1 as the learning experience tuple (S t , a t , r t , S t+1 ), and store the learning experience tuple (S t , a t , r t , S t+1 ) in the experience pool of the computing power resource allocation model, and determine the second environmental state as the new second environmental state. At the same time, execute Step 4 after each iteration, and increment the iteration count by 1. The initial value of the iteration count is 0.

[0078] During the above iteration process, the parameters of the target network are updated as follows: Obtain the iteration count; When the iteration count is a multiple of the preset update frequency (for example, represented by U, which represents the frequency of updating the parameters of the target network, and U specifies how many iterations to perform before copying the parameters of the prediction network to the target network), update the parameters of the target network to the parameters of the prediction network. This can regularly update the target network to make it close to the prediction network.

[0079] Meanwhile, during the iteration process, for performance stability, experience replay is carried out, and the loss value of the computing power resource allocation model is determined based on the loss function. For example, the loss function is represented by the following formula:

[0080] Loss = CrossEntropy(output, action),

[0081] where Loss represents the loss value of the computing power resource allocation model, CrossEntropy refers to the loss function (such as the cross-entropy loss function), output refers to the output of the prediction network (i.e., the prediction Q-network) (specifically the first reward score r i ), and action represents the action actually executed in the current state. In reinforcement learning, the agent needs to select an action from the action space to execute according to a certain policy, and action is the selected action, such as the first action mentioned above.

[0082] Based on the loss value, the learning experience tuples in the experience pool are sorted in descending order (i.e., the learning experience tuples are arranged in descending order of the loss value) to obtain a sequence of learning experience tuples; the learning experience tuples ranked at a preset position (for example, the m - nth position, where m is in the middle of the sequence of learning experience tuples, and n is greater than m but less than or equal to the last position of the sequence of learning experience tuples. Different values of m and n are selected in different stages. In the initial stage of model learning, the neural network needs to focus on learning new knowledge, and at this time, the values of m and n are taken as smaller values; as learning progresses, the neural network should focus on replaying historical experiences to stabilize performance, and at this time, the values of m and n should be taken as larger values) in the sequence of learning experience tuples are determined as the set of target learning experience tuples. After selecting the set of target learning experience tuples, experience replay is carried out, that is, these learning experience tuples are used to update the parameters of the prediction network to reduce the loss value. This process usually lasts for multiple replay cycles until the number of iterations reaches the preset number of experience replay cycles (which is set in advance). When the number of iterations is equal to the preset number of experience replay cycles, the learning experience tuples in the experience pool except the set of target learning experience tuples are cleared (another optional method is to directly clear the experience pool).

[0083] Step S210, allocate computing power resources to the computing power nodes corresponding to a preset number of subtasks based on the computing power resource allocation result.

[0084] Once the computing power resource allocation result is determined, computing power resources are allocated to the computing power nodes corresponding to a preset number of subtasks based on the result. To continuously optimize the computing power resource allocation model, observe the actual effects generated, including costs, latency, energy consumption, etc. Subsequently, the computing power resource allocation model will adjust its parameters according to the difference between the actual effect and the expected reward, and continuously optimize. Through this mechanism, the computing power resource allocation model can intelligently handle the dynamic computing power network environment, respond to complex user service requirements, and achieve efficient resource allocation. In practical applications, this process will continue, and the model will continuously learn and adapt to environmental changes to achieve the optimal computing power resource allocation strategy. This strategy can not only optimize resource utilization but also improve the user's service processing efficiency and reduce the operating cost of the entire system.

[0085] Through the above steps, using the improved DQN algorithm for computing power resource allocation, matching the user's computing tasks (target services) with computing power nodes, and then scheduling the computing tasks to the corresponding computing power nodes, can achieve effective computing power resource allocation, solve the feedback loop problem in the training process, improve the stability and training efficiency of the algorithm, and can dynamically sense the computing power node resource information, perform intelligent matching according to different service requirements and computing power network resources, comprehensively consider latency and energy consumption, and improve the utilization rate of computing resources and network resources as much as possible to reduce system costs.

[0086] Figure 4 It is a schematic structural diagram of an allocation device for computing power resources provided by an embodiment of the present application, including:

[0087] An acquisition module 402, configured to acquire the service requirements of a user, where the service requirements at least include a target service and the size of the computing power resources corresponding to the target service;

[0088] A division module 404, configured to divide the service requirements into a preset number of subtasks and acquire the environmental resource information of the computing power network, where each subtask at least includes the subtask data volume and the size of the subtask computing power resources, and the environmental resource information at least includes the computing power resource information of the computing power nodes in the computing power network.

[0089] A determination module 406, configured to determine the preset number of subtasks and the environmental resource information as environmental state data.

[0090] An analysis module 408, configured to analyze the environmental state data by using a computing power resource allocation model to obtain a computing power resource allocation result, where the computing power resource allocation model is used to determine the computing power resource allocation result based on the environmental state data.

[0091] An allocation module 410, configured to allocate computing power resources to the computing power nodes corresponding to a preset number of subtasks based on the computing power resource allocation result.

[0092] It should be noted that Figure 4 the shown computing power resource allocation device is used to execute Figure 2 the shown computing power resource allocation method. Therefore Figure 2 the relevant explanations in the computing power resource allocation method in also apply to this computing power resource allocation device and will not be elaborated here.

[0093] It should be noted that each module in the above-mentioned computing power resource allocation device can be a program module (for example, a set of program instructions that implements a specific function), or a hardware module. For the latter, it can be presented in the following forms, but not limited to this: the presentation form of each of the above modules is a processor, or the functions of each of the above modules are implemented by a processor.

[0094] The embodiment of the present application also provides a non-volatile storage medium. The non-volatile storage medium includes a stored program. When the program runs, it controls the device where the non-volatile storage medium is located to execute the above-mentioned computing power resource allocation method. For example, obtain the service requirements of the user, where the service requirements at least include the target service and the size of the computing power resources corresponding to the target service; divide the service requirements into a preset number of subtasks, and obtain the environmental resource information of the computing power network. Each subtask at least includes the subtask data volume and the subtask computing power resource size, and the environmental resource information at least includes the computing power resource information of the computing power nodes in the computing power network; determine the preset number of subtasks and the environmental resource information as environmental state data; use the computing power resource allocation model to analyze the environmental state data to obtain a computing power resource allocation result, where the computing power resource allocation model is used to determine the computing power resource allocation result based on the environmental state data; allocate computing power resources to the computing power nodes corresponding to the preset number of subtasks based on the computing power resource allocation result.

[0095] The embodiment of the present application also provides an electronic device. The electronic device includes a processor, and the processor is used to run a program. When the program runs, it executes the above-mentioned computing power resource allocation method. For example, obtain the service requirements of the user, where the service requirements at least include the target service and the size of the computing power resources corresponding to the target service; divide the service requirements into a preset number of subtasks, and obtain the environmental resource information of the computing power network. Each subtask at least includes the subtask data volume and the subtask computing power resource size, and the environmental resource information at least includes the computing power resource information of the computing power nodes in the computing power network; determine the preset number of subtasks and the environmental resource information as environmental state data; use the computing power resource allocation model to analyze the environmental state data to obtain a computing power resource allocation result, where the computing power resource allocation model is used to determine the computing power resource allocation result based on the environmental state data; allocate computing power resources to the computing power nodes corresponding to the preset number of subtasks based on the computing power resource allocation result.

[0096] According to another aspect of the embodiments of the present application, a computer program product is further provided, including a computer program which, when executed by a processor, implements the above-mentioned computing power resource allocation method. For example, obtain the service requirements of a user, where the service requirements at least include a target service and the size of the computing power resources corresponding to the target service; divide the service requirements into a preset number of subtasks, and obtain the environmental resource information of the computing power network, where each subtask at least includes the subtask data volume and the size of the computing power resources of the subtask, and the environmental resource information at least includes the computing power resource information of the computing power nodes in the computing power network; determine the preset number of subtasks and the environmental resource information as environmental state data; analyze the environmental state data by using a computing power resource allocation model to obtain a computing power resource allocation result, where the computing power resource allocation model is used to determine the computing power resource allocation result based on the environmental state data; allocate computing power resources to the computing power nodes corresponding to the preset number of subtasks based on the computing power resource allocation result.

[0097] In the above embodiments of the present application, the descriptions of the respective embodiments have their own emphases. For parts not detailed in a certain embodiment, reference may be made to the relevant descriptions of other embodiments.

[0098] In several embodiments provided by the present application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are merely illustrative. For example, the division of the units can be a logical function division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection to each other can be through some interfaces, and the indirect coupling or communication connection of units or modules can be in an electrical or other form.

[0099] The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they can be located in one place, or can be distributed to multiple units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0100] In addition, the functional units in the various embodiments of the present application can be integrated in one processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

[0101] When the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the related technology, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of this application. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, read-only memories (ROMs), random access memories (RAMs), mobile hard disks, magnetic disks, or optical discs.

[0102] The above are only the preferred embodiments of this application. It should be noted that for those of ordinary skill in the art, without departing from the principle of this application, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of this application.

Claims

1. A method for allocating computing resources, characterized in that: include: Obtaining the user's business requirements, wherein the business requirements at least include the target business and the size of computing resources corresponding to the target business; Divide the business demand into a preset number of subtasks, and obtain environmental resource information of the computing network, wherein each subtask includes at least a subtask data volume and a subtask computing resource size, and the environmental resource information includes at least computing resource information of computing nodes in the computing network; Determining the preset number of subtasks and the environmental resource information as environmental status data; The environment status data is analyzed using a computing power resource allocation model to obtain a computing power resource allocation result, wherein the computing power resource allocation model is used to determine the computing power resource allocation result based on the environment status data; Based on the computing power resource allocation result, computing power resources are allocated to the computing power nodes corresponding to the preset number of subtasks.

2. The method according to claim 1, characterized in that The computing power resource information of the computing power node includes at least the available computing power resource size of the computing power node and the available storage resource size of the computing power node.

3. The method according to claim 1, characterized in that The computing power resource allocation model is used to analyze the environmental status data to obtain computing power resource allocation results, including: Determine a first environmental state of a computing resource allocation model based on the environmental state data; Under the constraint of the objective function, determining the reward score corresponding to the computing power allocation strategy under the first environmental state through the prediction network of the computing power resource allocation model, wherein the objective function is used to minimize the consumption cost of all subtasks, and the consumption cost includes the user terminal processing cost and the computing power node processing cost corresponding to the subtask; The computing power allocation strategy with the highest reward score is determined as the computing power resource allocation result.

4. The method according to claim 3, characterized in that When the Boolean variable corresponding to the subtask is a first identifier, the subtask is executed by a user terminal, and the processing cost of the user terminal is determined based on a first delay, a first preset coefficient corresponding to the first delay, a first energy consumption, and a second preset coefficient corresponding to the first energy consumption.

5. The method according to claim 4, characterized in that When the Boolean variable corresponding to the subtask is the second identifier, the subtask is executed by the computing power node, and the processing cost of the computing power node is determined based on the second delay, the third preset coefficient corresponding to the second delay, the second energy consumption and the fourth preset coefficient corresponding to the second energy consumption.

6. The method according to claim 5, characterized in that The consumption cost is determined based on the first identifier, the second identifier, the computing power node processing cost, the user terminal processing cost and a fifth preset coefficient.

7. The method according to claim 1, characterized in that The computing resource allocation model is trained in the following way: Determining a second environmental state of the initial computing power resource allocation model based on historical environmental state data of the computing power network; Iterate the following steps until the preset number of iterations is reached, stop iterating, and obtain the computing power resource allocation model: Step 1: Under the second environment state, determine a first action through the prediction network of the computing resource allocation model, wherein the first action is used to indicate one or a group of computing resource allocation strategies under the second environment state; Step 2: Execute the first action in the computing power network to obtain a first reward score and a third environment state, and determine a second reward score under the third environment state through the target network of the computing power resource allocation model; Step 3: Determine the first action, the first reward score, the second environment state, and the third environment state as a learning experience tuple, store the learning experience tuple in the experience pool of the computing resource allocation model, and determine the second environment state as a new second environment state; Step 4: Add 1 to the number of iterations, where the initial value of the number of iterations is 0.

8. The method according to claim 7, characterized in that The method further comprises: Obtaining the number of iterations; When the number of iterations is a multiple of a preset update frequency, the parameters of the target network are updated to the parameters of the prediction network.

9. The method according to claim 7, characterized in that: The method further comprises: Determine a loss value of the computing power resource allocation model based on a loss function; Based on the loss value, the learning experience tuples in the experience pool are sorted in descending order to obtain a learning experience tuple sequence; Determine the learning experience tuple ranked at a preset position in the learning experience tuple sequence as a target learning experience tuple set; When the number of iterations is equal to the preset number of experience playback cycles, the learning experience tuples in the experience pool except for the target learning experience tuple set are cleared.

10. A computing resource allocation device, characterized in that: include: An acquisition module is used to acquire the business requirements of the user, wherein the business requirements at least include the target business and the size of computing resources corresponding to the target business; A division module, used to divide the business demand into a preset number of subtasks, and obtain environmental resource information of the computing network, wherein each subtask includes at least a subtask data volume and a subtask computing resource size, and the environmental resource information includes at least computing resource information of computing nodes in the computing network; A determination module, used to determine the preset number of subtasks and the environmental resource information as environmental status data; An analysis module, configured to analyze the environmental status data using a computing power resource allocation model to obtain a computing power resource allocation result, wherein the computing power resource allocation model is used to determine the computing power resource allocation result based on the environmental status data; An allocation module is used to allocate computing resources to computing nodes corresponding to the preset number of subtasks based on the computing resource allocation result.

11. A non-volatile storage medium, characterized in that: The non-volatile storage medium stores a program, wherein when the program is running, the device where the non-volatile storage medium is located is controlled to execute the computing resource allocation method described in any one of claims 1 to 9.

12. An electronic device, characterized in that: include: A memory and a processor, wherein the processor is used to run a program stored in the memory, wherein the program, when running, executes the method for allocating computing resources described in any one of claims 1 to 9.

13. A computer program product comprising computer instructions, characterized in that: When the computer instructions are executed by the processor, the computing power resource allocation method described in any one of claims 1 to 9 is implemented.

Images