Computing Power Scheduling Method, Device, System and Storage Medium

Abstract

An embodiment of the present invention provides a computing power scheduling method, device, system, and storage medium. The method includes: if the current working mode is the inference mode, perform the following operations: obtain the computing power weight information of each computing power device in the target computing power device group; determine the task allocation order according to the computing power weight information of each computing power device; and allocate each task in the task flow to the corresponding computing power device in the target computing power device group in sequence according to the task allocation order information. Through the above computing power scheduling method, the problem of waste of computing resources caused by some computing power devices being idle while waiting for another part of the computing power devices to complete calculations in the prior art can be solved, and tasks are dynamically allocated according to the real-time computing power attributes of the computing power devices, effectively utilizing the computing capabilities of each device.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular, to a computing power scheduling method, device, system, and storage medium. Background Art

[0002] In the prior art, after a user submits corresponding requirement information, a computing power network system can schedule idle computing power devices according to the user's requirements to provide corresponding computing services for the user. However, in the existing computing power scheduling method, during the computing process of the computing power devices providing computing power services for the user, some computing power devices are still in the computing state, while another part has completed the computing and is in an idle state for a period of time, that is, waiting for the above-mentioned computing power devices still in the computing state to complete the current round of computing tasks. Therefore, the existing computing power scheduling method has the problem of waste of computing power device resources. Summary of the Invention

[0003] Embodiments of the present invention provide a computing power scheduling method, device, system, and storage medium. Through the above-mentioned computing power scheduling method, the problem of waste of computing resources caused by some computing power devices being in an idle state while waiting for another part of the computing power devices to complete the computing in the prior art can be solved. By dynamically allocating tasks according to the real-time computing power attributes of the computing power devices, the computing capabilities of each device can be effectively utilized.

[0004] Embodiments of the present invention provide a computing power scheduling method, the method comprising: if the current working mode is the inference mode, then perform the following operations: obtain the computing power weight information of each computing power device in a target computing power device group; determine the task allocation order according to the computing power weight information of each computing power device; and sequentially allocate each task in a task stream to the corresponding computing power device in the target computing power device group according to the task allocation order information.

[0005] Further, determining the task allocation order according to the computing power weight information of each computing power device includes: performing multiple rounds of computing power device selection operations, obtaining the computing power device selection order after the multiple rounds of computing power device selection operations, and using the computing power device selection order as the task allocation order; wherein, each round of computing power device selection operation includes: taking the computing power device corresponding to the highest weight in the first weight information of this round as the computing power device selected in this round, wherein the weight information includes the computing power weight values of each computing power device in the target computing power device group, and the first weight information in the first round of computing power device selection operation includes the initial computing power weight values of each computing power device in the target computing power device group; calculating the difference between the current computing power weight value of the computing power device selected in this round and the sum of the weights of all computing power devices in the target computing power device group to obtain a correction value, and updating the first weight information according to the correction value to obtain the second weight information; and adding the current computing power weight values of each computing power device in the second weight information to the initial computing power weight values of the corresponding computing power devices respectively to obtain the third weight information, and using the third weight information as the first weight information for the next round of computing power device selection operation.

[0006] Further, before performing the multiple rounds of computing power device selection operations, it further includes: configuring a target selection rule according to the selection rule of the first round of computing power device selection operation; wherein the selection rule of the first round of computing power device selection operation includes: taking the computing power device corresponding to the highest weight in the first weight information of this round as the computing power device selected in this round; and / or randomly selecting a target weight in the first weight information of the first round, and taking the computing power device corresponding to the target weight as the computing power device selected in the first round.

[0007] Further, before obtaining the computing power weight information of each computing power device in the target computing power device group, it further includes: obtaining task description information, and determining the current working mode according to the task description information, and the working mode includes a training mode and an inference mode.

[0008] Further, after determining the current working mode according to the task description information, if the current working mode is the training mode, then perform the following operations: obtaining the computing power ratio information of each computing power device in the target computing power device group; dividing the first-round training data according to the computing power ratio information of each computing power device, and respectively allocating the divided first-round training data to the corresponding computing power devices; determining the allocation information of the (i + 1)-th round of training data according to the training time of each computing power device in the i-th round, dividing the (i + 1)-th round of training data according to the allocation information, and respectively allocating the divided (i + 1)-th round of training data to the corresponding computing power devices, where i is an integer greater than or equal to 1.

[0009] Further, dividing the first-round training data according to the computing power ratio information of each computing power device and distributing the divided first-round training data to the corresponding computing power devices respectively includes: according to the computing power ratio information {P1, P2, P3,..., P n}, dividing the first-round training data into multiple training sub-data in equal proportion where P n represents the computing power of the nth computing power device among each computing power device, represents the data volume of the training sub-data divided for the nth computing power device in the first-round training, and the value range of n includes {1, 2, 3,..., N}, where N represents the number of computing power devices in the target computing power device group; respectively distributing the training sub-data D n in the multiple training sub-data to the computing power device with the corresponding computing power P n .

[0010] Further, determining the distribution information of the (i + 1)-th round training data according to the training time of the i-th round of each computing power device, dividing the (i + 1)-th round training data according to the distribution information, and distributing the divided (i + 1)-th round training data to the corresponding computing power devices respectively includes: obtaining the training time of the i-th round of each computing power device where represents the training time of the i-th round of the nth computing power device; obtaining the longest training time among the training times of the i-th round of each computing power device and determining the computing power devices to be configured whose training time of the i-th round is greater than or equal to the preset time compared with the longest training time , where the preset time includes the training time of the training sub-data of the unit data volume for the computing power device to be configured; obtaining the distribution information of the i-th round training data where represents the data volume of the training sub-data divided for the nth computing power device in the i-th round of training; performing an update operation on the data volume corresponding to the computing power device to be configured in the distribution information of the i-th round training data to obtain the distribution information of the (i + 1)-th round training data where the update operation includes adding the training sub-data of the unit data volume on the basis of the training sub-data divided by the computing power device to be configured; dividing the (i + 1)-th round training data according to the distribution information of the (i + 1)-th round training data and distributing the divided (i + 1)-th round training data to the corresponding computing power devices respectively.

[0011] In a second aspect, an embodiment of the present application further provides a computing power scheduling device, where the device includes: a processor and a memory. The memory is used to store at least one instruction, and when the instruction is loaded and executed by the processor, it is used to implement the computing power scheduling method provided in the first aspect.

[0012] In a third aspect, an embodiment of the present application further provides a distributed system, where the system includes: a plurality of computing power devices for providing computing services; a user interface for obtaining user demand information of a user and determining task description information according to the user demand information, where the task description information includes one or more of the following: task type, latency requirement, accuracy requirement, memory occupancy, and working mode; a gateway for obtaining the task description information provided by the user interface and matching a target algorithm model that meets the task description information; a computing power device matching module for matching a target computing power device group that meets the task description information among the plurality of computing power devices; and the computing power scheduling device provided in the second aspect.

[0013] Further, matching the target computing power device group that meets the task description information among the plurality of computing power devices includes: screening out a plurality of candidate computing power devices that meet the task description information from the idle computing power devices among the plurality of computing power devices; and selecting some of the candidate computing power devices as the target computing power device group.

[0014] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored, and characterized in that when the computer program is executed by a processor, it implements the computing power scheduling method provided in the first aspect.

[0015] Through the above technical solution, in the inference stage, tasks are dynamically allocated according to different computing power resources according to a certain weight, avoiding some computing power devices from being idle all the time, that is, solving the problem of waste of computing resources caused by some computing power devices being idle while waiting for another part of the computing power devices to complete calculations in the prior art. By dynamically allocating tasks according to the real-time computing power attributes of the computing power devices, the computing power of each device is effectively utilized. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0017] Figure 1 It is a comparison diagram of edge computing and computing power network;

[0018] Figure 2 Flowchart of user requirement processing provided for an embodiment of this application;

[0019] Figure 3 Schematic diagram of user requirement parsing provided for an embodiment of this application;

[0020] Figure 4 Flowchart of computing power scheduling method in training mode provided for an embodiment of this application;

[0021] Figure 5 Schematic diagram of dynamically correcting task allocation amount in training mode provided for an embodiment of this application;

[0022] Figure 6 Flowchart of computing power scheduling method in inference mode provided for an embodiment of this application;

[0023] Figure 7 Schematic diagram of the structure of the computing power scheduling device provided for another embodiment of this application;

[0024] Figure 8 Distributed system architecture diagram provided for another embodiment of this application. Detailed implementation manners

[0025] To make the objectives, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Apparently, the described embodiments are some but not all of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0026] With the development of artificial intelligence, AI deployment inevitably has extensive requirements in all walks of life. The essence of the edge is the extension and evolution of cloud computing near the computing demand side. One of the core capabilities of edge computing is the network, which can integrate capabilities such as network, computing, storage, and applications at the network edge. Low-computing-power tasks are directly completed at the edge device, while high-computing-power calculations are arranged to be processed in the cloud.

[0027] Figure 1 It is a comparison diagram of edge computing and computing power network, as Figure 1 shown. In the edge computing mode, each edge device processes low-computing-power tasks locally and executes high-computing-power tasks in the cloud if encountered. In the computing power network mode, various computing devices are networked to form a unified computing network, which can automatically schedule computing tasks to the devices in the network for execution according to the task type and computing power requirements of the user.

[0028] Among them, when processing computing tasks through edge computing, due to the general conditional limitations of the deployment computer rooms for edge computing, computing-intensive operations such as model training cannot be implemented at the edge side and can only be uploaded to the cloud platform for processing. However, uploading user data to the cloud platform for model training is likely to cause security problems such as leakage of user privacy data. And the method of training the model in the cloud and then performing inference on the edge side will occupy a large amount of network bandwidth, which is likely to cause congestion in the central server line when the training data is large or there are many users. On the edge side, the computing power network not only needs to perform inference but also needs to perform online model learning. Therefore, the computing power requirement for the edge side is higher, and various computing devices in the computing power network need to be used comprehensively.

[0029] Currently, the edge computing + cloud platform model faces many problems that need to be solved, such as privacy, network bandwidth congestion, and difficult computing power scheduling. Therefore, placing the computing center locally and building a computing power network locally can solve the above problems. However, during the application of computing power, there are still some problems. For example, in the computing power scheduling problem in the computing power network, the existing computing power scheduling has the problem of waste of computing device resources.

[0030] At least to overcome the computing power scheduling problem existing in the above computing power network, the embodiments of the present application provide the following technical solutions:

[0031] Figure 2 The flowchart of user demand processing provided by an embodiment of the present application is shown in Figure 2 As shown, the user demand processing flow includes the following steps:

[0032] Step S1: Obtain the user demand information of the user, and determine the task description information by analyzing the user demand information. The task description information includes one or more of the following: task type, latency requirement, accuracy requirement, memory occupancy, and working mode. In one implementation, the task description information includes task type, latency requirement, accuracy requirement, memory occupancy, and working mode. Figure 3 The schematic diagram of user demand analysis provided by an embodiment of the present application is shown in Figure 3 As shown, the user demand information sent by the user may include the following several demands: object detection, face recognition, difference detection, or speech recognition, and the specific demands of the user are analyzed. For example, two demand information of the user (demand 1 and demand 2) are obtained, and the two demand information are analyzed respectively, and then the task description information 1 corresponding to demand 1 and the task description information 2 corresponding to demand 2 are obtained respectively. As shown in Figure 3As shown, the task description information 1 includes task type: object detection; latency requirement: 0.5s; working mode: inference mode. The task description information 2 includes task type: object detection; latency requirement: none; algorithm model: YOLOv4; working mode: training mode; data source: / path / to / source.

[0033] Step S2: An appropriate AI algorithm can be selected from the algorithm library according to the task description information provided in Step S1, and the trained model parameters can be loaded into the computing power device. Among them, the selection principle of the appropriate AI algorithm can include user requirements and hardware resource limitations. For example, an algorithm that meets the user requirements in terms of latency requirement and memory requirement is selected from the model library and loaded onto the parameter server.

[0034] Among them, before executing Step S2, the following preset steps can also be executed. Specifically, before networking multiple computing power devices, a network discovery operation can be performed on the existing multiple heterogeneous computing power devices. Network discovery combines ICMP (Internet Control Message Protocol), ARP (Address Resolution Protocol), and SNMP (Simple Network Management Protocol) to check for active devices on the network, obtain all active devices, then obtain the basic information of the devices through SNMP, determine the type of the devices according to the basic information, and then obtain the detailed information of the corresponding devices according to the type of the devices and perform unified management.

[0035] The resource information in the entire network is searched by using a centralized search method, and then the location of the resources in the grid is located. When discovering network resources, the system defines a unified interface and protocol for different types of devices, devices produced by different manufacturers, and devices of different models, so that the system can manage the devices with a unified name and descriptor.

[0036] In one implementation, whenever a new device is connected to the network, a resource descriptor can be assigned to the newly discovered computing power device. The descriptor is organized in a Key-Value manner, which is convenient for future expansion of new description features. The descriptor has four basic features: device name, computing efficiency of the device in double precision, single precision, half precision, and integer type, the latency required for data to be sent from the sending end to the device end, and the device memory. After the newly connected device is assigned a descriptor, it is sent to the s10 heterogeneous resource pool for unified management. Through the network, the system can manage devices located in different physical spaces, greatly improving the efficiency of network management.

[0037] Step S3: According to the algorithm model selected in Step S2, configure the corresponding parameter file on the parameter server. The corresponding parameter file may include one or more of the following: task ID, deep learning framework, required hardware resources, network structure, network parameters, and task-related hyperparameters. In one implementation, the parameter file may include all of the above information (task ID, deep learning framework, required hardware resources, network structure, network parameters, and task-related hyperparameters).

[0038] Step S4: Determine the working mode according to the task description information provided in Step S1. The working mode includes a training mode or an inference mode. Since the parameter server stores the training parameters of the model, when the determined working mode is the training mode, the parameter server performs a parameter synchronization operation among multiple computing power devices in the resource pool. When the determined working mode is the inference mode, the parameter server provides network parameters for the computing power devices performing inference. Specifically, when configuring network hyperparameters, a flexible, efficient, and automated mechanism of proto buffer can be used to serialize training samples and model parameters.

[0039] Step S5: According to the working mode determined in Step S4, when the determined working mode is the training mode, poll the computing power resources in the resource pool, screen out multiple candidate computing power devices that meet the task description information from the idle computing power devices in the resource pool, and select some of the candidate computing power devices as the target computing power device group. It is necessary to reserve a part of the computing power for other tasks. Whether it meets the requirements (whether it satisfies the task description information) is considered from aspects such as whether it meets the memory requirement (more than 1.5 times the model size) and the selection of heterogeneous device computing power levels.

[0040] After selecting the target computing power device group, perform the corresponding computing power scheduling operation. In the training mode, the batch processing sample size of each device can be dynamically adjusted according to the device memory and computing power. Specifically, Figure 4 is a flowchart of the computing power scheduling method in the training mode provided by an embodiment of the present application. As Figure 4 shown, the computing power scheduling method in the training mode includes the following steps:

[0041] Step 401: Obtain the computing power ratio information of each computing power device in the target computing power device group.

[0042] Step 402: Divide the first-round training data according to the computing power ratio information of each computing power device, and distribute the divided first-round training data to the corresponding computing power devices respectively.

[0043] Step 403: Determine the allocation information of the (i + 1)-th round of training data according to the training time of each computing power device in the i-th round. Divide the (i + 1)-th round of training data according to the allocation information, and allocate the divided (i + 1)-th round of training data to the corresponding computing power devices respectively, where i is an integer greater than or equal to 1.

[0044] It can be assumed that the training time is roughly inversely proportional to the computing power. Therefore, when allocating resources for the first time, a certain amount of data is allocated to each device according to the ratio of the computing power of each device. Let the set of selected computing power devices be {P1, P2, P3,..., P n}, that is, the ratio information of the computing power of each computing power device in the target computing power device group is {P1, P2, P3,..., P n}, where P n represents the computing power of the n-th computing power device among each computing power device. Among them, the ratio of the computing power of each of the above computing power devices is the ratio information obtained by comparison under the same computing precision.

[0045] Among them, the step of dividing the first-round training data according to the ratio information of the computing power of each computing power device and allocating the divided first-round training data to the corresponding computing power devices in step 402 can specifically include dividing the first-round training data into multiple training sub-data in proportion according to the ratio information of the computing power of each computing power device {P1, P2, P3,..., P n}, where represents the data volume of the training sub-data divided for the n-th computing power device in the first-round training. The value range of n includes {1, 2, 3,..., N}, and N represents the number of computing power devices in the target computing power device group. Then, the training sub-data D n in the multiple training sub-data is respectively allocated to the computing power device with the corresponding computing power P n .

[0046] For example, if the target computing power device group contains 3 computing power devices, and the ratio information of the computing power of these three computing power devices is {P1, P2, P3}, then the first-round training data can be divided into multiple training sub-data in proportion according to the ratio information of these three computing power devices where the training sub-data is allocated to the computing power device with the computing power P1, and so on.

[0047] The method of allocating the training data volume according to the computing power in the above step 402 is only an initial value, which will be dynamically adjusted according to the actual training time. After all devices have completed one round of data calculation for allocation, the network parameters are updated. Calculate the network parameters: where λ(D j ) is a quantity related to the size of each batch of training data volume. W i represents the parameters of the parameter server node during the i-th round of synchronization. During the initial training, the initial training data volume is determined according to the proportion of the computing power of the participating training devices in the computing power pool, and then the allocated training volume is dynamically adjusted according to the actual training time.

[0048] Figure 5 It is a schematic diagram of dynamically correcting the task allocation volume in the training mode provided by an embodiment of the present application. As Figure 5 shown, in each training, each device can complete the training within approximately the same time, so as to uniformly update the parameters in the parameter server. It not only solves the drawback that the efficiency of multi-device parameter update depends on the slowest device when fixing the training data volume of each batch, but also avoids the problem of message lag when updating devices asynchronously. After each round of calculation is completed, the scheduling module can automatically correct the allocated task volume according to the time difference of the devices in the previous round of calculation, so as to ensure that the calculation time of each device is similar.

[0049] That is, the above dynamic adjustment is realized according to the operation steps of step 403. Specifically, the operation of determining the allocation information of the (i + 1)-th round of training data according to the i-th round of training time of each computing power device in step 403, dividing the (i + 1)-th round of training data according to the allocation information, and respectively allocating the divided (i + 1)-th round of training data to the corresponding computing power devices includes: obtaining the i-th round of training time of each computing power device wherein, represents the i-th round of training time of the n-th computing power device; obtaining the longest training time among the i-th round of training times of each computing power device and determining the computing power devices to be configured whose time difference between the i-th round of training time of each computing power device and the longest training time is greater than or equal to a preset time, where the preset time includes the training time of the training sub-data of the unit data volume for the computing power device to be configured. Specifically, for each device j among the n computing power devices, the inequality is calculated in turn. If the inequality holds, the training batch data volume of device j in the next round is increased by 1 (training sub-data of the unit data volume), that is, where the specific implementation steps include: obtaining the allocation information of the i-th round of training data wherein, represents the data volume size of the training sub-data divided for the n-th computing power device in the i-th round of training; performing an update operation on the data volume corresponding to the computing power device to be configured in the allocation information of the i-th round of training data to obtain the allocation information of the (i + 1)-th round of training data Among them, the update operation includes adding training sub - data with a unit data volume based on the training sub - data divided for the to - be - configured computing power device; according to the allocation information of the (i + 1)-th round of training data divide the (i + 1)-th round of training data, and distribute the divided (i + 1)-th round of training data to the corresponding computing power devices respectively.

[0050] According to the above - mentioned working mode confirmation operation, when it is confirmed that the current working mode is the inference mode, poll the computing power resources in the resource pool, screen out multiple candidate computing power devices that meet the task description information from the idle computing power devices in the resource pool, and select some of the candidate computing power devices as the target computing power device group. It is necessary to reserve a part of the computing power for the use of other tasks. Among them, whether it meets the requirements (whether it meets the task description information) is considered from whether it meets the latency requirements.

[0051] After selecting the target computing power device group, perform the corresponding computing power scheduling operation. In the inference mode, tasks can be assigned to computing power machines based on the weights of the devices. Specifically, Figure 6 is the flowchart of the computing power scheduling method in the inference mode provided by an embodiment of this application. As Figure 6 shown, the computing power scheduling method in the inference mode includes the following steps:

[0052] Step 601: Obtain the computing power weight information of each computing power device in the target computing power device group.

[0053] Step 602: Determine the task allocation order according to the computing power weight information of each computing power device.

[0054] Step 603: Allocate each task in the task flow to the corresponding computing power device in the target computing power device group according to the task allocation order information.

[0055] Let the set of computing power devices selected to work in the inference mode be {P1, P2, P3,..., P n}, where P n represents the computing power of the n - th computing power device among each computing power device. Correspondingly, the set of computing power weights of each computing power device selected to work in the inference mode is {w1, w2, w3,..., w n}, where. w n represents the computing power weight of the computing power device corresponding to the computing power P n

[0056] When the task flow is transmitted from the scheduling center, the tasks are allocated according to the following principles:

[0057] i. Select the device with the largest weight from the weights of the alternative devices, and allocate the task to this device.​

[0058] ii. Subtract the weight of the selected device from the sum of the weights of all devices, and then add the initial weight value of each device to its own weight.

[0059] iii. Repeat steps i and ii until the weight is 0.

[0060] iv. Reset all weights to the initial weights and continue the task allocation from step i.

[0061] In other words, the allocation operation of the above allocation principle can be implemented through the operation of step 602. Specifically, the determination of the task allocation order according to the computing power weight information of each computing power device described in step 602 includes: performing multiple rounds of computing power device selection operations, obtaining the computing power device selection order after the multiple rounds of computing power device selection operations, and using the computing power device selection order as the task allocation order. Among them, each round of computing power device selection operation includes: taking the computing power device corresponding to the highest weight in the first weight information of this round as the computing power device selected in this round, where the weight information includes the computing power weight values of each computing power device in the target computing power device group, and the first weight information in the first round of computing power device selection operation includes the initial computing power weight values of each computing power device in the target computing power device group; subtracting the current computing power weight value of the computing power device selected in this round from the sum of the weights of all computing power devices in the target computing power device group to obtain a correction value, and updating the first weight information according to the correction value to obtain the second weight information; and adding the current computing power weight values of each computing power device in the second weight information to the corresponding initial computing power weight values of the computing power devices respectively to obtain the third weight information, and using the third weight information as the first weight information for the next round of computing power device selection operation.

[0062] For example, the target computing power device group described in step 601 includes three devices: computing power device A, computing power device B, and computing power device C, and the initial computing power weights of computing power device A, computing power device B, and computing power device C are obtained as 5, 2, and 1 respectively. Then the device selection method is shown in Table 1.

[0063] Table 1

[0064]

[0065] Among them, in the first round of device selection process, the computing power device with the highest weight in the first-round first weight information (5, 2, 1) is computing power device A, and its current computing power weight is 5. Further, subtract the current computing power weight (5) of computing power device A from the sum of the weights of the three computing power devices (5 + 2 + 1), that is, 5 - 8 = -3. The correction value (-3) of this round is obtained through the above subtraction calculation. The first-round first weight information (5, 2, 1) is updated by this correction value (-3). This update operation may include replacing the weight value of the selected computing power device A in the first weight information with the correction value (-3), that is, updating the "5" in the first weight information (5, 2, 1) to "-3", and then obtaining the second-round weight information (-3, 2, 1) of this round. Further, the current computing power weight values of each computing power device in the second-round weight information (-3, 2, 1) of this round are respectively added to the initial computing power weight values of the corresponding computing power devices to obtain the third weight information. Specifically, the above addition operations are (-3) + 5 = 2, 2 + 2 = 4, 1 + 1 = 2 respectively. Then the third-round weight information of this round is (2, 4, 2), and the third-round weight information (2, 4, 2) of this round is used as the first weight information for the next round of device selection, and so on.

[0066] In one implementation, when performing high-concurrency tasks, to avoid tasks being simultaneously assigned to the one with the highest weight, it can be considered to randomly select a weight first and then continue to allocate downward. In other words, before performing the multi-round computing power device selection operation, it further includes: configuring a target selection rule according to the selection rule of the first-round computing power device selection operation; where the selection rule of the first-round computing power device selection operation includes: using the computing power device corresponding to the highest weight in the first-round first weight information as the computing power device selected in this round; and / or randomly selecting a target weight in the first-round first weight information and using the computing power device corresponding to the target weight as the computing power device selected in the first round.

[0067] Figure 7 The structural schematic diagram of the computing power scheduling device provided for another embodiment of the present application is as Figure 7 shown. The computing power scheduling device includes a processor 701 and a memory 702. The memory 702 is used to store at least one instruction, and when the instruction is loaded and executed by the processor 701, it is used to implement Figure 4 the computing power scheduling method provided by the embodiment shown and / or Figure 6 the computing power scheduling method provided by the embodiment shown.

[0068] Figure 8 The distributed system provided for another embodiment of the present application is as Figure 8 shown. The distributed system may include multiple computing power devices, a user interface, a gateway, a computing power device matching module, and Figure 7The computing power scheduling device provided by the illustrated embodiment (such as Figure 8 the computing power scheduling module shown).

[0069] Among them, the user interface (user API) is used to receive the user demand information sent by the user terminal, and the task description information includes one or more of the following: task type, latency requirement, accuracy requirement, memory occupancy, and working mode; in one implementation, the task description information includes: task type, latency requirement, accuracy requirement, memory occupancy, and working mode. The user passes the user demand into the computing power network of the distributed system through the given user API, and the network automatically deploys.

[0070] In one implementation, the distributed system may further include an AI algorithm library (SOTA library), and the AI algorithm library may include algorithm libraries in many deep learning fields such as natural language processing, computer vision, recommendation systems, and knowledge graphs. Each algorithm has a supporting network parameter and model. So if the user has a demand, the user can also customize the training data for training. Each model in the AI algorithm library also has corresponding description information, mainly including: model type, model input and output, running mode (accuracy priority / calculation speed priority), average accuracy, accuracy rate, calculation time per average unit input amount, etc. After the user transmits the demand to the computing power network, the algorithm selection will automatically select a candidate model group according to the model type, and then select the optimal algorithm for deployment according to the user's specific requirements, such as latency, accuracy requirement, data volume size, etc.

[0071] The gateway (algorithm selection shown in the figure) can be a computing power network gateway, which is used to obtain the task description information provided by the user interface and match the target algorithm model that meets the task description information in the AI algorithm library. After selecting the computing power device, the gateway can also load the trained model parameters into each computing power device. Specifically, use OpenVINO or libtorch to convert the model into an easily deployable model description file.xml file and model parameter file.bin, and then transmit the parameters and model files to the computing power device for calculation.

[0072] Among them, OpenVINO is a tool suite developed by Intel based on its existing hardware platform, which can accelerate the development speed of high-performance computer vision and deep learning vision applications, support deep learning on hardware accelerators of various Intel platforms, and allow direct heterogeneous execution. It supports Windows and Linux systems, and Python / C++ languages. The C++ API provided by Pytorch official is called Libtorch, which has supported Windows since the Pytorch 1.0 version, and the Pytorch model can be directly deployed with Libtorch.

[0073] The computing power device matching module is used to match a target computing power device group that meets the task description information among the multiple computing power devices.

[0074] In one implementation, the distributed network further includes a resource discovery and management module, which is used to check active devices in the network in combination with SNMP to obtain all active devices, then obtain the basic information of the devices through SNMP, determine the types of the devices according to the basic information, and then obtain the detailed information of the corresponding devices according to the types of the devices, and summarize them into descriptors and send them to heterogeneous computing power. When the scheduling module requests to use computing power resources, it performs addressing and calls. Network administrators can easily manage the devices in the network that support the SNMP protocol. Based on the unified interface provided by the SNMP protocol, the differences between different devices can be masked, and there is no need to care about what type of device it is or which manufacturer produced it, thus realizing automated network management.

[0075] The multiple computing power devices constitute the above-mentioned resource pool. Real heterogeneous resource devices refer to the entities of various resources, which can be computers, mobile phones, routers, intelligent terminals, etc., or NAS, private clouds, etc. In one implementation, the computing power devices in the resource pool can provide heterogeneous computing power. Specifically, the heterogeneous computing power stores information about various computing powers in the network, including computing power in double precision, single precision, and half precision, the latency of data arriving at the computing power device, the memory of the computing power device, and the MAC address, etc. Double precision, single precision, and half precision: Floating-point numbers are one of the most commonly used data types on computers. Commonly used floating-point numbers include double precision and single precision. Half precision is more for reducing data transmission and storage costs. Double precision has 64 bits, single precision has 32 bits, and half precision has 16 bits. In distributed training, using half precision saves half of the transmission cost compared to single precision.

[0076] In one implementation, the distributed system may further include a network scheduling module: When the network scale is large, each router needs to obtain the entire network information for each application service and then independently calculate the path. At this time, the maintenance workload of the entire network is unacceptable. Therefore, for the feasibility of the computing power network operation, we need to centrally manage the computing power network, centralize the synchronization of information and the calculation of paths, and then send the service routing table entries to the routers after calculation. The routers are only responsible for forwarding service traffic at the data layer.

[0077] An embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored, and characterized in that when the computer program is executed by a processor, it implements Figure 4 The computing power scheduling method provided by the illustrated embodiment and / or Figure 6 The computing power scheduling method provided by the illustrated embodiment.

[0078] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the systems, devices, and units described above can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated herein.

[0079] In several embodiments provided by the present invention, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division, and there can be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed mutual coupling, direct coupling, or communication connection can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be in electrical, mechanical, or other forms.

[0080] The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0081] In addition, in each embodiment of the present invention, the functional units can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of hardware plus software functional units.

[0082] The above-mentioned integrated units implemented in the form of software functional units can be stored in a computer-readable storage medium. The software functional units stored in a storage medium include several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to execute some steps of the methods described in each embodiment of the present invention. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical discs that can store program codes.

[0083] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

[0084] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A computing power scheduling method, characterized in that, The method includes: If the current working mode is the inference mode, then perform the following operations: Obtain the computing power weight information of each computing power device in the target computing power device group; Determine the task allocation order according to the computing power weight information of each computing power device; and Allocate each task in the task flow to the corresponding computing power device in the target computing power device group in sequence according to the task allocation order information; Among them, the determining the task allocation order according to the computing power weight information of each computing power device includes: Perform multiple rounds of computing power device selection operations. After the multiple rounds of computing power device selection operations, obtain the computing power device selection order, and use the computing power device selection order as the task allocation order; Among them, each round of computing power device selection operation includes: Use the computing power device corresponding to the highest weight in the first weight information of this round as the computing power device selected in this round. Among them, the first weight information includes the computing power weight values of each computing power device in the target computing power device group. The first weight information in the first round of computing power device selection operation includes the initial computing power weight values of each computing power device in the target computing power device group; Subtract the current computing power weight value of the computing power device selected in this round from the sum of the weights of all computing power devices in the target computing power device group to obtain a correction value, and update the first weight information according to the correction value to obtain second weight information; and Add the current computing power weight values of each computing power device in the second weight information to the corresponding initial computing power weight values of the computing power devices respectively to obtain third weight information, and use the third weight information as the first weight information for the next round of computing power device selection operation.

2. The method according to claim 1, characterized in that, Before performing the multiple rounds of computing power device selection operations, it further includes: Configure the target selection rule according to the selection rule of the first round of computing power device selection operation; Among them, the selection rule of the first round of computing power device selection operation includes: Using the computing power device corresponding to the highest weight in the first weight information of this round as the computing power device selected in this round; and / or Randomly select a target weight in the first weight information of the first round, and use the computing power device corresponding to the target weight as the computing power device selected in the first round.

3. The method according to claim 1, characterized in that Before the obtaining the computing power weight information of each computing power device in the target computing power device group, it further includes: Obtain task description information, and determine the current working mode according to the task description information. The working mode includes the training mode and the inference mode.

4. The method according to claim 3, characterized in that, After determining the current working mode according to the task description information, if the current working mode is the training mode, then perform the following operations: Obtain the computing power ratio information of each computing power device in the target computing power device group; Divide the first-round training data according to the computing power ratio information of each computing power device, and allocate the divided first-round training data to the corresponding computing power devices respectively; Determine the allocation information of the (i + 1)-th round of training data according to the training time of the i-th round of each computing power device, divide the (i + 1)-th round of training data according to the allocation information, and allocate the divided (i + 1)-th round of training data to the corresponding computing power devices respectively, where i is an integer greater than or equal to 1.

5. The method according to claim 4, characterized in that Dividing the first-round training data according to the computing power ratio information of each computing power device, and respectively allocating the divided first-round training data to the corresponding computing power devices includes: According to the computing power ratio information {P1, P2, P3,..., P n} of each computing power device, equally divide the first-round training data into multiple training sub-data where P n represents the computing power of the nth computing power device among each computing power device, represents the data volume of the training sub-data allocated to the nth computing power device in the first-round training, and the value range of n includes {1, 2, 3,..., N}, and N represents the number of computing power devices in the target computing power device group; Allocate the training sub-data D in the multiple training sub-data respectively n to the computing power devices with the corresponding computing power P n .

6. The method according to claim 5, wherein Determining the allocation information of the (i + 1)-th round of training data according to the training time of the i-th round of each computing power device, dividing the (i + 1)-th round of training data according to the allocation information, and respectively allocating the divided (i + 1)-th round of training data to the corresponding computing power devices includes: Obtain the training time of the i-th round of each computing power device wherein represents the training time of the i-th round of the n-th computing power device; Obtain the longest training time in the i-th round of training of each computing power device And determine the time difference between the i-th round of training time of each computing power device and the longest training time of the computing power devices to be configured whose time difference is greater than or equal to a preset time, where the preset time includes the training time of the training sub-data of the unit data volume by the computing power devices to be configured; Obtain the allocation information of the training data for the i-th round wherein represents the data volume size of the training sub-data allocated to the n-th computing power device in the i-th round of training; Allocation information for the i-th round of training data Perform an update operation on the data volume corresponding to the computing power device to be configured in the above to obtain the allocation information for the (i + 1)-th round of training data Among them, the update operation includes adding training sub-data with a unit data volume on the basis of the training sub-data allocated to the computing power device to be configured According to the allocation information of the (i + 1)-th round of training data Partition the (i + 1)-th round of training data and allocate the partitioned (i + 1)-th round of training data to the corresponding computing power devices respectively.

7. A computing power scheduling device, characterized in that, The device includes: A processor and a memory, the memory is used to store at least one instruction, and when the instruction is loaded and executed by the processor, it is used to implement the computing power scheduling method described in any one of claims 1-6.

8. A distributed system, characterized in that, The system includes: Multiple computing power devices for providing computing services; A user interface for obtaining the user's user demand information and determining task description information according to the user demand information, the task description information including one or more of the following: task type, latency requirement, accuracy requirement, memory occupancy, and working mode; A gateway for obtaining the task description information provided by the user interface and matching a target algorithm model that meets the task description information; A computing power device matching module for matching a target computing power device group that meets the task description information among the multiple computing power devices; and The computing power scheduling device described in claim 7.

9. The system according to claim 8, wherein Matching a target computing power device group that meets the task description information among the multiple computing power devices includes: Screening out multiple candidate computing power devices that meet the task description information from the idle computing power devices among the multiple computing power devices; and Selecting some of the multiple candidate computing power devices as the target computing power device group.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the computing power scheduling method described in any one of claims 1-6.

Images