A large model computing resource coupling scheduling method, device and medium

Abstract

The embodiment of the specification discloses a large model computing resource coupling scheduling method and device and medium, relates to the technical field of computers, and is used for solving the problem that the existing scheduling mode causes the computing center not to be fully utilized. The method comprises the following steps: splitting each computing task based on an algorithm and required data to obtain a computing subtask; establishing a global data flow topology graph based on the computing topology relationship between each computing subtask; determining the data processing concurrency of each edge in the global data flow topology graph according to the computing speed corresponding to each computing task, and determining the total concurrency of each algorithm based on the data processing concurrency of each edge; determining the optimal computing center based on the total concurrency of each algorithm and a preset hierarchical constraint condition, and creating an algorithm instance of each algorithm based on the optimal computing center, so as to realize resource allocation of the algorithm instance.

Description

TECHNICAL FIELD

[0001] The present specification relates to the technical field of computer technology, and in particular to a large model computing resource coupling scheduling method, device and medium. BACKGROUND

[0002] With the development of artificial intelligence technology, especially the emergence of a new generation of algorithms represented by large language models, more and more industry fields begin to introduce AI algorithms to assist and improve production efficiency. At the same time, the development and application of artificial intelligence technology cannot be separated from the support of data and computing power. With the rapid development of the data element market, massive data is constantly generated and traded, and the computing power servers being built in various places provide support for a large amount of computing analysis. However, the current data, computing power and algorithms are still relatively fragmented and exist independently without forming a closed loop of mutual communication. A large amount of valuable data has not been effectively utilized and quickly circulated, and a large amount of computing power servers are idle or in excess and have not been fully used, resulting in resource waste, so the scheduling of computing resources is very important.

[0003] When the current existing framework splits computing tasks, the tasks are split by operator splitting and inter-layer segmentation, etc. However, the static splitting strategy cannot dynamically adapt to heterogeneous computing power, and for computing tasks with complex dependency relationships and computing logic, it is difficult to effectively split and easy to cause resource waste. In addition, the traditional topology depends on manual preset and only supports point-to-point communication between homogeneous devices. When the task has complex serial-parallel hybrid, the communication path needs to be hard-coded and cannot be dynamically optimized. In addition, the scheduling of computing resources cannot fully consider the correlation between the current data, computing power and algorithms, and simply allocating computing power according to simple rules can easily lead to high data scheduling cost and serious fragmentation of computing nodes. SUMMARY

[0004] To solve the above technical problems, one or more embodiments of the present specification provide a large model computing resource coupling scheduling method, device and medium.

[0005] One or more embodiments of the present specification adopt the following technical solutions:

[0006] One or more embodiments of the present specification provide a large model computing resource coupling scheduling method, which comprises:

[0007] Determine the algorithm corresponding to each computing task of the current large model and the required data, and split each computing task based on the algorithm and the required data to obtain computing subtasks;

[0008] Based on the computing topology relationship between each computing subtask, a global data flow topology graph is established; wherein the computing topology relationship includes parallel relationship, serial relationship;

[0009] determine data processing concurrency of each edge in the global data flow topology graph according to a calculation speed corresponding to each calculation task, to determine total concurrency of each algorithm based on the data processing concurrency of each edge;

[0010] determine an optimal computing power center based on the total concurrency of each algorithm and a preset hierarchical constraint condition, and create an algorithm instance of each algorithm based on the optimal computing power center, to realize resource allocation of the algorithm instance.

[0011] Optionally, in one or more embodiments of the present specification, the algorithm corresponding to each calculation task of the current large model and the required data are determined, and each calculation task is split based on the algorithm and the required data to obtain a calculation subtask, specifically including:

[0012] The algorithm corresponding to each calculation task of the current large model and the required data are determined to obtain a minimum granularity data set corresponding to each algorithm; wherein the minimum granularity data set includes algorithm input data and algorithm output data.

[0013] Each calculation task is split based on each algorithm and the minimum granularity data set corresponding to each algorithm to obtain a plurality of calculation subtasks.

[0014] Optionally, in one or more embodiments of the present specification, a global data flow topology graph is established based on the calculation topology relationship between each calculation subtask, specifically including:

[0015] A data flow topology graph corresponding to each calculation task is established based on the calculation topology relationship between each calculation subtask.

[0016] Based on the minimum granularity data set, it is determined whether there are multiple input data corresponding to each calculation subtask of each calculation task.

[0017] If there are multiple input data, a merging operator corresponding to the calculation subtask is created, and an edge corresponding to the merging operator is obtained to obtain an initial global data flow topology graph.

[0018] The vertices belonging to the same data and the same edges in the initial global data flow topology graph are merged to obtain a global data flow topology graph.

[0019] If there are no multiple input data, the vertices belonging to the same data and the same edges in the data flow topology graph are merged to obtain a global data flow topology graph.

[0020] Optionally, in one or more embodiments of the present specification, the data processing concurrency of each edge in the global data flow topology graph is determined according to the computing speed corresponding to each computing task, and the total concurrency of each algorithm is determined based on the data processing concurrency of each edge, specifically comprising:

[0021] The data flow path corresponding to each computing task is identified based on the global data flow topology graph, and the computing speed corresponding to the computing task is determined according to the data flow path;

[0022] It is determined whether there is a merging operator in the computing subtask, and the merging operator is removed to determine the to-be-assigned partial topology graph of each computing subtask;

[0023] Based on the computing speed corresponding to the computing task, the data concurrency of the to-be-assigned partial topology graph of the computing subtask corresponding to the computing task is allocated;

[0024] The edges of the global data flow topology graph are traversed to determine the data processing concurrency of each edge in the global data flow topology graph;

[0025] The algorithm corresponding to each edge is determined, and the data processing concurrency of the corresponding edge is merged based on the algorithm to obtain the total concurrency of each algorithm.

[0026] Optionally, in one or more embodiments of the present specification, the edges of the global data flow topology graph are traversed to determine the data processing concurrency of each edge in the global data flow topology graph, specifically comprising:

[0027] Each edge is classified based on the computing subtask corresponding to each edge to determine the edge type of each edge; wherein the edge type includes: single-task type, multi-task type;

[0028] If it is determined that the edge type is a single-task type, the data concurrency of the edge is determined based on the data concurrency of the computing subtask corresponding to the edge;

[0029] If it is determined that the edge type is a multi-task type, the data concurrency of the edge is determined based on the data concurrency of the computing subtask corresponding to the edge;

[0030] Optionally, in one or more embodiments of the present specification, the optimal algorithm center is determined based on the total concurrency of each algorithm and the preset hierarchical constraint condition, specifically comprising:

[0031] Based on the algorithm power specification vector corresponding to each algorithm and the total concurrency of each algorithm, the required number of algorithm instances of each algorithm under different algorithm power specifications is determined;

[0032] sort the algorithms corresponding to each required algorithm instance based on the total concurrency corresponding to each algorithm, and sort the algorithm power specifications based on resource information corresponding to each algorithm power specification to obtain an algorithm power specification priority queue; wherein the resource information includes: a key resource type, a unit resource data processing concurrency;

[0033] analyze each algorithm based on a preset hierarchical constraint condition to determine the optimal algorithm power center corresponding to each algorithm; wherein the preset hierarchical constraint condition includes: a data constraint condition, an algorithm constraint condition, and an algorithm constraint condition.

[0034] Optionally, in one or more embodiments of the present specification, the optimal algorithm power center corresponding to each algorithm is determined based on a preset hierarchical constraint condition, specifically including:

[0035] determining the constraint factors corresponding to each constraint condition in the preset hierarchical constraint condition, and aggregating the constraint factors and the preset resource constraint factors as the to-be-screened constraint factors;

[0036] quantifying each to-be-screened constraint factor based on the index corresponding to each to-be-screened constraint factor;

[0037] determining the weight of the quantized to-be-screened factor based on a preset priority strategy, determining the screening priority of each preset hierarchical constraint condition according to the weight corresponding to each to-be-screened constraint factor, and obtaining the optimal algorithm power center corresponding to each algorithm based on the priority hierarchical screening calculation node.

[0038] Optionally, in one or more embodiments of the present specification, algorithm instances of each algorithm are created based on the optimal algorithm power center to realize resource allocation of the algorithm instances, specifically including:

[0039] determining the optimal algorithm power specification corresponding to the algorithm according to the algorithm power specification priority queue, and creating algorithm instances of the algorithm based on each optimal algorithm power specification at the optimal algorithm power center;

[0040] obtaining the concurrency corresponding to the optimal algorithm power specification, determining the remaining concurrency based on the concurrency and the total concurrency, and determining whether to terminate the algorithm instance allocation of the algorithm based on the remaining concurrency and a preset termination condition.

[0041] One or more embodiments of the present specification provide a large model computing resource coupling scheduling device, the device comprising:

[0042] at least one processor; and,

[0043] a memory in communication connection with the at least one processor; wherein,

[0044] The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform any of the above-described methods.

[0045] The one or more embodiments of the specification provide a non-volatile computer storage medium storing computer executable instructions configured to enable the execution of any of the above-described methods.

[0046] The above at least one technical solution adopted by the embodiments of the specification can achieve the following beneficial effects:

[0047] By splitting the computing task into subtasks and establishing a global data flow topology graph based on the computing topology relationship, the dependencies between tasks are clearly revealed, and the problem of resource waste caused by the difficulty of effective splitting of computing tasks with complex dependency relationships and computing logic is avoided. Through the total algorithm concurrency and the preset hierarchical constraint condition, the precise matching of resources and demand is realized, and the problems of resource waste or resource idleness are avoided. And the joint optimization and scheduling of data, computing power and algorithm are realized in a data-centric manner, reducing the repeated transmission and copying of data and the repeated calculation of algorithms, and improving the utilization efficiency of resources and data. BRIEF DESCRIPTION OF DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the specification or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments described in the specification, and those skilled in the art can also obtain other drawings according to these drawings without creative labor. In the drawings:

[0049] Figure 1 A large model computing resource coupling scheduling method flowchart is provided for the embodiments of the specification;

[0050] Figure 2 A computing task schematic diagram in an application scenario is provided for the embodiments of the specification;

[0051] Figure 3 A computing power center information schematic diagram of computing power resources in an application scenario is provided for the embodiments of the specification;

[0052] Figure 4 A global data flow topology graph in an application scenario is provided for the embodiments of the specification.

[0053] Figure 5A computing power specification diagram corresponding to each algorithm in an application scenario provided by an embodiment of the present specification is shown in the following table.

[0054] Figure 6 An algorithm instance diagram provided in an application scenario provided by an embodiment of the present specification is shown in the following table.

[0055] Figure 7 A structure diagram of a coupling scheduling device of a large model computing resource provided by an embodiment of the present specification is shown in the following table.

[0056] Figure 8 A structure diagram of a non-volatile storage medium provided by an embodiment of the present specification is shown in the following table. DETAILED DESCRIPTION

[0057] An embodiment of the present specification provides a large model computing resource coupling scheduling method, device and medium.

[0058] In order for those skilled in the art to better understand the technical solutions in the present specification, the technical solutions in the present specification will be described clearly and completely in the following with reference to the drawings in the present specification. Obviously, the described embodiments are only part of the embodiments of the present specification, not all embodiments. Based on the embodiments of the present specification, all other embodiments obtained by those skilled in the art without creative labor should be within the scope of protection of the present specification.

[0059] As shown in the following table, an embodiment of the present specification provides a method flow diagram of a large model computing resource coupling scheduling method. It can be known from the following table that in one or more embodiments of the present specification, a large model computing resource coupling scheduling method specifically includes the following steps: Figure 1 Figure 1 As shown in the following table, an embodiment of the present specification provides a method flow diagram of a large model computing resource coupling scheduling method. It can be known from the following table that in one or more embodiments of the present specification, a large model computing resource coupling scheduling method specifically includes the following steps:

[0060] S101: Determine the algorithm corresponding to each computing task of the current large model and the required data, to split each computing task based on the algorithm and the required data, and obtain a computing subtask.

[0061] In order to accurately identify the computing power specification such as CPU / GPU type, memory capacity, storage IO demand and other information of the computing task, so as to improve the accuracy of subsequent resource scheduling. In an embodiment of the present specification, the algorithm corresponding to each computing task of the current large model and the required data are determined, so that each computing task is split according to the algorithm and the required data, and a computing subtask is obtained. As shown in the following table, an embodiment of the present specification provides a computing power specification diagram corresponding to each algorithm in an application scenario. Figure 2 ​As shown, there are 3 computing tasks in a certain application scenario that need to be allocated computing resources, task 1 is city waterlogging early warning monitoring, which detects the water depth of the video data collected by the city road camera, and then predicts the time and location of waterlogging through the mechanism model, the calculation speed is 20qps. Task 2 is traffic OD flow analysis, which identifies vehicle license plates through video data collected by city road cameras, and combines road network topology data to restore vehicle trajectories, and then outputs OD paths and flow with larger traffic flow through OD aggregation sorting, the calculation speed is 1qps. Task 3 is a multi-modal AI agent based on a large model to search for people and vehicles, which analyzes user input text through a large model, determines whether the object the user is looking for is a person or a vehicle, and filters out the city road camera data that needs to be analyzed through the provided time and location information, and then uses face recognition algorithm or license plate recognition algorithm and the result to output the find person or find vehicle result, the calculation speed is 10qps. In this process, complex tasks are divided into computing subtasks, which can make the goal of each computing subtask more single, facilitate the acquisition of complex dependencies in the computing task, and make the scheduling of computing resources more accurate.

[0062] Specifically, in one or more embodiments of the present specification, the algorithms corresponding to each computing task of the current large model and the required data are determined, and each computing task is split based on the algorithms and the required data to obtain computing subtasks, specifically including:

[0063] First, the algorithms corresponding to each computing task of the current large model and the required data are determined, thereby obtaining the minimum granularity data set corresponding to each algorithm. It should be noted that the minimum granularity data set includes: algorithm input data, algorithm output data. That is, the data is the minimum granularity data set required for one calculation of the algorithm, is the input data, is the output data. Then, based on each algorithm and the minimum granularity data set corresponding to each algorithm, each computing task is split to obtain multiple computing subtasks. The algorithm is a computing program with a function operator as the smallest module. It should be noted that the computing topology relationship between each computing subtask includes parallel and serial, where two subtasks and serial then parallel is divided into deterministic parallel and probabilistic parallel, where deterministic parallel is , and probabilistic parallel means that different algorithms are triggered to calculate according to different values of the input data in this computing step, but only one algorithm will be triggered each time the task runs, which can be represented as .

[0064] S102: Establish a global data flow topology graph based on the computing topology relationship between each computing subtask; wherein, the computing topology relationship includes: parallel relationship, serial relationship.

[0065] After the above step S101 realizes the splitting of the computing task, a global data flow topology graph is established according to the computing topology relationship between each computing subtask, that is, the parallel relationship or serial relationship between the computing subtasks. In this process, by converting the computing task into a global data flow topology graph, the data flow path can be clearly revealed, thereby avoiding the problem of deadlock or resource contention caused by chaotic dependence when there is complex dependence in the computing task.

[0066] Specifically, in one or more embodiments of the present specification, the global data flow topology graph is established based on the computing topology relationship between each computing subtask, specifically including the following processes:

[0067] First, based on the computing topology relationship between each computing subtask, a data flow topology graph corresponding to each computing task is established. Then, according to the minimum granularity data set, it is determined whether there are multiple input data corresponding to each computing subtask of the computing task. If there are multiple input data, a merging operator corresponding to the computing subtask is created, and an edge corresponding to the merging operator is obtained to obtain an initial global data flow topology graph. Then, the vertices belonging to the same data and the same edges in the initial global data flow topology graph are merged to obtain the global data flow topology graph. If there is no multiple input data, the vertices belonging to the same data and the same edges in the data flow topology graph are merged to obtain the global data flow topology graph.

[0068] In a feasible embodiment, the global data flow topology graph is established based on the computing topology relationship between each computing subtask, which can specifically be established according to the serial and parallel relationship of each computing subtask, wherein the data flow topology graph has vertices and directed edges . Each computing task data flow topology graph is merged to generate a global data flow topology graph, wherein vertices of the same data are merged into one vertex, and edges of the same data , and algorithm are merged into one edge. If the input data of the subtask is multiple, that is, , a merging operator is created, and k edges are created to realize data merging operation. For example, Figure 4As shown, three tasks are subtasked according to data and algorithms in a certain application scenario, a directed acyclic data flow topology graph is established, the vertex is data, and the edge is algorithm. The dashed line in task 3 represents that face recognition and license plate recognition are probabilistic parallel subtasks. Then the same data and algorithm subtasks are merged to form a global data flow topology graph.

[0069] S103: According to the computing speed corresponding to each computing task, the data processing concurrency of each edge in the global data flow topology graph is determined, and the total concurrency of each algorithm is determined based on the data processing concurrency of each edge.

[0070] In order to provide a direct basis for subsequent resource matching of the computing power center, and to ensure the accurate alignment of computing power resources and algorithm demand, in the embodiments of the present specification, the data processing concurrency of each edge in the global data flow topology graph is determined according to the computing speed corresponding to each computing task, and the total concurrency of each algorithm is determined based on the data processing concurrency of each edge. That is, through the mapping of computing speed and concurrency, the blindness of traditional schemes based on personnel experience in estimating resources is avoided, and the situation of resource redundancy or deficiency is reduced. And taking the concurrency as one of the core indicators of subsequent scheduling makes the process of screening and resource allocation of the computing power center more interpretable, which is convenient for decision makers to optimize resource strategies in a data-driven manner rather than relying on subjective judgment.

[0071] Specifically, in one or more embodiments of the present specification, according to the computing speed corresponding to each computing task, the data processing concurrency of each edge in the global data flow topology graph is determined, and the total concurrency of each algorithm is determined based on the data processing concurrency of each edge, which specifically includes the following processes:

[0072] Firstly, the data flow path corresponding to each computing task is identified based on the global data flow topology graph, so as to determine the computing speed corresponding to the computing task according to the data flow path. Then it is determined whether there is a merging operator in the computing subtask as proposed in step S102. Since the merging operator does not calculate the concurrency, the merging operator is removed, and the to-be-allocated partial topology graph of each computing subtask is determined. Then, based on the computing speed corresponding to the computing task, the data concurrency of the to-be-allocated partial topology graph of the computing subtask corresponding to the computing task is allocated; wherein the computing speed of each computing subtask is consistent, and the sum of the data concurrency of each computing subtask is greater than or equal to the data concurrency of the computing task. By traversing the edges of the global data flow topology graph, the data processing concurrency of each edge in the global data flow topology graph can be determined. Then, in order to realize the reuse of results multiple times for one-time calculation of an algorithm, the algorithm corresponding to each edge can be determined, and the data processing concurrency of the corresponding edge is merged based on the algorithm to obtain the total concurrency of each algorithm.

[0073] Further, in one or more embodiments of the present specification, traversing edges of the global data flow topology graph to determine data processing concurrency of each edge in the global data flow topology graph specifically comprises:

[0074] classifying each edge based on the computing sub-tasks corresponding to each edge to determine the edge type of each edge; wherein the edge type includes: single task type, multi-task type. If it is determined that the edge type is a single task type, i.e., there is no multiple tasks sharing, the data concurrency of the edge will be determined based on the data concurrency of the computing sub-tasks corresponding to the edge. If it is determined that the edge type is a multi-task type, i.e., there are multiple tasks sharing, the data concurrency of the edge will be determined by taking the maximum data concurrency of the multiple computing sub-tasks corresponding to the edge.

[0075] In a feasible embodiment, according to the computing speed requirement of each computing task, i.e., the data processing concurrency, the computing speed of each sub-task is determined to be consistent and at least reach the concurrency of the overall task, and the merging operator does not calculate the concurrency. Then according to the global data flow topology graph, when an edge is shared by multiple tasks, the concurrency of this edge is calculated according to the maximum value of the concurrency of all tasks by caching the calculation results of the edge, so that the algorithm is calculated once and the results are reused multiple times. Finally, the concurrency of the edges of the same algorithm is merged to obtain the total concurrency of each algorithm . For example, in the application scenario as shown in Figure 4 , according to the above example, the computing speed of task 1 is 20qps, the computing speed of task 2 is 1qps, and the computing speed of task 3 is 10qps. At this time, according to the computing speed of each task, the concurrency requirement of each edge is calculated , , , , , . Among them the edge is shared by task 2 and task 3, so the concurrency is the maximum value of task 2 and task 3. Since the algorithm of each edge is different, the concurrency requirement of each algorithm is consistent with the concurrency requirement of the corresponding edge.

[0076] S104: determining an optimal computing power center based on the total concurrency of each algorithm and a preset hierarchical constraint condition, and creating an algorithm instance of each algorithm based on the optimal computing power center to realize resource allocation of the algorithm instance.

[0077] ​According to the total concurrency of each algorithm obtained according to the above steps and the preset hierarchical constraint condition, the optimal computing power center is determined, and algorithm instances of each algorithm are created based on the optimal computing power center to realize resource allocation of the algorithm instances. Specifically, in one or more embodiments of the present specification, the optimal computing power center is determined based on the total concurrency of each algorithm and the preset hierarchical constraint condition, specifically including the following steps:

[0078] Firstly, the required number of algorithm instances of each algorithm under different computing power specifications is determined based on the computing power specification vector corresponding to each algorithm and the total concurrency of each algorithm. Then, the algorithms corresponding to each required algorithm instance are sorted according to the total concurrency corresponding to each algorithm, so that the computing power specifications are sorted according to the resource information corresponding to each computing power specification, and a computing power specification priority queue is obtained. The resource information includes: key resource type, unit resource data processing concurrency. Then, the preset hierarchical constraint condition is used to analyze each algorithm to determine the optimal computing power center corresponding to each algorithm. It should be noted that the preset hierarchical constraint condition includes: data constraint condition, computing power constraint condition, and algorithm constraint condition.

[0079] In a feasible embodiment, each algorithm supports different chip architectures and has certain differences in computing performance under different architectures. In a multi-architecture and multi-algorithm scenario, the computing power specification of an algorithm is set as a vector: , wherein is the th specification of the algorithm , and is the consumption of different computing power resources under the specification , and the computing power resources at least include one of cpu, memory, computing power card (gpu, tpu, npu, etc.), hard disk, and network bandwidth is the maximum algorithm concurrency that can be supported by the specification. According to the computing power specification and the total concurrency requirement of the algorithm, the required instance number under each specification is calculated . According to the concurrency of each edge in the global data flow topology graph and the maximum algorithm concurrency supported by the computing power specification of the algorithm corresponding to the edge, the data splitting number is calculated, wherein when , data splitting is not performed. That is, according to the computing power specification and the concurrency requirement of each algorithm, the required instance number under each specification and the corresponding data slice number are calculated, and it can be seen from Figure 5 that the required instance number is large due to the high computing speed requirement of task 1, and the camera video data needs to be split into equal parts for parallel computing to meet the computing speed requirement.

[0080] Further, in one or more embodiments of the present specification, each algorithm is analyzed based on preset hierarchical constraint conditions to determine the optimal computing power center corresponding to each algorithm, specifically including:

[0081] First, the constraint factors corresponding to each constraint condition in the preset hierarchical constraint condition are determined, and the constraint factors and preset resource constraint factors such as network bandwidth cost, energy consumption efficiency, security authentication level, geographical location delay, etc. are summarized as the to-be-screened constraint factors. According to the index corresponding to each to-be-screened constraint factor, the to-be-screened constraint factor is quantified. For example, when the to-be-screened constraint factor is a hardware resource factor, it is quantified based on the CPU computing power and memory bandwidth, etc. as shown in the figure. That is, the weight of the quantized to-be-screened factor is determined based on the preset priority strategy, so as to determine the screening priority of each preset hierarchical constraint condition according to the weight corresponding to each to-be-screened constraint factor, and the computing node is screened based on the priority, and the optimal computing power center corresponding to each algorithm is obtained. Figure 3

[0082] In a feasible embodiment, the resource allocation of the algorithm instance needs to consider the scheduling constraints of data, computing power and algorithm, and screen the computing nodes that meet all the constraints. The data constraints include data security level restriction, network domain out-domain restriction, etc., the computing power constraints include private domain network restriction, exclusive node use restriction, etc., and the algorithm constraints include chip architecture model, use authorization restriction, etc. Each edge in the global data flow topology graph is screened according to the constraints to obtain the candidate computing power center that can create an instance . Then, all algorithms that need to create an instance are sorted in descending order according to the total concurrent demand, and the algorithm with large demand is preferentially allocated with computing power resources. At the same time, all computing power specifications of each algorithm are sorted in descending order according to its key resource type and unit resource data processing concurrency, wherein the key resource refers to one or more resource types that have the greatest impact on the computing performance of the algorithm. First, the computing power specifications are sorted in descending order according to their key resources , and the unit resource data processing concurrency is calculated . Different specifications are sorted in descending order according to , and a priority queue is established . When two specifications have , , the one with large sorting is in front, and finally the final sorting of the computing power specifications is obtained, that is, the larger the computing amount provided by the unit resource, the higher the sorting of the specification. The is selected as the computing power center with data scheduling cost as the first priority, wherein the data scheduling cost of the computing power center is . When the computing power center has been allocated with the specification containing ​subtasks ,or ,and When the computing center It does not contain Subtasks, that is, tasks that are prioritized Assigned to contain The computing power center reduces duplicate copying or network transmission of data, thereby reducing data scheduling costs.

[0083] For computing power centers with the same first priority, the computing power center with the lowest utilization rate is selected as the second priority. The computing power utilization rate , For computing power center resource Utilization rate For the corresponding weighting coefficients ( For the computing centers selected according to the first and second priorities. Algorithm computing power specification queue The corresponding chip architecture uses a third-priority selection of computing node sets. This means prioritizing the selection of computing nodes that match the chip architecture of the specification that provides the highest computational load per unit of resource within the computing power specification, thereby maximizing resource utilization efficiency. For computing centers selected according to the first, second, and third priorities... and node set The computing node with the highest fragmentation rate is selected with computing power fragmentation rate as the third priority. The fragmentation rate of computing power , , For computing nodes resource Usage For computing nodes resource The total amount, For the corresponding weighting coefficients ( This means prioritizing the selection of the node with the least remaining resources after instance creation, thereby maximizing the use of the entire node and reducing fragmentation.

[0084] In a feasible embodiment, the weights corresponding to the quantified constraints to be screened are determined based on a preset priority strategy, which can be obtained through the following process:

[0085] Firstly, historical scheduling data of a plurality of computing nodes corresponding to the current large model is acquired to cluster the to-be-screened constraint factors corresponding to the historical scheduling data to obtain a clustered set of quantized to-be-screened constraint factors, so as to identify a constraint factor combination that frequently appears in historical scheduling based on clustering. Then, the quantized to-be-screened constraint factors are sorted based on the number of the quantized to-be-screened constraint factors in each clustered set to obtain a first sequence. Then, the historical scheduling data of adjacent historical unit time periods are compared with each other based on time sequence to obtain a set of differential scheduling data, so as to sort the quantized to-be-screened constraint factors according to the frequency of the quantized to-be-screened factors corresponding to the differential scheduling data to obtain a second sequence. The first sequence and the second sequence are fused according to a preset rule, for example, the first sequence accounts for 60% of the weight and the second sequence accounts for 40% of the weight, and then the weight corresponding to each quantized to-be-screened constraint factor is determined. This process generates the weight based on clustering statistics and differential analysis of historical scheduling data, rather than relying on artificial experience presets, which can avoid overemphasizing hardware computing power and ignoring network delay and other implicit constraints. In addition, by clustering and quantifying the frequency of constraint factors, different dimensions of indicators can be unified, and the scientificity of weight calculation is improved. The weight fusion of the first sequence of high-frequency constraint factors and the second sequence of high-risk constraint factors can make the scheduling system give priority to factors of high-frequency needs and sudden bottlenecks.

[0086] Further, in one or more embodiments of the present specification, based on the optimal computing power center, algorithm instances of each algorithm are created to realize resource allocation of the algorithm instances, specifically including:

[0087] According to the computing power specification priority queue, the optimal computing power specification corresponding to the algorithm is determined, and the algorithm instances of the algorithm are created based on the optimal computing power specifications in the optimal computing power center. Then, the concurrency corresponding to the optimal computing power specification is acquired, and the remaining concurrency is determined based on the concurrency and the total concurrency. According to the remaining concurrency and the preset termination condition, it is determined whether to terminate the algorithm instance allocation of the algorithm. That is, in a certain application scenario, the node with the smallest remaining resource after instance creation is preferentially selected, so as to preferentially occupy the entire node to reduce the fragmentation rate. According to the above method, the algorithm instance is created until the total concurrency required by the algorithm is met or the resource is fully occupied.

[0088] As shown in Figure 6 In a certain application scenario, the allocation can be performed based on the following process: first, according to the concurrency demand sorting, the resource allocation is preferentially performed for the waterlogging detection (e2). According to the data out-domain constraint and the algorithm chip architecture constraint, center1 is selected for allocation. According to the third priority, for specifications G2 and G3, the unit resource concurrency is the same, So according to the maximum concurrent quantity ranking priority selection G2, it is determined to create 2 instances of node2 in center1. For the algorithm mechanism model calculation (e3), according to the algorithm chip architecture constraint and the algorithm authorization constraint, it is determined to create 1 instance of node4 in center3. For the algorithm AI agent (e1), according to the algorithm chip architecture constraint, it is determined to create 1 instance of node1 in center1. For the face recognition (e4), according to the data out-domain constraint and the algorithm chip architecture constraint, center1 and center2 are selected for allocation, and according to the first priority rule and the third priority rule, since the camera data has been allocated in center1, the unit resource concurrent quantity of the algorithm power specification G6 is greater than G5, so it is determined to create 1 instance of node1 in center1. For the license plate recognition (e5), according to the algorithm chip architecture constraint, it is determined to create 1 instance of node2 in center1. For the feature screening (e6) and the license plate screening (e7), according to the algorithm chip architecture constraint, center1, center2 and center3 are selected for allocation, and according to the first priority rule, the output data of the upstream task is used as the input data, and since the upstream task has been allocated in center1, it is determined to create 1 instance of node1 in center1. For the trajectory restoration (e11) and the aggregation sorting (e12), according to the algorithm chip architecture constraint, center1, center2 and center3 are selected for allocation, and according to the first priority rule, the output data of the upstream task is used as the input data, and since the upstream task has been allocated in center1, it is determined to create 1 instance of node1 in center1. Figure 6

[0089] As shown in Figure 7 , the embodiment of the present specification provides an internal structure diagram of a large model computing resource coupling scheduling device. As can be seen from Figure 7 , in one or more embodiments of the present specification, a large model computing resource coupling scheduling device, the device comprises:

[0090] at least one processor; and,

[0091] a memory in communication connection with the at least one processor; wherein,

[0092] the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform any of the above methods.

[0093] As shown in Figure 8 , the embodiment of the present specification provides a structure diagram of a non-volatile storage medium. As can be seen from Figure 8 ​It can be known that in one or more embodiments of the present specification, a nonvolatile storage medium stores computer executable instructions 801, and the computer executable instructions 801 can execute any of the above-mentioned methods.

[0094] Each of the embodiments in the present specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other. Each of the embodiments mainly describes the difference from other embodiments. In particular, for the device, equipment, and nonvolatile computer storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the part of the method embodiments.

[0095] The above describes specific embodiments of the present specification. Other embodiments are within the scope of the appended claims. In some cases, the acts or steps recited in the claims can be performed in a different order than the order in which they are recited and still achieve desirable results. In addition, the processes depicted in the figures do not necessarily require the particular order shown, or sequential order, to achieve the desired results. In some implementations, multitasking and parallel processing can be advantageous or necessary.

[0096] The above only describes one or more embodiments of the present specification and does not limit the present specification. One or more embodiments of the present specification can have various modifications and changes for those skilled in the art. Any modification, equivalent replacement, improvement, etc. within the spirit and principle of one or more embodiments of the present specification shall be included in the scope of the claims of the present specification.

Claims

1. A method for coupling scheduling of large model computing resources, characterized in that, The method comprises: determining the algorithms and required data corresponding to each computing task of the current large model to split each computing task based on the algorithms and the required data to obtain computing subtasks; based on the computing topology relationship between each computing subtask, a global data flow topology graph is established; wherein the computing topology relationship includes: parallel relationship, serial relationship; determine the data processing concurrency of each edge in the global data flow topology graph according to the computing speed corresponding to each computing task, to determine the total concurrency of each algorithm based on the data processing concurrency of each edge; based on the total concurrency of each algorithm and the preset hierarchical constraint condition, determine the optimal computing power center, and create algorithm instances of each algorithm based on the optimal computing power center to realize resource allocation of the algorithm instances; based on the total concurrency of each algorithm and the preset hierarchical constraint condition, determine the optimal computing power center, specifically comprising: based on the computing power specification vector corresponding to each algorithm and the total concurrency of each algorithm, determine the required algorithm instance number of each algorithm under different computing power specifications; based on the total concurrency corresponding to each algorithm, sort the algorithms corresponding to each required algorithm instance, and sort the computing power specifications based on the resource information corresponding to each computing power specification to obtain a computing power specification priority queue; wherein the resource information includes: key resource type, unit resource data processing concurrency; analyze each algorithm based on the preset hierarchical constraint condition to determine the optimal computing power center corresponding to each algorithm; wherein the preset hierarchical constraint condition includes: data constraint condition, computing power constraint condition, algorithm constraint condition; analyze each algorithm based on the preset hierarchical constraint condition to determine the optimal computing power center corresponding to each algorithm, specifically comprising: determine the constraint factors corresponding to each constraint condition in the preset hierarchical constraint condition, and aggregate the constraint factors and preset resource constraint factors as to-be-screened constraint factors; quantify each to-be-screened constraint factor based on the index corresponding to each to-be-screened constraint factor; determine the weight of the quantized to-be-screened factor based on the preset priority strategy, to determine the screening priority of each preset hierarchical constraint condition according to the weight corresponding to each to-be-screened constraint factor, and screen computing nodes based on the priority to obtain the optimal computing power center corresponding to each algorithm.

2. The coupling scheduling method of large model computing resources according to claim 1, characterized in that, determining the algorithms and required data corresponding to each computing task of the current large model to split each computing task based on the algorithms and the required data to obtain computing subtasks, specifically comprising: determining the algorithms and required data corresponding to each computing task of the current large model to obtain the minimum granularity data set corresponding to each algorithm; wherein the minimum granularity data set includes: algorithm input data, algorithm output data; based on each algorithm and the minimum granularity data set corresponding to each algorithm, split each computing task to obtain multiple computing subtasks.

3. The coupling scheduling method of large model computing resources according to claim 2, characterized in that, based on the computing topology relationship between each computing subtask, a global data flow topology graph is established, specifically comprising: establish a data flow topology graph corresponding to each of the computing tasks based on a computing topology relationship between each of the computing sub-tasks; determine whether there are multiple input data corresponding to each of the computing sub-tasks based on the minimum granularity data set; if there are multiple input data, create a merging operator corresponding to the computing sub-task, and obtain an initial global data flow topology graph based on an edge corresponding to the merging operator; merge vertices belonging to the same data and the same edge in the initial global data flow topology graph to obtain a global data flow topology graph; if there are no multiple input data, merge vertices belonging to the same data and the same edge in the data flow topology graph to obtain a global data flow topology graph.

4. The coupling scheduling method of large model computing resources according to claim 1, characterized in that, determine data processing concurrency of each edge in the global data flow topology graph based on the computing speed corresponding to each of the computing tasks, to determine total concurrency of each of the algorithms based on the data processing concurrency of each edge, specifically including: identify data flow paths corresponding to each of the computing tasks based on the global data flow topology graph, to determine the computing speed corresponding to the computing task according to the data flow paths; determine whether there is a merging operator in the computing sub-task, to remove the merging operator and determine a to-be-assigned partial topology graph of each of the computing sub-tasks; assign data concurrency to the to-be-assigned partial topology graph of the computing sub-task corresponding to the computing task based on the computing speed corresponding to the computing task; traverse edges of the global data flow topology graph to determine data processing concurrency of each edge in the global data flow topology graph; determine algorithms corresponding to each edge, to merge data processing concurrency of the corresponding edge based on the algorithms to obtain total concurrency of each of the algorithms.

5. The coupling scheduling method of large model computing resources according to claim 4, characterized in that, traverse edges of the global data flow topology graph to determine data processing concurrency of each edge in the global data flow topology graph, specifically including: classify each edge based on the computing sub-task corresponding to the edge to determine an edge type of each edge; wherein the edge type includes a single-task type and a multi-task type; if it is determined that the edge type is the single-task type, determine data concurrency of the edge based on data concurrency of the computing sub-task corresponding to the edge; if it is determined that the edge type is the multi-task type, obtain data concurrency of multiple computing sub-tasks corresponding to the edge to take the maximum data concurrency as the data concurrency of the edge.

6. The coupling scheduling method of large model computing resources according to claim 1, characterized in that, create algorithm instances of each of the algorithms based on the optimal computing power center to implement resource allocation of the algorithm instances, specifically including: determine optimal computing power specifications corresponding to the algorithm according to the computing power specification priority queue, to create algorithm instances of the algorithm based on each of the optimal computing power specifications at the optimal computing power center; obtain concurrency corresponding to the optimal computing power specification, to determine a remaining concurrency based on the concurrency and the total concurrency, to determine whether to terminate the algorithm instance allocation of the algorithm based on the remaining concurrency and a preset termination condition.

7. A large model computing resource coupling scheduling device, characterized in that, The device includes: at least one processor; and a memory connected in communication with the at least one processor; wherein The memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-6.

8. A non-transitory storage medium storing computer-executable instructions, the computer-executable instructions comprising: The computer executable instructions are capable of performing the method of any one of claims 1-6.

Images