Coupling scheduling method and device for large model computing resources and medium

Abstract

The embodiment of the invention discloses a coupling scheduling method and device for large model computing resources and a medium, relates to the technical field of computers, and is used for solving the problem that a computing power center is not fully utilized due to an existing scheduling mode. The method comprises the steps that all computing tasks are split based on an algorithm and needed data, and computing subtasks are obtained; establishing a global data flow topological graph based on the calculation topological relation among the calculation sub-tasks; according to the calculation speed corresponding to each calculation task, determining the data processing concurrency of each edge in the global data flow topological graph, so as to determine the total concurrency of each algorithm based on the data processing concurrency of each edge; and determining an optimal computing power center based on the total concurrency of each algorithm and a preset hierarchical constraint condition, and creating an algorithm instance of each algorithm based on the optimal computing power center so as to realize resource allocation of the algorithm instance.

Description

Technical Field

[0001] This specification relates to the field of computer technology, and in particular to a coupling scheduling method, device, and medium for large-scale model computing resources. Background Art

[0002] With the development of artificial intelligence (AI), particularly the emergence of new-generation algorithms represented by large language models, an increasing number of industries are adopting AI algorithms to assist and improve production efficiency. Furthermore, the development and application of AI technology rely heavily on data and computing power. The rapid growth of the data market is driving the continuous generation and trading of vast amounts of data, while the proliferation of computing servers across the globe provides the necessary infrastructure for massive computational analysis. However, data, computing power, and algorithms currently exist in relatively isolated and disconnected forms, lacking a mutually connected closed loop. This leaves a vast amount of valuable data unutilized and uncirculated, while a significant number of computing servers remain idle or oversupplied, leading to wasted resources. Therefore, the effective scheduling of computing resources is crucial.

[0003] Currently, when computing tasks are split based on existing frameworks, tasks are split through operator splitting, inter-layer segmentation, etc. However, static splitting strategies cannot dynamically adapt to heterogeneous computing power. For computing tasks with complex dependencies and computing logic, it is difficult to effectively split them, which can easily lead to resource waste. In addition, traditional topologies rely on manual presets and only support point-to-point communication between homogeneous devices. When tasks involve a complex mix of serial and parallel tasks, communication paths need to be hard-coded and cannot be dynamically optimized. In addition, it is difficult to fully consider the relationship between current data, computing power, and algorithms when scheduling computing resources. Allocating computing power based on simple rules can easily lead to high data scheduling costs and severe fragmentation of computing nodes. Summary of the Invention

[0004] In order to solve the above technical problems, one or more embodiments of this specification provide a coupling scheduling method, device and medium for large-model computing resources.

[0005] One or more embodiments of this specification adopt the following technical solutions: One or more embodiments of this specification provide a coupling scheduling method for large-scale model computing resources, the method comprising: Determine the algorithm and required data corresponding to each computing task of the current large model, and split each computing task based on the algorithm and the required data to obtain computing subtasks; Based on the computing topology relationship between each of the computing subtasks, a global data flow topology graph is established; wherein the computing topology relationship includes: a parallel relationship and a serial relationship; Determine the data processing concurrency of each edge in the global data flow topology diagram according to the computing speed corresponding to each computing task, and determine the total concurrency of each algorithm based on the data processing concurrency of each edge; Based on the total concurrency of each algorithm and the preset hierarchical constraints, the optimal computing power center is determined, and algorithm instances of each algorithm are created based on the optimal computing power center to achieve resource allocation of the algorithm instances.

[0006] Optionally, in one or more embodiments of this specification, determining the algorithm and required data corresponding to each computing task of the current large model, and splitting each computing task based on the algorithm and the required data to obtain computing subtasks specifically includes: Determine the algorithms and required data corresponding to each computing task of the current large model to obtain the minimum granularity data set corresponding to each algorithm; wherein the minimum granularity data set includes: algorithm input data and algorithm output data; Based on the algorithms and the minimum granularity data sets corresponding to the algorithms, each computing task is split to obtain a plurality of computing subtasks.

[0007] Optionally, in one or more embodiments of this specification, a global data flow topology graph is established based on the computing topology relationship between the computing subtasks, specifically including: Based on the computing topology relationship between the computing subtasks, a data flow topology graph corresponding to each computing task is established; Based on the minimum granularity data set, determining whether a computing subtask corresponding to each computing task has multiple input data; If there are multiple input data, create a merge operator corresponding to the calculation subtask and the edge corresponding to the merge operator to obtain the initial global data flow topology graph; Merging vertices belonging to the same data and the same edges in the initial global data flow topology graph to obtain a global data flow topology graph; If there are no multiple input data, the vertices belonging to the same data and the same edges in the data flow topology graph are merged to obtain a global data flow topology graph.

[0008] Optionally, in one or more embodiments of this specification, determining the data processing concurrency of each edge in the global data flow topology graph according to the computing speed corresponding to each computing task, and determining the total concurrency of each algorithm based on the data processing concurrency of each edge, specifically includes: Identifying a data flow path corresponding to each of the computing tasks based on the global data flow topology graph, so as to determine a computing speed corresponding to the computing task according to the data flow path; Determine whether there is a merging operator in the computing subtask, so as to remove the merging operator and determine a topological graph of a portion to be allocated for each computing subtask; Based on the computing speed corresponding to the computing task, allocating data concurrency to the to-be-allocated portion of the topology graph of the computing subtask corresponding to the computing task; Traversing the edges of the global data flow topology graph to determine the data processing concurrency of each edge in the global data flow topology graph; The algorithms corresponding to the edges are determined, and the data processing concurrency of the corresponding edges is merged based on the algorithms to obtain the total concurrency of the algorithms.

[0009] Optionally, in one or more embodiments of the present specification, traversing the edges of the global data flow topology graph to determine the data processing concurrency of each edge in the global data flow topology graph specifically includes: Classifying each edge based on the computing subtask corresponding to each edge to determine the edge type of each edge; wherein the edge type includes: single-task type and multi-task type; If it is determined that the edge type is a single-task type, the data concurrency of the edge is determined based on the data concurrency of the computing subtask corresponding to the edge; If it is determined that the edge type is a multi-task type, the data concurrency of multiple computing subtasks corresponding to the edge is obtained, and the maximum data concurrency is used as the data concurrency of the edge.

[0010] Optionally, in one or more embodiments of this specification, determining the optimal computing power center based on the total concurrency of each algorithm and preset layer constraints specifically includes: Determine the number of algorithm instances required for each algorithm at different computing power specifications based on the computing power specification vector corresponding to each algorithm and the total concurrency of each algorithm; Based on the total concurrency corresponding to each algorithm, the algorithms corresponding to each required algorithm instance are sorted, and based on the resource information corresponding to each computing power specification, the computing power specifications are sorted to obtain a computing power specification priority queue; wherein the resource information includes: key resource type and unit resource data processing concurrency; Based on preset hierarchical constraints, each of the algorithms is analyzed to determine the optimal computing power center corresponding to each algorithm; wherein the preset hierarchical constraints include: data constraints, computing power constraints, and algorithm constraints.

[0011] Optionally, in one or more embodiments of this specification, each algorithm is analyzed based on preset hierarchical constraints to determine the optimal computing power center corresponding to each algorithm, specifically including: Determining the constraint factors corresponding to each constraint condition in the preset hierarchical constraint conditions, and summarizing the constraint factors and the preset resource constraint factors as constraint factors to be screened; quantifying each of the constraint factors to be screened based on the indicators corresponding to the constraint factors to be screened; Based on the preset priority strategy, the weights corresponding to the quantified factors to be screened are determined, and the screening priority of each preset hierarchical constraint condition is determined according to the weights corresponding to each constraint factor to be screened, and the computing nodes are hierarchically screened based on the priority to obtain the optimal computing power center corresponding to each algorithm.

[0012] Optionally, in one or more embodiments of this specification, creating an algorithm instance of each algorithm based on the optimal computing power center to implement resource allocation of the algorithm instance specifically includes: Determining the optimal computing power specification corresponding to the algorithm according to the computing power specification priority queue, so as to create an algorithm instance of the algorithm based on each of the optimal computing power specifications in the optimal computing power center; Obtain the concurrency corresponding to the optimal computing power specification, determine the remaining concurrency based on the concurrency and the total concurrency, and determine whether to terminate the algorithm instance allocation of the algorithm based on the remaining concurrency and a preset termination condition.

[0013] One or more embodiments of this specification provide a coupling scheduling device for large-model computing resources, the device including: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to: perform any of the above methods.

[0014] One or more embodiments of this specification provide a non-volatile computer storage medium storing computer-executable instructions, wherein the computer-executable instructions are configured to execute any of the above-described methods.

[0015] At least one of the above technical solutions adopted in the embodiments of this specification can achieve the following beneficial effects: By splitting computing tasks into subtasks and establishing a global data flow topology based on computing topology relationships, the dependencies between tasks are clearly revealed, avoiding the problem that static splitting strategies cannot dynamically adapt to heterogeneous computing power. For computing tasks with complex dependencies and computing logic, it is difficult to effectively split them, which easily leads to resource waste. Through the total concurrency of the algorithm and the preset hierarchical constraints, resources are accurately matched with demand, avoiding the problem of resource waste or idle resources. It also realizes the data-centric joint optimization scheduling of data, computing power and algorithms, reducing the repeated transmission and copying of data, reducing the repeated calculation of algorithms, and improving the efficiency of resource and data utilization. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the embodiments of this specification or the technical solutions in the prior art, the following briefly introduces the drawings required for the embodiments or the description of the prior art. Obviously, the drawings described below are only some of the embodiments described in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without inventive work. In the drawings: Figure 1 A schematic flow chart of a coupling scheduling method for large-scale model computing resources provided in an embodiment of this specification; Figure 2 A schematic diagram of a computing task in an application scenario provided by an embodiment of this specification; Figure 3 A schematic diagram of computing center information of computing resources in an application scenario provided by an embodiment of this specification; Figure 4 This is a global data flow topology diagram for an application scenario provided in an embodiment of this specification.

[0017] Figure 5 A schematic diagram of the computing power specifications corresponding to each algorithm in an application scenario provided in an embodiment of this specification; Figure 6 A schematic diagram of an algorithm example provided in an application scenario provided in an embodiment of this specification; Figure 7 A schematic diagram of the structure of a coupling scheduling device for large-scale model computing resources provided in an embodiment of this specification; Figure 8 A schematic diagram of the structure of a non-volatile storage medium provided in an embodiment of this specification. DETAILED DESCRIPTION

[0018] The embodiments of this specification provide a coupling scheduling method, device and medium for large-scale model computing resources.

[0019] To help those skilled in the art better understand the technical solutions in this specification, the following will provide a clear and complete description of the technical solutions in the embodiments of this specification, in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of this specification, not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this specification without creative work should fall within the scope of protection of this specification.

[0020] like Figure 1 As shown, the embodiment of this specification provides a method flow diagram of a coupling scheduling method for large model computing resources. Figure 1 It can be seen that in one or more embodiments of this specification, a coupling scheduling method for large model computing resources specifically includes the following steps: S101: Determine the algorithm and required data corresponding to each computing task of the current large model, and split each computing task based on the algorithm and the required data to obtain computing subtasks.

[0021] In order to accurately identify the computing power specifications of computing tasks such as CPU / GPU type, memory capacity, storage IO requirements, etc., so as to improve the accuracy of subsequent resource scheduling. In the embodiment of this specification, the algorithm and required data corresponding to each computing task of the current large model will be determined, and each computing task will be split according to the algorithm and required data to obtain computing subtasks. Figure 2 As shown in the figure, a certain application scenario involves three computing tasks requiring resource allocation. Task 1 involves urban flooding early warning monitoring. This involves detecting water depth using video data collected by road cameras and then predicting the time and location of flooding using a mechanism model. The computation speed is 20 qps. Task 2 involves traffic OD flow analysis. This involves recognizing vehicle license plates from video data collected by road cameras, combining it with road network topology data to restore vehicle trajectories. OD aggregation and sorting are then used to output OD paths and flows with high traffic volume. The computation speed is 1 qps. Task 3 involves a multimodal AI agent based on a large model performing person and vehicle search. The large model parses user input to determine whether the user is looking for a person or a vehicle. The agent then uses the provided time and location information to filter the road camera data that needs to be analyzed. The agent then uses a facial recognition algorithm or a license plate recognition algorithm, along with the results, to output the person or vehicle search results. The computation speed is 10 qps. By splitting complex tasks into subtasks, this process allows for a more focused objective within each subtask, facilitating the understanding of complex dependencies within the tasks and enabling more accurate resource scheduling.

[0022] Specifically, in one or more embodiments of this specification, the algorithm and required data corresponding to each computing task of the current large model are determined, and each computing task is split based on the algorithm and the required data to obtain computing subtasks, specifically including: First, determine the algorithms and required data corresponding to each computing task of the current large model, so as to obtain the minimum granularity data set corresponding to each algorithm. It should be noted that the minimum granularity data set includes: algorithm input data and algorithm output data. It is the minimum granularity data set required for the algorithm to perform a calculation. For input data, Then, based on each algorithm and the minimum granularity data set corresponding to each algorithm, each computing task is split to obtain multiple computing subtasks. .algorithm It is a computing program with a function operator as the smallest module. It should be noted that the computing topology relationship between each computing subtask includes parallel and serial, where two subtasks and Serial , parallelism is divided into deterministic parallelism and probabilistic parallelism, among which deterministic parallelism is , while probabilistic parallelism means that different algorithms are probabilistically triggered to perform calculations according to different values of input data in the calculation step, but only one algorithm will be triggered each time the task runs, which can be expressed as .

[0023] S102: Based on the computing topology relationship between the computing subtasks, a global data flow topology graph is established; wherein the computing topology relationship includes: a parallel relationship and a serial relationship.

[0024] After the computing task is split based on step S101, a global data flow topology diagram is established based on the computing topology relationships between the various computing subtasks, i.e., the parallel or serial relationships between the computing subtasks. This process, by converting the computing tasks into a global data flow topology diagram, can clearly reveal the data flow path, thereby avoiding deadlock or resource contention caused by dependency confusion when complex computing tasks exist.

[0025] Specifically, in one or more embodiments of this specification, a global data flow topology graph is established based on the computational topology relationship between the computational subtasks, specifically including the following process: First, based on the computational topology relationship between each computational subtask, a data flow topology graph corresponding to each computational task is established. Then, based on the minimum granularity data set, it is determined whether the computational subtask corresponding to each computational task has multiple input data. If multiple input data exist, a merge operator corresponding to the computational subtask is created, and the edges corresponding to the merge operator are used to obtain an initial global data flow topology graph. Then, the vertices and edges belonging to the same data in the initial global data flow topology graph are merged to obtain a global data flow topology graph. If multiple input data do not exist, the vertices and edges belonging to the same data in the data flow topology graph are merged to obtain a global data flow topology graph.

[0026] In a feasible embodiment, a global data flow topology diagram is established based on the computational topology relationship between each computational subtask. Specifically, a data flow topology diagram can be established based on the serial and parallel relationship of each computational subtask, where the vertices , directed edges Merge the data flow topology graphs of each computing task to generate a global data flow topology graph, where the same data The vertices of , and algorithm The edges are merged into one edge. If the input data of the subtask is multiple, that is, , create a merge operator , and create k edges , to realize data merging operation. Figure 4 As shown in the figure, in a specific application scenario, three tasks are broken down into subtasks based on data and algorithms, creating a directed acyclic data flow topology graph, with vertices representing data and edges representing algorithms. The dashed edges in Task 3 indicate that face recognition and license plate recognition are probabilistically parallel subtasks. Subtasks with the same data and algorithms are then merged to form a global data flow topology graph.

[0027] S103: Determine the data processing concurrency of each edge in the global data flow topology diagram according to the computing speed corresponding to each computing task, and determine the total concurrency of each algorithm based on the data processing concurrency of each edge.

[0028] In order to provide a direct basis for the resource matching of the subsequent computing power center and ensure the precise alignment of computing power resources with algorithm requirements. In the embodiment of this specification, the data processing concurrency of each edge in the global data flow topology diagram will be determined according to the computing speed corresponding to each computing task, so as to determine the total concurrency of each algorithm based on the data processing concurrency of each edge. That is, by mapping the computing speed and concurrency, the blindness of estimating resources based only on personnel experience in the traditional solution is avoided, and the situation of resource redundancy or shortage is reduced. And taking concurrency as one of the core indicators of subsequent scheduling makes the computing power center screening and resource allocation process more explainable, which is convenient for decision makers to optimize resource strategies in a data-driven manner rather than relying on subjective judgment.

[0029] Specifically, in one or more embodiments of this specification, the data processing concurrency of each edge in the global data flow topology graph is determined according to the computing speed corresponding to each computing task, so as to determine the total concurrency of each algorithm based on the data processing concurrency of each edge. Specifically, the process includes the following: First, based on the global data flow topology graph, the data flow path corresponding to each computing task is identified to determine the computing speed corresponding to the computing task based on the data flow path. Then, it is determined whether the merge operator proposed in the above step S102 exists in the computing subtask. Since the merge operator does not calculate the concurrency, the merge operator is removed to determine the topology graph of the portion to be allocated of each computing subtask. Then, based on the computing speed corresponding to the computing task, the data concurrency of the topology graph of the portion to be allocated of the computing subtask corresponding to the computing task is allocated; wherein, the computing speed of each computing subtask is consistent during allocation, and the sum of the data concurrency of each computing subtask is greater than or equal to the data concurrency of the computing task. By traversing the edges of the global data flow topology graph, the data processing concurrency of each edge in the global data flow topology graph can be determined. Then, in order to realize one-time calculation of the algorithm and multiple reuse of the results, the algorithm corresponding to each edge can be determined, and the data processing concurrency of the corresponding edge can be merged based on the algorithm to obtain the total concurrency of each algorithm.

[0030] Furthermore, in one or more embodiments of the present specification, traversing the edges of the global data flow topology graph to determine the data processing concurrency of each edge in the global data flow topology graph specifically includes: Each edge is classified based on the computing subtask corresponding to it, and its edge type is determined. Edge types include single-task and multi-task. If the edge type is determined to be single-task, meaning that multiple tasks do not share the edge, the data concurrency of the edge is determined based on the data concurrency of the computing subtask corresponding to the edge. If the edge type is determined to be multi-task, meaning that multiple tasks share the edge, the data concurrency of the multiple computing subtasks corresponding to the edge is obtained, with the maximum data concurrency used as the data concurrency of the edge.

[0031] In a feasible embodiment, according to the computational speed requirement of each computational task, that is, the data processing concurrency, it is determined that the computational speed of each subtask is consistent and at least reaches the concurrency of the overall task, and the merging operator does not calculate the concurrency. Then, according to the global data flow topology, when an edge is shared by multiple tasks, by caching the computational results of the edge, the algorithm can be calculated once and the results can be reused multiple times. Therefore, the concurrency of this edge is calculated according to the maximum concurrency of all tasks. Finally, the concurrency of the edges of the same algorithm is calculated. Merge to get the total concurrency of each algorithm For example, in Figure 4 In the application scenario shown, continuing with the above example, the computing speed of task 1 is 20qps, the computing speed of task 2 is 1qps, and the computing speed of task 3 is 10qps. Then, based on the computing speed of each task, calculate the concurrency requirement of each edge. , , , , , .in It is shared by Task 2 and Task 3, so the concurrency requirement is the maximum of Task 2 and Task 3. Since the algorithm for each edge is different, the concurrency requirement of each algorithm is consistent with the concurrency requirement of the corresponding edge.

[0032] S104: Based on the total concurrency of each algorithm and the preset hierarchical constraints, determine the optimal computing power center, and create algorithm instances of each algorithm based on the optimal computing power center to achieve resource allocation of the algorithm instances.

[0033] Based on the total concurrency of each algorithm obtained in the above steps and the pre-set hierarchical constraints, the optimal computing power center is determined, and algorithm instances of each algorithm are created based on the optimal computing power center to implement resource allocation for the algorithm instances. Specifically, in one or more embodiments of this specification, based on the total concurrency of each algorithm and the pre-set hierarchical constraints, the optimal computing power center is determined, which specifically includes the following steps: First, based on the computing power specification vector corresponding to each algorithm and the total concurrency of each algorithm, the number of algorithm instances required for each algorithm at different computing power specifications is determined. Then, based on the total concurrency corresponding to each algorithm, the algorithms corresponding to each required algorithm instance are sorted. This is then done based on the resource information corresponding to each computing power specification, resulting in a computing power specification priority queue. This resource information includes: key resource types and unit resource data processing concurrency. Each algorithm is then analyzed based on pre-set hierarchical constraints to determine the optimal computing power center for each algorithm. It should be noted that the pre-set hierarchical constraints include: data constraints, computing power constraints, and algorithm constraints.

[0034] In a feasible embodiment, since each algorithm supports different chip architectures and the computing performance under different architectures has certain differences, in a multi-architecture and multi-algorithm scenario, the computing power specification of an algorithm is set as a vector: ,in For the algorithm No. Specifications For specifications The consumption of different computing resources is as follows. Computing resources include at least one of the following: CPU, memory, computing card (GPU, TPU, NPU, etc.), hard disk, and network bandwidth. The maximum algorithm concurrency supported by this specification. Calculate the number of instances required for each specification based on the computing power specification and the total algorithm concurrency requirements. According to the concurrency of each edge in the global data flow topology graph , and the computing power specifications of the algorithm corresponding to this edge Maximum number of concurrent algorithms supported , calculate the number of data splits , among which No data segmentation is performed. That is, according to the computing power specifications and concurrency requirements of each algorithm, the number of instances required for each specification and the number of corresponding data slices are calculated. Figure 5 It can be seen that due to the high computing speed requirements of Task 1, a large number of instances are required. At the same time, the camera video data needs to be divided into equal parts for parallel computing to meet the computing speed requirements.

[0035] Furthermore, in one or more embodiments of this specification, each algorithm is analyzed based on preset hierarchical constraints to determine the optimal computing power center corresponding to each algorithm, specifically including: First, determine the constraint factors corresponding to each constraint condition in the preset hierarchical constraint conditions, and summarize the constraint factors and preset resource constraint factors such as network bandwidth cost, energy consumption efficiency, security certification level, geographical location delay, etc. as the constraint factors to be screened. According to the indicators corresponding to each constraint factor to be screened, quantify each constraint factor to be screened. For example, when the constraint factor to be screened is a hardware resource factor, it will be based on the following: Figure 3 The CPU computing power and memory bandwidth shown are quantified. That is, the weights corresponding to the quantified factors to be screened are determined based on the preset priority strategy. Based on the weights corresponding to the constraints to be screened, the screening priorities of the preset hierarchical constraints are determined. The computing nodes are then screened hierarchically based on the priority to obtain the optimal computing power center corresponding to each algorithm.

[0036] In a feasible embodiment, resource allocation for algorithm instances needs to consider data, computing power, and algorithm scheduling constraints, and select computing nodes that meet all constraints. Data constraints include data confidentiality level restrictions, network domain exit restrictions, etc., computing power constraints include private domain network restrictions, exclusive node usage restrictions, etc., and algorithm constraints include chip architecture model, usage authorization restrictions, etc. For each edge in the global data flow topology diagram, candidate computing power centers that can create instances are selected according to the constraints. Then, all algorithms that need to create instances are sorted in descending order according to the total concurrent demand, and computing resources are allocated first to algorithms with large demands. At the same time, all computing power specifications of each algorithm are sorted in descending order according to their key resource types and unit resource data processing concurrency, where key resources refer to one or more resource types that have the greatest impact on the algorithm's computing performance. First, sort the computing power specifications in descending order according to their key resources. , calculate the concurrent amount of data processing per unit resource , the different specifications are arranged in order according to Sort in descending order and create a priority queue , when the two specifications and have hour, The larger the amount of computing power provided by a unit resource, the higher the ranking of the specification. The computing center is selected with data scheduling cost as the first priority, among which the computing center The data scheduling cost is , when the computing power center The Subtasks ,or ,and , when the computing power center Not included Subtasks, that is, give priority to tasks Assign to include The computing center can reduce duplicate data copying or network transmission, thereby reducing data scheduling costs.

[0037] For computing centers with the same first priority, the computing center with the lowest utilization rate is selected as the second priority, where the computing center The computing power utilization rate is , For the computing center resource The utilization rate, is the corresponding weight coefficient ( ). For the computing centers selected according to the first and second priority , based on the algorithm computing power specification queue The corresponding chip architecture is the third priority screening computing node set , that is, give priority to selecting computing nodes that match the chip architecture corresponding to the specification that provides the largest amount of computing per unit resource in the computing power specification, thereby maximizing resource utilization efficiency. For the computing power centers selected according to the first, second and third priorities, and node collection , taking computing power fragmentation rate as the third priority, select the computing node with the highest fragmentation rate, among which node The computing power fragmentation rate is , , For computing nodes resource The amount of use, For computing nodes resource The total amount, is the corresponding weight coefficient ( ), that is, the node with the least remaining resources after the instance is created is preferentially selected, so as to occupy the entire node first to reduce the fragmentation rate.

[0038] In a feasible embodiment, the weights corresponding to the quantified constraint factors to be screened are determined based on the preset priority strategy, and can be obtained based on the following process: First, the historical scheduling data of multiple computing nodes corresponding to the current large model is obtained to cluster the constraints to be screened corresponding to the historical scheduling data to obtain a cluster set of quantified constraints to be screened, so as to identify the combination of constraints that appear frequently in historical scheduling based on clustering. Based on the number of each quantized constraint to be screened in each cluster set, the quantized constraint to be screened is sorted to obtain a first sequence. Then, the historical scheduling data of adjacent historical unit time periods are compared pairwise based on the time sequence to obtain a differential scheduling data set, so as to sort the quantized constraint to be screened according to the frequency of occurrence of the quantized constraint to be screened corresponding to the differential scheduling data to obtain a second sequence. The first sequence and the second sequence are fused according to preset rules. For example, after the fusion of the first sequence with a weight of 60% and the second sequence with a weight of 40%, the weight corresponding to each quantized constraint to be screened is determined. This process generates weights based on cluster statistics and differential analysis of historical scheduling data, rather than relying on empirical assumptions. This avoids overemphasizing hardware computing power while neglecting implicit constraints such as network latency. Furthermore, by quantifying constraints through clustering and differential frequency, indicators across different dimensions can be unified, improving the scientific nature of weight calculations. Furthermore, the weighted fusion of the first-order high-frequency constraints and the second-order high-risk constraints enables the scheduling system to prioritize high-frequency rigid demands and sudden bottlenecks.

[0039] Furthermore, in one or more embodiments of this specification, algorithm instances of each algorithm are created based on the optimal computing power center to implement resource allocation of the algorithm instances, specifically including: Based on the computing power specification priority queue, the optimal computing power specification corresponding to the algorithm is determined. Algorithm instances are then created at the optimal computing power center based on each optimal computing power specification. The concurrency corresponding to the optimal computing power specification is then obtained. Based on this concurrency and the total concurrency, the remaining concurrency is determined. Based on the remaining concurrency and a preset termination condition, whether to terminate the algorithm instance allocation is determined. Specifically, in certain application scenarios, the node with the least remaining resources after instance creation is prioritized, thereby prioritizing full node occupation to reduce fragmentation. Algorithm instance creation continues according to this method until the required total concurrency is met or all resources are fully occupied.

[0040] like Figure 6 As shown, in a certain application scenario, allocation can be based on the following process: First, sort by concurrency requirements and prioritize resource allocation for water accumulation detection (e2). Based on data out-of-domain constraints and algorithm chip architecture constraints, center1 is selected for allocation. Based on the third priority, for specifications G2 and G3, the unit resource concurrency is the same. Therefore, G2 is prioritized based on the maximum concurrency sorting, and therefore two instances are created in node 2 of center 1. For algorithm mechanism model calculation (e3), based on the algorithm chip architecture constraints and algorithm authorization constraints, one instance is created in node 4 of center 3. For algorithm AI agent (e1), based on the algorithm chip architecture constraints, one instance is created in node 1 of center 1. For face recognition (e4), based on the data out-of-domain constraints and algorithm chip architecture constraints, center 1 and center 2 are selected for allocation. According to the first and third priority rules, since camera data has been allocated in center 1, the unit resource concurrency of computing power specification G6 is greater than G5, so one instance is created in node 1 of center 1. For license plate recognition (e5), based on the algorithm chip architecture constraints, one instance is created in node 2 of center 1. For feature screening (e6) and license plate screening (e7), according to the algorithm chip architecture constraints, center1, center2, and center3 are selected for allocation. According to the first priority rule, the output data of the upstream task is used as the input data. Since the upstream task has been assigned to center1, it is determined that one instance is created for each node1 in center1. For trajectory restoration (e11) and aggregation sorting (e12), according to the algorithm chip architecture constraints, center1, center2, and center3 are selected for allocation. According to the first priority rule, the output data of the upstream task is used as the input data. Since the upstream task has been assigned to center1, it is determined that one instance is created for each node1 in center1. Figure 6 The example shown.

[0041] like Figure 7 As shown, the embodiment of this specification provides a schematic diagram of the internal structure of a coupling scheduling device for large-scale model computing resources. Figure 7 It can be seen that in one or more embodiments of this specification, a coupling scheduling device for large model computing resources includes: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to: perform any of the above methods.

[0042] like Figure 8 As shown in FIG, the embodiment of this specification provides a structural diagram of a non-volatile storage medium. Figure 8It can be seen that in one or more embodiments of this specification, a non-volatile storage medium stores computer-executable instructions 801, and the computer-executable instructions 801 can execute any of the above-described methods.

[0043] The various embodiments in this specification are described in a progressive manner. Similar portions between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from the other embodiments. In particular, the device, apparatus, and non-volatile computer storage medium embodiments are generally similar to the method embodiments, so their descriptions are relatively simplified. For relevant details, refer to the descriptions of the method embodiments.

[0044] The foregoing description of this specification describes specific embodiments. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0045] The foregoing description is merely one or more embodiments of this specification and is not intended to limit this specification. It will be apparent to those skilled in the art that various modifications and variations may be made to one or more embodiments of this specification. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of one or more embodiments of this specification are intended to be within the scope of the claims of this specification.

Claims

1. A coupling scheduling method for large model computing resources, characterized in that: The method comprises: Determine the algorithm and required data corresponding to each computing task of the current large model, and split each computing task based on the algorithm and the required data to obtain computing subtasks; Based on the computing topology relationship between each of the computing subtasks, a global data flow topology graph is established; wherein the computing topology relationship includes: a parallel relationship and a serial relationship; Determine the data processing concurrency of each edge in the global data flow topology diagram according to the computing speed corresponding to each computing task, and determine the total concurrency of each algorithm based on the data processing concurrency of each edge; Based on the total concurrency of each algorithm and the preset hierarchical constraints, the optimal computing power center is determined, and algorithm instances of each algorithm are created based on the optimal computing power center to achieve resource allocation of the algorithm instances.

2. A coupling scheduling method for large model computing resources according to claim 1, characterized in that: Determine the algorithm and required data corresponding to each computing task of the current large model, and split each computing task based on the algorithm and the required data to obtain computing subtasks, specifically including: Determine the algorithms and required data corresponding to each computing task of the current large model to obtain the minimum granularity data set corresponding to each algorithm; wherein the minimum granularity data set includes: algorithm input data and algorithm output data; Based on the algorithms and the minimum granularity data sets corresponding to the algorithms, each computing task is split to obtain a plurality of computing subtasks.

3. The coupling scheduling method for large-scale computing resources according to claim 2, characterized in that: Based on the computational topology relationship between the computational subtasks, a global data flow topology diagram is established, specifically including: Based on the computing topology relationship between the computing subtasks, a data flow topology graph corresponding to each computing task is established; Based on the minimum granularity data set, determining whether a computing subtask corresponding to each computing task has multiple input data; If there are multiple input data, create a merge operator corresponding to the calculation subtask and the edge corresponding to the merge operator to obtain the initial global data flow topology graph; Merging vertices belonging to the same data and the same edges in the initial global data flow topology graph to obtain a global data flow topology graph; If there are no multiple input data, the vertices belonging to the same data and the same edges in the data flow topology graph are merged to obtain a global data flow topology graph.

4. The coupling scheduling method for large-scale model computing resources according to claim 1, characterized in that: Determining the data processing concurrency of each edge in the global data flow topology diagram according to the computing speed corresponding to each computing task, and determining the total concurrency of each algorithm based on the data processing concurrency of each edge, specifically including: Identifying a data flow path corresponding to each of the computing tasks based on the global data flow topology graph, so as to determine a computing speed corresponding to the computing task according to the data flow path; Determine whether there is a merging operator in the computing subtask, so as to remove the merging operator and determine a topological graph of a portion to be allocated for each computing subtask; Based on the computing speed corresponding to the computing task, allocating data concurrency to the to-be-allocated portion of the topology graph of the computing subtask corresponding to the computing task; Traversing the edges of the global data flow topology graph to determine the data processing concurrency of each edge in the global data flow topology graph; The algorithms corresponding to the edges are determined, and the data processing concurrency of the corresponding edges is merged based on the algorithms to obtain the total concurrency of the algorithms.

5. A coupling scheduling method for large model computing resources according to claim 4, characterized in that: Traversing the edges of the global data flow topology graph to determine the data processing concurrency of each edge in the global data flow topology graph specifically includes: Classifying each edge based on the computing subtask corresponding to each edge to determine the edge type of each edge; wherein the edge type includes: single-task type and multi-task type; If it is determined that the edge type is a single-task type, the data concurrency of the edge is determined based on the data concurrency of the computing subtask corresponding to the edge; If it is determined that the edge type is a multi-task type, the data concurrency of multiple computing subtasks corresponding to the edge is obtained, and the maximum data concurrency is used as the data concurrency of the edge.

6. The coupling scheduling method for large model computing resources according to claim 1, characterized in that: Based on the total concurrency of each algorithm and the preset layer constraints, the optimal computing power center is determined, specifically including: Determine the number of algorithm instances required for each algorithm at different computing power specifications based on the computing power specification vector corresponding to each algorithm and the total concurrency of each algorithm; Based on the total concurrency corresponding to each algorithm, the algorithms corresponding to each required algorithm instance are sorted, and based on the resource information corresponding to each computing power specification, the computing power specifications are sorted to obtain a computing power specification priority queue; wherein the resource information includes: key resource type and unit resource data processing concurrency; Based on preset hierarchical constraints, each of the algorithms is analyzed to determine the optimal computing power center corresponding to each algorithm; wherein the preset hierarchical constraints include: data constraints, computing power constraints, and algorithm constraints.

7. A coupling scheduling method for large model computing resources according to claim 6, characterized in that: Analyze each algorithm based on preset hierarchical constraints to determine the optimal computing power center corresponding to each algorithm, specifically including: Determining the constraint factors corresponding to each constraint condition in the preset hierarchical constraint conditions, and summarizing the constraint factors and the preset resource constraint factors as constraint factors to be screened; quantifying each of the constraint factors to be screened based on the indicators corresponding to the constraint factors to be screened; Based on the preset priority strategy, the weights corresponding to the quantified factors to be screened are determined, and the screening priority of each preset hierarchical constraint condition is determined according to the weights corresponding to each constraint factor to be screened, and the computing nodes are hierarchically screened based on the priority to obtain the optimal computing power center corresponding to each algorithm.

8. The coupling scheduling method for large-scale model computing resources according to claim 6, characterized in that: Creating algorithm instances of each algorithm based on the optimal computing power center to implement resource allocation for the algorithm instance specifically includes: Determining the optimal computing power specification corresponding to the algorithm according to the computing power specification priority queue, so as to create an algorithm instance of the algorithm based on each of the optimal computing power specifications in the optimal computing power center; Obtain the concurrency corresponding to the optimal computing power specification, determine the remaining concurrency based on the concurrency and the total concurrency, and determine whether to terminate the algorithm instance allocation of the algorithm based on the remaining concurrency and a preset termination condition.

9. A coupling scheduling device for large-scale computing resources, characterized in that: The device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to: execute the method according to any one of claims 1 to 8.

10. A non-volatile storage medium storing computer-executable instructions, characterized in that: The computer executable instructions can execute the method according to any one of claims 1 to 8.

Images