A computing resource allocation system for improving model reasoning efficiency

Abstract

The present invention provides a computing resource allocation system for improving model reasoning efficiency, which relates to the field of electronic digital data processing, and includes a task rating module, an inference management module, an inference monitoring module and a resource allocation module. The task rating module is used to perform priority rating on tasks in the model, the inference management module is used to control and manage the execution of tasks, the inference monitoring module is used to monitor the execution process of tasks, and the resource allocation module is used to allocate computing cores and video memories to tasks. The system rates tasks in the reasoning model and allocates computing resources based on the rating results, so as to effectively control the processing speed of different tasks, thereby improving the reasoning efficiency of the entire reasoning model.

Description

Technical Field

[0001] The present invention relates to the field of electronic digital data processing, and in particular to a computing resource allocation system for improving model reasoning efficiency. Background Art

[0002] The inference model is an important component of artificial intelligence and machine learning, and is widely used in various intelligent systems. The core of the inference model is to predict or judge unknown data based on pre-trained models by analyzing input data. The inference model needs to process a large number of computing tasks. How to reasonably allocate computing resources for tasks is the key to improving inference efficiency.

[0003] The foregoing discussion of the background art is intended only to facilitate an understanding of the present invention. This discussion does not acknowledge or admit that any of the material referred to is part of the common general knowledge.

[0004] Many resource allocation systems have been developed. After a lot of searching and reference, it is found that the existing resource allocation systems are such as the system disclosed in the publication number CN116820784B. These system methods generally include: obtaining reasoning tasks; inputting reasoning tasks into the trained task performance prediction model to obtain prediction results; the prediction results include execution time, global memory bandwidth and number of instructions per cycle; allocating resources according to reasoning tasks, execution time, global memory bandwidth and number of instructions per cycle to obtain the optimal resource allocation plan. However, this system only allocates resources based on the individual characteristics of reasoning tasks, and does not consider the relationship between tasks in the entire reasoning model. There is still room for improvement in reasoning efficiency. Summary of the invention

[0005] The purpose of the present invention is to propose a computing resource allocation system for improving model reasoning efficiency in response to the existing deficiencies.

[0006] The present invention adopts the following technical solution:

[0007] A computing resource allocation system for improving model reasoning efficiency, comprising a task rating module, a reasoning management module, a reasoning monitoring module and a resource allocation module;

[0008] The task rating module is used to rate the priority of tasks in the model, the reasoning management module is used to control and manage the execution of tasks, the reasoning monitoring module is used to monitor the execution process of tasks, and the resource allocation module is used to allocate computing cores and video memory to tasks;

[0009] The task rating module includes a task parsing unit, a task statistics unit and a rating calculation unit. The task parsing unit is used to parse multiple tasks from the model and form a task tree. The task statistics unit is used to count the parameters of each task in the task tree. The rating calculation unit is used to calculate the rating index of each task.

[0010] The inference management module includes a task management unit, a data management unit and a task running unit. The task management unit is used to receive and release tasks, the data management unit is used to communicate data with the model, and the task running unit uses computing resources to execute tasks;

[0011] The inference monitoring module includes a progress monitoring unit, a video memory monitoring unit and a monitoring feedback unit. The progress monitoring unit is used to monitor the execution progress of each task, the video memory monitoring unit is used to monitor the video memory occupancy of each task, and the monitoring feedback unit is used to feed back the monitoring data to the resource allocation module;

[0012] The resource allocation module includes a core allocation unit, a video memory allocation unit and a task pairing unit, wherein the core allocation unit is used to allocate computing cores to running tasks, the video memory allocation unit is used to allocate video memory to running tasks, and the task pairing unit is used to provide a pairing relationship between a task identifier and a running task;

[0013] Furthermore, the rating calculation unit includes a parameter receiving processor, a parameter calculation processor and a rating output processor, wherein the parameter receiving processor is used to receive a task identifier and corresponding statistical parameters, the parameter calculation processor is used to process the statistical parameters to obtain a rating index, and the rating output processor is used to package the task identifier and the rating index and output them to the resource allocation module;

[0014] The parameter calculation processor calculates the task rating index P according to the following formula:

[0015] ;

[0016] Among them, Lv is the level in the task statistical parameters, and n is the number of subtask individuals in the task statistical parameters;

[0017] Further, the core allocation unit includes a core information manager, an allocation calculation processor and an allocation execution processor, wherein the core information manager is used to manage the address information of the computing core, the allocation calculation processor is used to calculate the number of computing cores allocated to each task, and the allocation execution processor is used to allocate the corresponding number of computing cores to the corresponding task;

[0018] Furthermore, the allocation calculation processor calculates the allocation index Q of the target task according to the following formula:

[0019] ;

[0020] Where k is the scaling factor, V is the total computational effort of the target task, and s is the progress value of the target task;

[0021] The allocation calculation processor calculates the number of computing cores N(i) allocated to the i-th target task according to the following formula:

[0022] ;

[0023] Where m is the number of target tasks, M is the total number of computing cores, and Q(i) represents the allocation index of the i-th target task;

[0024] Furthermore, the video memory allocation unit includes a video memory information manager, a scheduling calculation processor and a scheduling execution processor. The video memory information manager is used to manage the address information of the video memory, the scheduling calculation processor is used to adjust and analyze the size of the video memory allocated to each task, and the scheduling execution processor is used to schedule the video memory between different tasks.

[0025] The beneficial effects achieved by the present invention are:

[0026] This system establishes a task tree for tasks in the reasoning model and rates tasks based on the relationships in the task tree. It uses the rating information to control the resources allocated during task execution, which enables key tasks to be completed faster, thereby improving the reasoning efficiency of the entire reasoning model. At the same time, real-time allocation of video memory allocated to tasks can effectively adapt to changes in the demand for video memory during the processing stage of tasks, thereby improving the average processing speed of tasks.

[0027] To further understand the features and technical contents of the present invention, please refer to the following detailed description and drawings of the present invention. However, the drawings provided are only for reference and description and are not intended to limit the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 It is a schematic diagram of the overall structural framework of the present invention;

[0029] Figure 2 This is a schematic diagram of the task rating module of the present invention;

[0030] Figure 3 This is a schematic diagram of the reasoning management module of the present invention;

[0031] Figure 4 This is a schematic diagram of the reasoning monitoring module of the present invention;

[0032] Figure 5This is a schematic diagram of the resource allocation module of the present invention. DETAILED DESCRIPTION

[0033] The following is an explanation of the embodiments of the present invention through specific embodiments. Those skilled in the art can understand the advantages and effects of the present invention from the contents disclosed in this specification. The present invention can be implemented or applied through other different specific embodiments, and the details in this specification can also be modified and changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. In addition, the drawings of the present invention are only simple schematic illustrations and are not depicted according to actual sizes. It is stated in advance. The following embodiments will further explain the relevant technical contents of the present invention in detail, but the disclosed contents are not intended to limit the scope of protection of the present invention.

[0034] Embodiment 1: This embodiment provides a computing resource allocation system for improving model reasoning efficiency, combined with Figure 1 , including task rating module, reasoning management module, reasoning monitoring module and resource allocation module;

[0035] The task rating module is used to rate the priority of tasks in the model, the reasoning management module is used to control and manage the execution of tasks, the reasoning monitoring module is used to monitor the execution process of tasks, and the resource allocation module is used to allocate computing cores and video memory to tasks;

[0036] The task rating module includes a task parsing unit, a task statistics unit and a rating calculation unit. The task parsing unit is used to parse multiple tasks from the model and form a task tree. The task statistics unit is used to count the parameters of each task in the task tree. The rating calculation unit is used to calculate the rating index of each task.

[0037] The inference management module includes a task management unit, a data management unit and a task running unit. The task management unit is used to receive and release tasks, the data management unit is used to communicate data with the model, and the task running unit uses computing resources to execute tasks;

[0038] The inference monitoring module includes a progress monitoring unit, a video memory monitoring unit and a monitoring feedback unit. The progress monitoring unit is used to monitor the execution progress of each task, the video memory monitoring unit is used to monitor the video memory occupancy of each task, and the monitoring feedback unit is used to feed back the monitoring data to the resource allocation module;

[0039] The resource allocation module includes a core allocation unit, a video memory allocation unit and a task pairing unit, wherein the core allocation unit is used to allocate computing cores to running tasks, the video memory allocation unit is used to allocate video memory to running tasks, and the task pairing unit is used to provide a pairing relationship between a task identifier and a running task;

[0040] The rating calculation unit includes a parameter receiving processor, a parameter calculation processor and a rating output processor, wherein the parameter receiving processor is used to receive a task identifier and corresponding statistical parameters, the parameter calculation processor is used to process the statistical parameters to obtain a rating index, and the rating output processor is used to package the task identifier and the rating index and output them to the resource allocation module;

[0041] The parameter calculation processor calculates the task rating index P according to the following formula:

[0042] ;

[0043] Among them, Lv is the level in the task statistical parameters, and n is the number of subtask individuals in the task statistical parameters;

[0044] The core allocation unit includes a core information manager, an allocation calculation processor and an allocation execution processor, wherein the core information manager is used to manage address information of the computing cores, the allocation calculation processor is used to calculate the number of computing cores allocated to each task, and the allocation execution processor is used to allocate a corresponding number of computing cores to the corresponding tasks;

[0045] The allocation calculation processor calculates the allocation index Q of the target task according to the following formula:

[0046] ;

[0047] Where k is the scaling factor, V is the total computational effort of the target task, and s is the progress value of the target task;

[0048] The allocation calculation processor calculates the number of computing cores N(i) allocated to the i-th target task according to the following formula:

[0049] ;

[0050] Where m is the number of target tasks, M is the total number of computing cores, and Q(i) represents the allocation index of the i-th target task;

[0051] The video memory allocation unit includes a video memory information manager, a deployment calculation processor and a deployment execution processor. The video memory information manager is used to manage the address information of the video memory, the deployment calculation processor is used to adjust and analyze the size of the video memory allocated to each task, and the deployment execution processor is used to schedule the video memory between different tasks.

[0052] Embodiment 2: This embodiment includes all the contents of Embodiment 1, and provides a computing resource allocation system for improving model reasoning efficiency, including a task rating module, a reasoning management module, a reasoning monitoring module, and a resource allocation module;

[0053] The task rating module is used to rate the priority of tasks in the model, the reasoning management module is used to control and manage the execution of tasks, the reasoning monitoring module is used to monitor the execution process of tasks, and the resource allocation module is used to allocate computing cores and video memory to tasks;

[0054] Combination Figure 2 The task rating module includes a task parsing unit, a task statistics unit and a rating calculation unit. The task parsing unit is used to parse multiple tasks from the model and form a task tree. The task statistics unit is used to count the parameters of each task in the task tree. The rating calculation unit is used to calculate the rating index of each task.

[0055] Combination Figure 3 The reasoning management module includes a task management unit, a data management unit and a task running unit. The task management unit is used to receive and release tasks, the data management unit is used to communicate data with the model, and the task running unit uses computing resources to execute tasks;

[0056] Combination Figure 4 The inference monitoring module includes a progress monitoring unit, a video memory monitoring unit and a monitoring feedback unit. The progress monitoring unit is used to monitor the execution progress of each task, the video memory monitoring unit is used to monitor the video memory occupancy of each task, and the monitoring feedback unit is used to feed back the monitoring data to the resource allocation module;

[0057] Combination Figure 5 The resource allocation module includes a core allocation unit, a video memory allocation unit and a task pairing unit, wherein the core allocation unit is used to allocate computing cores to running tasks, the video memory allocation unit is used to allocate video memory to running tasks, and the task pairing unit is used to provide a pairing relationship between a task identifier and a running task;

[0058] The task parsing unit includes a task identification processor and a one-way connection processor, wherein the task identification processor is used to add an identification to each task and record it, and the one-way connection processor is used to connect the identifications one-way based on the dependency relationship of the tasks;

[0059] The tasks and their dependencies are directly given by the model;

[0060] The task statistics unit includes a hierarchical statistics processor and an individual statistics processor, wherein the hierarchical statistics processor is used to count the level of each task in the task tree, and the individual statistics processor is used to count the number of individual subtasks of each task in the task tree;

[0061] In the task tree, if task A can retrieve task B through a one-way connection relationship, then task B is called a subtask of task A. A task without subtasks is called a root task. The minimum number of one-way connections between a task and the root task is the level of the task. The level of the root task is 0, and the level of the task directly connected to the root task is 1, and so on.

[0062] The rating calculation unit includes a parameter receiving processor, a parameter calculation processor and a rating output processor, wherein the parameter receiving processor is used to receive a task identifier and corresponding statistical parameters, the parameter calculation processor is used to process the statistical parameters to obtain a rating index, and the rating output processor is used to package the task identifier and the rating index and output them to the resource allocation module;

[0063] The parameter calculation processor calculates the task rating index P according to the following formula:

[0064] ;

[0065] Among them, Lv is the level in the task statistical parameters, and n is the number of subtask individuals in the task statistical parameters;

[0066] The task management unit includes a task receiving processor, a queue management processor and a task releasing processor. The task receiving processor is used to receive executable tasks obtained from the model. The queue management processor places the executable tasks in a queue according to the receiving order and sends the executable tasks to the task running unit in sequence. The task releasing processor is used to release the related stored data after the executable tasks are processed.

[0067] The tasks in the task rating module are unexecutable tasks, which become executable tasks after adding necessary data;

[0068] The data management unit includes a task data register and a result feedback processor, wherein the task data register is used to store necessary data in the executable task, and the result feedback processor is used to feed back the processing result of the executable task to the model;

[0069] The task running unit includes a resource connection processor and an operation control processor, the resource connection processor is used to connect the allocated computing core and video memory, and the operation control processor runs the task based on the process information in the executable task;

[0070] The progress monitoring unit includes a monitoring pairing processor and a progress calculation processor, wherein the monitoring pairing processor is used to perform pairing processing on each task running in the task running unit, and the progress calculation processor calculates the progress value of the task based on the amount of calculation completed in the task and the total amount of calculation;

[0071] The video memory monitoring unit includes a real-time monitoring processor and a feature calculation processor, wherein the real-time monitoring processor is used to monitor the size of the video memory occupied by each task in real time, and the feature calculation processor is used to calculate feature data of the video memory occupied by each task;

[0072] The characteristic data includes the mean and residual of the occupancy, and the residual is the difference between the maximum value and the mean;

[0073] The monitoring feedback unit includes a progress feedback processor and a video memory feedback processor, wherein the progress feedback processor is used to feed back progress value information to the core allocation unit, and the video memory feedback processor is used to feed back feature data of the video memory to the video memory allocation unit;

[0074] The core allocation unit includes a core information manager, an allocation calculation processor and an allocation execution processor, wherein the core information manager is used to manage address information of the computing cores, the allocation calculation processor is used to calculate the number of computing cores allocated to each task, and the allocation execution processor is used to allocate a corresponding number of computing cores to the corresponding tasks;

[0075] The allocation calculation processor calculates the allocation index Q of the target task according to the following formula:

[0076] ;

[0077] Where k is the scaling factor, V is the total computational effort of the target task, and s is the progress value of the target task;

[0078] The allocation calculation processor calculates the number of computing cores N(i) allocated to the i-th target task according to the following formula:

[0079] ;

[0080] Where m is the number of target tasks, M is the total number of computing cores, and Q(i) represents the allocation index of the i-th target task;

[0081] The target task is the task currently being run in the running task unit;

[0082] When a target task is completed and a new target task is added, the calculation processor is assigned to perform a calculation according to the above content, and the execution processor is assigned to be reallocated based on the calculation result;

[0083] The video memory allocation unit includes a video memory information manager, a deployment calculation processor and a deployment execution processor, wherein the video memory information manager is used to manage the address information of the video memory, the deployment calculation processor is used to adjust and analyze the size of the video memory allocated to each task, and the deployment execution processor is used to schedule the video memory between different tasks;

[0084] The allocation calculation processor calculates the video memory usage index U according to the following formula:

[0085] ;

[0086] in, is the mean value in the feature data, is the residual in the characteristic data;

[0087] The allocation calculation processor allocates part of the video memory of the task with a video memory usage index of 0 to the task with a video memory usage index of 2. The allocation calculation processor calculates the allocated video memory size X according to the following formula:

[0088] ;

[0089] Among them, Y is the size of video memory that has been allocated to the task;

[0090] The allocation execution processor allocates the video memory to be allocated to the tasks with a video memory usage index of 2 on average;

[0091] The i and j appearing in the above text are both ordinal numbers used to indicate serial numbers.

[0092] The contents disclosed above are only preferred feasible embodiments of the present invention, and do not limit the protection scope of the present invention. Therefore, all equivalent technical changes made using the contents of the present invention specification and drawings are included in the protection scope of the present invention. In addition, the elements therein can be updated as technology develops.

Claims

1. A computing resource allocation system for improving model reasoning efficiency, characterized in that: It includes task rating module, reasoning management module, reasoning monitoring module and resource allocation module; The task rating module is used to rate the priority of tasks in the model, the reasoning management module is used to control and manage the execution of tasks, the reasoning monitoring module is used to monitor the execution process of tasks, and the resource allocation module is used to allocate computing cores and video memory to tasks; The task rating module includes a task parsing unit, a task statistics unit and a rating calculation unit. The task parsing unit is used to parse multiple tasks from the model and form a task tree. The task statistics unit is used to count the parameters of each task in the task tree. The rating calculation unit is used to calculate the rating index of each task. The inference management module includes a task management unit, a data management unit and a task running unit. The task management unit is used to receive and release tasks, the data management unit is used to communicate data with the model, and the task running unit uses computing resources to execute tasks; The inference monitoring module includes a progress monitoring unit, a video memory monitoring unit and a monitoring feedback unit. The progress monitoring unit is used to monitor the execution progress of each task, the video memory monitoring unit is used to monitor the video memory occupancy of each task, and the monitoring feedback unit is used to feed back the monitoring data to the resource allocation module; The resource allocation module includes a core allocation unit, a video memory allocation unit and a task pairing unit, wherein the core allocation unit is used to allocate computing cores to running tasks, the video memory allocation unit is used to allocate video memory to running tasks, and the task pairing unit is used to provide a pairing relationship between a task identifier and a running task; The rating calculation unit includes a parameter receiving processor, a parameter calculation processor and a rating output processor, wherein the parameter receiving processor is used to receive a task identifier and corresponding statistical parameters, the parameter calculation processor is used to process the statistical parameters to obtain a rating index, and the rating output processor is used to package the task identifier and the rating index and output them to the resource allocation module; The parameter calculation processor calculates the task rating index P according to the following formula: ； Among them, Lv is the level in the task statistical parameters, and n is the number of subtask individuals in the task statistical parameters; The core allocation unit includes a core information manager, an allocation calculation processor and an allocation execution processor. The core information manager is used to manage the address information of the computing core, the allocation calculation processor is used to calculate the number of computing cores allocated to each task, and the allocation execution processor is used to allocate the corresponding number of computing cores to the corresponding tasks.

2. A computing resource allocation system for improving model reasoning efficiency as claimed in claim 1, characterized in that: The allocation calculation processor calculates the allocation index Q of the target task according to the following formula: ； Where k is the scaling factor, V is the total computational effort of the target task, and s is the progress value of the target task; The allocation calculation processor calculates the number of computing cores N(i) allocated to the i-th target task according to the following formula: ； Where m is the number of target tasks, M is the total number of computing cores, and Q(i) represents the allocation index of the i-th target task.

3. A computing resource allocation system for improving model reasoning efficiency as claimed in claim 2, characterized in that: The video memory allocation unit includes a video memory information manager, a deployment calculation processor and a deployment execution processor. The video memory information manager is used to manage the address information of the video memory, the deployment calculation processor is used to adjust and analyze the size of the video memory allocated to each task, and the deployment execution processor is used to schedule the video memory between different tasks.

Images