Model arrangement method and device, equipment, product and storage medium
By obtaining the time and space occupancy table of the cross-array unit in the multi-core memory and computing integrated accelerator, dynamically optimized task allocation, solving the problem of in-core storage space limitations and array characteristics not being utilized, improving task allocation efficiency and simplifying the process.
Patent Information
- Application Number
- CN202411038024.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-30
- Publication Date
- 2024-11-08
AI Technical Summary
In the multi-core integrated accelerator scenario, the existing technology fails to effectively consider the storage space limitations in the core and the integrated array characteristics of the memory, resulting in low task allocation efficiency and independent sorting and allocation processes, increasing the implementation complexity.
By obtaining the time and space occupancy table of the cross-array unit of each core, based on the data dependencies and resource requirements of the task, dynamically determine the allocation time and space of the task, optimize the task allocation, and make full use of the parallel optimization space within the kernel.
Under the condition that the array storage space limitations are met, the overall completion time efficiency is improved, the task allocation process is simplified, and the complexity is reduced.
Smart Images

Figure CN118916138A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of computing power technology, and in particular to a model compilation method, apparatus, model compilation equipment, product and storage medium. Background Art
[0002] In the current multi-core processor scenario, when allocating tasks, the processing time of each task is often used as a constraint to perform heuristic scheduling and allocation, and the overall completion time is minimized by minimizing the task processing time and the communication time between tasks and increasing the parallelism between cores. However, these solutions still have the following problems when used in multi-core storage and computing accelerator scenarios:
[0003] 1. Using the processing time of each task and the data dependencies between tasks as constraints, without considering the in-core storage space limitation;
[0004] 2. The goal of minimizing the overall completion time is achieved by minimizing the task processing time and the communication time between tasks and increasing the degree of parallelism between cores, without considering the intra-core parallel optimization space brought by the characteristics of the storage-computing integrated array;
[0005] 3. The scheduling process of sorting-allocation-optimization is often adopted. The sorting and allocation processes are independent. The influence of the allocation process on the sorting parameters can only be introduced through subsequent optimization or overall iteration, which increases the implementation complexity. Summary of the invention
[0006] To solve related technical problems, the embodiments of the present application provide a model compilation method, device, equipment, product and storage medium.
[0007] The technical solution of the embodiment of the present application is implemented as follows: The embodiment of the present application provides a model arrangement method, which is applied to a multi-core storage and computing integrated accelerator, including:
[0008] Obtaining a time occupancy table and a space occupancy table of each cross array unit in each core; the time occupancy table is used to record the occupied time period of each cross array unit; the space occupancy table is used to record the occupied space resource position of each cross array unit;
[0009] Determining tasks to be assigned based on each task in the model;
[0010] Determine the actual completion time of allocating the task to be assigned to each core according to the arrival time of the latest input data of the task to be assigned among all the data input to the task to be assigned, the space resource requirement of the task to be assigned, the earliest start time of the successor task of the task to be assigned, and the time occupancy table and the space occupancy table;
[0011] Determine a space resource reservation table based on the number of remaining space resources in each core and the space resource requirements of unassigned tasks;
[0012] The task to be assigned is assigned and deployed to at least one array unit on at least one core based on the actual completion time of the task to be assigned on each core, the number of remaining space resources in each core and the space resource reservation table.
[0013] In the above scheme, the method further comprises:
[0014] Recording the resource allocation of the task to be assigned;
[0015] The time occupancy table and the space occupancy table are updated based on the resource allocation situation.
[0016] In the above solution, the step of determining the tasks to be assigned based on the tasks in the model includes:
[0017] Determining the degree of freedom parameters, processing time, and inter-core communication latency of each task in the model;
[0018] Determine a list of assigned tasks based on data dependencies among the tasks in the model;
[0019] Determine the priority parameter of each task in the to-be-assigned task list in the assigned task list according to the degree of freedom parameter, the processing time and the delay time;
[0020] Prioritize each task in the task list to be assigned using the priority parameter to obtain the task with the highest priority in the task list to be assigned;
[0021] The task with the highest priority in the list of tasks to be assigned is used as the task to be assigned.
[0022] In the above solution, the step of determining the assigned task list based on the data dependency relationship of each task in the model includes:
[0023] Determine a first target task in the model; the first target task is any task among the tasks;
[0024] Determine whether the first target task has been assigned;
[0025] If the first target task has been assigned, adding the first target task to the assigned list;
[0026] If the first target task has not been assigned, determining whether the first target task has a predecessor task or whether the predecessor task has been assigned;
[0027] If the first target task has no predecessor task or the predecessor tasks have been assigned, adding the first target task to a to-be-assigned list;
[0028] If the first target task has a predecessor task and not all predecessor tasks have been assigned, adding the first target task to an unassigned list;
[0029] The allocated list, the to-be-allocated list and the unallocated list are used as the allocated task list.
[0030] In the above scheme, the method further comprises:
[0031] In response to completion of the assignment of the to-be-assigned task, the assigned task list is updated.
[0032] In the above scheme, the method further comprises:
[0033] Determine whether the list of tasks to be assigned in the updated assigned task list is empty;
[0034] When the list of tasks to be assigned is not empty, re-determine the degree of freedom parameters of each task in the model;
[0035] When the to-be-allocated task list is empty, each task in the allocated task list in the updated allocated task list is deployed to at least one array unit on at least one core according to the resource allocation situation.
[0036] In the above solution, determining the degree of freedom parameters of each task in the model includes:
[0037] Determine the earliest start time and the latest finish time of each task in the model;
[0038] The degree of freedom parameter is determined based on the earliest start time and the latest finish time.
[0039] In the above solution, determining the earliest start time and the latest completion time of each task in the model includes:
[0040] Obtaining the processing time and data dependency of each task in the model on the cross array unit;
[0041] Determine an initial earliest start time and an initial latest finish time of each task in the model based on the processing time and the data dependency;
[0042] In the process of allocating each task in the model, determining the actual completion time of the current task according to the resource allocation of the current task to be allocated;
[0043] The earliest start time and the latest finish time of the current task to be assigned are updated based on the actual finish time of the current task.
[0044] In the above scheme, the method further comprises:
[0045] In the case where the updated latest completion time is greater than the earliest start time of the successor task of the current task to be assigned, the earliest start time and the latest completion time of each task in the to-be-assigned and unassigned task lists in the assigned task list are updated based on the updated earliest start time and the latest completion time of the current task to be assigned;
[0046] The degree of freedom parameter of each task in the to-be-assigned and unassigned task lists in the assigned task list is updated based on the earliest start time and the latest finish time of each task in the to-be-assigned and unassigned task lists in the updated assigned task list.
[0047] In the above scheme, the unallocated tasks are located in the list of tasks to be allocated and unallocated; and the determining of the space resource reservation table based on the number of remaining space resources in each core and the space resource requirements of the unallocated tasks includes:
[0048] Determine the space resource reservation threshold parameter based on the number of remaining space resources in each core;
[0049] The space resource reservation table is determined according to the space resource demand of the unassigned task and the space resource reservation threshold parameter.
[0050] In the above scheme, the determining the space resource reservation table according to the space resource requirements of each task in the to-be-allocated and unallocated task lists and the space resource reservation threshold parameter comprises:
[0051] Determine a second target task in the list of to-be-allocated and unallocated tasks, each of which has a space resource demand exceeding the space resource reservation threshold parameter;
[0052] Counting the number of tasks in the list of to-be-allocated and unallocated tasks whose space resource requirements are greater than or equal to the space resource requirements of the second target task;
[0053] The space resource reservation table is determined based on the space resource demand of the second target task and the number of tasks.
[0054] In the above scheme, the method further comprises:
[0055] Determine the actual completion time of each array unit based on the actual start time of the task to be assigned on each array unit of each core;
[0056] Sort all array units on each core according to the actual completion time and number of remaining space resources of each array unit to obtain a first sorting result;
[0057] Determine all array units that need to be added to the allocation array list to meet the space resource requirements of the task to be allocated based on the first sorting result;
[0058] Obtain the target array unit with the latest actual completion time among all the array units;
[0059] The actual completion time of the target array unit is used as the actual completion time of the task to be assigned on each core.
[0060] In the above scheme, the step of allocating and deploying the task to be allocated to at least one array unit on at least one core based on the actual completion time of the task to be allocated on each core, the number of remaining space resources in each core, and the space resource reservation table includes:
[0061] Traversing each core of the multi-core storage-computing integrated accelerator to determine the actual completion time of the task to be assigned on each core;
[0062] Sorting all the cores in the multi-core storage-computing integrated accelerator based on the actual completion time on each core, the number of remaining space resources in each core, and the space resource reservation table to obtain a second sorting result;
[0063] Based on the second sorting result, determining whether the remaining space resources in the first-ranked core among the multi-cores meet the space resource requirements of the task to be assigned;
[0064] When the remaining space resources in the first-ranked core among the multiple cores meet the space resource requirements of the task to be assigned, assigning and deploying the task to be assigned to at least one array unit on one core;
[0065] When the remaining space resources in the first-ranked core among the multiple cores do not meet the space resource requirements of the task to be assigned, the task to be assigned is assigned and deployed to multiple array units on multiple cores.
[0066] The embodiment of the present application further provides a model arrangement device, which is arranged on a multi-core storage and computing integrated accelerator, and includes:
[0067] An acquisition unit, used to acquire a time occupancy table and a space occupancy table of each cross array unit in each core; the time occupancy table is used to record the occupied time period of each cross array unit; the space occupancy table is used to record the occupied space resource position of each cross array unit;
[0068] A determination unit, configured to determine the to-be-assigned task based on each task in the model; and determine the actual completion time of allocating the to-be-assigned task to each core according to the arrival time of the latest input data of the to-be-assigned task among all the data input to the to-be-assigned task, the space resource requirement of the to-be-assigned task, the earliest start time of the successor task of the to-be-assigned task, and the time occupancy table and the space occupancy table; and determine the space resource reservation table based on the number of remaining space resources in each core and the space resource requirement of unassigned tasks;
[0069] An allocation unit is used to allocate and deploy the task to be allocated to at least one array unit on at least one core based on the actual completion time of the task to be allocated on each core, the number of remaining space resources in each core and the space resource reservation table.
[0070] The present application also provides a model arrangement device, including:
[0071] A memory for storing executable instructions;
[0072] The processor is used to implement any step of the above-mentioned method when executing the executable instructions stored in the memory.
[0073] An embodiment of the present application also provides a computer program product, which implements any step of the above-mentioned method when executed by a processor.
[0074] The embodiment of the present application also provides a computer-readable storage medium storing executable instructions for implementing any step of the above-described method when executed by a processor.
[0075] The model arrangement method, apparatus, model arrangement equipment, product and storage medium provided by the embodiments of the present application, wherein the method is applied to a multi-core storage and computing integrated accelerator, comprising: obtaining a time occupancy table and a space occupancy table for each cross array unit in each core; the time occupancy table is used to record the occupied time period of each cross array unit; the space occupancy table is used to record the occupied space resource position of each cross array unit; determining the task to be assigned based on each task in the model; determining the actual completion time of assigning the task to be assigned to each core according to the arrival time of the latest input data of the task to be assigned among all the data input to the task to be assigned, the space resource requirement of the task to be assigned, the earliest start time of the successor task of the task to be assigned, and the time occupancy table and the space occupancy table; determining a space resource reservation table based on the number of remaining space resources in each core and the space resource requirement of unassigned tasks; assigning and deploying the task to be assigned to at least On at least one array unit on a core, the solution of an embodiment of the present application obtains the time occupancy table and space occupancy table of each cross array unit in each core; determines the task to be assigned based on each task in the model; determines the actual completion time of assigning the task to be assigned to each core according to the arrival time of the latest input data of the task to be assigned among all the data input to the task to be assigned, the space resource requirement of the task to be assigned, the earliest start time of the successor task of the task to be assigned, and the time occupancy table and the space occupancy table; determines the space resource reservation table based on the number of remaining space resources in each core and the space resource requirement of unassigned tasks; assigns and deploys the task to be assigned to at least one array unit on at least one core based on the actual completion time of the task to be assigned on each core, the number of remaining space resources in each core, and the space resource reservation table, that is, according to the hardware characteristics of the multi-core storage and computing integrated accelerator, the dual constraints of time and space resources are considered when performing model task assignment, so as to fully utilize the intra-core parallel optimization space brought by the storage and computing integrated array characteristics under the condition of meeting the array storage space limitation, so as to minimize the overall completion time and make the implementation more simple. BRIEF DESCRIPTION OF THE DRAWINGS
[0076] Figure 1 A schematic diagram of a multi-core architecture in a storage-computing integrated accelerator;
[0077] Figure 2 A schematic diagram of a model arrangement method flow is provided for an embodiment of the present application;
[0078] Figure 3 A schematic diagram of the application scenario for the model orchestration process;
[0079] Figure 4A schematic diagram of a model arrangement device according to an embodiment of the present application;
[0080] Figure 5 A schematic diagram of a hardware entity structure of a model arrangement device in an embodiment of the present application. DETAILED DESCRIPTION
[0081] The present application is further described in detail below in conjunction with the accompanying drawings and embodiments.
[0082] As a new computing technology, integrated storage and computing can perform calculations directly in the memory, breaking through the storage wall and power consumption wall problems caused by the separation of storage and computing in the von Neumann architecture from an architectural perspective, greatly improving computing power and energy efficiency. It is particularly suitable for scenarios such as neural networks that require large amounts of memory access and computing, and is expected to become the mainstream computing architecture in the era of artificial intelligence (AI).
[0083] There are currently many different development paths for integrated storage and computing chips. Different types of integrated storage and computing chips have different device structures, but their storage and computing principles are basically similar. They all use pressure to maintain different states of charge in the integrated storage and computing device to achieve the representation of different resistance states, that is, the storage of different weight values, and then convert the inference input into voltage and obtain current to achieve multiplication and addition operations. Preparing storage and computing devices in a cross array can achieve highly parallel matrix multiplication and addition operations in the memory, which is particularly suitable for inference acceleration of deep neural network models.
[0084] Considering performance, flexibility and cost, typical storage-computing accelerators can be combined with Figure 1 To understand, Figure 1 This is a schematic diagram of the multi-core architecture in the storage-computing integrated accelerator. Figure 1 As shown, each core includes multiple storage and computing integrated crossbar array units (Crossbar, XB) to complete matrix multiplication and addition operations and a special function unit (Special Function Unit, SFU) to complete non-matrix multiplication and addition operations. Since each XB can only perform one matrix multiplication and addition operation at the same time, multiple matrix operation tasks deployed on the same XB cannot be executed in parallel, while multiple matrix operations deployed on different XBs can be executed in parallel as long as there is no data dependency between each other. The XBs in the same core exchange data through the local cache (Local Memory) in the core, and different cores exchange data through the common shared cache (Shared Memory).
[0085] As the scale of neural network models grows, before the model is deployed, it is usually necessary to orchestrate complex neural network models and reasonably distribute the computing tasks in the model to multiple XBs or even multiple cores. Since the storage-computing accelerator is data-flow driven, a computing process can be transformed into a process of computing in which data flows between multiple XBs after orchestration and corresponding deployment. The efficiency of data flow determines the computing efficiency of the storage-computing accelerator. Therefore, how to achieve efficient orchestration of neural network models on the storage-computing accelerator has become the key to unleashing the performance and energy efficiency of the neural network accelerator.
[0086] At present, the task allocation scheme of multi-core processors is mainly based on the following two allocation algorithms:
[0087] (1) Random search algorithm.
[0088] Typical random search algorithms include genetic algorithms, simulated annealing algorithms, ant colony algorithms, etc. The core of these algorithms is to randomly select samples uniformly in a certain space, compare the size of random function values, and then gradually narrow the search area according to the function value. Since the solution space is relatively large, the search range is relatively large and the search itself is also very complex, its time complexity is very high.
[0089] (2) Heuristic algorithm.
[0090] Heuristic algorithms are the most commonly used task allocation algorithms due to their simple implementation and low time complexity. They generally combine indicators such as earliest completion time, minimum completion time, and load balancing, and use the idea of greedy algorithms to find suboptimal solutions to task allocation.
[0091] In the current multi-core processor scenario, when allocating tasks, the processing time of each task is often used as a constraint to perform heuristic scheduling and allocation, and the overall completion time is minimized by minimizing the task processing time and the communication time between tasks and increasing the parallelism between cores. However, these solutions still have the following problems when used in multi-core storage and computing accelerator scenarios:
[0092] 1. Using the processing time of each task and the data dependencies between tasks as constraints, without considering the in-core storage space limitation;
[0093] 2. The goal of minimizing the overall completion time is achieved by minimizing the task processing time and the communication time between tasks and increasing the degree of parallelism between cores, without considering the intra-core parallel optimization space brought by the characteristics of the storage-computing integrated array;
[0094] 3. The scheduling process of sorting-allocation-optimization is often adopted. The sorting and allocation processes are independent. The influence of the allocation process on the sorting parameters can only be introduced through subsequent optimization or overall iteration, which increases the implementation complexity.
[0095] Based on this, the embodiment of the present application provides a model arrangement method, which is applied to a model arrangement device. The function implemented by the method can be implemented by calling a program code by a processor in the model arrangement device. Of course, the program code can be stored in a computer storage medium. It can be seen that the model arrangement device at least includes a processor and a storage medium. As an example, the model arrangement device can be a mobile phone, a computer, a terminal, an information transceiver device, a tablet device, a personal digital assistant, etc.
[0096] Figure 2 A schematic diagram of a model arrangement method flow is provided for an embodiment of the present application; Figure 2 As shown, the method is applied to a multi-core storage and computing integrated accelerator, including:
[0097] Step 201: Obtain a time occupancy table and a space occupancy table of each cross array unit in each core; the time occupancy table is used to record the occupied time period of each cross array unit; the space occupancy table is used to record the occupied space resource position of each cross array unit;
[0098] Step 202: Determine the tasks to be assigned based on each task in the model;
[0099] Step 203: determining the actual completion time of allocating the task to be allocated to each core according to the arrival time of the latest input data of the task to be allocated among all the data input to the task to be allocated, the space resource requirement of the task to be allocated, the earliest start time of the successor task of the task to be allocated, and the time occupancy table and the space occupancy table;
[0100] Step 204: Determine a space resource reservation table based on the number of remaining space resources in each core and the space resource requirements of unassigned tasks;
[0101] Step 205: Allocate and deploy the task to be allocated to at least one array unit on at least one core based on the actual completion time of the task to be allocated on each core, the number of remaining space resources in each core and the space resource reservation table.
[0102] It should be noted that the multi-core storage-computing integrated accelerator can be determined according to actual conditions and is not limited here. As an example, the multi-core storage-computing integrated accelerator may include a multi-core architecture, each core including multiple storage-computing integrated cross array units to complete matrix multiplication and addition operations and a special function unit to complete non-matrix multiplication and addition operations.
[0103] In step 201, the time occupancy table is used to record the occupied time period of each cross array unit and the space occupancy table is used to record the occupied space resource position of each cross array unit. It can be understood that for each cross array unit, a time occupancy table and a space occupancy table are maintained; the time occupancy table records the occupied time period of the cross array unit, and the time period outside the occupied time period is an idle time period; the space occupancy table records the occupied space resource position in the cross array unit, and the time period outside the occupied space resource position is an available space resource position.
[0104] This embodiment can be understood as the maintenance of the time and space resource occupancy table of the array unit. When performing task allocation, the task needs to be allocated to one or more cross arrays. Once the allocation is completed, the task is exclusive to the array unit in time and space. On the other hand, each array unit is independent and can execute computing tasks in parallel. Therefore, for each cross array unit, a time occupancy table and a space occupancy table are maintained. The time occupancy table records the occupied time periods of the cross array unit, and the time periods outside the occupied time periods are idle time periods; the space occupancy table records the occupied space resource positions in the cross array unit, and the space resource positions outside the occupied space resource positions are available space resource positions.
[0105] In step 202, the specific determination process in determining the tasks to be assigned based on the tasks in the model can be determined according to the actual situation and is not limited here. As an example, determining the tasks to be assigned based on the tasks in the model may include: determining the degree of freedom parameters, processing time, and delay time of inter-core communication of each task in the model; determining the assigned task list based on the data dependency of each task in the model; determining the priority parameters of each task in the list of tasks to be assigned in the assigned task list according to the degree of freedom parameters, the processing time, and the delay time; prioritizing each task in the list of tasks to be assigned using the priority parameters to obtain the task with the highest priority in the list of tasks to be assigned; and using the task with the highest priority in the list of tasks to be assigned as the task to be assigned.
[0106] In step 203, the specific determination process of determining the actual completion time of allocating the task to be assigned to each core is determined based on the arrival time of the latest input data of the task to be assigned among all the data input to the task to be assigned, the space resource requirement of the task to be assigned, the earliest start time of the successor task of the task to be assigned, and the time occupancy table and the space occupancy table. The specific determination process can be determined according to the actual situation and is not limited here. As an example, determining the actual completion time of allocating the task to be assigned to each core is determined based on the arrival time of the latest input data of the task to be assigned among all the data input to the task to be assigned, the space resource requirement of the task to be assigned, the earliest start time of the successor task of the task to be assigned, and the time occupancy table and the space occupancy table. It can be understood as determining the actual completion time of allocating the task to each core based on the space resource requirement of the task to be assigned, the ready time of the input data, the earliest start time of the successor task, and the time occupancy table and the space occupancy table of each array.
[0107] In step 204, the specific determination process in the space resource reservation table based on the number of remaining space resources in each core and the space resource demand of unassigned tasks can be determined according to actual conditions and is not limited here. As an example, the unassigned tasks are located in the list of tasks to be assigned and unassigned; the determination of the space resource reservation table based on the number of remaining space resources in each core and the space resource demand of unassigned tasks may include determining the space resource reservation threshold parameter based on the number of remaining space resources in each core; and determining the space resource reservation table according to the space resource demand of the unassigned tasks and the space resource reservation threshold parameter.
[0108] In practical applications, determining the space resource reservation table based on the number of remaining space resources in each core and the space resource requirements of unassigned tasks can be understood as dynamically maintaining the space resource reservation table based on the number of remaining space resources in each core and the space resource requirements of unassigned tasks.
[0109] In step 205, the specific allocation process of allocating and deploying the task to be allocated to at least one array unit on at least one core based on the actual completion time of the task to be allocated on each core, the number of remaining space resources in each core and the space resource reservation table can be determined according to actual conditions and is not limited here. As an example, allocating and deploying the task to be allocated to at least one array unit on at least one core based on the actual completion time of the task to be allocated on each core, the number of remaining space resources in each core, and the space resource reservation table may include traversing each core of the multi-core storage and computing integrated accelerator to determine the actual completion time of the task to be allocated on each core; sorting all the cores in the multi-core storage and computing integrated accelerator based on the actual completion time on each core, the number of remaining space resources in each core, and the space resource reservation table to obtain a second sorting result; judging whether the remaining space resources in the first-ranked core among the multi-cores meet the space resource requirements of the task to be allocated based on the second sorting result; if the remaining space resources in the first-ranked core among the multi-cores meet the space resource requirements of the task to be allocated, allocating and deploying the task to be allocated to at least one array unit on one core; if the remaining space resources in the first-ranked core among the multi-cores do not meet the space resource requirements of the task to be allocated, allocating and deploying the task to be allocated to multiple array units on multiple cores.
[0110] In practical applications, the allocation and deployment of the tasks to be assigned to at least one array unit on at least one core based on the actual completion time of the tasks to be assigned on each core, the number of remaining space resources in each core and the space resource reservation table can be understood as allocating and deploying the tasks to one or more array units on one or more cores in combination with the actual completion time on each core, the remaining space resources in each core and the space resource reservation requirements of the unassigned tasks.
[0111] This embodiment can be understood as a task allocation process based on the dual constraints of time and space resources of the storage-computing integrated array. When performing task allocation, the storage space limitation in the core is taken into consideration, and the parallel optimization space in the core brought by the characteristics of the storage-computing integrated array is fully utilized, including maintaining a time occupancy table and a space occupancy table for each cross array unit; determining the actual completion time of assigning tasks to each core based on the space resource requirements of the tasks to be assigned, the input data ready time, the earliest start time of the successor tasks, and the time occupancy table and space occupancy table of each array; dynamically maintaining a space resource reservation table based on the number of remaining space resources in each core and the space resource requirements of unassigned tasks; and assigning and deploying tasks to one or more array units on one or more cores in combination with the actual completion time on each core, the remaining space resource situation in each core, and the space resource reservation requirements of unassigned tasks.
[0112] In an embodiment of the present application, a time occupancy table and a space occupancy table of each cross array unit in each core are obtained; tasks to be assigned are determined based on each task in the model; the actual completion time of assigning the tasks to be assigned to each core is determined according to the arrival time of the latest input data of the tasks to be assigned among all the input data of the tasks to be assigned, the space resource requirements of the tasks to be assigned, the earliest start time of the successor tasks of the tasks to be assigned, and the time occupancy table and the space occupancy table; a space resource reservation table is determined based on the number of remaining space resources in each core and the space resource requirements of unassigned tasks; and the tasks to be assigned are assigned and deployed to at least one array unit on at least one core based on the actual completion time of the tasks to be assigned on each core, the number of remaining space resources in each core, and the space resource reservation table. That is, in view of the hardware characteristics of the multi-core storage-computing integrated accelerator, the dual constraints of time and space resources are considered when performing model task assignment, so as to fully utilize the intra-core parallel optimization space brought by the storage-computing integrated array characteristics while satisfying the array storage space limitation, so as to minimize the overall completion time and make the implementation more convenient.
[0113] In one embodiment, the method further comprises:
[0114] Recording the resource allocation of the task to be assigned;
[0115] The time occupancy table and the space occupancy table are updated based on the resource allocation situation.
[0116] In this embodiment, the resource allocation situation can be determined according to actual conditions, and is not limited here. As an example, the resource allocation situation may include the allocated core number and the XB number on the corresponding core.
[0117] The specific updating process of updating the time occupancy table and the space occupancy table based on the resource allocation situation can be determined according to the actual situation and is not limited here. As an example, the specific updating process of updating the time occupancy table and the space occupancy table based on the resource allocation situation can be to update the time occupancy table and the space occupancy table of the corresponding XB according to the resource allocation situation, and update the corresponding remaining space resource number and space resource utilization rate.
[0118] In actual applications, the resource allocation of the task is recorded, including the allocated core number and the XB number on the corresponding core; the time occupancy table and space occupancy table of the corresponding XB are updated according to the resource allocation, and the corresponding remaining space resource number and space resource utilization are updated.
[0119] In one embodiment, determining the tasks to be assigned based on the tasks in the model includes:
[0120] Determining the degree of freedom parameters, processing time, and inter-core communication latency of each task in the model;
[0121] Determine a list of assigned tasks based on data dependencies among the tasks in the model;
[0122] Determine the priority parameter of each task in the to-be-assigned task list in the assigned task list according to the degree of freedom parameter, the processing time and the delay time;
[0123] Prioritize each task in the task list to be assigned using the priority parameter to obtain the task with the highest priority in the task list to be assigned;
[0124] The task with the highest priority in the list of tasks to be assigned is used as the task to be assigned.
[0125] In this embodiment, the specific determination process of determining the degree of freedom parameters of each task in the model can be determined according to actual conditions and is not limited here. As an example, determining the degree of freedom parameters of each task in the model may include determining the earliest start time and the latest completion time of each task in the model; determining the degree of freedom parameters based on the earliest start time and the latest completion time. For ease of understanding, the earliest start time can be recorded as EST (T i ); the latest completion time can be recorded as LFT (T i ); the degree of freedom parameter can be recorded as DOF(T i ).
[0126] The specific determination process in determining the processing time of each task in the model can be determined according to the actual situation and is not limited here. As an example, determining the processing time of each task in the model can be understood as determining the processing time of the task itself on the storage and computing array. For ease of understanding, the processing time can be recorded as PT (T i ), represents the processing time of Ti on the storage array. As an example, PT(T i )=H input ×T MVM ; Among them, T MVM It can be understood as the time of a matrix-vector multiplication operation; H input It can be understood as the number of rows of the input matrix of the storage-computing integrated accelerator.
[0127] In practical applications, for a storage-computing integrated accelerator, assuming that task T i The input matrix dimension is H input ×W input , the weight matrix dimension is H weight ×W weightDue to the parallel computing characteristics of the storage-computation integrated array, each multiplication operation of the input row vector and the weight matrix can be completed in parallel by converting the input row vector into a single input voltage. Since the arrays in each core of the storage-computation integrated accelerator are the same, the time for a matrix-vector multiplication operation is basically the same, which is set as T MVM Therefore, as long as the array space resources are sufficient, T i The processing time on the storage array is: PT(T i )=H input ×T MVM It can be seen that task T i The processing time on the storage array is only related to the number of rows of the input matrix, and has nothing to do with which core the task is assigned to execute.
[0128] The specific determination process of determining the delay time of the inter-core communication of each task in the model can be determined according to the actual situation and is not limited here. As an example, the delay time of the inter-core communication can be simply referred to as the inter-core communication delay. In practical applications, That is T i The earliest start time delay caused by the failure of its predecessor task to be assigned to the same core is the possible delay under the most unfavorable allocation situation. Among them, DCT(T k ,T i ) is task T k and T i The inter-core data transfer time is determined by task T k and T i The amount of data transferred between
[0129] The specific determination process of determining the assigned task list based on the data dependency relationship of each task in the model can be determined according to actual conditions and is not limited here. As an example, the determination of the assigned task list based on the data dependency relationship of each task in the model may include determining the first target task in the model; the first target task is any task among the tasks; judging whether the first target task has been assigned; if the first target task has been assigned, adding the first target task to the assigned list; if the first target task has not been assigned, judging whether the first target task has a predecessor task or whether the predecessor task has been assigned; if the first target task has no predecessor task or all predecessor tasks have been assigned, adding the first target task to the to-be-assigned list; if the first target task has a predecessor task and not all predecessor tasks have been assigned, adding the first target task to the unassigned list; using the assigned list, the to-be-assigned list and the unassigned list as the assigned task list.
[0130] The specific determination process of determining the priority parameters of each task in the to-be-assigned task list in the assigned task list according to the degree of freedom parameter, the processing time and the delay time can be determined according to actual conditions and is not limited here. As an example, the priority parameters of each task in the to-be-assigned task list in the assigned task list determined according to the degree of freedom parameter, the processing time and the delay time can be determined by a preset algorithm according to the degree of freedom parameter, the processing time and the delay time; wherein the preset algorithm can be determined according to actual conditions and is not limited here. As an example, the preset algorithm can be a formula. In actual applications, the priority parameter can be a definition of task T i The priority parameter is priority(T i ).
[0131]
[0132] Among them, DCT (T k ,T i ) is task T k and T i The inter-core data transfer time is determined by task T k and T i The amount of data transferred between
[0133] That is T i The earliest start time delay caused by the failure of its predecessor task to be assigned to the same core is the possible delay under the most unfavorable assignment situation.
[0134] It can be seen that the task priority takes into account both the task freedom and the inter-core communication delay: the greater the task freedom, the lower the priority; the greater the delay caused by the task not being able to be assigned to the same core as its predecessor task due to constraints, the higher the priority.
[0135] Since the task priority is calculated based on the task freedom, each task freedom update will result in an update of the task priority.
[0136] Using the priority parameter to prioritize each task in the list of tasks to be assigned, and obtaining the task with the highest priority in the list of tasks to be assigned can be understood as using the priority parameter to prioritize each task in the list of tasks to be assigned from high to low or from low to high, and obtaining the task with the highest priority in the list of tasks to be assigned.
[0137] Taking the task with the highest priority in the task list to be assigned as the task to be assigned can be understood as calling the task with the highest priority in the task list to be assigned as the task to be assigned. In practical applications, before assigning each task, the tasks in the task list to be assigned are sorted, and the task with the highest priority is selected for assignment. If there are multiple tasks in the task list to be assigned with the same priority, the task node that occupies more space resources is placed in front. If there are multiple tasks in the task list to be assigned with the same priority and the same space resources, the order of these tasks is random.
[0138] In one embodiment, determining the assigned task list based on the data dependency relationship of each task in the model includes:
[0139] Determine a first target task in the model; the first target task is any task among the tasks;
[0140] Determine whether the first target task has been assigned;
[0141] If the first target task has been assigned, adding the first target task to the assigned list;
[0142] If the first target task has not been assigned, determining whether the first target task has a predecessor task or whether the predecessor task has been assigned;
[0143] If the first target task has no predecessor task or the predecessor tasks have been assigned, adding the first target task to a to-be-assigned list;
[0144] If the first target task has a predecessor task and not all predecessor tasks have been assigned, adding the first target task to an unassigned list;
[0145] The allocated list, the to-be-allocated list and the unallocated list are used as the allocated task list.
[0146] In this embodiment, the specific determination process in determining the first target task in the model can be determined according to actual conditions and is not limited here. As an example, determining the first target task in the model can be understood as traversing all computing tasks in the model.
[0147] In actual applications, during the task allocation process, three task lists are maintained, namely, an allocated task list, a to-be-allocated task list, and an unallocated task list.
[0148] Before starting the scheduling, traverse all computing tasks in the model. If a task node has no predecessor task, add the task to the list of tasks to be assigned; otherwise, add the task to the list of unassigned tasks. The assigned task list is initially an empty list.
[0149] After each task assignment is completed, the assigned task will be deleted from the list of tasks to be assigned, added to the list of assigned tasks, and all successor tasks of the task will be traversed. If all predecessor tasks of a successor task are already in the assigned list, the successor task will be deleted from the list of unassigned tasks and added to the list of tasks to be assigned.
[0150] It can be seen that only when all the predecessor tasks of a task have been assigned can it enter the list of tasks to be assigned. For any task in the list of tasks to be assigned, its predecessor task either does not exist or is in the list of assigned tasks, and its successor task either does not exist or is in the list of unscheduled tasks. Therefore, during the entire task assignment process, at any time, there is no data dependency between the tasks in the list of tasks to be assigned, and as long as they are not deployed on the same XB, they can be executed in parallel.
[0151] In one embodiment, the method further comprises:
[0152] In response to completion of the assignment of the to-be-assigned task, the assigned task list is updated.
[0153] In this embodiment, in response to the completion of the assignment of the to-be-assigned task, the specific update process of updating the assigned task list can be determined according to the actual situation, and is not limited here. As an example, in response to the completion of the assignment of the to-be-assigned task, updating the assigned task list can be understood as in response to the completion of the assignment of the to-be-assigned task, re-determining the assigned task list based on the data dependency relationship of each task in the model, and then updating the assigned task list. That is, the first target task in the model can be re-determined; the first target task is any task among the tasks; re-determining whether the first target task has been assigned; if the first target task has been assigned, adding the first target task to the assigned list; if the first target task has not been assigned, re-determining whether the first target task has a predecessor task or whether the predecessor task has been assigned; if the first target task has no predecessor task or the predecessor tasks have been assigned, adding the first target task to the to-be-assigned list; if the first target task has a predecessor task and not all predecessor tasks have been assigned, adding the first target task to the unassigned list; and then updating the assigned task list.
[0154] In actual applications, the assigned, to-be-assigned, and unassigned task lists are updated: the task is deleted from the to-be-assigned task list, added to the assigned task list, and all successor tasks of the task are traversed. If all predecessor tasks of a successor task are already in the assigned list, the successor task is deleted from the unassigned task list and added to the to-be-assigned task list.
[0155] In one embodiment, the method further comprises:
[0156] Determine whether the list of tasks to be assigned in the updated assigned task list is empty;
[0157] When the list of tasks to be assigned is not empty, re-determine the degree of freedom parameters of each task in the model;
[0158] When the to-be-allocated task list is empty, each task in the allocated task list in the updated allocated task list is deployed to at least one array unit on at least one core according to the resource allocation situation.
[0159] In this embodiment, the step of determining whether the list of tasks to be assigned in the updated assigned task list is empty may be referred to as determining whether the list of tasks to be assigned is empty.
[0160] When the list of tasks to be assigned is not empty, redetermining the degree of freedom parameters of each task in the model can be understood as redetermining the earliest start time and the latest completion time of each task in the model when the list of tasks to be assigned is not empty; and redetermining the degree of freedom parameters based on the earliest start time and the latest completion time.
[0161] When the to-be-allocated task list is empty, each task in the allocated task list in the updated allocated task list is deployed to at least one array unit on at least one core according to the resource allocation situation.
[0162] In practical applications, determine whether the list of tasks to be assigned is empty; if it is empty, deploy each task in the assigned task list in the updated assigned task list to at least one array unit on at least one core according to the resource allocation situation; if it is not empty, redetermine the degree of freedom parameters of each task in the model.
[0163] In one embodiment, determining the degree of freedom parameters of each task in the model includes:
[0164] Determine the earliest start time and the latest finish time of each task in the model;
[0165] The degree of freedom parameter is determined based on the earliest start time and the latest finish time.
[0166] In this embodiment, the specific determination process of determining the earliest start time and the latest completion time of each task in the model can be determined according to actual conditions and is not limited here. As an example, determining the earliest start time and the latest completion time of each task in the model may include obtaining the processing time and data dependency of each task in the model on the cross array unit; determining the initial earliest start time and the initial latest completion time of each task in the model based on the processing time and the data dependency; in the process of assigning each task in the model, determining the actual completion time of the current task according to the resource allocation of the current task to be assigned; and updating the earliest start time and the latest completion time of the current task to be assigned based on the actual completion time of the current task.
[0167] The specific determination process of determining the degree of freedom parameter based on the earliest start time and the latest finish time can be determined according to actual conditions and is not limited here. As an example, the determination of the degree of freedom parameter based on the earliest start time and the latest finish time can be determined by a preset algorithm based on the earliest start time and the latest finish time. The earliest start time can be recorded as EST (T i ); the latest completion time can be recorded as LFT (T i ); the degree of freedom parameter can be recorded as DOF(T i ); the preset algorithm can be determined according to actual conditions and is not limited here. As an example, the preset algorithm may include a formula, for example, DOF(T i )=LFT(T i )-EST(T i ).
[0168] In one embodiment, determining the earliest start time and the latest finish time of each task in the model includes:
[0169] Obtaining the processing time and data dependency of each task in the model on the cross array unit;
[0170] Determine an initial earliest start time and an initial latest finish time of each task in the model based on the processing time and the data dependency;
[0171] In the process of allocating each task in the model, determining the actual completion time of the current task according to the resource allocation of the current task to be allocated;
[0172] The earliest start time and the latest finish time of the current task to be assigned are updated based on the actual finish time of the current task.
[0173] In this embodiment, for tasks to be assigned and unassigned:
[0174] T i The earliest start time is T i The maximum value of the earliest completion time of all predecessor tasks, if T i If there is no predecessor task, the value is 0.
[0175] T i The latest completion time is T i The minimum of the latest start time of all subsequent tasks, if T i If there is no successor task, then take T i The earliest completion time.
[0176] LST(T i )=LFT(T i )-PT(T i ), is T i The latest start time is T i The difference between the latest finish time and the processing time.
[0177] EFT(T i )=EST(T i )+PT(T i ), is T i The earliest completion time is T i The sum of the earliest start time and the processing time.
[0178] Therefore, task T i The degree of freedom is that T i The maximum time that can be occupied, in addition to the processing time of the task itself on the storage and computing array, should also include the communication time for receiving data from the predecessor task and the waiting time caused by hardware conflicts.
[0179] In practical applications, for a storage-computing integrated accelerator, assuming that task T i The input matrix dimension is H input ×W input , the weight matrix dimension is H weight ×W weight Due to the parallel computing characteristics of the storage-computation integrated array, each multiplication operation of the input row vector and the weight matrix can be completed in parallel by converting the input row vector into a single input voltage. Since the arrays in each core of the storage-computation integrated accelerator are the same, the time for a matrix-vector multiplication operation is basically the same, which is set as T MVM Therefore, as long as the array space resources are sufficient, Ti The processing time on the storage array is: PT(T i )=H input ×T MVM It can be seen that task T i The processing time on the storage array is only related to the number of rows of the input matrix, and has nothing to do with which core the task is assigned to execute.
[0180] For assigned tasks: LST(T i )=EST(T i )=AST(T i ), T i The latest start time and the earliest start time are equal to the actual start time.
[0181] EFT(T i )=LFT(T i )=AFT(T i )=AST(T i )+PT(T i ), T i The earliest completion time and the latest completion time are equal to the actual completion time, and T i The sum of the actual start time and the processing time.
[0182] In the task T i After the allocation, according to the allocation result, T i The earliest and latest start times of T are updated to the actual start time, and T i The earliest and latest finish times are updated to the actual finish times.
[0183] In one embodiment, the method further comprises:
[0184] In the case where the updated latest completion time is greater than the earliest start time of the successor task of the current task to be assigned, the earliest start time and the latest completion time of each task in the to-be-assigned and unassigned task lists in the assigned task list are updated based on the updated earliest start time and the latest completion time of the current task to be assigned;
[0185] The degree of freedom parameter of each task in the to-be-assigned and unassigned task lists in the assigned task list is updated based on the earliest start time and the latest finish time of each task in the to-be-assigned and unassigned task lists in the updated assigned task list.
[0186] In practical applications, in the process of task allocation, if the constraints lead to T i The actual completion time of a task exceeds the earliest start time of its successor task. Then the degrees of freedom of all currently allocated and unallocated nodes need to be updated.
[0187] In one embodiment, the unallocated tasks are located in a list of tasks to be allocated and unallocated; and determining the space resource reservation table based on the number of remaining space resources in each core and the space resource requirements of the unallocated tasks includes:
[0188] Determine the space resource reservation threshold parameter based on the number of remaining space resources in each core;
[0189] The space resource reservation table is determined according to the space resource demand of the unassigned task and the space resource reservation threshold parameter.
[0190] In this embodiment, the specific determination process of determining the space resource reservation threshold parameter based on the number of remaining space resources in each core can be determined according to actual conditions and is not limited here. As an example, the determination of the space resource reservation threshold parameter based on the number of remaining space resources in each core can be determined by a preset algorithm based on the number of remaining space resources in each core. The number of remaining space resources in each core can be recorded as R_rem m , represents the number of remaining space resources in the current m-th core; the space resource reservation threshold parameter can be recorded as R_th; the preset algorithm can be determined according to actual conditions and is not limited here. As an example, the preset algorithm may include a formula, for example, Where M is the number of cores of the storage-computing integrated accelerator, R_rem m is the number of remaining space resources in the current mth core, h is a reservation coefficient ranging from 0 to 1, and the value of h can be determined according to actual conditions and is not limited here. As an example, h can be 0.5. The space resource reservation threshold decreases with task allocation.
[0191] The specific determination process of determining the space resource reservation table according to the space resource demand of the unallocated task and the space resource reservation threshold parameter can be determined according to actual conditions and is not limited here. As an example, determining the space resource reservation table according to the space resource demand of each task in the to-be-allocated and unallocated task list and the space resource reservation threshold parameter may include determining each second target task in the to-be-allocated and unallocated task list whose space resource demand exceeds the space resource reservation threshold parameter; counting the number of tasks in the to-be-allocated and unallocated task list whose space resource demand is greater than or equal to the space resource demand of the second target task; and determining the space resource reservation table based on the space resource demand of the second target task and the number of tasks.
[0192] In actual applications, the cross-array unit in the storage-computing accelerator is both a computing unit and a storage unit. The model weights need to be deployed in advance to the storage-computing array. When performing model orchestration, if the number of task weights assigned to a core exceeds the storage-computing array space limit in the core, the model cannot be deployed to the core. If the space resource demand of a task exceeds the remaining space resources in all current cores, the weight matrix of the task needs to be split and deployed to multiple cores respectively. This will require cross-core synchronization during the calculation process, introducing additional overhead. In order to avoid this situation as much as possible, resources are reserved for tasks whose space resource demand exceeds the threshold. The specific resource reservation method is as follows: Space resource reservation threshold Where M is the number of cores of the storage-computing integrated accelerator, R_rem m is the number of remaining space resources in the current mth core, h is the reservation coefficient ranging from 0 to 1, and the value of h can be determined according to the actual situation, and the default value is 0.5. It can be seen that the space resource reservation threshold decreases with the task allocation.
[0193] In the list of all currently allocated and unallocated tasks, the tasks whose space resource requirements exceed the resource reservation threshold are sorted from high to low according to the resource requirements and numbered starting from 1. The corresponding resource requirements res and sorting numbers num are filled into the space resource reservation table in sequence. If there are multiple tasks with the same resource requirements, only the one with the largest number is filled. Each pair of res and num in the space resource reservation table indicates that the number of remaining resources greater than res is required to be reserved on num cores. The space resource reservation condition is met only when all requirements in the space resource reservation table are met.
[0194] In one embodiment, determining the space resource reservation table according to the space resource requirements of each task in the to-be-allocated and unallocated task lists and the space resource reservation threshold parameter comprises:
[0195] Determine a second target task in the list of to-be-allocated and unallocated tasks, each of which has a space resource demand exceeding the space resource reservation threshold parameter;
[0196] Counting the number of tasks in the list of to-be-allocated and unallocated tasks whose space resource requirements are greater than or equal to the space resource requirements of the second target task;
[0197] The space resource reservation table is determined based on the space resource demand of the second target task and the number of tasks.
[0198] In this embodiment, the specific determination process in the second target task of determining that each space resource demand in the list of to-be-allocated and unallocated tasks exceeds the space resource reservation threshold parameter can be determined according to actual conditions and is not limited here. As an example, the second target task of determining that each space resource demand in the list of to-be-allocated and unallocated tasks exceeds the space resource reservation threshold parameter can be to sort each space resource demand in the list of to-be-allocated and unallocated tasks from high to low or from low to high, and obtain a sorting result of sorting the space resource demand from high to low or from low to high, and then obtain the second target task of each space resource demand in the list of to-be-allocated and unallocated tasks exceeding the space resource reservation threshold parameter based on the sorting result.
[0199] The specific statistical process of counting the number of tasks in the list of to-be-allocated and unallocated tasks whose space resource demand is greater than or equal to the space resource demand of the second target task can be determined according to the actual situation and is not limited here. As an example, the process of counting the number of tasks in the list of to-be-allocated and unallocated tasks whose space resource demand is greater than or equal to the space resource demand of the second target task can be to number the number of tasks in the list of to-be-allocated and unallocated tasks whose space resource demand is greater than or equal to the space resource demand of the second target task starting from 1. The space resource demand can be recorded as res; the number can be recorded as num.
[0200] The specific determination process of determining the space resource reservation table based on the empty resource demand of the second target task and the number of tasks can be determined according to actual conditions and is not limited here. As an example, the determination of the space resource reservation table based on the empty resource demand of the second target task and the number of tasks can be to fill the empty resource demand of the second target task and the number of tasks into the space resource reservation table in sequence.
[0201] In actual applications, in the current list of all tasks to be assigned and unassigned, the tasks whose space resource requirements exceed the resource reservation threshold are sorted from high to low according to the resource requirements and numbered from 1, and the corresponding resource requirements res and sorting numbers num are filled into the space resource reservation table in sequence. If there are multiple tasks with the same resource requirements, only the one with the largest number is filled. Each pair of res and num in the space resource reservation table indicates that the number of remaining resources greater than res is required to be reserved on num cores. The space resource reservation condition is met only when all requirements in the space resource reservation table are met.
[0202] In one embodiment, the method further comprises:
[0203] Determine the actual completion time of each array unit based on the actual start time of the task to be assigned on each array unit of each core;
[0204] Sort all array units on each core according to the actual completion time and number of remaining space resources of each array unit to obtain a first sorting result;
[0205] Determine all array units that need to be added to the allocation array list to meet the space resource requirements of the task to be allocated based on the first sorting result;
[0206] Obtain the target array unit with the latest actual completion time among all the array units;
[0207] The actual completion time of the target array unit is used as the actual completion time of the task to be assigned on each core.
[0208] In the embodiment of the present application, the actual start time of each array unit can be recorded as AST (T i ,XB n ); The actual completion time on each array unit can be recorded as AFT(T i ,XB n ); The specific determination process of determining the actual completion time of each array unit based on the actual start time of the task to be assigned on each array unit of each core can be determined according to actual conditions, and is not limited here. As an example, the actual completion time of each array unit is determined by a preset algorithm based on the actual start time of the task to be assigned on each array unit of each core; wherein the preset algorithm can be determined according to actual conditions, and is not limited here. As an example, the preset algorithm may include a formula, for example, AFT(T i ,XB n )=AST(T i ,XB n )+PT(T i ); where AST(T i ,XB n ) is T i In array element XB n The actual start time on XB n Among all idle time periods, the start time of the earliest time period that meets the following two conditions: the duration is greater than PT (T i ); start time is greater than T i The time when all the input data it depends on will be ready.
[0209] All array units on each core are sorted respectively according to the actual completion time of each array unit and the number of remaining space resources. The first sorting result can be understood as when the actual completion time is less than the earliest start time of the successor task, the array units are sorted from most to least according to the number of remaining space resources; otherwise, the array units are sorted from least to greatest according to the actual completion time.
[0210] In practical applications, the task T i Assign to a core C m The actual completion time is calculated as follows: m For all array elements on XB n , calculate T i The actual completion time AFT(T i ,XB n ): AFT(T i ,XB n )=AST(T i ,XB n )+PT(T i ); where AST(T i ,XB n ) is T i In array element XB n The actual start time on XB n The start time of the earliest time period among all available time periods that meets the following two conditions:
[0211] a) Duration is greater than PT(T i );
[0212] b) The start time is greater than T i The time when all the input data it depends on will be ready.
[0213] When T i When there is no predecessor task, all data ready time is 0; otherwise, T i The ready time of all the input data that it depends on is Here DCT(T k ,T i , C m ) is T i Assigned to C m When and its predecessor task T k The inter-core data transfer time, when T k Also assigned in C m When the task is on, the inter-core data transfer time is 0, otherwise, the inter-core data transfer time is determined by task T k and T i The amount of data transferred between That is T iThe time when all the dependent input data are ready,
[0214] C m Sort all array elements on , for any two array elements, the sorting rules are as follows:
[0215] a) If the AFT(T i ,XB n ) are satisfied The two array units are sorted in descending order according to the remaining space resources.
[0216] b) If the AFT(T i ,XB n ) are not satisfied, then the two array elements are Sort in order from smallest to largest.
[0217] Select arrays from the sorted array cells one by one and add them to T i In C m Allocation array list on XBList i,m , until the task requirements are met.
[0218] Since different array units in the core can be computed in parallel, task T i Assigned to C m The actual completion time is: That is T i In C m The latest actual completion time of all array elements assigned to it.
[0219] In one embodiment, the allocating and deploying the task to be allocated to at least one array unit on at least one core based on the actual completion time of the task to be allocated on each core, the number of remaining space resources in each core, and the space resource reservation table includes:
[0220] Traversing each core of the multi-core storage-computing integrated accelerator to determine the actual completion time of the task to be assigned on each core;
[0221] Sorting all the cores in the multi-core storage-computing integrated accelerator based on the actual completion time on each core, the number of remaining space resources in each core, and the space resource reservation table to obtain a second sorting result;
[0222] Based on the second sorting result, determining whether the remaining space resources in the first-ranked core among the multi-cores meet the space resource requirements of the task to be assigned;
[0223] When the remaining space resources in the first-ranked core among the multiple cores meet the space resource requirements of the task to be assigned, assigning and deploying the task to be assigned to at least one array unit on one core;
[0224] When the remaining space resources in the first-ranked core among the multiple cores do not meet the space resource requirements of the task to be assigned, the task to be assigned is assigned and deployed to multiple array units on multiple cores.
[0225] In this embodiment, traversing each core of the multi-core storage and computing integrated accelerator to determine the actual completion time of the task to be assigned on each core can be understood as calculating the actual completion time of each task to be assigned T i , traverse each core of the storage-computing integrated accelerator, for each core C m , calculate T i The actual completion time AFT(T i ,C m ).
[0226] Based on the actual completion time on each core, the number of remaining space resources in each core, and the space resource reservation table, all the cores in the multi-core storage-computing integrated accelerator are sorted, and the specific sorting process in the second sorting result can be determined according to actual conditions and is not limited here. As an example, all the cores on the storage-computing integrated accelerator are sorted. For any two cores, the sorting process can be as follows:
[0227] a) If the remaining space resources in both cores can satisfy the task T i Space resource demand AFT(T i ,C m ), and both satisfy Then sort the two cores in order of remaining space resources from more to less.
[0228] b) If the remaining space resources in both cores can satisfy the task T i space resource requirements, but AFT(T i ,C m ) are not satisfied Then these two cores are divided according to AFT(T i ,C m ) are sorted in order from smallest to largest.
[0229] c) If the remaining space resources in a core satisfy the task T i The space resource requirements of task T are not met by the remaining space resources on the other core. i If the space resource demand is met, the two cores are sorted in order from most to least remaining space resources.
[0230] d) If the remaining space resources in both cores do not satisfy task T i The space resource requirements of AFT(T i ,C m ) are satisfied Then sort the two cores in order of the remaining space resources from most to least.
[0231] e) If the remaining space resources in both cores do not satisfy task T i The space resource requirements of AFT(T i ,C m ) are not satisfied Then these two cores are divided according to AFT(T i ,C m ) are sorted in order from smallest to largest.
[0232] In the case where the remaining space resources in the first-ranked core in the multi-cores meet the space resource requirements of the task to be assigned, the task to be assigned is assigned and deployed to at least one array unit on a core. It can be understood that if the remaining space resources in the first-ranked core meet the space resource requirements of the task T i If the space resource requirements are met, each core is traversed in turn to determine whether the space resource reservation conditions are met after the task is assigned to the core. If so, the core is selected for allocation. Otherwise, the traversal continues until the conditions are met or all cores are traversed. If all cores cannot meet the conditions, the first core is selected for allocation.
[0233] In the case that the remaining space resources in the first-ranked core among the multiple cores do not meet the space resource requirements of the task to be assigned, the task to be assigned is assigned and deployed to multiple array units on multiple cores. It can be understood that if the remaining space resources in the first-ranked core do not meet the space resource requirements of task Ti, multiple cores are selected from the ranked cores in turn for allocation until the task requirements are met.
[0234] In practical applications, for each task to be assigned T i , traverse each core of the storage-computing integrated accelerator, for each core C m , calculate T i The actual completion time AFT(T i ,C m ).
[0235] Sort all the cores on the storage-computing accelerator. For any two cores, the sorting rules are as follows:
[0236] f) If the remaining space resources in both cores can satisfy the task T i The space resource requirements of AFT(T i ,Cm ) are satisfied Then sort the two cores in order of remaining space resources from more to less.
[0237] g) If the remaining space resources in both cores can satisfy the task T i space resource requirements, but AFT(T i ,C m ) are not satisfied Then these two cores are divided according to AFT(T i ,C m ) are sorted in order from smallest to largest.
[0238] h) If the remaining space resources in a core satisfy the task T i The space resource requirements of task T are not met by the remaining space resources on the other core. i If the space resource demand is met, the two cores are sorted in order from most to least remaining space resources.
[0239] i) If the remaining space resources in both cores do not satisfy the task T i The space resource requirements of AFT(T i ,C m ) are satisfied Then sort the two cores in order of the remaining space resources from most to least.
[0240] j) If the remaining space resources in both cores do not satisfy task T i The space resource requirements of AFT(T i ,C m ) are not satisfied Then these two cores are divided according to AFT(T i ,C m ) are sorted in order from smallest to largest.
[0241] If the remaining space resources in the first core after sorting satisfy the task T i If the space resource requirements are met, each core is traversed in turn to determine whether the space resource reservation conditions are met after the task is assigned to the core. If so, the core is selected for allocation. Otherwise, the traversal continues until the conditions are met or all cores are traversed. If all cores cannot meet the conditions, the first core is selected for allocation.
[0242] If the remaining space resources in the first core after sorting do not meet the requirements of task T i , indicating that there is no core with remaining space resources that can meet the task requirements. In this case, multiple cores are selected from the sorted cores for allocation in sequence until the task requirements are met.
[0243] For ease of understanding, the model orchestration method exemplified here is specifically a model orchestration method for a multi-core storage and computing integrated accelerator.
[0244] Since the storage-computing accelerator performs calculations directly in the memory, the model weights need to be deployed in advance to the storage-computing array. When arranging the model, if the number of task weights assigned to a core exceeds the storage space limit of the storage-computing array in the core, the model cannot be deployed. At the same time, since each core of the storage-computing accelerator includes multiple cross-array units, even tasks assigned to the same core can be calculated in parallel, which has a larger parallel optimization space than traditional multi-core processors with separate storage and computing.
[0245] The present invention proposes a model arrangement method for a multi-core storage-computing integrated accelerator, including dynamic task sorting, timely updating the priority sorting of tasks to be assigned according to the actual completion time of the assigned tasks during the assignment process, and reducing the complexity of subsequent processing, including:
[0246] 1) Dynamically maintain the list of tasks to be assigned based on the data dependencies between tasks;
[0247] 2) Prioritize the tasks in the scheduling list based on the dynamically updated task freedom and inter-core communication delay.
[0248] The present invention proposes a model arrangement method for a multi-core storage-computing integrated accelerator, including task allocation based on the dual constraints of time and space resources of the storage-computing integrated array, taking into account the in-core storage space limitation when performing task allocation, and making full use of the in-core parallel optimization space brought by the characteristics of the storage-computing integrated array, including:
[0249] 1) Maintaining a time occupancy table and a space occupancy table for each cross array unit;
[0250] 2) Determine the actual completion time of allocating tasks to each core based on the space resource requirements of the tasks to be allocated, the ready time of input data, the earliest start time of the subsequent tasks, and the time occupancy table and space occupancy table of each array;
[0251] 3) Dynamically maintain the space resource reservation table based on the number of remaining space resources in each core and the space resource requirements of unassigned tasks;
[0252] 4) Based on the actual completion time on each core, the remaining space resources in each core, and the space resource reservation requirements of unassigned tasks, the tasks are assigned and deployed to one or more array units on one or more cores.
[0253] This application proposes a model arrangement method for a multi-core storage-computing integrated accelerator. In view of the hardware characteristics of the multi-core storage-computing integrated accelerator, the dual constraints of time and space resources are considered when allocating model tasks, so as to fully utilize the intra-core parallel optimization space brought by the storage-computing integrated array characteristics under the condition of satisfying the array storage space limitation, so as to minimize the overall completion time. To achieve the above purpose, the specific technical scheme of the present invention is as follows:
[0254] 1. Dynamic task sorting
[0255] 1. Task freedom calculation
[0256] Define Task T i The degree of freedom is DOF(T i )=LFT(T i )-EST(T i ).
[0257] For pending and unassigned tasks:
[0258] T i The earliest start time is T i The maximum value of the earliest completion time of all predecessor tasks, if T i If there is no predecessor task, the value is 0.
[0259] T i The latest completion time is T i The minimum of the latest start time of all subsequent tasks, if T i If there is no successor task, then take T i The earliest completion time.
[0260] LST(T i )=LFT(T i )-PT(T i ), is T i The latest start time is T i The difference between the latest finish time and the processing time.
[0261] EFT(T i )=EST(T i )+PT(T i ), is T i The earliest completion time is T i The sum of the earliest start time and the processing time.
[0262] Therefore, task T i The degree of freedom is that T iThe maximum time that can be occupied, in addition to the processing time of the task itself on the storage and computing array, should also include the communication time for receiving data from the predecessor task and the waiting time caused by hardware conflicts.
[0263] For the storage-computing integrated accelerator, suppose task T i The input matrix dimension is H input ×W input , the weight matrix dimension is H weight ×W weight Due to the parallel computing characteristics of the storage-computation integrated array, each multiplication operation of the input row vector and the weight matrix can be completed in parallel by converting the input row vector into a single input voltage. Since the arrays in each core of the storage-computation integrated accelerator are the same, the time for a matrix-vector multiplication operation is basically the same, which is set as T MVM Therefore, as long as the array space resources are sufficient, T i The processing time on the storage array is: PT(T i )=H input ×T MVM It can be seen that task T i The processing time on the storage array is only related to the number of rows of the input matrix, and has nothing to do with which core the task is assigned to execute.
[0264] For assigned tasks:
[0265] LST(T i )=EST(T i )=AST(T i ), T i The latest start time and the earliest start time are equal to the actual start time.
[0266] EFT(T i )=LFT(T i )=AFT(T i )=AST(T i )+PT(T i ), T i The earliest completion time and the latest completion time are equal to the actual completion time, and T i The sum of the actual start time and the processing time.
[0267] In the task T i After the allocation, according to the allocation result, T i The earliest and latest start times of T are updated to the actual start time, and T i The earliest and latest finish times are updated to the actual finish times.
[0268] In the task allocation process, if the constraints lead to Ti The actual completion time of a task exceeds the earliest start time of its successor task. Then the degrees of freedom of all currently allocated and unallocated nodes need to be updated.
[0269] 2. Task priority calculation.
[0270] Define Task T i The priorities are:
[0271]
[0272] Among them, DCT (T k ,T i ) is task T k and T i The inter-core data transfer time is determined by task T k and T i The amount of data transferred between
[0273] That is T i The earliest start time delay caused by the failure of its predecessor task to be assigned to the same core is the possible delay under the most unfavorable assignment situation.
[0274] It can be seen that the task priority takes into account both the task freedom and the inter-core communication delay: the greater the task freedom, the lower the priority; the greater the delay caused by the task not being able to be assigned to the same core as its predecessor task due to constraints, the higher the priority.
[0275] Since the task priority is calculated based on the task freedom, each task freedom update will result in an update of the task priority.
[0276] 3. Task list maintenance and dynamic sorting.
[0277] During the task assignment process, three task lists are maintained, namely the assigned task list, the to-be-assigned task list, and the unassigned task list.
[0278] Before starting the scheduling, traverse all computing tasks in the model. If a task node has no predecessor task, add the task to the list of tasks to be assigned; otherwise, add the task to the list of unassigned tasks. The assigned task list is initially an empty list.
[0279] Each time a task is assigned, all tasks in the task list to be assigned are sorted by priority. If there are multiple tasks in the task list with the same priority, the task node that occupies more space resources is placed first. If there are multiple tasks in the task list with the same priority and space resources, the order of these tasks is random.
[0280] After each task assignment is completed, the assigned task will be deleted from the list of tasks to be assigned, added to the list of assigned tasks, and all successor tasks of the task will be traversed. If all predecessor tasks of a successor task are already in the assigned list, the successor task will be deleted from the list of unassigned tasks and added to the list of tasks to be assigned.
[0281] It can be seen that only when all the predecessor tasks of a task have been assigned can it enter the list of tasks to be assigned. For any task in the list of tasks to be assigned, its predecessor task either does not exist or is in the list of assigned tasks, and its successor task either does not exist or is in the list of unscheduled tasks. Therefore, during the entire task assignment process, at any time, there is no data dependency between the tasks in the list of tasks to be assigned, and as long as they are not deployed on the same XB, they can be executed in parallel.
[0282] 2. Task allocation based on the dual constraints of time and space resources of the storage and computing integrated array.
[0283] Each time a task is assigned, the tasks in the task list to be assigned are sorted, and the task with the highest ranking is selected for assignment.
[0284] 1. Maintain the array unit time and space resource occupancy table.
[0285] When performing task allocation, tasks need to be allocated to one or more cross arrays. Once the allocation is completed, the task has exclusive ownership of the array unit in both time and space. On the other hand, each array unit is independent and can execute computing tasks in parallel. Therefore, for each cross array unit, a time occupancy table and a space occupancy table are maintained. The time occupancy table records the occupied time periods of the cross array unit, and the time periods outside the occupied time periods are idle time periods; the space occupancy table records the occupied space resource positions in the cross array unit, and the space resource positions outside the occupied space resource positions are available space resource positions.
[0286] 2. Calculation of actual completion time.
[0287] Task T i Assign to a core C m The actual completion time is calculated as follows:
[0288] Traversing C m For all array elements on XB n , calculate T i The actual completion time AFT(T i ,XB n ):
[0289] AFT(T i,XB n )=AST(T i ,XB n )+PT(T i )
[0290] in,
[0291] AST(T i ,XB n ) is T i In array element XB n The actual start time on XB n The start time of the earliest time period among all available time periods that meets the following two conditions:
[0292] c) Duration is greater than PT (T i )
[0293] d) Start time is greater than T i The time when all the input data it depends on will be ready.
[0294] When T i When there is no predecessor task, all data ready time is 0; otherwise, T i The ready time of all the input data that it depends on is Here DCT(T k ,T i , C m ) is T i Assigned to C m When and its predecessor task T k The inter-core data transfer time, when T k Also assigned in C m When the task is on, the inter-core data transfer time is 0, otherwise, the inter-core data transfer time is determined by task T k and T i The amount of data transferred between That is T i The time when all the dependent input data are ready,
[0295] C m Sort all array elements on , for any two array elements, the sorting rules are as follows:
[0296] c) If the AFT(T i ,XB n ) are satisfied The two array units are sorted in descending order according to the remaining space resources.
[0297] d) If the AFT(T i ,XB n ) are not satisfied Then the two array elements are connected according to AFT(T i ,XB n ) are sorted in order from smallest to largest.
[0298] Select arrays from the sorted array cells one by one and add them to T i In C m Allocation array list on XBList i,m , until the task requirements are met.
[0299] Since different array units in the core can be computed in parallel, task T i Assigned to C m The actual completion time is: That is T i In C m The latest actual completion time of all array elements assigned to it.
[0300] 3. Space resource reservation.
[0301] The cross-array unit in the storage-computing accelerator is both a computing unit and a storage unit. The model weights need to be deployed in advance to the storage-computing array. When performing model orchestration, if the number of task weights assigned to a core exceeds the storage-computing array space limit in the core, the model cannot be deployed to the core. If the space resource demand of a task exceeds the remaining space resources in all current cores, the weight matrix of the task needs to be split and deployed to multiple cores respectively. This will require cross-core synchronization during the calculation process, introducing additional overhead. In order to avoid this situation as much as possible, resources are reserved for tasks whose space resource demand exceeds the threshold. The specific resource reservation method is as follows:
[0302] Space resource reservation threshold
[0303] Where M is the number of cores of the storage-computing integrated accelerator, R_rem m is the number of remaining space resources in the current mth core. It can be seen that the space resource reservation threshold decreases with task allocation.
[0304] In the list of all currently allocated and unallocated tasks, the tasks whose space resource requirements exceed the resource reservation threshold are sorted from high to low according to the resource requirements and numbered starting from 1. The corresponding resource requirements res and sorting numbers num are filled into the space resource reservation table in sequence. If there are multiple tasks with the same resource requirements, only the one with the largest number is filled. Each pair of res and num in the space resource reservation table indicates that the number of remaining resources greater than res is required to be reserved on num cores. The space resource reservation condition is met only when all requirements in the space resource reservation table are met.
[0305] 4. The nuclear option.
[0306] For each task Ti to be assigned, traverse each core of the storage-computation integrated accelerator, and for each core Cm, calculate the actual completion time AFT (T i ,C m ).
[0307] Sort all the cores on the storage-computing accelerator. For any two cores, the sorting rules are as follows:
[0308] If the remaining space resources in both cores meet the space resource requirements of task Ti, and AFT(T i ,C m ) are satisfied Then sort the two cores in order of remaining space resources from more to less.
[0309] k) If the remaining space resources in both cores meet the space resource requirements of task Ti, but AFT(T i ,C m ) are not satisfied Then these two cores are divided according to AFT(T i ,C m ) are sorted in order from smallest to largest.
[0310] l) If the remaining space resources in one core meet the space resource requirements of task Ti, and the remaining space resources on another core do not meet the space resource requirements of task Ti, then the two cores are sorted in order from most to least remaining space resources.
[0311] m) If the remaining space resources in both cores do not meet the space resource requirements of task Ti, and AFT(T i ,C m ) are satisfied Then sort the two cores in order of the remaining space resources from most to least.
[0312] If the remaining space resources in both cores do not meet the space resource requirements of task Ti, and AFT(T i ,C m ) are not satisfied Then these two cores are divided according to AFT(T i ,C m ) are sorted in order from smallest to largest.
[0313] If the remaining space resources in the first core after sorting satisfy the task T iIf the space resource requirements are met, each core is traversed in turn to determine whether the space resource reservation conditions are met after the task is assigned to the core. If so, the core is selected for allocation. Otherwise, the traversal continues until the conditions are met or all cores are traversed. If all cores cannot meet the conditions, the first core is selected for allocation.
[0314] If the remaining space resources in the first core after sorting do not meet the requirements of task T i , indicating that there is no core with remaining space resources that can meet the task requirements. In this case, multiple cores are selected from the sorted cores for allocation in sequence until the task requirements are met.
[0315] Record the resource allocation of the task, including the allocated core number and the XB number on the corresponding core, update the time occupancy table and space occupancy table of the corresponding XB according to the resource allocation, and update the corresponding remaining space resource number and space resource utilization.
[0316] Delete the task from the list of tasks to be assigned, add it to the list of assigned tasks, traverse all the successor tasks of the task, if all the predecessor tasks of a successor task are already in the assigned list, delete the successor task from the list of unassigned tasks and add it to the list of tasks to be assigned.
[0317] The specific model arrangement process is as follows: Figure 3 As shown, Figure 3 Schematic diagram of the application scenario of model orchestration process.
[0318] Step 1: Initialize the degrees of freedom of each task in the model: traverse each task in the model in depth-first-search (DFS) order, calculate the EST and EFT of each task in turn, then traverse each task in reverse order, and calculate the LST, LFT, and DOF of each task in turn.
[0319] Step 2: Initialize the assigned, pending and unassigned task lists: traverse all computing tasks in the model. If a task node has no predecessor task, add the task to the pending task list; otherwise, add the task to the unassigned task list. The assigned task list is initially an empty list.
[0320] Step 3 calculates the priority of each task in the list of tasks to be assigned: traverse all computing tasks in the list of tasks to be assigned. If a task has no predecessor task, calculate the priority of the task based on the task freedom; otherwise, calculate the EST delay caused by the failure of the task and its predecessor task to be assigned to the same core, and calculate the priority of the task based on the delay and the task freedom.
[0321] Step 4: Sort the tasks in the task list to be assigned: Sort all the tasks in the task list to be assigned by priority. If there are multiple tasks in the task list with the same priority, the task node that occupies more space resources is placed first. If there are multiple tasks in the task list with the same priority and space resources, the order of these tasks is random.
[0322] Step 5 calculates the actual completion time of the first-ranked task on each core: for the first-ranked task, traverse each core of the storage-computing integrated accelerator and sort all array units on a core based on the remaining space resources of the array and the AFT for allocating tasks to the array. Select arrays from the sorted array units in turn to add the task to the allocation array list on the core until the task requirements are met. The AFT of the task on the core is the latest AFT on all array units in the allocation array list.
[0323] Step 6 performs core sorting on the first ranked task: sort all the cores on the storage-computing integrated accelerator based on the remaining space resources on the core and the AFT that allocates the task to the core.
[0324] Step 7 determines whether the remaining space resources in the first-ranked core meet the task requirements: if yes, execute step 8; otherwise, execute step 9.
[0325] Step 8: Select a core for allocation: traverse each core in turn to determine whether the space resource reservation condition is met after the task is allocated to the core. If so, select the core for allocation. Otherwise, continue traversing until the condition is met or all cores are traversed. If all cores cannot meet the condition, select the first core for allocation. Execute step 10.
[0326] Step 9: Select multiple cores for allocation: Select multiple cores from the sorted cores for allocation in sequence until the task requirements are met.
[0327] Step 10 records the resource allocation of the task: including the allocated core number and one or more XB numbers that meet the minimum AFT of the task on the corresponding core, and updates the remaining space resources on the corresponding core according to the resource allocation.
[0328] Step 11 updates the assigned, to-be-assigned, and unassigned task lists: delete the task from the to-be-assigned task list, add it to the assigned task list, traverse all successor tasks of the task, and if all predecessor tasks of a successor task are already in the assigned list, delete the successor task from the unassigned task list and add it to the to-be-assigned task list.
[0329] Step 12 determines whether the list of tasks to be assigned is empty; if it is empty, execute step 15; otherwise, execute step 13.
[0330] Step 13 determines whether the task freedom needs to be updated: determine whether the AFT of the task exceeds the EST of its successor task. If so, execute step 14; otherwise, execute step 3.
[0331] Step 14 updates the degrees of freedom of all tasks to be assigned and unassigned; updates the EST and LST of the assigned tasks to AST, updates the EFT and LFT to AFT, and recalculates the EST, LST, EFT, LFT and DOF for all tasks in the list of assigned and unassigned tasks.
[0332] Step 15: Deploy the model according to the task allocation result: Deploy the weight of each task to the allocated cross array unit according to the task allocation result.
[0333] The present application proposes a model scheduling method, including alternating execution of dynamic task sorting and task allocation based on the dual constraints of time and space resources of a storage-computing integrated array; and dynamic task sorting, timely updating the priority sorting of tasks to be assigned according to the actual completion time of the assigned tasks during the allocation process, reducing the complexity of subsequent processing, including: dynamically maintaining a list of tasks to be assigned based on the data dependency relationship between tasks; and prioritizing tasks in a list to be scheduled based on dynamically updated task freedom and inter-core communication delay.
[0334] This application proposes a model scheduling method, including task allocation based on the dual constraints of time and space resources of the storage-computing integrated array. The dual constraints of time and space resources are considered when allocating tasks. Under the condition of satisfying the storage space limit of the array, the in-core parallel optimization space brought by the characteristics of the storage-computing integrated array is fully utilized to minimize the overall completion time, including:
[0335] 1) Maintaining a time occupancy table and a space occupancy table for each cross array unit;
[0336] 2) Determine the actual completion time of allocating tasks to each core based on the space resource requirements of the tasks to be allocated, the ready time of input data, the earliest start time of the subsequent tasks, and the time occupancy table and space occupancy table of each array;
[0337] 3) Dynamically maintain the space resource reservation table based on the number of remaining space resources in each core and the space resource requirements of unassigned tasks;
[0338] 4) Based on the actual completion time on each core, the remaining space resources in each core, and the space resource reservation requirements of unassigned tasks, the tasks are assigned and deployed to one or more array units on one or more cores.
[0339] This application aims at the characteristics of the storage-computing integrated array, maintains a time occupancy table and a space occupancy table for each cross-array unit, considers the dual constraints of time and space resources when performing task assignment, and fully utilizes the intra-core parallel optimization space brought by the characteristics of the storage-computing integrated array under the condition of satisfying the array storage space limitation, so as to minimize the overall completion time. When performing task assignment, in addition to considering the space resource requirements of the current tasks to be assigned and the remaining space resources in the core, the space resource reservation requirements of the unassigned tasks are also considered, and the cross-core overhead caused by the subsequent splitting and deployment of the task weight matrix to multiple cores due to insufficient space resources is avoided as much as possible. Dynamic task sorting and task assignment based on the dual constraints of time and space resources of the storage-computing integrated array are executed alternately. During the assignment process, the priority order of the tasks to be assigned is updated in time according to the actual completion time of the assigned tasks, reducing the complexity of subsequent processing.
[0340] In order to implement the method of the embodiment of the present application, the embodiment of the present application further provides a model arrangement device 400, which is arranged on a multi-core storage and computing integrated accelerator. Figure 4 Schematic diagram of a model arrangement device according to an embodiment of the present application; Figure 4 As shown, including:
[0341] The acquisition unit 401 is used to acquire a time occupancy table and a space occupancy table of each cross array unit in each core; the time occupancy table is used to record the occupied time period of each cross array unit; the space occupancy table is used to record the occupied space resource position of each cross array unit;
[0342] The determining unit 402 is used to determine the to-be-assigned task based on each task in the model; and determine the actual completion time of allocating the to-be-assigned task to each core according to the arrival time of the latest input data of the to-be-assigned task among all the data input to the to-be-assigned task, the space resource requirement of the to-be-assigned task, the earliest start time of the successor task of the to-be-assigned task, and the time occupancy table and the space occupancy table; and determine the space resource reservation table based on the number of remaining space resources in each core and the space resource requirement of the unassigned task;
[0343] The allocation unit 403 is used to allocate and deploy the task to be allocated to at least one array unit on at least one core based on the actual completion time of the task to be allocated on each core, the number of remaining space resources in each core and the space resource reservation table.
[0344] Here, in one embodiment, the device 400 further includes a recording unit and an updating unit; wherein,
[0345] The recording unit is used to record the resource allocation status of the task to be assigned;
[0346] The updating unit is used to update the time occupancy table and the space occupancy table based on the resource allocation situation.
[0347] Here, in one embodiment, the determination unit 402 is also used to determine the degree of freedom parameters, processing time and delay time of inter-core communication of each task in the model; determine the assigned task list based on the data dependency relationship of each task in the model; determine the priority parameters of each task in the to-be-assigned task list in the assigned task list according to the degree of freedom parameters, the processing time and the delay time; use the priority parameters to prioritize each task in the to-be-assigned task list to obtain the task with the highest priority in the to-be-assigned task list; and use the task with the highest priority in the to-be-assigned task list as the to-be-assigned task.
[0348] Here, in one embodiment, the determination unit 402 is also used to determine the first target task in the model; the first target task is any task among the tasks; whether the first target task has been assigned is determined; if the first target task has been assigned, the first target task is added to the assigned list; if the first target task has not been assigned, it is determined whether the first target task has a predecessor task or whether the predecessor task has been assigned; if the first target task has no predecessor task or all predecessor tasks have been assigned, the first target task is added to the to-be-assigned list; if the first target task has a predecessor task and not all predecessor tasks have been assigned, the first target task is added to the unassigned list; the assigned list, the to-be-assigned list and the unassigned list are used as the assigned task list.
[0349] Here, in one embodiment, the updating unit is further configured to update the assigned task list in response to completion of the assignment of the to-be-assigned task.
[0350] Here, in one embodiment, the device 400 further includes a judgment unit; wherein,
[0351] The judging unit is used to judge whether the list of tasks to be assigned in the updated assigned task list is empty;
[0352] The determining unit 402 is further configured to re-determine the degree of freedom parameter of each task in the model when the list of tasks to be assigned is not empty;
[0353] The allocation unit 403 is further configured to deploy each task in the allocated task list in the updated allocated task list to at least one array unit on at least one core according to resource allocation conditions when the to-be-allocated task list is empty.
[0354] Here, in one embodiment, the determining unit 402 is further used to determine the earliest start time and the latest completion time of each task in the model; and determine the degree of freedom parameter based on the earliest start time and the latest completion time.
[0355] Here, in one embodiment, the determining unit 402 is further used to obtain the processing time and data dependency of each task in the model on the cross array unit; determine the initial earliest start time and the initial latest completion time of each task in the model based on the processing time and the data dependency; in the process of allocating each task in the model, determine the actual completion time of the current task according to the resource allocation of the current task to be allocated;
[0356] The updating unit is further configured to update the earliest start time and the latest completion time of the current task to be assigned based on the actual completion time of the current task.
[0357] Here, in one embodiment, the updating unit is also used to update the earliest start time and latest finish time of each task in the to-be-assigned and unassigned task lists in the assigned task list based on the updated earliest start time and latest finish time of the current to-be-assigned task, when the updated latest finish time is greater than the earliest start time of the successor task of the current to-be-assigned task; and update the degree of freedom parameter of each task in the to-be-assigned and unassigned task lists in the assigned task list based on the updated earliest start time and latest finish time of each task in the to-be-assigned and unassigned task lists in the assigned task list.
[0358] Here, in one embodiment, the unallocated task is located in the list of tasks to be allocated and unallocated; the determination unit 402 is also used to determine the space resource reservation threshold parameter based on the number of remaining space resources in each core; and determine the space resource reservation table according to the space resource demand of the unallocated task and the space resource reservation threshold parameter.
[0359] Here, in one embodiment, the determination unit 402 is also used to determine the second target task in the list of to-be-allocated and unallocated tasks for each space resource demand that exceeds the space resource reservation threshold parameter; count the number of tasks in the list of to-be-allocated and unallocated tasks whose space resource demand is greater than or equal to the space resource demand of the second target task; and determine the space resource reservation table based on the space resource demand of the second target task and the number of tasks.
[0360] Here, in one embodiment, the device 402 further includes a sorting unit; wherein,
[0361] The determining unit 402 is further configured to determine the actual completion time of each array unit based on the actual start time of the task to be assigned on each array unit of each core;
[0362] The sorting unit is used to sort all array units on each core according to the actual completion time and the number of remaining space resources of each array unit to obtain a first sorting result;
[0363] The determining unit 402 is further configured to determine, based on the first sorting result, all array units that need to be added to the allocation array list to meet the space resource requirements of the task to be allocated;
[0364] The acquisition unit 401 is further configured to acquire the target array unit with the latest actual completion time among all the array units; and use the actual completion time of the target array unit as the actual completion time of the task to be assigned on each core.
[0365] Here, in one embodiment, the allocation unit 403 is also used to traverse each core of the multi-core storage and computing integrated accelerator to determine the actual completion time of the task to be allocated on each core; sort all the cores in the multi-core storage and computing integrated accelerator based on the actual completion time on each core, the number of remaining space resources in each core and the space resource reservation table to obtain a second sorting result; based on the second sorting result, determine whether the remaining space resources in the core ranked first among the multi-cores meet the space resource requirements of the task to be allocated; if the remaining space resources in the core ranked first among the multi-cores meet the space resource requirements of the task to be allocated, allocate and deploy the task to be allocated to at least one array unit on one core; if the remaining space resources in the core ranked first among the multi-cores do not meet the space resource requirements of the task to be allocated, allocate and deploy the task to be allocated to multiple array units on multiple cores.
[0366] It should be noted that: the model arrangement device provided in the above embodiment only uses the division of the above program modules as an example when performing model arrangement. In actual applications, the above processing can be assigned to different program modules as needed, that is, the internal structure of the device is divided into different program modules to complete all or part of the processing described above. In addition, the model arrangement device provided in the above embodiment and the model arrangement method embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.
[0367] Based on the hardware implementation of the above-mentioned program module, an embodiment of the present application also provides a model orchestration device, including a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the program, the steps in the model orchestration method provided in the above-mentioned embodiment are implemented.
[0368] Correspondingly, an embodiment of the present application provides a computer program product, which, when executed by a processor, implements the steps in the model orchestration method provided in the above embodiment.
[0369] Correspondingly, an embodiment of the present application provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps in the model arrangement method provided in the above embodiment are implemented.
[0370] It should be noted here that the description of the above storage medium and device embodiments is similar to the description of the above method embodiments, and has similar beneficial effects as the method embodiments. For technical details not disclosed in the storage medium and device embodiments of this application, please refer to the description of the method embodiments of this application for understanding.
[0371] It should be noted that Figure 5 Schematic diagram of a hardware entity structure of a model arrangement device in an embodiment of the present application, such as Figure 5 As shown, the hardware entity of the model arrangement device 500 includes: a processor 501 and a memory 503 . Optionally, the model arrangement device 500 may also include a communication interface 502 .
[0372] It can be understood that the memory 503 can be a volatile memory or a non-volatile memory, and can also include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM); the magnetic surface memory can be a disk memory or a tape memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM, SyncLink Dynamic Random Access Memory), and direct RAM bus random access memory (DRRAM, Direct Rambus Random Access Memory).The memory 503 described in the embodiments of the present application is intended to include but is not limited to these and any other suitable types of memories.
[0373] The method disclosed in the above embodiment of the present application can be applied to the processor 501, or implemented by the processor 501. The processor 501 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 501. The above processor 501 may be a general processor, a digital signal processor (DSP, Digital Signal Processor), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The processor 501 can implement or execute the methods, steps and logic block diagrams disclosed in the embodiments of the present application. The general processor can be a microprocessor or any conventional processor, etc. In combination with the steps of the method disclosed in the embodiment of the present application, it can be directly embodied as a hardware decoding processor to execute, or it can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium, which is located in the memory 503, and the processor 501 reads the information in the memory 503 and completes the steps of the above method in combination with its hardware.
[0374] In an exemplary embodiment, the device may be implemented by one or more application specific integrated circuits (ASIC), DSP, programmable logic device (PLD), complex programmable logic device (CPLD), field programmable gate array (FPGA), general processor, controller, microcontroller (MCU), microprocessor, or other electronic components to execute the aforementioned method.
[0375] It should be understood that "one embodiment" or "an embodiment" mentioned throughout the specification means that specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in various embodiments of the present application, the size of the sequence number of the above-mentioned processes does not mean the order of execution, and the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application. The above-mentioned sequence numbers of the embodiments of the present application are only for description and do not represent the advantages and disadvantages of the embodiments.
[0376] It should be noted that, in this application, the terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.
[0377] The methods disclosed in several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.
[0378] The features disclosed in several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.
[0379] The features disclosed in several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.
[0380] The above is only an implementation method of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.
Claims
1. A model arrangement method, It is characterized in that Applied to multi-core storage and computing accelerators, including: Obtaining a time occupancy table and a space occupancy table of each cross array unit in each core; the time occupancy table is used to record the occupied time period of each cross array unit; the space occupancy table is used to record the occupied space resource position of each cross array unit; Determining tasks to be assigned based on each task in the model; Determine the actual completion time of allocating the task to be assigned to each core according to the arrival time of the latest input data of the task to be assigned among all the data input to the task to be assigned, the space resource requirement of the task to be assigned, the earliest start time of the successor task of the task to be assigned, and the time occupancy table and the space occupancy table; Determine a space resource reservation table based on the number of remaining space resources in each core and the space resource requirements of unassigned tasks; The task to be assigned is assigned and deployed to at least one array unit on at least one core based on the actual completion time of the task to be assigned on each core, the number of remaining space resources in each core and the space resource reservation table.
2. The method according to claim 1, It is characterized in that The method further comprises: Recording the resource allocation of the task to be assigned; The time occupancy table and the space occupancy table are updated based on the resource allocation situation.
3. The method according to claim 2, It is characterized in that The determining of tasks to be assigned based on each task in the model includes: Determining the degree of freedom parameters, processing time, and inter-core communication latency of each task in the model; Determine a list of assigned tasks based on data dependencies among the tasks in the model; Determine the priority parameter of each task in the to-be-assigned task list in the assigned task list according to the degree of freedom parameter, the processing time and the delay time; Prioritize each task in the task list to be assigned using the priority parameter to obtain the task with the highest priority in the task list to be assigned; The task with the highest priority in the list of tasks to be assigned is used as the task to be assigned.
4. The method according to claim 3, It is characterized in that The step of determining the assigned task list based on the data dependency relationship between the tasks in the model includes: Determine a first target task in the model; the first target task is any task among the tasks; Determine whether the first target task has been assigned; If the first target task has been assigned, adding the first target task to the assigned list; If the first target task has not been assigned, determining whether the first target task has a predecessor task or whether the predecessor task has been assigned; If the first target task has no predecessor task or the predecessor tasks have been assigned, adding the first target task to a to-be-assigned list; If the first target task has a predecessor task and not all predecessor tasks have been assigned, adding the first target task to an unassigned list; The allocated list, the to-be-allocated list and the unallocated list are used as the allocated task list.
5. The method according to claim 4, It is characterized in that The method further comprises: In response to the completion of the assignment of the to-be-assigned task, the assigned task list is updated.
6. The method according to claim 5, It is characterized in that The method further comprises: Determine whether the list of tasks to be assigned in the updated assigned task list is empty; When the list of tasks to be assigned is not empty, re-determine the degree of freedom parameters of each task in the model; When the to-be-allocated task list is empty, each task in the allocated task list in the updated allocated task list is deployed to at least one array unit on at least one core according to the resource allocation situation.
7. The method according to claim 3, It is characterized in that Determining the degree of freedom parameters of each task in the model includes: Determine the earliest start time and the latest finish time of each task in the model; The degree of freedom parameter is determined based on the earliest start time and the latest finish time.
8. The method according to claim 7, It is characterized in that Determining the earliest start time and the latest completion time of each task in the model includes: Obtaining the processing time and data dependency of each task in the model on the cross array unit; Determine an initial earliest start time and an initial latest finish time of each task in the model based on the processing time and the data dependency; In the process of allocating each task in the model, determining the actual completion time of the current task according to the resource allocation of the current task to be allocated; The earliest start time and the latest finish time of the current task to be assigned are updated based on the actual finish time of the current task.
9. The method according to claim 8, It is characterized in that The method further comprises: In the case where the updated latest completion time is greater than the earliest start time of the successor task of the current task to be assigned, the earliest start time and the latest completion time of each task in the to-be-assigned and unassigned task lists in the assigned task list are updated based on the updated earliest start time and the latest completion time of the current task to be assigned; The degree of freedom parameter of each task in the to-be-assigned and unassigned task lists in the assigned task list is updated based on the earliest start time and the latest finish time of each task in the to-be-assigned and unassigned task lists in the updated assigned task list.
10. The method according to claim 1, It is characterized in that The unallocated tasks are located in a list of tasks to be allocated and unallocated; and the space resource reservation table is determined based on the number of remaining space resources in each core and the space resource requirements of the unallocated tasks, including: Determine the space resource reservation threshold parameter based on the number of remaining space resources in each core; The space resource reservation table is determined according to the space resource demand of the unassigned task and the space resource reservation threshold parameter.
11. The method according to claim 10, It is characterized in that The determining the space resource reservation table according to the space resource requirements of each task in the to-be-allocated and unallocated task lists and the space resource reservation threshold parameter comprises: Determine a second target task in the list of to-be-allocated and unallocated tasks, each of which has a space resource demand exceeding the space resource reservation threshold parameter; Counting the number of tasks in the list of to-be-allocated and unallocated tasks whose space resource requirements are greater than or equal to the space resource requirements of the second target task; The space resource reservation table is determined based on the space resource demand of the second target task and the number of tasks.
12. The method according to claim 1, It is characterized in that The method further comprises: Determine the actual completion time of each array unit based on the actual start time of the task to be assigned on each array unit of each core; Sort all array units on each core according to the actual completion time and number of remaining space resources of each array unit to obtain a first sorting result; Determine all array units that need to be added to the allocation array list to meet the space resource requirements of the task to be allocated based on the first sorting result; Obtain the target array unit with the latest actual completion time among all the array units; The actual completion time of the target array unit is used as the actual completion time of the task to be assigned on each core.
13. The method according to claim 1, It is characterized in that The method of allocating and deploying the task to be allocated to at least one array unit on at least one core based on the actual completion time of the task to be allocated on each core, the number of remaining space resources in each core, and the space resource reservation table comprises: Traversing each core of the multi-core storage-computing integrated accelerator to determine the actual completion time of the task to be assigned on each core; Sorting all the cores in the multi-core storage-computing integrated accelerator based on the actual completion time on each core, the number of remaining space resources in each core, and the space resource reservation table to obtain a second sorting result; Based on the second sorting result, determining whether the remaining space resources in the first-ranked core among the multi-cores meet the space resource requirements of the task to be assigned; When the remaining space resources in the first-ranked core among the multiple cores meet the space resource requirements of the task to be assigned, assigning and deploying the task to be assigned to at least one array unit on one core; When the remaining space resources in the first-ranked core among the multiple cores do not meet the space resource requirements of the task to be assigned, the task to be assigned is assigned and deployed to multiple array units on multiple cores.
14. A model arrangement device, It is characterized in that include: An acquisition unit, used to acquire a time occupancy table and a space occupancy table of each cross array unit in each core; the time occupancy table is used to record the occupied time period of each cross array unit; The space occupancy table is used to record the occupied space resource position of each cross array unit; A determination unit, configured to determine a task to be assigned based on each task in the model; and determining the actual completion time of allocating the task to be allocated to each core according to the arrival time of the latest input data of the task to be allocated among all the data input to the task to be allocated, the space resource requirement of the task to be allocated, the earliest start time of the successor task of the task to be allocated, and the time occupancy table and the space occupancy table; and determining a space resource reservation table based on the number of remaining space resources in each core and the space resource requirements of unassigned tasks; An allocation unit is used to allocate and deploy the task to be allocated to at least one array unit on at least one core based on the actual completion time of the task to be allocated on each core, the number of remaining space resources in each core and the space resource reservation table.
15. A model arrangement device, It is characterized in that include: A memory for storing executable instructions; The processor is used to implement the model arrangement method described in any one of claims 1 to 13 when executing the executable instructions stored in the memory.
16. A computer program product comprising a computer program, It is characterized in that When the computer program is executed by a processor, the model compilation method according to any one of claims 1 to 13 is implemented.
17. A computer-readable storage medium, It is characterized in that Executable instructions are stored, which are used to implement the model arrangement method described in any one of claims 1 to 13 when executed by a processor.
