Data processing system, task scheduling method and device, chip and electronic equipment

Abstract

The present disclosure provides a task processing system, a task scheduling method and device, a chip, electronic equipment and a storage medium, wherein the method comprises: in each scheduling period of a plurality of scheduling periods, performing: in response to triggering a current scheduling period, determining a candidate memory access task based on a plurality of neural networks respectively corresponding to a memory access task; based on the first execution cycle number corresponding to the candidate memory access task, the second execution cycle number of the calculation task corresponding to the candidate memory access task, and the first execution cycle total number of the current executable calculation task, determining a target memory access task from the candidate memory access task, and scheduling the determined target memory access task; based on the first execution cycle of the target memory access task and the second execution cycle number of the current schedulable calculation task, determining a target calculation task from the current schedulable calculation task, and scheduling the target calculation task.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of computer, and in particular, to a data processing system, a task scheduling method and device, a chip, and an electronic device. BACKGROUND

[0002] In related technologies, in the field of deep learning, when an electronic device runs a deep learning model, a deep learning framework deployed on the electronic device parses the deep learning model to obtain operators in the deep learning model, and sends the operators to an Artificial Intelligence (AI) accelerator card, a Graphics Processing Unit (GPU), or the like, which is an execution device. After receiving the operators sent by the deep learning framework, the execution device is responsible for scheduling and executing the operators.

[0003] An AI computing cloud platform can deploy a large number of execution devices. When receiving an inference task initiated by a terminal device, the AI computing cloud platform can distribute the inference task to each execution device for execution, and send an inference result to the terminal device. However, the current AI computing cloud platform has a low utilization rate of execution devices when performing a computing task. SUMMARY

[0004] The present disclosure at least provides a data processing system, a task scheduling method and device, a chip, an electronic device, and a storage medium.

[0005] The present disclosure provides a data processing system, comprising:

[0006] A control unit is configured to distribute task execution information corresponding to a plurality of neural networks to an execution unit. The task execution information includes an execution period corresponding to at least one to-be-executed task. The to-be-executed task includes a memory access task and a calculation task corresponding to the memory access task.

[0007] An operator execution unit is configured to, based on the received task execution information, perform the following in each current scheduling period in a plurality of scheduling periods: determining a candidate memory access task from a plurality of memory access tasks corresponding to a plurality of neural networks; determining a target memory access task from the candidate memory access task based on a first execution period number corresponding to the candidate memory access task, a second execution period number of a calculation task corresponding to the candidate memory access task, and a first execution period total number of a current executable calculation task, and scheduling the determined target memory access task; determining a target calculation task from the current schedulable calculation task based on a first execution period of the target memory access task and a second execution period number of the current schedulable calculation task, and scheduling the target calculation task.

[0008] In a possible implementation, before the control unit issues the task execution information corresponding to each of the plurality of neural networks to the execution unit, the control unit is further configured to:

[0009] In response to receiving the plurality of task processing requests, the control unit determines the plurality of neural networks corresponding to the plurality of task processing requests based on the plurality of task processing requests, parses each of the plurality of neural networks, and determines, from each of the plurality of neural networks, an operator corresponding to a target network layer allocated to the execution unit for processing; and the control unit generates the task execution information based on the operator.

[0010] In a possible implementation, the operator execution unit includes a scheduler and a plurality of operation units.

[0011] The scheduler is configured to, in response to receiving the task execution information, execute, based on the task execution information, in each of a plurality of scheduling periods: in response to triggering a current scheduling period, determining a candidate memory access task based on memory access tasks corresponding to the plurality of neural networks; determining, from the candidate memory access task, a target memory access task based on a first number of execution cycles of the candidate memory access task, a second number of execution cycles of a calculation task corresponding to the candidate memory access task, and a first total number of execution cycles of a currently executable calculation task, and scheduling the determined target memory access task; and determining, from a currently schedulable calculation task, a target calculation task based on a first execution cycle of the target memory access task and a second number of execution cycles of the currently schedulable calculation task, and scheduling the target calculation task.

[0012] The operation unit is configured to execute the scheduled target memory access task and the scheduled target calculation task.

[0013] In a possible implementation, the operator execution unit is further configured to, before the plurality of scheduling periods:

[0014] The operator execution unit is further configured to: obtain task execution information corresponding to a target network layer in each of the plurality of neural networks; the task execution information includes a number of to-be-executed tasks and an execution cycle corresponding to each of the to-be-executed tasks; the to-be-executed tasks include the memory access task and a calculation task corresponding to the memory access task; and the execution cycle includes the first execution cycle of the memory access task and the second execution cycle of the calculation task.

[0015] In a possible implementation, in response to the current scheduling period being the first scheduling period, the current scheduling period is triggered in the following manner:

[0016] The to-be-executed task corresponding to any of the plurality of neural networks is received in an idle state.

[0017] In response to the current scheduling period being a non-first scheduling period, the current scheduling period is triggered in the following manner:

[0018] The target memory access task determined in the previous scheduling period corresponding to the current scheduling period is executed and / or the target computing task determined in the history scheduling period is executed.

[0019] In a possible implementation, the operator execution unit, when determining the candidate memory access task based on the memory access tasks corresponding to the plurality of neural networks respectively, is configured to:

[0020] For each neural network in the plurality of neural networks, the candidate memory access task corresponding to the neural network is determined from the current unscheduled memory access task corresponding to the neural network.

[0021] In a possible implementation, the operator execution unit, when determining the candidate memory access task corresponding to each neural network in the plurality of neural networks from the current unscheduled memory access task corresponding to the neural network in response to the current scheduling period being a first scheduling period, is configured to:

[0022] The candidate memory access task of the first neural network in the current scheduling period is determined from the current unscheduled memory access task corresponding to the first neural network to which the target memory access task determined in the previous scheduling period belongs;

[0023] and the candidate memory access tasks other than the target memory access task in the previous scheduling period are determined as the candidate memory access tasks of the neural networks other than the first neural network in the current scheduling period.

[0024] In a possible implementation, the candidate memory access task has a plurality of candidate memory access tasks;

[0025] The operator execution unit, when determining the target memory access task from the candidate memory access tasks based on the first execution cycle number corresponding to the candidate memory access task, the second execution cycle number of the computing task corresponding to the candidate memory access task, and the first execution cycle total number of the currently executable computing task, is configured to:

[0026] The current candidate memory access task is determined from the plurality of candidate memory access tasks;

[0027] Based on the first execution cycle number corresponding to the current candidate memory access task, the second execution cycle number of the computing task corresponding to the current candidate memory access task, and the first execution cycle total number of the currently executable computing task, it is determined whether the current candidate memory access task satisfies a preset condition;

[0028] In response to determining that the current candidate memory access task satisfies the preset condition, the current candidate memory access task is determined as the target memory access task;

[0029] In response to determining that the current candidate memory access task does not satisfy the preset condition, returning to the step of determining the current candidate memory access task from the plurality of candidate memory access tasks.

[0030] In a possible implementation, when determining whether the current candidate memory access task satisfies the preset condition based on the first execution cycle number corresponding to the current candidate memory access task, the second execution cycle number of the computing task corresponding to the current candidate memory access task, and the total first execution cycle number of the currently executable computing task, the operator execution unit is configured to:

[0031] determining whether a storage space storing the data to be accessed corresponding to the memory access task satisfies a storage condition of the data to be accessed corresponding to the current candidate memory access task;

[0032] In response to satisfying the storage condition of the data to be accessed corresponding to the current candidate memory access task, determining whether the total first execution cycle number of the currently executable computing task is less than a preset execution cycle number threshold;

[0033] In response to the total first execution cycle number being less than the execution cycle number threshold, comparing the first execution cycle number corresponding to the candidate memory access task and the second execution cycle number of the computing task corresponding to the candidate memory access task;

[0034] In response to the first execution cycle number corresponding to the candidate memory access task being less than the second execution cycle number of the computing task corresponding to the candidate memory access task, determining that the current candidate memory access task satisfies the preset condition.

[0035] In a possible implementation, when determining whether the current candidate memory access task satisfies the preset condition based on the first execution cycle number corresponding to the current candidate memory access task, the second execution cycle number of the computing task corresponding to the current candidate memory access task, and the total first execution cycle number of the currently executable computing task, the operator execution unit is further configured to:

[0036] In response to the total first execution cycle number being greater than or equal to the execution cycle number threshold, determining that the current candidate memory access task satisfies the preset condition.

[0037] In a possible implementation, the operator execution unit is further configured to:

[0038] In response to determining that the target memory access task fails from the plurality of candidate memory access tasks, determining a first computing task to be preferentially executed from the currently executable computing task based on the second execution cycle number of the currently executable computing task and the first execution cycle number of the memory access task corresponding to the currently executable computing task;

[0039] in response to ending a current computing task being executed, executing the first computing task.

[0040] In a possible implementation, the operator execution unit, when ending a current computing task being executed, is configured to:

[0041] execute the current computing task being executed to the end;

[0042] or, release the current computing task being executed.

[0043] In a possible implementation, the operator execution unit, when releasing a current computing task being executed, is configured to:

[0044] determine whether to release the current computing task being executed, based on a second execution period number corresponding to the current computing task being executed, and a first execution period number of a memory access task corresponding to the current computing task being executed;

[0045] in response to determining to release the current computing task being executed, release the current computing task being executed.

[0046] In a possible implementation, the operator execution unit, before releasing the current computing task being executed, is further configured to:

[0047] record execution progress information of the current task being executed;

[0048] the execution progress information is used to, when rescheduling the current computing task being executed, continue to execute the current computing task being executed based on the execution progress information.

[0049] In a possible implementation, the current schedulable computing task includes at least one; the operator execution unit, when determining a target computing task from the current schedulable computing task based on a first execution period of the target memory access task and a second execution period number of the current schedulable computing task, is configured to:

[0050] determine a target first parameter of a current scheduling period based on the first execution period number of the target memory access task determined in the current scheduling period and a current first parameter; the current first parameter is determined based on a first execution period number of a historical target memory access task;

[0051] iteratively execute the following determination process until the current second parameter is greater than or equal to the target first parameter:

[0052] determine a current computing task from the at least one current schedulable computing task;

[0053] The target first parameter is compared with a current second parameter, wherein the current second parameter is determined based on a second execution period of a historical target computing task;

[0054] In response to the current second parameter being less than the target first parameter, the current computing task is determined as a target computing task; and

[0055] Based on a second execution period corresponding to the current computing task, the second parameter is updated to obtain a new second parameter;

[0056] The new second parameter is taken as a new current second parameter, and the step of comparing the target first parameter with the current second parameter is returned to.

[0057] In a second aspect, the embodiments of the present disclosure provide a task scheduling method, including: in each scheduling period of a plurality of scheduling periods, performing: in response to triggering a current scheduling period, determining a candidate memory access task based on a plurality of neural networks respectively corresponding to a memory access task; based on a first execution period number corresponding to the candidate memory access task, a second execution period number of a computing task corresponding to the candidate memory access task, and a first execution period total number of a current executable computing task, determining a target memory access task from the candidate memory access task, and scheduling the determined target memory access task; based on a first execution period of the target memory access task and a second execution period number of a current schedulable computing task, determining a target computing task from the current schedulable computing task, and scheduling the target computing task.

[0058] In a possible implementation, before the plurality of scheduling periods, the method further includes: obtaining task execution information corresponding to a target network layer in each neural network in the plurality of neural networks; the task execution information includes: a task quantity of a to-be-executed task, and an execution period corresponding to each to-be-executed task; the to-be-executed task includes: the memory access task and a computing task corresponding to the memory access task; and the execution period includes: the first execution period of executing the memory access task and the second execution period of executing the computing task.

[0059] In a possible implementation, in response to the current scheduling period being a first scheduling period, triggering the current scheduling period includes: receiving, in an unloaded state, a to-be-executed task corresponding to any neural network in the plurality of neural networks; in response to the current scheduling period being a non-first scheduling period, triggering the current scheduling period includes: a target memory access task determined by a previous scheduling period corresponding to the current scheduling period being executed and / or a target computing task determined by a historical scheduling period being executed.

[0060] In a possible implementation, the determining the candidate memory access task based on the memory access tasks corresponding to the plurality of neural networks respectively comprises: for each neural network in the plurality of neural networks, determining, from the current unscheduled memory access tasks corresponding to the neural network, a candidate memory access task corresponding to the neural network.

[0061] In a possible implementation, in response to the current scheduling period being the first scheduling period, the determining the candidate memory access task corresponding to each neural network in the plurality of neural networks from the current unscheduled memory access tasks corresponding to the neural network comprises: determining, from the current unscheduled memory access tasks corresponding to a first neural network to which the target memory access task determined in the previous scheduling period belongs, a candidate memory access task of the first neural network in the current scheduling period; and determining, as the candidate memory access tasks of the neural networks other than the first neural network in the current scheduling period, the candidate memory access tasks other than the target memory access task in the previous scheduling period.

[0062] In a possible implementation, the candidate memory access tasks are multiple; and the determining the target memory access task from the candidate memory access tasks based on the first execution cycle numbers of the candidate memory access tasks, the second execution cycle numbers of the computing tasks corresponding to the candidate memory access tasks, and the first execution cycle total number of the currently executable computing tasks comprises: determining a current candidate memory access task from the multiple candidate memory access tasks; determining whether the current candidate memory access task meets a preset condition based on the first execution cycle number of the current candidate memory access task, the second execution cycle number of the computing task corresponding to the current candidate memory access task, and the first execution cycle total number of the currently executable computing tasks; in response to determining that the current candidate memory access task meets the preset condition, determining the current candidate memory access task as the target memory access task; and in response to determining that the current candidate memory access task does not meet the preset condition, returning to the step of determining the current candidate memory access task from the multiple candidate memory access tasks.

[0063] In a possible implementation, the determining whether the current candidate memory access task satisfies the preset condition based on the first execution cycle number corresponding to the current candidate memory access task, the second execution cycle number of the computing task corresponding to the current candidate memory access task, and the total first execution cycle number of the currently executable computing task includes: determining whether a storage space storing data to be accessed corresponding to the memory access task satisfies a storage condition of the data to be accessed corresponding to the current candidate memory access task; in response to satisfying the storage condition of the data to be accessed corresponding to the current candidate memory access task, determining whether the total first execution cycle number of the currently executable computing task is less than a preset execution cycle number threshold; in response to the total first execution cycle number being less than the execution cycle number threshold, comparing the first execution cycle number corresponding to the candidate memory access task with the second execution cycle number of the computing task corresponding to the candidate memory access task; and in response to the first execution cycle number corresponding to the candidate memory access task being less than the second execution cycle number of the computing task corresponding to the candidate memory access task, determining that the current candidate memory access task satisfies the preset condition.

[0064] In a possible implementation, the determining whether the current candidate memory access task satisfies the preset condition based on the first execution cycle number corresponding to the current candidate memory access task, the second execution cycle number of the computing task corresponding to the current candidate memory access task, and the total first execution cycle number of the currently executable computing task further includes: in response to the total first execution cycle number being greater than or equal to the execution cycle number threshold, determining that the current candidate memory access task satisfies the preset condition.

[0065] In a possible implementation, the method further includes: in response to failing to determine the target memory access task from the plurality of candidate memory access tasks, determining a first computing task to be executed preferentially from the currently executable computing task based on the second execution cycle number of the currently executable computing task and the first execution cycle number of the memory access task corresponding to the currently executable computing task; and in response to ending the currently executing computing task, executing the first computing task.

[0066] In a possible implementation, the ending the currently executing computing task includes: completing execution of the currently executing computing task; or releasing the currently executing computing task.

[0067] In a possible implementation, the releasing the currently executing computing task includes: determining whether to release the currently executing computing task based on the second execution cycle number corresponding to the currently executing computing task and the first execution cycle number of the memory access task corresponding to the currently executing computing task; and in response to determining to release the currently executing computing task, releasing the currently executing computing task.

[0068] In a possible implementation, the releasing the currently-executing computing task further includes: recording execution progress information of the currently-executing task; and the execution progress information is used to continue executing the currently-executing computing task based on the execution progress information when the currently-executing computing task is rescheduled.

[0069] In a possible implementation, the currently-schedulable computing tasks include at least one; and the determining, from the currently-schedulable computing tasks, a target computing task based on the first execution period of the target memory access task and the second execution period of the currently-schedulable computing tasks includes: determining a target first parameter of a current scheduling period based on the first execution period of the target memory access task determined in the current scheduling period and a current first parameter; the current first parameter is determined based on the first execution period of the historical target memory access task; iteratively performing the following determining process until a current second parameter is greater than or equal to the target first parameter: determining a current computing task from the at least one currently-schedulable computing task; comparing the target first parameter and the current second parameter; the current second parameter is determined based on the second execution period of the historical target computing task; in response to the current second parameter being less than the target first parameter, determining the current computing task as the target computing task; and updating the second parameter based on the second execution period corresponding to the current computing task to obtain a new second parameter; and returning to the step of comparing the target first parameter and the current second parameter with the new second parameter as a new current second parameter.

[0070] In a third aspect, the optional implementation of the present disclosure further provides an operator execution apparatus, including:

[0071] The scheduler is configured to, in response to receiving task execution information, perform, in each scheduling period of a plurality of scheduling periods, the following based on the task execution information: in response to triggering a current scheduling period, determining candidate memory access tasks based on memory access tasks corresponding to a plurality of neural networks respectively; determining, from the candidate memory access tasks, a target memory access task based on a first execution period of the candidate memory access tasks, a second execution period of a computing task corresponding to the candidate memory access tasks, and a total number of first execution periods of currently-executable computing tasks, and scheduling the determined target memory access task; determining, from currently-schedulable computing tasks, a target computing task based on a first execution period of the target memory access task and a second execution period of the currently-schedulable computing tasks, and scheduling the target computing task.

[0072] The computing unit is configured to execute the scheduled target memory access task and the scheduled target computing task.

[0073] In a fourth aspect, the optional implementation of the present disclosure further provides a chip comprising the task processing system according to the first aspect or the task scheduling apparatus according to the third aspect.

[0074] In a fifth aspect, the optional implementation of the present disclosure further provides an electronic device comprising the chip according to the fourth aspect.

[0075] In a sixth aspect, the optional implementation of the present disclosure further provides a computer readable storage medium, which stores a computer program. When the computer program is run, the steps of the first aspect or any possible implementation of the first aspect are executed.

[0076] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, but not limiting the technical solutions of the present disclosure.

[0077] The embodiments of the present disclosure provide data prefetching through memory access tasks, and balanced control of execution cycles between corresponding computing tasks and the prefetching of to-be-accessed data, thereby achieving load balancing between computing tasks and memory access tasks, and improving the utilization of hardware resources.

[0078] In order to make the above objectives, features and advantages of the present disclosure more apparent, the following will describe a preferred embodiment in detail, and the accompanying drawings will be described as follows. BRIEF DESCRIPTION OF DRAWINGS

[0079] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following will briefly introduce the drawings needed to be used in the embodiments. The drawings herein are incorporated into the specification and form a part of the specification, which show the embodiments consistent with the present disclosure, and are used to explain the technical solutions of the present disclosure together with the specification. It should be understood that the following drawings only show some embodiments of the present disclosure, and therefore should not be considered as a limitation to the scope, and for those skilled in the art, other related drawings can also be obtained without creative labor.

[0080] Figure 1 Fig. 1 shows a schematic diagram of a specific example of a data processing system provided by the embodiments of the present disclosure;

[0081] Figure 2 Fig. 2 shows a flowchart of a task scheduling method provided by the embodiments of the present disclosure;

[0082] Figure 3 Fig. 3 shows a flowchart of a specific way of determining a target memory access task from candidate memory access tasks provided by the embodiments of the present disclosure;

[0083] Figure 4An example of specific execution of a computing task and a memory access task in different scheduling manners provided by the embodiments of the present disclosure is shown.

[0084] Figure 5 A specific schematic diagram of a task scheduling apparatus provided by the embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0085] To make the objectives, technical solutions, and advantages of the embodiments of the present disclosure clearer, the technical solutions in the embodiments of the present disclosure will be described clearly and completely below with reference to the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure and are not all the embodiments. The components of the embodiments of the present disclosure described and shown herein can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure is not intended to limit the scope of the claimed present disclosure, but only represents selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without creative work belong to the scope of protection of the present disclosure.

[0086] It is found through research that low-cost multi-tenant neural network execution is becoming one of the most important design goals of modern neural network accelerators. An AI computing cloud platform is a kind of cloud platform that provides multi-tenant neural networks; the artificial intelligence services it provides include the execution of many heterogeneous neural networks; the data processing tasks of different neural networks are issued to the same neural network accelerator; the same neural network accelerator uses a first-in, first-out or time-slice polling manner to process the data processing tasks issued by different neural networks. However, in the above processing manner, the load imbalance between the computing and memory access tasks of different neural networks causes a lot of idle time for hardware circuits used for computing and memory access bandwidth, resulting in a serious problem of insufficient utilization of hardware resources when running multiple neural networks.

[0087] Based on the above research, the present disclosure provides a task scheduling method, which pre-fetches data to be accessed by a memory access task and merges the execution of a computing task, to achieve load balancing between the computing task and the memory access task and improve the utilization of hardware resources.

[0088] The defects of the above solutions are the results of the inventors after practice and careful research, and therefore, the discovery process of the above problems and the solutions proposed by the present disclosure to solve the above problems in the following should be the contributions of the inventors to the present disclosure.

[0089] It should be noted that: similar reference numerals and letters represent similar items in the following drawings, and therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings.

[0090] To facilitate the understanding of the present embodiment, first, a data processing system disclosed by the present embodiment is introduced in detail. The data processing system provided by the present embodiment can be deployed in a server, can receive data processing requests respectively sent by multiple users, and can execute the data processing requests respectively sent by multiple users in parallel. It can also be deployed in a terminal device of a user, receive multiple data processing requests sent by the user, and execute multiple data processing requests sent by the user in parallel. The terminal device can be a user equipment (UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. In some possible implementation manners, the task processing device has a hardware structure that can execute the task scheduling method provided by the present embodiment. The hardware structure can realize the task scheduling method by calling computer readable instructions stored in a memory through a processor.

[0091] Referring to Figure 1 The present embodiment provides a specific example of a data processing system. It includes a control unit 10 and an operator execution unit 20.

[0092] In the operator execution unit 20, it includes a scheduler 21 and multiple operation units 22.

[0093] The control unit is configured to, in response to receiving multiple task processing requests, determine a neural network corresponding to each of the multiple task processing requests based on the multiple task processing requests, parse each of the neural networks, determine target network layers from each of the neural networks that can be allocated to the same artificial intelligence chip for processing, and then distribute task execution information of data processing tasks corresponding to the target network layers to the operation units 22.

[0094] The control unit includes, for example, any one of a computer device, a virtual machine, a central processing unit (CPU) in a computer device, and a graphics processing unit (GPU).

[0095] It should be noted that the multiple operators can be distributed to the same artificial intelligence chip at the same time period (here, the same time period corresponds to a short time period, which can be considered as being distributed to the same artificial intelligence chip at the same time); in addition, due to different times of receiving the task processing requests and different loads of each artificial intelligence chip at the same time, the multiple operators can also be distributed to the same artificial intelligence chip at different time periods.

[0096] In the embodiments of the present disclosure, the operators corresponding to the plurality of target network layers include, for example, a computation-intensive task and a memory-intensive task.

[0097] The computation-intensive task refers to a number of execution cycles consumed by a computation task being greater than a number of execution cycles consumed by a corresponding memory access task. The memory-intensive task refers to a number of execution cycles consumed by a computation task being less than a number of execution cycles consumed by a corresponding memory access task.

[0098] In this way, the balance between memory access and computation can be achieved more easily during scheduling, and the utilization of hardware resources can be improved.

[0099] In the embodiments of the present disclosure, each network layer of the neural network corresponds to an operator, and the types of the operators are different for different network layers. For example, a convolution layer corresponds to a convolution operator, and a fully connected layer corresponds to a fully connected operator.

[0100] In addition, the control unit can also split the operator into a plurality of data processing tasks. When the operator is issued to the scheduler, the task execution information determined after the operator is split into a plurality of data processing tasks can be issued to the scheduler.

[0101] The task execution information includes a number of to-be-executed tasks corresponding to the operator and an execution cycle corresponding to each to-be-executed task. The to-be-executed tasks include the memory access task and the computation task corresponding to the memory access task.

[0102] The memory access task is used to read data required for executing the corresponding computation task from the memory and store the read data into a cache of the artificial intelligence chip for storing the data.

[0103] The data can be an operator parameter, such as a convolution kernel of a convolution operator or a fully connected weight of a fully connected operator. The data can also be to-be-processed data, such as a feature map input to the convolution operator or the fully connected operator. In the embodiments of the present disclosure, the data required for executing the corresponding computation task is referred to as to-be-accessed data.

[0104] The computation task is, for example, used to execute the computation task corresponding to the operator by using the to-be-accessed data read from the memory by the memory access task. For example, if the operator is a convolution operator, the computation task is convolution processing. If the operator is a fully connected operator, the computation task is fully connected processing.

[0105] The execution cycle corresponding to the to-be-executed task includes the first execution cycle for executing the memory access task and the second execution cycle for executing the computation task.

[0106] The operator execution unit 20, for example, includes an Artificial Intelligence (AI) chip, a Graphics Processing Unit (GPU), and the like.

[0107] The scheduler 21 is configured to schedule the to-be-executed task based on the task scheduling method provided in the embodiments of the present disclosure in response to receiving the plurality of neural network subgraphs corresponding to the operators issued by the control unit. When scheduling the to-be-executed task, for example, the to-be-scheduled to-be-executed task can be stored in the corresponding task queue.

[0108] The operation unit 22 acquires the scheduled to-be-executed task and executes the acquired to-be-executed task.

[0109] Each operation unit is generally composed of a Processing Engine (PE) array and a local register file. In each PE, there are calculation elements such as multipliers for executing specific calculation tasks. Each operation unit can simultaneously process multiple data elements. For example, the PE array in a certain operation unit includes n×n PEs (the PEs are arranged in an n×n array, and the n×n PEs can simultaneously process n×n data elements; that is, the operation unit can simultaneously process at most n×n data elements. The task of processing each data element is called a subtask, and n×n subtasks completed by n×n PEs in an operation unit constitute a calculation task.

[0110] For example, when a feature map is used as the input data element of the operation unit, one data element refers to one feature element in the feature map.

[0111] The task of reading n×n data elements required to execute the calculation task constitutes a memory access task corresponding to the calculation task.

[0112] The number of calculation tasks corresponding to an operator is related to the configuration of the operation unit. Assuming that the number of data elements in the to-be-processed data corresponding to the operator includes H, and the number of PEs in the PE array in each operation unit is S, then the number of calculation tasks corresponding to the operator is an integer obtained by rounding up H / (n×n). For example, the to-be-processed data is image data, and the size of the image data is w×h×c, where w represents the image width, h represents the image height, and c represents the number of image channels. The number of data elements H in the corresponding to-be-processed data satisfies: H=w×h×c.

[0113] The control unit can split the operator into a plurality of to-be-executed tasks based on a type of the operator, a size of the to-be-processed data, a size of an operator parameter, and a size of the operation unit.

[0114] When splitting one operator into a plurality of to-be-executed tasks, it is assumed that there are M PE arrays, that is, M operation units. Each PE array has n×n PEs. The to-be-processed data is an input feature map, and the result data obtained after processing the to-be-processed data according to the operator is an output feature map.

[0115] The size of the input feature map is ih×iw×ic, where ih, iw, and ic respectively represent the height, width, and channel number of the input feature map. The size of the output feature map is oh×ow×oc, where oh, ow, and oc respectively represent the height, width, and channel number of the output feature map.

[0116] (1) If the type of the operator is a convolution operator, the corresponding network layer is a convolution layer, the operator parameter is a convolution kernel, and the size of the convolution kernel is: × × × ic: where, represents the number of convolution kernels, and =oc, k represents the length and width of the convolution kernel, and ic represents the channel number of the convolution kernel. When the operator is split, the feature map is split according to the convolution kernel to obtain a plurality of feature sub-maps. Different feature sub-maps have the same convolution kernel, and different PE arrays use the same convolution kernel to respectively perform convolution processing on different feature sub-maps. The convolution results obtained by each PE array are spliced according to the positions of the corresponding feature sub-maps in the feature map to obtain the convolution processing result of the feature map. Therefore, when the operator is split, the number of calculation tasks can be determined based on the size of the convolution kernel and the size of the PE array.

[0117] The number of calculation tasks, and the number of memory tasks satisfy: .

[0118] where, indicates that the weights of each operator are divided into n parts in the dimension of the number of convolution kernels, and indicates that the size of the convolution kernel is divided into n parts in the dimension of the size of the convolution kernel, and the product of the two is the number of calculation tasks.

[0119] Each calculation task needs n weight values for calculation. For the same calculation task, a plurality of PE arrays have the same weight element. However, the input feature maps of different PE arrays are different.

[0120] ​(2): If the operator type is full connection, the corresponding network layer is a full connection layer, the operator parameter is a full connection weight, and the size of the full connection weight of the full connection layer is: ih x iw x ic. Wherein, ih x iw x ic. Wherein, ih x iw x ic. Wherein, ih x iw x ic. Wherein, ih x iw x ic. Wherein, ih x iw x ic. Wherein,

[0121] In the splitting of the full connection operator, the number of calculation tasks and the number of memory access tasks satisfy: .

[0122] Wherein, ih x iw x ic. Wherein,

[0123] ih x iw x ic. Wherein,

[0124] The control unit can also estimate the first execution period of the memory access task and the second execution period of the calculation task based on the memory required by the calculation task when executed and the computing power information of the operation unit.

[0125] ​In a specific implementation, the computing power information of the operation unit includes, for example, the number of operations per second (ops) performed by the operation unit per second. The computing power information of each operation unit can be determined in the following manner: the number of PEs included in the operation unit is determined, where each PE can perform f operations per second, representing the computing power information of one PE; the computing power information of the operation unit, i.e., the total number of ops of one operation unit, is determined according to the number of PEs included in the operation unit and the computing power information of each PE, where the computing power information of the operation unit satisfies (n×n)×M×f. Where n×n represents the number of PEs in one operation unit, and M represents the number of operation units included in the operator execution unit.

[0126] When estimating the execution duration required for executing the computing task based on the memory required by the computing task during execution and the computing power information of the operation unit, for example, the first execution period corresponding to the memory access task can be determined based on the memory required by the operator during execution and the memory access bandwidth of the operation unit; and the second execution period of the computing task can be determined based on the computing power information required by each computing step in each subtask in the computing task and the computing power information of each operation unit.

[0127] The memory accessed by the computing task during execution can represent the amount of data processed by the computing task during execution. Since the type of the operator is known, the computing steps of each subtask during the execution of the operator can be determined, and thus the ops required for executing one subtask can be determined according to the ops required by the PE when executing one computing step. For example, the PE is a multiplier-adder that can be used to perform multiplication and addition operations, and each multiplication and addition operation requires 2 ops. The computing steps of each subtask are known, i.e., the specific computing steps of the computing task are known, and thus the ops consumed by each computing step can be determined according to the ops required by each computing step. Then, the computing duration of one subtask is determined according to the specific computing steps of the subtask. One computing task includes multiple subtasks executed in parallel by multiple PEs, for example, one computing task includes convolution processing of 4×4 feature elements using a 4×4 PE array, and thus the computing duration of each subtask, i.e., the computing duration of the computing task, i.e., the second execution period.

[0128] The memory access duration of the computing task is determined according to the amount of data accessed by the memory access task corresponding to one computing task and the access bandwidth.

[0129] The execution duration required for executing one computing task is determined based on the computing duration and the memory access duration.

[0130] Exemplarily, the operator is, for example, a matrix multiplication operator, and a matrix multiplication task in each calculation task satisfies CMN = AMK × BKN, where M and K respectively represent the size of the operand A; K and N respectively represent the size of the operand B; and each data needs to occupy b bits, and the data amount (memory access data amount) required by the operator to access the memory when performing the operator is (M × K + K × N + M × N) × b / 8 bytes. Wherein, M × K represents the data amount of the operand A that needs to be read from the memory; K × N represents the data amount of the operand B that needs to be read from the memory; and M × N represents the data amount of the calculation result to be stored in the memory. The number of multipliers (i.e., PEs) in each operation unit in the operator execution unit for executing the operator is M × K × N.

[0131] Therefore, the calculation time required for performing one calculation task of the operator is f × M × K × N ÷ total OPS number of operation units.

[0132] The memory access time required by the operation unit for performing one calculation task of the operator, i.e., the first execution period, is the memory access data amount ÷ memory access bandwidth, i.e., ((M × K + K × N + M × N) × b / 8) ÷ memory access bandwidth.

[0133] The control unit can split the operator into a plurality of to-be-executed tasks, generate a scheduling table corresponding to the operator, save the task execution information in the scheduling table, and issue the scheduling table to the scheduler.

[0134] The scheduler can perform a task scheduling process based on the scheduling table.

[0135] Referring to FIG. 1, Figure 2 The embodiment of the present disclosure provides a flowchart of a specific method for the scheduler to perform task scheduling, and steps S201-S203 are performed in each scheduling period in a plurality of scheduling periods, wherein:

[0136] S201: in response to triggering a current scheduling period, determining a candidate memory access task based on a plurality of neural networks respectively corresponding to memory access tasks;

[0137] S202: determining a target memory access task from the candidate memory access tasks based on a first execution period number corresponding to the candidate memory access tasks, a second execution period number of a calculation task corresponding to the candidate memory access tasks, and a first execution period total number of currently executable calculation tasks, and scheduling the determined target memory access task;

[0138] S203: determining a target calculation task from candidate calculation tasks corresponding to the current scheduling period based on a second execution period total number of currently scheduled target memory access tasks and a first execution period total number of currently executable calculation tasks, and scheduling the target calculation task.

[0139] The above S201-S203 are described in detail as follows.

[0140] For S201, the scheduling period refers to a period for scheduling a memory access task and a computing task. The scheduling of the memory access task, for example, is to determine a new memory access task to be executed after the end of a previous memory access task (at this time, there can be a memory access task waiting to be executed, or there can be no memory access task being executed). The scheduling of the computing task, for example, is to schedule at least one computing task that can be currently scheduled to be executed after determining to schedule a certain memory access task. Here, there are currently executable computing tasks and currently schedulable computing tasks; the currently schedulable computing task refers to a computing task that has not been scheduled but the corresponding memory access task has been executed; the currently executable computing task refers to determining a target computing task from the currently schedulable computing task, and after the target computing task is determined, it is scheduled and stored in the execution queue of the computing task, waiting for a computing unit to execute the currently executing computing task, and then obtaining a new executable computing task from the execution queue.

[0141] In this way, it is ensured that after the current computing task is executed, the memory access task corresponding to another computing task has been executed, and the computing unit does not need to wait and can directly use the to-be-accessed data obtained from the memory by the executed memory access task to execute the computing task. Thus, the pre-fetching of the to-be-accessed data by scheduling the memory access task is realized, and the computing unit continuously executes multiple scheduled computing tasks, so as to minimize the bandwidth resources of memory access and the computing resources of the computing unit in an idle state, thereby improving the utilization of hardware resources in the operator execution unit.

[0142] In a specific implementation, for multiple scheduling periods:

[0143] A: in response to the current scheduling period being the first scheduling period, triggering the current scheduling period includes:

[0144] receiving a to-be-executed task corresponding to any of the plurality of neural networks in an idle state.

[0145] The idle state refers to that the operator execution unit 20 is currently not loaded with any to-be-executed task. The idle state may occur in any of the following situations: after the data processing system is started and no to-be-executed task is assigned; all assigned to-be-executed tasks have been processed and no new to-be-executed task is assigned.

[0146] In the above cases, the operator execution unit 20 can be in an idle state.

[0147] If the scheduler 21 is in an idle state at the operator execution unit 20, if any task corresponding to a neural network to be executed is received, the first scheduling period is triggered.

[0148] B: in response to the current scheduling period being a non-first scheduling period, triggering the current scheduling period, including:

[0149] The target memory access task determined by the previous scheduling period corresponding to the current scheduling period is executed and / or the target computing task determined by the historical scheduling period is executed.

[0150] If the target memory access task determined by the previous scheduling period corresponding to the current scheduling period is executed, the next memory access task can be executed, in which case the current scheduling period can be triggered, thereby realizing continuous prefetching of data to be accessed and improving the utilization of access bandwidth by the operator execution unit 20.

[0151] If the target computing task determined by the historical scheduling period is executed, the data to be accessed corresponding to the executed target computing task is released from the register, so that there is storage space in the register to store new data to be accessed. At this time, the current scheduling period can be triggered, so that the storage space for storing data to be accessed can be fully utilized, the efficiency and timeliness of prefetching data to be accessed can be ensured, the waiting situation of the PE array due to the delay in prefetching data to be accessed can be reduced, and the hardware utilization of the operator execution unit 20 can be improved.

[0152] After triggering the current scheduling period, candidate memory access tasks corresponding to the current scheduling period can be determined based on the memory access tasks corresponding to the plurality of neural networks, respectively.

[0153] In specific implementation, for each neural network in the plurality of neural networks, a candidate memory access task corresponding to the neural network can be determined from the current unscheduled memory access task corresponding to the neural network. For example, if there are 4 neural networks, there are 4 candidate memory access tasks determined by the current scheduling period, respectively from the 4 neural networks.

[0154] In response to the current scheduling period being a first scheduling period, for each neural network, a memory access task required by the first executed computing task in the target network layer corresponding to the neural network can be determined as the candidate memory access task corresponding to the neural network.

[0155] In response to the current scheduling period being a non-first scheduling period, for each neural network in the plurality of neural networks, a candidate memory access task corresponding to the neural network can be determined from the current unscheduled memory access task corresponding to the neural network.

[0156] For example, assuming that there are v memory access tasks for a target network layer of a neural network, and one corresponding calculation task for each of the v memory access tasks. The v memory access tasks are U1~Uv. For the U1~Uv memory access tasks, U1~Uv can be sequentially taken as candidate memory access tasks. In the first scheduling period, U1 is taken as the candidate memory access task of the neural network. If U1 is determined as the target memory access task to be scheduled in the scheduling period, then in the second scheduling period, U2~Uv are the current unscheduled memory access tasks, and therefore U2 is determined as the candidate memory access task in the second scheduling period.

[0157] Here, the control unit can issue a scheduling table to the scheduler. In the scheduling table, the number of tasks to be executed and the execution period corresponding to each task to be executed can be saved, and the identifier of the next memory access task to be taken as a candidate memory access task in the current unscheduled memory access task and the address information of the to-be-accessed data in the memory corresponding to the next memory access task to be taken as a candidate memory access task can also be saved. Assuming that the identifiers of all n memory access tasks corresponding to an operator are 1~n, for the first scheduling period, the current unscheduled memory access task is all the memory access tasks corresponding to the operator, and the next memory access task to be taken as a candidate memory access task in the current unscheduled memory access task is the memory access task with identifier 1. If the current unscheduled memory access task is part of the memory access tasks corresponding to the operator, and the identifier of the previous scheduled memory access task is j, then the next memory access task to be taken as a candidate memory access task in the current unscheduled memory access task is the memory access task with identifier j+1.

[0158] The scheduler can determine the candidate memory access task of the current scheduling period according to the scheduling table.

[0159] In addition, in response to the current scheduling period being the first scheduling period, the candidate memory access task corresponding to each neural network in the plurality of neural networks is determined from the current unscheduled memory access task corresponding to the neural network. For example, the candidate memory access task of the first neural network in the current scheduling period can be determined from the current unscheduled memory access task of the first neural network to which the target memory access task determined in the previous scheduling period belongs;

[0160] and the other candidate memory access tasks of the first neural network in the previous scheduling period except the target memory access task are determined as the candidate memory access tasks of the other neural networks in the current scheduling period except the first neural network.

[0161] For example, a candidate memory access task queue can be determined for the candidate memory access tasks. The candidate memory access task queue stores all the candidate memory access tasks determined in the current scheduling period. The memory access tasks can be arranged in the order of the candidate memory access tasks entering the queue. When the candidate memory access task queue is empty, for example, in the first scheduling period, in the first scheduling period, the candidate memory access tasks required in the first scheduling period can be determined from the memory access tasks corresponding to the neural networks in the order of the task receiving order of the neural networks, and placed in the candidate memory access task queue. In a non-first scheduling period, if a memory access task in the candidate memory access task queue is determined as a target memory access task in the last scheduling period, the target memory access task is deleted from the candidate memory access task queue, and a new candidate memory access task is determined in the current scheduling period and placed at the tail of the candidate memory access task queue.

[0162] In addition, a candidate computing task queue can also be generated for the schedulable computing tasks. The current schedulable computing tasks are stored in the candidate computing task queue. When a target computing task needs to be determined from the schedulable computing tasks, the schedulable computing tasks can be determined in turn as the target computing task according to the order of the schedulable computing tasks in the candidate computing task queue. In each scheduling period, the determined target computing task can be 0, 1 or more.

[0163] The position of the schedulable computing task in the candidate computing task queue can be determined based on the order in which the memory access tasks corresponding to the schedulable computing tasks are scheduled. For example, the earlier the corresponding memory access task is scheduled, the higher the schedulable computing task is ranked in the candidate computing task queue, and the greater the possibility of being determined as a target computing task.

[0164] If a candidate memory access task is determined as a target memory access task in the current scheduling period, the target memory access task can be deleted from the candidate memory access task queue after being scheduled. In addition, a new memory access task can be determined from the scheduling table of the neural network L corresponding to the target memory access task in the above manner, the newly determined memory access task is used as the candidate memory access task corresponding to the neural network L in the next scheduling period, and is stored in the candidate memory access task queue. At the same time, the other candidate memory access tasks remaining in the candidate memory access task queue are the candidate memory access tasks corresponding to other neural networks, respectively.

[0165] In addition, after the target memory access task is determined in the current scheduling period and scheduled, the scheduling table corresponding to the target memory access task can be updated to record the identifier of the next memory access task to be used as a candidate memory access task, and the address information of the to-be-accessed data corresponding to the next memory access task to be used as a candidate memory access task in the memory.

[0166] Here, with the increase or decrease of the neural networks to which the tasks to be executed assigned to the operator execution unit 20 belong, the candidate memory tasks determined by the scheduling period also increase or decrease.

[0167] For example, if there are 4 neural networks to which the tasks to be executed assigned to the operator execution unit 20 belong for the i-th scheduling period, there are 4 candidate memory tasks determined by the i-th scheduling period; if a new task to be executed corresponding to a neural network is assigned to the operator execution unit 20 for the i+1-th scheduling period, there are 5 candidate memory tasks determined by the i+1-th scheduling period. If the tasks to be executed corresponding to a neural network in the operator execution unit 20 are all executed in the i+1-th scheduling period for the i+2-th scheduling period, there are 4 candidate memory tasks determined by the i+2-th scheduling period.

[0168] After determining the candidate memory tasks corresponding to the current scheduling period, the target memory task can be determined from the candidate memory tasks.

[0169] For S202 described above, referring to FIG. 8, the embodiment of the disclosure also provides a specific example of determining a target memory task from candidate memory tasks, including: Figure 3 S301: determining a current candidate memory task from a plurality of candidate memory tasks;

[0170] S302: determining whether the current candidate memory task meets a preset condition based on a first execution period number corresponding to the current candidate memory task, a second execution period number of a calculation task corresponding to the current candidate memory task, and a first total execution period number of executable calculation tasks;

[0171] S303: in response to determining that the current candidate memory task meets the preset condition, determining the current candidate memory task as the target memory task;

[0172] S304: in response to determining that the current candidate memory task does not meet the preset condition, returning to the step of determining a current candidate memory task from a plurality of candidate memory tasks.

[0173] In a specific implementation, not all candidate memory tasks can be target memory tasks.

[0174]

[0175] ​Therefore, in 301, the current candidate memory access task can be determined based on the position of each candidate memory access task in the candidate memory access task queue, and then it is determined whether the current candidate memory access task meets the preset condition. If yes, the current candidate memory access task is determined as the target memory access task; if not, it is determined again whether a current candidate memory access task meets the preset condition. Until the target memory access task is determined or it is determined that all current candidate memory access tasks do not meet the preset condition.

[0176] When it is determined whether the current candidate memory access task meets the preset condition, the following method can be used, for example.

[0177] In step 3021, it is determined whether the storage space storing the to-be-accessed data corresponding to the memory access task meets the storage condition of the to-be-accessed data corresponding to the current candidate memory access task.

[0178] Here, the storage space includes, for example, a register in the operator execution unit for storing the to-be-accessed data. If the size of the free space in the register can accommodate the to-be-accessed data corresponding to the current candidate memory access task, it is indicated that the current candidate memory access task meets the storage condition. If not, it is indicated that the current candidate memory access task does not meet the storage condition.

[0179] In the determination of the storage condition, the first data amount of the to-be-accessed data corresponding to the current candidate memory access task and the current free space of the storage space storing the to-be-accessed data corresponding to the memory access task are determined, and the first data amount and the current free space are compared. If the first data amount is less than or equal to the current free space, it is indicated that the storage condition is met. If the first data amount is greater than the current free space, it is indicated that the storage condition is not met.

[0180] In another embodiment of the present disclosure, the execution cycle number RM_C required to fill the remaining register capacity in the current scheduling period can also be recorded in each scheduling period. When it is required to determine whether a certain candidate memory access task meets the storage condition, the first execution cycle number corresponding to the candidate memory access task and the RM_C are compared. If the first execution cycle number is less than or equal to the RM_C, it is indicated that the storage condition is met. If the first execution cycle number is greater than the RM_C, it is indicated that the storage condition is not met.

[0181] With this scheme, only the RM_C of each scheduling period needs to be updated based on the first execution cycle number corresponding to the target memory access task after the target memory access task is determined in each scheduling period. The RM_C decreases with the determination of the target memory access task. The decreased value is the same as the first execution cycle number corresponding to the determined target memory access task.

[0182] In addition, after a certain computing task is executed, the remaining capacity of the register increases because the to-be-accessed data corresponding to the executed computing task is released from the register. Therefore, the number of execution cycles required to fill the remaining register capacity, RM_C, also increases. The value of RM_C increases by the first execution cycle number of the access task corresponding to the released to-be-accessed data.

[0183] Step 3022: In response to the storage condition of the to-be-accessed data corresponding to the current candidate access task being met, determining whether the first execution cycle total number of the currently executable computing task is less than a preset execution cycle number threshold.

[0184] Here, when determining the execution cycle number threshold, the following process can be performed in multiple iteration periods: in the first iteration period, set an original execution cycle number threshold, and based on the original execution cycle number threshold, perform inference tasks of multiple neural networks by using a data processing system to determine the execution duration of the inference tasks in the first iteration period; in the ith iteration period other than the first iteration period, adjust the execution cycle number threshold determined in the first iteration period to obtain a new execution cycle number threshold corresponding to the current iteration period, and then based on the new execution cycle number threshold of the current iteration period, perform inference tasks of multiple neural networks by using a data processing system to determine the execution duration of the inference tasks in the ith iteration period; compare the execution duration of the (i-1)th iteration period and the execution duration of the ith iteration period, and based on the comparison result, adjust the execution cycle number determined in the (i-1)th iteration period to obtain the execution cycle number threshold of the ith iteration period; perform multiple iteration periods in this way until the execution duration no longer changes in the multiple iteration periods, and determine the execution cycle number determined in the last iteration period as the execution cycle number threshold.

[0185] The currently executable computing task refers to a computing task that is determined to be a target computing task to be scheduled in a historical scheduling period but has not been executed. The currently executable computing task can exist or can not exist; in the case where the currently executable computing task exists, the first execution cycle total number corresponding to all currently executable computing tasks can be obtained by adding the second execution cycle numbers corresponding to all currently executable computing tasks.

[0186] Step 3023: In response to the first execution cycle total number being less than the execution cycle number threshold, comparing the first execution cycle number corresponding to the candidate access task and the second execution cycle number of the computing task corresponding to the candidate access task.

[0187] Step 3024: In response to the first execution cycle number corresponding to the candidate memory access task being less than the second execution cycle number of the computing task corresponding to the candidate memory access task, it is determined that the current candidate memory access task meets the preset condition.

[0188] Here, steps 3022, 3023 and 3024 cooperate with each other. In the case that the first total execution cycle number corresponding to the current executable computing task is too small, in order to ensure that the data to be accessed corresponding to other computing tasks is prepared before the current executable computing task is executed, the candidate memory access task with the first execution cycle number less than the second execution cycle number of the corresponding computing task needs to be selected from all current candidate memory access tasks. After the candidate memory access task is determined as the target memory access task and is scheduled, the increase in the first total execution cycle number corresponding to the current executable computing task is more than the increase in the second total execution cycle number of the current scheduled target memory access task after the target memory access task is scheduled. Therefore, in terms of execution cycle, the computing task and the memory access task are balanced, and the situation that the data to be accessed corresponding to the next computing task to be executed is not prepared before the current executable computing task is executed is avoided.

[0189] In addition, in determining whether the current candidate memory access task meets the preset condition, step 3022 is further included:

[0190] Step 3025: In response to the first total execution cycle number being greater than or equal to the execution cycle number threshold, it is determined that the current candidate memory access task meets the preset condition.

[0191] Here, in the case that the first total execution cycle number corresponding to the current executable computing task is large (compared with the execution cycle number threshold), the demand for balancing the execution cycle of the computing task and the memory access task by using the target memory access task is weak. Therefore, the current candidate memory access task can be determined as the target memory access task. In this case, the situation that the data to be accessed corresponding to the next computing task to be executed is not prepared before the current executable computing task is executed is also unlikely to occur.

[0192] In addition, if it is determined that all current candidate memory access tasks do not meet the preset condition, that is, it is determined that the target memory access task is failed to be determined from the multiple candidate memory access tasks corresponding to the current scheduling period, the scheduling period can be exited until the currently executing computing task is executed, triggering the next scheduling period.

[0193] In another embodiment of the present disclosure, in order to more fully utilize the bandwidth resources of the operator execution unit, in response to determining that the target memory access task fails from the plurality of candidate memory access tasks, a first computing task that is to be executed preferentially can be determined from the current executable computing task based on the second execution cycle number of the current executable computing task and the first execution cycle number of the memory access task corresponding to the current executable computing task.

[0194] In response to ending the computing task that is currently being executed, the first computing task is executed.

[0195] In determining the first computing task that is to be executed preferentially from the current executable computing task, a computing task that satisfies a first screening condition can be determined from the plurality of executable computing tasks as the first computing task, and the first screening condition can include at least one of the following, for example:

[0196] a1: The second execution cycle number corresponding to the current executable computing task is less than the first execution cycle number of the memory access task corresponding to the current executable computing task. In this way, it is ensured that the computing task whose to-be-accessed data occupies a larger storage space will be executed preferentially, so as to quickly release the larger storage space.

[0197] a2: The second execution cycle number corresponding to the current executable computing task is less than a preset first execution cycle number threshold. In this way, it is ensured that the first computing task has less execution time, so that the storage space occupied by the to-be-accessed data corresponding to the current executable computing task can be quickly released.

[0198] a3: The first execution cycle number of the memory access task corresponding to the current executable computing task is less than a preset second execution cycle number threshold. In this way, it is ensured that after the first computing task is executed, a larger storage space can be released.

[0199] After the computing task that is currently being executed is ended, the storage space occupied by the to-be-accessed data corresponding to the computing task is released, so that the storage space can be left empty to store the to-be-accessed data corresponding to other computing tasks.

[0200] Ending the computing task that is currently being executed can include any of the following, for example:

[0201] The current computing task that is being executed is executed completely, and the current computing task that is being executed is released.

[0202] In a specific implementation, which way to end the computing task that is currently being executed can be determined based on the actual situation of the computing task that is currently being executed.

[0203] An example: whether to release the currently executing computing task can be determined based on the second execution cycle number corresponding to the currently executing computing task and the first execution cycle number of the memory access task corresponding to the currently executing computing task.

[0204] When determining, whether the currently executing computing task meets a second screening condition can be determined, such as meeting, it is determined that the currently executing computing task is released. The second screening condition, for example, includes at least one of the following:

[0205] b1: the second execution cycle number corresponding to the currently executing computing task is greater than the preset third execution cycle threshold. In this way, the computing task that needs to be executed for a long time can be released first to execute the first computing task that needs to be executed for a short time, so that the storage space occupied by the corresponding to-be-accessed data can be quickly released.

[0206] b2: the first execution cycle number of the memory access task corresponding to the currently executing computing task is greater than the preset fourth execution cycle threshold. In this way, the computing task with a larger storage space occupied by to-be-accessed data can be released, so that a larger storage space can be released.

[0207] b3: the remaining execution cycle number required by the currently executing computing task is greater than the preset fifth execution cycle threshold. Here, since the currently executing computing task has been executed for a certain period of time, the execution cycle number required is the remaining execution cycle number; if the remaining execution cycle number is less than or equal to the preset fifth execution cycle threshold, it is represented that even if the currently executing computing task is executed next, it does not need to spend too much time, so the executing computing task can be selected to be executed and ended. Conversely, if the remaining execution cycle number is greater than the preset fifth execution cycle threshold, it is represented that a longer time is required to execute the computing task, so that the storage space cannot be released, and therefore, the executing computing task can be released, and the determined first computing task is executed to ensure that the storage space occupied by the corresponding to-be-accessed data is quickly released.

[0208] In another embodiment of the present disclosure, after releasing the currently executing computing task, the execution progress information of the currently executing task is also recorded;

[0209] The execution progress information is used to continue executing the currently executing computing task based on the execution progress information when the currently executing computing task is rescheduled.

[0210] Here, the execution progress information, for example, includes: address information corresponding to the to-be-accessed data in the memory. The new address information can be new address information determined based on the current execution progress of the currently executing computing task.

[0211] In addition, the progress information further includes, for example, the state of each PE in the operation unit, and / or the generated intermediate data (which can be data or a storage address in the memory).

[0212] In this way, when the released computing task is recalculated, the execution progress can be restored according to the execution progress information, and the subsequent execution process of the computing task being executed can be continued.

[0213] In another embodiment of the present disclosure, after a computing task is released, the current candidate memory access task corresponding to the neural network to which the computing task belongs and having a dependency relationship with the released computing task is deleted from the candidate memory access task queue, and the released computing task is placed in the candidate memory access task queue, so as to ensure the correct execution of the to-be-executed task.

[0214] For S203 described above: after the target memory access task of the current scheduling period is determined, there is at least one schedulable computing task corresponding to the current scheduling period, and the at least one schedulable computing task includes the computing task corresponding to the target memory access task.

[0215] If there is a schedulable computing task that is not determined as the target computing task corresponding to the previous scheduling period in the previous scheduling period of the current scheduling period, there are multiple schedulable computing tasks of the current scheduling period.

[0216] In a specific implementation, the target computing task can be determined from the schedulable computing tasks corresponding to the current scheduling period in the following manner, for example:

[0217] The target first parameter of the current scheduling period is determined based on the first execution period number of the target memory access task determined for the current scheduling period and the current first parameter, wherein the current first parameter is determined based on the first execution period number of the historical target memory access task.

[0218] For example, the first parameter is used to track the total number of the first execution period corresponding to all the target memory access tasks that have been scheduled. Before starting the scheduling, the first parameter is assigned a preset value, for example, 0. Each time a candidate memory access task is determined as a target memory access task, the value of the first parameter is increased by the first execution period number corresponding to the target memory access task.

[0219] The determination process is iteratively performed until the current second parameter is greater than or equal to the target first parameter.

[0220] The current computing task is determined from the at least one schedulable computing task.

[0221] The target first parameter is compared with a current second parameter, wherein the current second parameter is determined based on a second execution period of a historical target computing task;

[0222] In response to the current second parameter being less than the target first parameter, the current computing task is determined as a target computing task; and

[0223] The second parameter is updated based on a second execution period corresponding to the current computing task to obtain a new second parameter;

[0224] The new second parameter is taken as a new current second parameter, and the step of comparing the target first parameter with the current second parameter is returned to.

[0225] In a specific implementation, the first parameter is used to track a total number of execution periods of all memory access tasks that have been scheduled currently, and the second parameter is used to track a total number of execution periods of all computing tasks that have been scheduled currently.

[0226] In this embodiment, if the first parameter is greater than the second parameter, it proves that the speed of pre-fetching to-be-accessed data by the memory access task is faster than the speed of scheduling and executing the computing task, so the number of the scheduled computing tasks can be continuously increased, and the data pre-fetching speed can be ensured to be greater than the computing speed, thereby reducing the probability of the idle operation unit.

[0227] If the first parameter is equal to or less than the second parameter, it proves that the speed of pre-fetching to-be-accessed data by the memory access task is basically flat with the speed of scheduling and executing the computing task, or the speed of pre-fetching to-be-accessed data by the memory access task is less than the speed of scheduling and executing the computing task. In this case, if the number of the scheduled computing tasks is continuously increased, it is likely that the to-be-accessed data required for executing the next computing task has not been prepared when the scheduled computing task is executed, so that the operation unit needs to wait until the to-be-accessed data is prepared before executing the next computing task, which causes the idle operation unit and the waste of computing resources.

[0228] Therefore, in the embodiment of the present disclosure, when determining whether the current computing task is determined as a target computing task that needs to be scheduled, the target first parameter and the current second parameter are compared. If the target first parameter is greater than the current second parameter, it means that the number of the scheduled computing tasks can be continuously increased, so the current computing task is determined as the target computing task. If the target first parameter is less than the current second parameter, it means that if the number of the scheduled computing tasks is continuously increased, the operation unit is likely to be idle, which causes the waste of computing resources, so the current computing task cannot be determined as the target computing task.

[0229] In this way, the above process can maximize the load balancing of the execution cycles between computing tasks and memory access tasks, reducing the waste of hardware resources.

[0230] This embodiment of the disclosure determines candidate memory access tasks based on memory access tasks corresponding to multiple neural networks. Then, based on the first execution cycle count of the candidate memory access tasks, the second execution cycle count of the corresponding computational tasks, and the total number of first execution cycles of currently executable computational tasks, a target memory access task is determined from the candidate memory access tasks, and the determined target memory access task is scheduled. Next, based on the first execution cycle of the target memory access task and the second execution cycle count of currently schedulable computational tasks, a target computational task is determined from the currently schedulable computational tasks, and the target computational task is scheduled. This achieves balanced control of the execution cycles between data prefetching through memory access tasks and executing corresponding computational tasks using prefetched data to be accessed, thereby achieving load balancing between computational tasks and memory access tasks and improving the utilization of hardware resources.

[0231] See Figure 4 As shown in the embodiments of this disclosure, the specific execution of memory access tasks and computation tasks under different scheduling methods is also provided. In this example, there are four neural networks: neural network 1, neural network 2, neural network 3, and neural network 4.

[0232] like Figure 4 As shown in Figure (a), the execution of memory access tasks and computation tasks is illustrated in the Round Robin (RR) scheduling mode. In this scheduling mode, one computation task and one memory access task are executed in a given time slice (the two are not directly related). Within that time slice, if the memory access task completes before the computation task, the memory access task enters a waiting state and is not scheduled for the next memory access task; similarly, if the computation task completes before the memory access task within the same time slice, the computation task enters a waiting state and is not scheduled for the next computation task.

[0233] In time slice Part-1, the memory access task corresponding to neural network 2 is executed, along with the computation task corresponding to neural network 1. After the memory access task corresponding to neural network 2 is completed, it enters a waiting state. In time slice Part-2, the memory access task corresponding to neural network 3 is executed, along with the computation task corresponding to neural network 2. After the computation task corresponding to neural network 2 is completed, the memory access task corresponding to neural network 3 has not yet been completed, and the relevant circuits used to execute the computation task enter a waiting state. It can be seen that under this scheduling method, both computation and memory access tasks have a large amount of waiting time, resulting in a significant waste of memory access bandwidth in the operator execution unit and computational resources of the PE array.

[0234] like Figure 4 As shown in Figure (b), the execution of memory access tasks and computation tasks is illustrated under the normal memory access data prefetching scheduling method. Under this scheduling method, although memory access tasks of different neural networks can be executed continuously, that is, memory access bandwidth is not wasted, there may be cases where the corresponding memory access tasks are not completed when executing computation tasks. It can be seen that under this scheduling method, although there is no waiting time for memory access tasks, there is still a lot of waiting time for computation tasks, resulting in a great waste of computing resources in the PE array.

[0235] like Figure 4 As shown in (c), the execution of memory access tasks and computation tasks is illustrated in the scheduling method of prefetching memory access data and merging computation tasks provided in this embodiment of the present disclosure. In this scheduling method, the scheduling order of computation tasks and memory access tasks can be flexibly determined using the task scheduling method provided in this embodiment of the present disclosure. After the target memory access task is scheduled in each scheduling cycle, the target computation task is scheduled, and the scheduled computation tasks are stored in the execution queue Avl.CBs. When the computing unit executes a computation task, it sequentially retrieves the target computation task from the execution queue according to the execution order of the computation tasks and executes it.

[0236] For example, in the first scheduling cycle, memory access task 1 of neural network 1 is scheduled, and the computation task 3 corresponding to memory access task 1 is stored in the execution queue.

[0237] In the second scheduling cycle, memory access task 2 of neural network 2 is scheduled, and the computation task 6 corresponding to memory access task 2 is stored in the execution queue as the scheduled computation task.

[0238] In the third scheduling cycle, the memory access task 4 of neural network 1 is scheduled, and the computation task 7 corresponding to the memory access task 4 is stored in the execution queue as the scheduled computation task.

[0239] In the fourth scheduling cycle, the memory access task 5 of neural network 3 is scheduled, and the computation task 11 corresponding to the memory access task 5 is stored in the execution queue as the scheduled computation task.

[0240] In the fifth scheduling cycle, the memory access task 8 of neural network 2 is scheduled, and the computation task 12 corresponding to the memory access task 8 is stored in the execution queue as the scheduled computation task.

[0241] In the process of executing each computing task in the execution queue: after the memory access task 1 ends, the computing task 3 is executed, and at this time, the computing task 3 is deleted from the execution queue; after the computing task 3 is executed, the memory access task 2 corresponding to the computing task 6 has been executed, and thus the computing task 6 can be executed, and in the process of executing the computing task 6, the computing task 6 is deleted from the execution queue; after the computing task 6 is executed, the memory access task 4 corresponding to the computing task 7 has been executed, and thus the computing task 7 can be executed, and in the process of executing the computing task 7, the computing task 7 is deleted from the execution queue; after the computing task 7 is executed, the memory access task 5 corresponding to the computing task 11 has been executed, and thus the computing task 11 can be executed, and in the process of executing the computing task 11, the computing task 11 is deleted from the execution queue.

[0242] It can be seen that, except for waiting for a certain time before the first computing task starts, there is no memory access task waiting and computing task waiting in the subsequent process. It can be seen that the scheduling manner provided by the embodiment of the disclosure fully utilizes the memory bandwidth in the operator execution unit and the computing resources of the PE array, and improves the utilization rate of hardware resources.

[0243] Those skilled in the art can understand that, in the above method of the specific implementation manner, the writing order of each step does not mean a strict execution order and does not constitute any limitation on the implementation process, and the specific execution order of each step should be determined according to its function and possible internal logic.

[0244] Based on the same inventive concept, the embodiment of the disclosure also provides a task scheduling device corresponding to the task scheduling method. Since the principle of solving problems by the device in the embodiment of the disclosure is similar to the above-mentioned task scheduling method of the embodiment of the disclosure, the implementation of the device can be referred to the implementation of the method, and the repeated parts will not be described.

[0245] Referring to Figure 5 FIG. 5 shows a schematic diagram of a task scheduling device provided by an embodiment of the disclosure, and the device includes a scheduler 51 and an operation unit 52; wherein,

[0246] The scheduler 51 is configured to, in response to receiving the task execution information, perform, in each of the plurality of scheduling periods, the following operations based on the task execution information: in response to triggering a current scheduling period, determining candidate memory access tasks based on memory access tasks corresponding to the plurality of neural networks respectively; determining, from the candidate memory access tasks, a target memory access task based on a first number of execution cycles corresponding to the candidate memory access task, a second number of execution cycles of a calculation task corresponding to the candidate memory access task, and a first total number of execution cycles of currently executable calculation tasks, and scheduling the determined target memory access task; determining, from currently schedulable calculation tasks, a target calculation task based on a first execution cycle of the target memory access task and a second number of execution cycles of the currently schedulable calculation tasks, and scheduling the target calculation task.

[0247] The operation unit 52 is configured to execute the scheduled target memory access task and the scheduled target calculation task.

[0248] In a possible implementation, the scheduler 51 is further configured to, before the plurality of scheduling periods:

[0249] obtain task execution information corresponding to a target network layer in each of the plurality of neural networks; the task execution information includes a number of to-be-executed tasks and an execution cycle corresponding to each to-be-executed task; the to-be-executed tasks include the memory access tasks and calculation tasks corresponding to the memory access tasks; and the execution cycle includes the first execution cycle of the memory access tasks and the second execution cycle of the calculation tasks.

[0250] In a possible implementation, in response to the current scheduling period being the first scheduling period, the scheduler 51 is configured to trigger the current scheduling period in the following manner:

[0251] receiving, in an idle state, to-be-executed tasks corresponding to any of the plurality of neural networks.

[0252] In response to the current scheduling period being a non-first scheduling period, the scheduler 51 is configured to trigger the current scheduling period in the following manner:

[0253] the target memory access task determined by a previous scheduling period corresponding to the current scheduling period is executed completely and / or the target calculation task determined by a historical scheduling period is executed completely.

[0254] In a possible implementation, the scheduler 51, when determining the candidate memory access tasks based on memory access tasks corresponding to the plurality of neural networks respectively, is configured to:

[0255] For each of the plurality of neural networks, determine, from the current unscheduled memory access tasks corresponding to the neural network, a candidate memory access task corresponding to the neural network.

[0256] In a possible implementation, in response to the current scheduling period being the first scheduling period, the scheduler 51, when determining, for each of the plurality of neural networks, a candidate memory access task corresponding to the neural network from the current unscheduled memory access tasks corresponding to the neural network, is configured to:

[0257] determine, from the current unscheduled memory access tasks of a first neural network corresponding to a target memory access task determined in the previous scheduling period, a candidate memory access task of the first neural network in the current scheduling period;

[0258] and determine, from the candidate memory access tasks other than the target memory access task in the previous scheduling period, candidate memory access tasks of neural networks other than the first neural network in the current scheduling period.

[0259] In a possible implementation, the candidate memory access tasks are multiple;

[0260] The scheduler 51, when determining, from the candidate memory access tasks, a target memory access task based on the first execution cycle number corresponding to the candidate memory access task, the second execution cycle number of the computing task corresponding to the candidate memory access task, and the total number of the first execution cycle of the currently executable computing task, is configured to:

[0261] determine a current candidate memory access task from the plurality of candidate memory access tasks;

[0262] determine whether the current candidate memory access task meets a preset condition based on the first execution cycle number corresponding to the current candidate memory access task, the second execution cycle number of the computing task corresponding to the current candidate memory access task, and the total number of the first execution cycle of the currently executable computing task;

[0263] In response to determining that the current candidate memory access task meets the preset condition, determine the current candidate memory access task as the target memory access task;

[0264] In response to determining that the current candidate memory access task does not meet the preset condition, return to the step of determining a current candidate memory access task from the plurality of candidate memory access tasks.

[0265] In a possible implementation, the scheduler 51, when determining whether the current candidate memory access task meets a preset condition based on the first execution cycle number corresponding to the current candidate memory access task, the second execution cycle number of the computing task corresponding to the current candidate memory access task, and the total number of the first execution cycle of the currently executable computing task, is configured to:

[0266] determining whether a storage condition of to-be-accessed data corresponding to the current candidate memory access task is satisfied;

[0267] in response to the storage condition of to-be-accessed data corresponding to the current candidate memory access task being satisfied, determining whether a total number of first execution cycles of currently executable computing tasks is less than a preset threshold of execution cycle number;

[0268] in response to the total number of first execution cycles being less than the threshold of execution cycle number, comparing a first execution cycle number corresponding to the candidate memory access task with a second execution cycle number of a computing task corresponding to the candidate memory access task;

[0269] in response to the first execution cycle number corresponding to the candidate memory access task being less than the second execution cycle number of the computing task corresponding to the candidate memory access task, determining that the current candidate memory access task satisfies the preset condition.

[0270] In a possible implementation, when determining whether the current candidate memory access task satisfies the preset condition based on the first execution cycle number corresponding to the current candidate memory access task, the second execution cycle number of the computing task corresponding to the current candidate memory access task, and the total number of first execution cycles of currently executable computing tasks, the scheduler 51 is further configured to:

[0271] in response to the total number of first execution cycles being greater than or equal to the threshold of execution cycle number, determining that the current candidate memory access task satisfies the preset condition.

[0272] In a possible implementation, the scheduler 51 is further configured to:

[0273] in response to failure of determining the target memory access task from the plurality of candidate memory access tasks, determining a first computing task to be preferentially executed from the currently executable computing tasks based on the second execution cycle number of the currently executable computing task and the first execution cycle number of a memory access task corresponding to the currently executable computing task;

[0274] in response to ending a currently executing computing task, executing the first computing task.

[0275] In a possible implementation, when ending the currently executing computing task, the scheduler 51 is configured to:

[0276] completing execution of the currently executing computing task;

[0277] or, releasing the currently executing computing task.

[0278] In a possible implementation, the scheduler 51 is configured to:

[0279] determine whether to release the currently-executing computing task based on the second execution cycle number corresponding to the currently-executing computing task and the first execution cycle number of the memory access task corresponding to the currently-executing computing task;

[0280] release the currently-executing computing task in response to a determination that the currently-executing computing task is to be released.

[0281] In a possible implementation, the scheduler 51 is further configured to:

[0282] record execution progress information of the currently-executing computing task;

[0283] the execution progress information is used to continue execution of the currently-executing computing task based on the execution progress information when the currently-executing computing task is rescheduled.

[0284] In a possible implementation, the currently-schedulable computing tasks include at least one, and the scheduler 51 is configured to:

[0285] determine a target first parameter of a current scheduling cycle based on the first execution cycle number of the target memory access task determined in the current scheduling cycle and a current first parameter, where the current first parameter is determined based on the first execution cycle number of the historical target memory access task;

[0286] perform the determination process iteratively until a current second parameter is greater than or equal to the target first parameter:

[0287] determine a current computing task from the at least one currently-schedulable computing task;

[0288] compare the target first parameter with the current second parameter, where the current second parameter is determined based on the second execution cycle of the historical target computing task;

[0289] in response to the current second parameter being less than the target first parameter, determine the current computing task as the target computing task; and

[0290] update the second parameter based on the second execution cycle corresponding to the current computing task to obtain a new second parameter;

[0291] The new second parameter is returned as a new current second parameter to the step of comparing the target first parameter and the current second parameter.

[0292] The description of the processing flow of each module in the device and the interaction flow between the modules can refer to the related description in the above method embodiments, and will not be described in detail here.

[0293] The embodiments of the present disclosure further provide a chip, which comprises the task processing system according to the embodiments of the present disclosure or the task scheduling device according to the embodiments of the present disclosure.

[0294] The optional implementation of the present disclosure further provides an electronic device, which comprises the chip according to the embodiments of the present disclosure.

[0295] The embodiments of the present disclosure further provide a computer readable storage medium, which stores a computer program. When the computer program is run by a processor, the steps of the task scheduling method described in the above method embodiments are executed. The storage medium can be a volatile or non-volatile computer readable storage medium.

[0296] The embodiments of the present disclosure further provide a computer program product, which carries a program code. The instructions included in the program code can be used to execute the steps of the task scheduling method described in the above method embodiments. For details, refer to the above method embodiments, which will not be described here.

[0297] The computer program product can be specifically implemented by hardware, software or a combination thereof. In an optional embodiment, the computer program product is specifically embodied as a computer storage medium. In another optional embodiment, the computer program product is specifically embodied as a software product, such as a software development kit (SDK) and the like.

[0298] If the technical solutions of the present disclosure involve personal information, the product applying the technical solutions of the present disclosure has explicitly informed the personal information processing rules before processing the personal information and has obtained the personal independent consent. If the technical solutions of the present disclosure involve sensitive personal information, the product applying the technical solutions of the present disclosure has obtained the personal independent consent before processing the sensitive personal information and at the same time meets the requirement of "explicit consent". For example, at the personal information collection device such as a camera, a clear and prominent mark is set to inform that the personal information collection range has been entered and the personal information will be collected. If the person voluntarily enters the collection range, it is regarded as consent to collect the personal information. Or on the device for processing personal information, the personal information processing rules are informed by using obvious marks / information, and the personal authorization is obtained by means of pop-up information or asking the person to upload his / her personal information, etc. The personal information processing rules can include the personal information processor, the processing purpose of personal information, the processing method, and the type of processed personal information, etc.

[0299] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system and device described above can refer to the corresponding process in the foregoing method embodiments, which will not be repeated here. In several embodiments provided by the present disclosure, it should be understood that the disclosed system, device and method can be implemented by other ways. The device embodiments described above are only schematic, for example, the division of the units is only a logical function division, and there can be another division way in actual implementation, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some communication interfaces, devices or units, and can be electrical, mechanical or other forms.

[0300] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e. they can be located in one place or distributed on multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the present embodiment.

[0301] In addition, each functional unit in each embodiment of the present disclosure can be integrated into one processing unit, or each unit can exist physically, or two or more units can be integrated into one unit.

[0302] If the functions are implemented in the form of software function units and sold or used as independent products, they can be stored in a nonvolatile computer readable storage medium executable by a processor. Based on this understanding, the technical solutions of the present disclosure essentially or the part that contributes to the prior art or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for making an electronic device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the method described in various embodiments of the present disclosure. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (Read-Only Memory, ROM), a random access memory (Random Access Memory, RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0303] Finally, it should be noted that: the above-described embodiments are only specific embodiments of the present disclosure, used to illustrate the technical solutions of the present disclosure, and not to limit them. The protection scope of the present disclosure is not limited thereto. Although the present disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can make modifications or easily think of changes to the technical solutions described in the foregoing embodiments within the technical scope disclosed by the present disclosure, or make equivalent replacements to some of the technical features. The modifications, changes or replacements do not make the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be covered within the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be subject to the protection scope of the claims.

Claims

1. A data processing system, characterized in that, include: The control unit is used to send task execution information corresponding to multiple neural networks to the operator execution unit; The task execution information includes: at least one execution cycle corresponding to a task to be executed; the task to be executed includes: a memory access task, and a computation task corresponding to the memory access task; The operator execution unit, based on the received task execution information, performs the following actions in each current scheduling cycle of multiple scheduling cycles: determining candidate memory access tasks from memory access tasks corresponding to multiple neural networks; determining a target memory access task from the candidate memory access tasks based on the first execution cycle number corresponding to the candidate memory access tasks, the second execution cycle number of the computation tasks corresponding to the candidate memory access tasks, and the total number of first execution cycles of currently executable computation tasks, and scheduling the determined target memory access task; determining a target computation task from the currently schedulable computation tasks based on the first execution cycle of the target memory access task and the second execution cycle number of currently schedulable computation tasks, and scheduling the target computation task. The operator execution unit, when determining the target memory access task from the candidate memory access tasks based on the first execution cycle number corresponding to the candidate memory access task, the second execution cycle number of the computation task corresponding to the candidate memory access task, and the total number of first execution cycles of the currently executable computation task, is used to: Determine whether the total number of first execution cycles of the currently executable computing task is less than a preset execution cycle threshold; in response to the total number of first execution cycles being less than the execution cycle threshold, compare the first execution cycle number corresponding to the candidate memory access task with the second execution cycle number of the computing task corresponding to the candidate memory access task; in response to the first execution cycle number corresponding to the candidate memory access task being less than the second execution cycle number of the computing task corresponding to the candidate memory access task, determine the candidate memory access task as the target memory access task; The currently schedulable computing tasks include at least one; when the operator execution unit determines the target computing task from the currently schedulable computing tasks based on the first execution cycle of the target memory access task and the second execution cycle number of the currently schedulable computing tasks, it is configured to: Based on the first execution cycle number of the target memory access task determined in the current scheduling cycle and the current first parameter, the target first parameter for the current scheduling cycle is determined; wherein, the current first parameter is determined based on the first execution cycle number of the target memory access task in the past. Iteratively execute the following determination process until the current second parameter is greater than or equal to the target first parameter: Among the at least one currently schedulable computing task, the current computing task is determined; The target first parameter is compared with the current second parameter; wherein the current second parameter is determined based on the second execution cycle of the historical target calculation task; In response to the current second parameter being less than the target first parameter, the current computation task is determined as the target computation task; and Based on the second execution cycle corresponding to the current computing task, update the second parameter to obtain a new second parameter; Then, the new second parameter is used as the new current second parameter, and the process returns to the step of comparing the target first parameter with the current second parameter.

2. The data processing system according to claim 1, characterized in that, Before sending task execution information corresponding to multiple neural networks to the operator execution unit, the control unit is further configured to: In response to receiving multiple task processing requests, based on the multiple task processing requests, determine the neural networks corresponding to the multiple task processing requests respectively; parse each neural network, and determine from each neural network the operator corresponding to the target network layer that is assigned to the operator execution unit for processing; Based on the operator, the task execution information is generated.

3. The data processing system according to claim 1 or 2, characterized in that, The operator execution unit includes: a scheduler and multiple arithmetic units; The scheduler is configured to, in response to receiving the task execution information, perform the following actions in each of multiple scheduling cycles based on the task execution information: in response to triggering the current scheduling cycle, determine candidate memory access tasks based on memory access tasks corresponding to multiple neural networks respectively; determine a target memory access task from the candidate memory access tasks based on the first execution cycle number corresponding to the candidate memory access task, the second execution cycle number of the computation tasks corresponding to the candidate memory access task, and the total number of first execution cycles of currently executable computation tasks, and schedule the determined target memory access task; determine a target computation task from the currently schedulable computation tasks based on the first execution cycle of the target memory access task and the second execution cycle number of currently schedulable computation tasks, and schedule the target computation task. The computing unit is used to execute the scheduled target memory access task and the scheduled target computation task.

4. The data processing system according to claim 1 or 2, wherein the operator execution unit, before multiple scheduling cycles, is further configured to: Obtain task execution information corresponding to the target network layer in each of multiple neural networks; The task execution information includes: the number of tasks to be executed, and the execution cycle corresponding to each task to be executed; the tasks to be executed include: the memory access task, and the computation task corresponding to the memory access task; the execution cycle includes: the first execution cycle for executing the memory access task, and the second execution cycle for executing the computation task.

5. The data processing system according to claim 1 or 2, characterized in that, In response to the current scheduling period being the first scheduling period, the current scheduling period is triggered in the following manner: In a no-load state, the task to be executed is received corresponding to any one of the multiple neural networks; In response to the fact that the current scheduling period is not the first scheduling period, the current scheduling period is triggered in the following manner: The target memory access task determined in the previous scheduling period corresponding to the current scheduling period has been completed and / or the target computation task determined in the historical scheduling period has been completed.

6. The data processing system according to claim 1 or 2, characterized in that, The operator execution unit, when determining candidate memory access tasks based on the memory access tasks corresponding to multiple neural networks, is used for: For each of the multiple neural networks, determine the candidate memory access task corresponding to that neural network from the currently unscheduled memory access tasks corresponding to that neural network.

7. The data processing system according to claim 6, characterized in that, The operator execution unit, in response to the current scheduling period being a non-first scheduling period, when determining the candidate memory access task corresponding to each of the multiple neural networks from the currently unscheduled memory access tasks corresponding to that neural network, is configured to: From the currently unscheduled memory access tasks corresponding to the first neural network to which the target memory access task determined in the previous scheduling cycle belongs, determine the candidate memory access tasks of the first neural network in the current scheduling cycle; And determine the other candidate memory access tasks in the previous scheduling cycle, excluding the target memory access task, as candidate memory access tasks for other neural networks other than the first neural network in the current scheduling cycle.

8. The data processing system according to claim 1, characterized in that, There are multiple candidate memory access tasks; The operator execution unit, when determining the target memory access task from the candidate memory access tasks based on the first execution cycle number corresponding to the candidate memory access task, the second execution cycle number of the computation task corresponding to the candidate memory access task, and the total number of first execution cycles of the currently executable computation task, is used to: The current candidate memory access task is determined from the plurality of candidate memory access tasks; Based on the number of first execution cycles corresponding to the current candidate memory access task, the number of second execution cycles of the computation task corresponding to the current candidate memory access task, and the total number of first execution cycles of the currently executable computation tasks, determine whether the current candidate memory access task meets the preset conditions; In response to determining that the current candidate memory access task meets the preset condition, the current candidate memory access task is determined as the target memory access task; In response to determining that the current candidate memory access task does not meet the preset condition, the process returns to the step of determining the current candidate memory access task from among the multiple candidate memory access tasks.

9. The data processing system according to claim 8, characterized in that, The operator execution unit, when determining whether the current candidate memory access task meets the preset conditions based on the first execution cycle number corresponding to the current candidate memory access task, the second execution cycle number of the computation task corresponding to the current candidate memory access task, and the total number of first execution cycles of the currently executable computation tasks, is configured to: Determine whether the storage space used to store the memory access data corresponding to the memory access task meets the storage conditions of the memory access data corresponding to the current candidate memory access task. In response to satisfying the storage conditions of the memory data to be accessed corresponding to the current candidate memory access task, determine whether the total number of the first execution cycles of the current executable computing task is less than a preset execution cycle number threshold. In response to the fact that the total number of the first execution cycles is less than the execution cycle number threshold, the number of the first execution cycles corresponding to the current candidate memory access task and the number of the second execution cycles corresponding to the current candidate memory access task are compared; In response to the fact that the first execution cycle number corresponding to the current candidate memory access task is less than the second execution cycle number of the computation task corresponding to the current candidate memory access task, it is determined that the current candidate memory access task meets the preset condition.

10. The data processing system according to claim 9, characterized in that, The operator execution unit, when determining whether the current candidate memory access task meets the preset conditions based on the first execution cycle number corresponding to the current candidate memory access task, the second execution cycle number of the computation task corresponding to the current candidate memory access task, and the total number of first execution cycles of the currently executable computation tasks, is further configured to: In response to the first total number of execution cycles being greater than or equal to the execution cycle number threshold, it is determined that the current candidate memory access task meets the preset condition.

11. The data processing system according to any one of claims 8-10, characterized in that, The operator execution unit is further configured to: In response to determining that the target memory access task has failed from a plurality of candidate memory access tasks, a first computing task to be executed is determined from the current executable computing tasks based on the second execution cycle number of the currently executable computing task and the first execution cycle number of the memory access task corresponding to the currently executable computing task; In response to the termination of the currently executing computation task, the first computation task is executed.

12. The data processing system according to claim 11, characterized in that, The operator execution unit, when ending the currently executing computation task, is configured to include: Complete the currently executing computation task; Alternatively, release the currently executing computing task.

13. The data processing system according to claim 12, characterized in that, The operator execution unit, when releasing the currently executing computation task, is used to: Based on the second execution cycle number corresponding to the currently executing computing task and the first execution cycle number of the memory access task corresponding to the currently executing computing task, determine whether to release the currently executing computing task; In response to determining to release the currently executing computing task, the currently executing computing task is released.

14. The data processing system according to claim 12 or 13, characterized in that, Before releasing the currently executing computation task, the operator execution unit is further configured to: Record the execution progress information of the currently executing task; The execution progress information is used to continue executing the currently executing computing task when it is rescheduled.

15. A task scheduling method, characterized in that, include: Execute in each of the multiple scheduling cycles: In response to triggering the current scheduling cycle, candidate memory access tasks are determined based on the memory access tasks corresponding to multiple neural networks. Based on the first execution cycle number corresponding to the candidate memory access task, the second execution cycle number of the computing task corresponding to the candidate memory access task, and the total number of first execution cycles of the currently executable computing tasks, a target memory access task is determined from the candidate memory access tasks, and the determined target memory access task is scheduled. Based on the first execution cycle of the target memory access task and the second execution cycle number of the currently schedulable computing tasks, the target computing task is determined from the currently schedulable computing tasks, and the target computing task is scheduled. The step of determining a target memory access task from the candidate memory access tasks based on the first execution cycle number corresponding to the candidate memory access task, the second execution cycle number of the computation task corresponding to the candidate memory access task, and the total number of first execution cycles of the currently executable computation task, and scheduling the determined target memory access task, includes: Determine whether the total number of first execution cycles of the currently executable computing task is less than a preset execution cycle threshold; in response to the total number of first execution cycles being less than the execution cycle threshold, compare the first execution cycle number corresponding to the candidate memory access task with the second execution cycle number of the computing task corresponding to the candidate memory access task; in response to the first execution cycle number corresponding to the candidate memory access task being less than the second execution cycle number of the computing task corresponding to the candidate memory access task, determine the candidate memory access task as the target memory access task; The currently schedulable computing tasks include at least one; determining the target computing task from the currently schedulable computing tasks based on the first execution cycle of the target memory access task and the second execution cycle number of the currently schedulable computing tasks includes: Based on the first execution cycle number of the target memory access task determined in the current scheduling cycle and the current first parameter, the target first parameter for the current scheduling cycle is determined; wherein, the current first parameter is determined based on the first execution cycle number of the target memory access task in the past. Iteratively execute the following determination process until the current second parameter is greater than or equal to the target first parameter: Among the at least one currently schedulable computing task, the current computing task is determined; The target first parameter is compared with the current second parameter; wherein the current second parameter is determined based on the second execution cycle of the historical target calculation task; In response to the current second parameter being less than the target first parameter, the current computation task is determined as the target computation task; and Based on the second execution cycle corresponding to the current computing task, update the second parameter to obtain a new second parameter; Then, the new second parameter is used as the new current second parameter, and the process returns to the step of comparing the target first parameter with the current second parameter.

16. A task scheduling device, characterized in that, include: A scheduler, in response to receiving task execution information, performs the following in each of multiple scheduling cycles based on the task execution information: in response to triggering the current scheduling cycle, it determines candidate memory access tasks based on memory access tasks corresponding to multiple neural networks; and determines a target memory access task from the candidate memory access tasks based on the first execution cycle number corresponding to the candidate memory access task, the second execution cycle number of the computation task corresponding to the candidate memory access task, and the total number of first execution cycles of the currently executable computation task, and schedules the determined target memory access task. Based on the first execution cycle of the target memory access task and the second execution cycle number of the currently schedulable computing tasks, the target computing task is determined from the currently schedulable computing tasks, and the target computing task is scheduled. The arithmetic unit is used to execute the scheduled target memory access task and the scheduled target computation task; When determining the target memory access task from the candidate memory access tasks based on the first execution cycle number corresponding to the candidate memory access task, the second execution cycle number of the computation task corresponding to the candidate memory access task, and the total number of first execution cycles of the currently executable computation task, the scheduler is configured to: Determine whether the total number of first execution cycles for the currently executable computing tasks is less than a preset threshold for the number of execution cycles; In response to the first total number of execution cycles being less than the execution cycle number threshold, the first execution cycle number corresponding to the candidate memory access task and the second execution cycle number of the computation task corresponding to the candidate memory access task are compared; in response to the first execution cycle number corresponding to the candidate memory access task being less than the second execution cycle number of the computation task corresponding to the candidate memory access task, the candidate memory access task is determined as the target memory access task. The currently schedulable computing tasks include at least one; when the scheduler determines the target computing task from the currently schedulable computing tasks based on the first execution cycle of the target memory access task and the second execution cycle number of the currently schedulable computing tasks, it is configured to: Based on the first execution cycle number of the target memory access task determined in the current scheduling cycle and the current first parameter, the target first parameter for the current scheduling cycle is determined; wherein, the current first parameter is determined based on the first execution cycle number of the target memory access task in the past. Iteratively execute the following determination process until the current second parameter is greater than or equal to the target first parameter: Among the at least one currently schedulable computing task, the current computing task is determined; The target first parameter is compared with the current second parameter; wherein the current second parameter is determined based on the second execution cycle of the historical target calculation task; In response to the current second parameter being less than the target first parameter, the current computation task is determined as the target computation task; and Based on the second execution cycle corresponding to the current computing task, update the second parameter to obtain a new second parameter; Then, the new second parameter is used as the new current second parameter, and the process returns to the step of comparing the target first parameter with the current second parameter.

17. A chip, characterized in that, include: The data processing system as described in any one of claims 1-14, or the task scheduling device as described in claim 16.

18. An electronic device, characterized in that, include: The chip as described in claim 17.

19. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by an electronic device, causes the electronic device to perform the steps of the task scheduling method as described in claim 15.

Images