Electronic device and data processing method

Abstract

The application provides an electronic device and a data processing method. The chip of the electronic device comprises a core queue module, a core scheduler, a tensor calculation unit and a vector calculation unit. The core queue module is configured to store kernel tasks of a tensor type and a first dependency relationship in a tensor kernel task queue, and store kernel tasks of a vector type and a second dependency relationship in a vector kernel task queue. The core scheduler is configured to determine a tensor kernel task to be scheduled from the tensor kernel task queue and determine a vector kernel task to be scheduled from the vector kernel task queue, and when it is determined that there is no dependency relationship between the tensor kernel task and the vector kernel task based on the first dependency relationship and the second dependency relationship, schedule the tensor kernel task to the tensor calculation unit and schedule the vector kernel task to the vector calculation unit. Through the application, the computing efficiency of the kernel task can be improved.

Description

Technical Field

[0001] This application relates to artificial intelligence technology in the field of computer applications, and more particularly to an electronic device and a data processing method. Background Technology

[0002] AI chips contain various types of computing units, such as tensor computing units and vector computing units. Each type of computing unit is used to execute a corresponding type of kernel task. When scheduling kernel tasks, AI chips typically schedule a type of kernel task to a corresponding type of computing unit for execution. However, during the execution of these kernel tasks, only computing units of the same type as the kernel task are active, while those of different types remain idle. This affects the utilization rate of computing units and consequently, the computational efficiency of the kernel tasks. Summary of the Invention

[0003] This application provides an electronic device and a data processing method that can improve the computational efficiency of kernel tasks.

[0004] The technical solution of this application embodiment is implemented as follows: This application provides an electronic device, which includes a chip, and the chip includes a kernel queue module, a kernel scheduler, a tensor computation unit, and a vector computation unit. The kernel queue module is used to store tensor type kernel tasks and a first dependency relationship in a tensor kernel task queue, and to store vector type kernel tasks and a second dependency relationship in a vector kernel task queue, wherein the first dependency relationship represents the dependency of the tensor type kernel task on the vector type kernel task, and the second dependency relationship represents the dependency of the vector type kernel task on the tensor type kernel task. The kernel scheduler is configured to determine tensor kernel tasks to be scheduled from the tensor kernel task queue and vector kernel tasks to be scheduled from the vector kernel task queue. When it is determined based on the first dependency relationship and the second dependency relationship that there is no dependency relationship between the tensor kernel task and the vector kernel task, the tensor kernel task is scheduled to the tensor computing unit and the vector kernel task is scheduled to the vector computing unit. The tensor computation unit is used to execute the tensor kernel task; The vector computation unit is used to execute the vector kernel task when the tensor computation unit executes the tensor kernel task.

[0005] This application provides a data processing method for an electronic device, the electronic device including a chip, the chip including a kernel queue module, a kernel scheduler, a tensor computation unit, and a vector computation unit, the method including: The kernel queue module stores tensor-type kernel tasks and a first dependency in the tensor kernel task queue, and vector-type kernel tasks and a second dependency in the vector kernel task queue. The first dependency indicates that the tensor-type kernel task depends on the vector-type kernel task, and the second dependency indicates that the vector-type kernel task depends on the tensor-type kernel task. The kernel scheduler determines the tensor kernel tasks to be scheduled from the tensor kernel task queue and the vector kernel tasks to be scheduled from the vector kernel task queue. When it is determined, based on the first dependency relationship and the second dependency relationship, that there is no dependency relationship between the tensor kernel task and the vector kernel task, the kernel scheduler schedules the tensor kernel task to the tensor computation unit and the vector kernel task to the vector computation unit. The tensor computation unit executes the tensor kernel task; The vector computation unit executes the vector kernel task while the tensor computation unit executes the tensor kernel task.

[0006] In this embodiment, the chip further includes an instruction queue module and a load memory unit. The kernel scheduler schedules the tensor kernel task to the tensor computation unit and the vector kernel task to the vector computation unit, including: The kernel scheduler sends the first scheduling instructions for the tensor kernel task and the vector kernel task to the instruction queue module; In response to the first scheduling instruction, the instruction queue module stores the execution instructions of the tensor kernel task in the tensor kernel instruction queue and the execution instructions of the vector kernel task in the vector kernel instruction queue, and sets the instruction queue working mode to independent kernel mode. In the independent kernel mode, the instruction queue module sends the execution instructions of the tensor kernel tasks in the tensor kernel instruction queue to the tensor computation unit and the load storage unit, and sends the execution instructions of the vector kernel tasks in the vector kernel instruction queue to the vector computation unit and the load storage unit; The tensor computation unit executes the tensor kernel task, and the vector computation unit executes the vector kernel task while the tensor computation unit executes the tensor kernel task, including: The tensor computation unit and the loading storage unit execute the execution instructions of the tensor kernel task; When the tensor computation unit executes the execution instructions of the tensor kernel task, the vector computation unit and the loading and storage unit execute the execution instructions of the vector kernel task.

[0007] In this embodiment of the application, the electronic device further includes a host side. Before the kernel queue module stores tensor-type kernel tasks and a first dependency in the tensor kernel task queue, and stores vector-type kernel tasks and a second dependency in the vector kernel task queue, the method further includes: The host starts the kernel of the tensor type, obtaining the kernel task of the tensor type, and starts the kernel of the vector type, obtaining the kernel task of the vector type.

[0008] In this embodiment of the application, the host starts a kernel of the tensor type so that the kernel queue module obtains kernel tasks of the tensor type, and starts a kernel of the vector type so that the kernel queue module obtains kernel tasks of the vector type, including: Based on the data dependency relationship between the tensor type kernel and the vector type kernel, the host starts the tensor type kernel so that the kernel queue module obtains the tensor type kernel task, and starts the vector type kernel so that the kernel queue module obtains the vector type kernel task.

[0009] In this embodiment of the application, before the kernel queue module stores tensor-type kernel tasks and a first dependency in the tensor kernel task queue, and stores vector-type kernel tasks and a second dependency in the vector kernel task queue, the method further includes: The host in the electronic device starts a combined kernel to obtain a kernel task of the tensor type and a kernel task of the vector type, wherein the combined kernel refers to the result of combining the kernel of the tensor type and the kernel of the vector type.

[0010] In this embodiment of the application, the method further includes: The kernel queue module stores fusion-type kernel tasks in the fusion kernel task queue. The fusion-type kernel tasks are obtained by starting a fusion kernel. The fusion kernel refers to the fusion result of the tensor type kernel and the vector type kernel. The kernel scheduler determines the fusion kernel task to be scheduled from the fusion kernel task queue and schedules the fusion kernel task to the tensor computation unit and the vector computation unit.

[0011] In this embodiment of the application, the kernel scheduler determines the tensor kernel tasks to be scheduled from the tensor kernel task queue and the vector kernel tasks to be scheduled from the vector kernel task queue, including: When the fusion kernel task is not scheduled, the kernel scheduler determines the tensor kernel task to be scheduled from the tensor kernel task queue and the vector kernel task to be scheduled from the vector kernel task queue.

[0012] In this embodiment of the application, scheduling the fusion kernel task to the tensor computation unit and the vector computation unit includes: The kernel scheduler sends the second scheduling instruction for the fused kernel task to the instruction queue module in the chip; In response to the second scheduling instruction, the instruction queue module stores the execution instructions of the fusion kernel task in the fusion kernel instruction queue and sets the instruction queue working mode to fusion kernel mode; In the fused kernel mode, the instruction queue module sends the execution instructions of the fused kernel task in the fused kernel instruction queue to the tensor computing unit, the vector computing unit, and the loading memory unit in the chip; The tensor computation unit, the vector computation unit, and the loading and storage unit execute the execution instructions of the fusion kernel task.

[0013] In this embodiment of the application, the method further includes: The host in the electronic device initiates the tensor type kernel, the vector type kernel, the composite kernel, and the fusion type kernel to different data streams; The kernel queue module obtains the tensor type kernel task, the vector type kernel task, and the fusion type kernel task from the different data streams.

[0014] In this embodiment of the application, after the kernel scheduler determines the tensor kernel tasks to be scheduled from the tensor kernel task queue and the vector kernel tasks to be scheduled from the vector kernel task queue, the method further includes: When it is determined that there is a dependency between the tensor kernel task and the vector kernel task based on the first dependency and the second dependency, the kernel scheduler determines the dependent kernel task from the tensor kernel task and the vector kernel task. The dependent kernel task is scheduled to the target computing unit, wherein the type of the target computing unit is the same as the type of the dependent kernel task.

[0015] This application provides a computer-readable storage medium storing computer-executable instructions or computer programs, which, when executed, implement the data processing method of the electronic device provided in this application.

[0016] This application provides a computer program product, including computer-executable instructions or a computer program. When the computer-executable instructions or the computer program are executed by a processor, they implement the data processing method of the electronic device provided in this application.

[0017] The embodiments of this application have at least the following beneficial effects: In the chip of an electronic device, the kernel queue module stores not only tensor-type kernel tasks in the tensor kernel task queue, but also the first dependency relationship between tensor-type kernel tasks and vector-type kernel tasks, and in the vector kernel task queue, not only vector-type kernel tasks, but also the second dependency relationship between vector-type kernel tasks and tensor-type kernel tasks. This enables the kernel scheduler to determine whether there is a dependency between the tensor kernel task to be scheduled and the vector kernel task to be scheduled based on the first and second dependencies when scheduling kernel tasks from the tensor kernel task queue and the vector kernel task queue. When there is no dependency relationship between the tensor kernel task and the vector kernel task, the tensor kernel task and the vector kernel task are scheduled to the tensor computing unit and the vector computing unit in parallel, so that the tensor kernel task and the vector kernel task can be executed in parallel in the tensor computing unit and the vector computing unit. This allows the vector computing unit to execute vector kernel tasks while the tensor computing unit is executing tensor kernel tasks, realizing the parallel execution of different types of kernel tasks in different types of computing units. Therefore, it can improve the utilization rate of the tensor computing unit and the vector computing unit and improve the computational efficiency of kernel tasks. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of the architecture of the data processing system provided in the embodiments of this application; Figure 2 This is a schematic diagram of a chip for data processing provided in an embodiment of this application. Figure 1 ; Figure 3 This is a schematic diagram of a chip for data processing provided in an embodiment of this application. Figure 2 ; Figure 4 This is a schematic diagram of an electronic device for data processing provided in an embodiment of this application; Figure 5 This is a flowchart of a data processing method for an electronic device provided in an embodiment of this application; Figure 6 This is an exemplary kernel task queue diagram provided in an embodiment of this application; Figure 7 This is a schematic diagram of the instruction queue corresponding to the scheduling mode provided in the embodiments of this application; Figure 8 This is a schematic diagram illustrating an exemplary parallel execution of kernel tasks provided in an embodiment of this application; Figure 9 This is an exemplary illustration of parallelized computing units provided in the embodiments of this application. Figure 1 ; Figure 10 This is an exemplary illustration of parallelized computing units provided in the embodiments of this application. Figure 2 ; Figure 11 This is an exemplary data dependency diagram provided in an embodiment of this application; Figure 12 This is a schematic diagram of kernel task processing provided in an embodiment of this application; Figure 13 This is a schematic diagram illustrating another kernel task processing method provided in an embodiment of this application; Figure 14 This is an exemplary illustration of parallelized computing units provided in the embodiments of this application. Figure 3 . Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0020] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0021] In the following description, the terms "first" and "second" are used to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first" and "second" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0022] Unless otherwise defined, all technical and scientific terms used in the embodiments of this application have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used in the embodiments of this application is for the purpose of describing the embodiments of this application only and is not intended to limit this application.

[0023] In the implementation of this application, the collection and processing of relevant data should strictly comply with the requirements of relevant national laws and regulations, obtain the informed consent or separate consent of the personal information subject, and carry out subsequent data use and processing within the scope of laws and regulations and the authorization of the personal information subject.

[0024] Before providing a further detailed description of the embodiments of this application, the nouns and terms involved in the embodiments of this application will be explained, and the nouns and terms involved in the embodiments of this application shall be interpreted as follows.

[0025] 1) Artificial Intelligence (AI) is a theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results. In other words, AI is a comprehensive technology in computer science used to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess perception, reasoning, and decision-making functions. The chip in the embodiments of this application is an artificial intelligence chip.

[0026] 2) A kernel, also known as a computation kernel or kernel function, is the code implementation of an operator on a specific hardware architecture (such as a CPU). Examples include the implementation code of a matrix multiplication operator, a convolution operator, an activation function (GELU, etc.) operator, and a normalization operator. In this embodiment, the kernel type includes tensor type and vector type. An operator (Op) is a logical operation type that defines the computational action, i.e., the action to be performed on the data. It is abstract information and does not involve specific processing methods. Examples include matrix multiplication (Matmul) operator, convolution operator, activation function (GELU, etc.) operator, and normalization operator.

[0027] 3) In response, used to indicate the conditions or states on which the operation performed depends. When the conditions or states on which it depends are met, one or more operations performed can be performed in real time or with a set delay. Unless otherwise specified, there is no restriction on the order in which the multiple operations are performed.

[0028] It's important to note that when scheduling kernel tasks, AI chips typically allocate one type of kernel task to a corresponding type of computing unit for execution. Furthermore, different types of kernel tasks are executed sequentially within the computing units. In other words, during the execution of a kernel task, only computing units of the same type as the task are active, while those of different types are idle. For instance, in an AI chip's computing units that include tensor and vector computing units, when a tensor-type kernel task is scheduled to the tensor computing unit, the vector computing unit is idle, and vice versa. This affects the utilization rate of the computing units, thus impacting the computational efficiency of the kernel tasks.

[0029] Furthermore, to improve the utilization of computing units, various types of computing units in the AI ​​chip can be utilized through kernel parallelization; for example, tensor computing units and vector computing units can be utilized through tensor-type and vector-type kernel parallelization. However, the process of obtaining the kernel is complex, affecting the efficiency of kernel acquisition; therefore, it affects the computational efficiency of kernel tasks.

[0030] Based on this, embodiments of this application provide an electronic device and a data processing method that can improve the utilization rate of computing units and enhance the computational efficiency of kernel tasks. The following describes exemplary applications of the electronic device provided in this application. The electronic device provided in this application can be implemented as various types of terminals such as robots, smartphones, smartwatches, laptops, tablets, desktop computers, smart home appliances, set-top boxes, smart vehicle devices, portable music players, personal digital assistants, dedicated messaging devices, intelligent voice interaction devices, portable gaming devices, and smart speakers. It can also be implemented as a server, or a combination of both. The following will describe exemplary applications when the electronic device is implemented as a server.

[0031] See Figure 1 , Figure 1 This is a schematic diagram of the architecture of the data processing system provided in the embodiments of this application; as shown Figure 1 As shown, to support a data processing application, in the data processing system 100, terminals 400 (terminals 400-1 and 400-2 are shown as examples) connect to server 200 via network 300. Network 300 can be a wide area network (WAN), a local area network (LAN), or a combination of both. Additionally, the data processing system 100 also includes a database 500 for providing data support to server 200; and... Figure 1The example shown illustrates a scenario where the database 500 is independent of the server 200. However, the database 500 can also be integrated into the server 200, and this embodiment does not limit this to any particular case.

[0032] Terminal 400 is used to send AI task execution requests (such as question-and-answer requests, image processing requests, etc.) to server 200 via network 300; it is also used to receive data processing results (such as question answers, image processing results, etc.) sent by server 200 in response to AI task execution requests via network 300, and to present the data processing results.

[0033] Server 200 is used to receive AI task execution requests sent by terminal 400 via network 300, execute AI tasks in response to AI task execution requests, obtain data processing results, and send the data processing results to terminal 400 via network 300.

[0034] In some embodiments, server 200 may be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. Terminals and servers can be connected directly or indirectly via wired or wireless communication, which is not limited in this embodiment.

[0035] The exemplary applications of the chips provided in the embodiments of this application will be described below.

[0036] See Figure 2 , Figure 2 This is a schematic diagram of a chip for data processing provided in an embodiment of this application. Figure 1 ;like Figure 2 As shown, the chip 2-1 of the electronic device includes a kernel queue module 2-11, a kernel scheduler 2-12, a tensor calculation unit 2-13, and a vector calculation unit 2-14.

[0037] In this embodiment, the kernel queue module 2-11 is used to store tensor type kernel tasks and first dependencies in the tensor kernel task queue, and to store vector type kernel tasks and second dependencies in the vector kernel task queue.

[0038] It should be noted that kernel queue module 2-11 stores kernel tasks into corresponding kernel task queues based on the type of the kernel task. Kernel task types include tensor and vector types, and kernel task queue types also include tensor and vector types. Therefore, when kernel queue module 2-11 obtains a kernel task of type tensor, it stores the tensor type kernel task into the tensor type kernel task queue. When storing a tensor type kernel task in the tensor type kernel task queue, it marks the vector type kernel task that the tensor type kernel task depends on. Similarly, when kernel queue module 2-11 obtains a kernel task of type vector, it stores the vector type kernel task into the vector type kernel task queue. When storing a vector type kernel task in the vector type kernel task queue, it marks the tensor type kernel task that the vector type kernel task depends on. Here, the kernel task queue for tensor types is called the tensor kernel task queue, and the kernel task queue for vector types is called the vector kernel task queue. Therefore, kernel queue module 2-11 includes both tensor kernel task queues and vector kernel task queues. The relationship between a tensor type kernel task and the vector type kernel task that the tensor type kernel task depends on is called the first dependency relationship. Thus, the first dependency relationship represents the dependency of a tensor type kernel task on a vector type kernel task. The relationship between a vector type kernel task and the tensor type kernel task that the vector type kernel task depends on is called the second dependency relationship. Thus, the second dependency relationship represents the dependency of a vector type kernel task on a tensor type kernel task. Furthermore, the first and second dependencies are determined based on the startup order of the kernels corresponding to the tensor type kernel task and the kernels corresponding to the vector type kernel task. For example, for two types of kernels, a kernel task of the type that starts earlier is depended upon by a kernel task of the type that starts later.

[0039] In this embodiment, kernel scheduler 2-12 is used to determine tensor kernel tasks to be scheduled from the tensor kernel task queue and vector kernel tasks to be scheduled from the vector kernel task queue. When it is determined based on the first dependency relationship and the second dependency relationship that there is no dependency relationship between the tensor kernel task and the vector kernel task, the tensor kernel task is scheduled to the tensor computation unit and the vector kernel task is scheduled to the vector computation unit. Tensor computation unit 2-13 is used to execute tensor kernel tasks; vector computation unit 2-14 is used to execute vector kernel tasks while the tensor computation unit is executing tensor kernel tasks.

[0040] It should be noted that when kernel scheduler 2-12 schedules kernel tasks from the tensor kernel task queue and the vector kernel task queue, it determines the kernel tasks to be scheduled from the tensor kernel task queue and the vector kernel task queue respectively, thus obtaining the tensor kernel tasks and vector kernel tasks to be scheduled. Kernel scheduler 2-12 determines that the vector type kernel tasks on which the tensor kernel task depends are unrelated to the vector kernel task (or there are no dependent vector type kernel tasks) based on the first dependency relationship, and determines that the tensor type kernel tasks on which the vector kernel task depends are unrelated to the tensor kernel task (or there are no dependent tensor type kernel tasks) based on the second dependency relationship, thus determining that there is no dependency relationship between the tensor kernel task and the vector kernel task. Here, "unrelated" means that there is no direct or indirect dependency relationship between the two types of kernel tasks; that is, the two types of kernel tasks are different, and the kernel tasks directly or indirectly dependent on by one type of kernel task do not include the other type of kernel task. Therefore, no dependency relationship means neither direct nor indirect dependency; otherwise, a dependency exists. For example, if a tensor-type kernel task T1 depends on a tensor-type kernel task T0, and a vector-type kernel task V1 depends on a tensor-type kernel task T1, then a tensor-type kernel task T0 and a vector-type kernel task V1 are dependent. Furthermore, a tensor kernel task is a tensor-type kernel task to be scheduled determined from the vector kernel task queue, and a vector kernel task is a vector-type kernel task to be scheduled determined from the vector kernel task queue.

[0041] It should also be noted that when there is no dependency between tensor kernel tasks and vector kernel tasks, kernel scheduler 2-12 can schedule tensor kernel tasks and vector kernel tasks in parallel. This means that while scheduling tensor kernel tasks to tensor computation unit 2-13, vector kernel tasks are simultaneously scheduled to vector computation unit 2-14. This ensures that the execution of tensor kernel tasks by tensor computation unit 2-13 and the execution of vector kernel tasks by vector computation unit 2-14 are parallel processes. Specifically, vector computation unit 2-14 executes vector kernel tasks in parallel while tensor computation unit 2-13 executes tensor kernel tasks, and vice versa. Furthermore, kernel scheduler 2-12 schedules tensor-type kernel tasks in the tensor kernel task queue in enqueue order, and schedules vector-type kernel tasks in the vector kernel task queue in enqueue order.

[0042] Understandably, in chip 2-1, the kernel queue module 2-11 stores not only tensor-type kernel tasks in the tensor kernel task queue, but also the first dependency relationship between tensor-type kernel tasks and vector-type kernel tasks; and in the vector kernel task queue, it stores not only vector-type kernel tasks, but also the second dependency relationship between vector-type kernel tasks and tensor-type kernel tasks. This enables the kernel scheduler 2-12 to determine the tensor kernel task to be scheduled and the vector kernel task to be scheduled based on the first and second dependencies when scheduling kernel tasks from the tensor kernel task queue and the vector kernel task queue. If there is no dependency between the tensor kernel task and the vector kernel task, the tensor kernel task and the vector kernel task are scheduled in parallel to tensor computation unit 2-13 and vector computation unit 2-14. This allows the tensor kernel task and the vector kernel task to be executed in parallel in tensor computation unit 2-13 and vector computation unit 2-14. This enables the parallel execution of different types of kernel tasks in different types of computation units, thereby improving the utilization of computation units and the computational efficiency of kernel tasks.

[0043] based on Figure 2 See Figure 3 , Figure 3 This is a schematic diagram of a chip for data processing provided in an embodiment of this application. Figure 2 ;like Figure 3 As shown, the chip 2-1 also includes an instruction queue module 3-1 and a load memory unit 3-3.

[0044] In this embodiment, the kernel scheduler 2-12 is further configured to send first scheduling instructions for the tensor kernel task and the vector kernel task to the instruction queue module 3-1 when there is no dependency between the tensor kernel task and the vector kernel task. The instruction queue module 3-1, in response to the first scheduling instructions, stores the execution instructions of the tensor kernel task in the tensor kernel instruction queue and the execution instructions of the vector kernel task in the vector kernel instruction queue, and sets the instruction queue operating mode to independent kernel mode. In independent kernel mode, the execution instructions of the tensor kernel task in the tensor kernel instruction queue are sent to the tensor computation unit 2-13 and the loading storage unit 3-3, and the execution instructions of the vector kernel task in the vector kernel instruction queue are sent to the vector computation unit 2-14 and the loading storage unit 3-3. The tensor computation unit 2-13 is configured to execute the execution instructions of the tensor kernel task in conjunction with the loading storage unit 3-3; the vector computation unit 2-14 is configured to execute the execution instructions of the vector kernel task in conjunction with the loading storage unit 3-3 when the tensor computation unit 2-13 executes the execution instructions of the tensor kernel task.

[0045] It should be noted that when there is no dependency between the tensor kernel task and the vector kernel task, the kernel scheduler 2-12 determines to schedule the tensor kernel task and the vector kernel task in parallel, thereby sending the first scheduling instruction for the tensor kernel task and the vector kernel task to the instruction queue module 3-1. The first scheduling instruction represents a signal to schedule the tensor kernel task and the vector kernel task in parallel to the tensor computation unit 2-13 and the vector computation unit 2-14. The instruction queue module 3-1 includes a tensor kernel instruction queue and a vector kernel instruction queue. The tensor kernel instruction queue represents the instruction queue for tensor-type kernel tasks, and the vector kernel instruction queue represents the instruction queue for vector-type kernel tasks. Upon receiving the first scheduling instruction, the instruction queue module 3-1, in response to the first scheduling instruction, generates execution instructions for executing the tensor kernel task (i.e., the execution instructions for the tensor kernel task) and execution instructions for executing the vector kernel task (i.e., the execution instructions for the vector kernel task), stores the execution instructions for the tensor kernel task in the tensor kernel instruction queue, and stores the execution instructions for the vector kernel task in the vector kernel instruction queue. It also sets the instruction queue operating mode to independent kernel mode. Since the independent kernel mode refers to the mode of issuing execution instructions for different types of kernel tasks in parallel to the tensor computation unit 2-13 and the vector computation unit 2-14, the tensor kernel instruction queue and the vector kernel instruction queue are independent in independent kernel mode. The instruction queue module 3-1 can simultaneously issue execution instructions for tensor kernel tasks in the tensor kernel instruction queue and execution instructions for vector kernel tasks in the vector kernel instruction queue, so that the execution instructions for tensor kernel tasks run in the tensor computation unit 2-13 and the load storage unit 3-3, while the execution instructions for vector kernel tasks run in parallel. The operation is carried out in the vector computation unit 2-14 and the loading storage unit 3-3. Thus, while the tensor computation unit 2-13 and the loading storage unit 3-3 execute the execution instructions of the tensor kernel task, the vector computation unit 2-14 and the loading storage unit 3-3 execute the execution instructions of the vector kernel task. Here, the tensor computation unit 2-13 and the loading storage unit 3-3 execute the tensor kernel task by executing the execution instructions of the tensor kernel task, and the vector computation unit 2-14 and the loading storage unit 3-3 execute the vector kernel task by executing the execution instructions of the vector kernel task.

[0046] It is understandable that when there is no dependency between the tensor kernel task and the vector kernel task, the execution instructions of the tensor kernel task and the vector kernel task are simultaneously sent to the tensor computation unit 2-13 and the vector computation unit 2-14 through independent tensor kernel instruction queues and vector kernel instruction queues. This allows the execution instructions of the tensor kernel task to be executed on the tensor computation unit 2-13 while the execution instructions of the vector kernel task are executed on the vector computation unit 2-14, thus realizing the parallel utilization of the tensor computation unit 2-13 and the vector computation unit 2-14.

[0047] based on Figure 3 See Figure 4 , Figure 4 This is a schematic diagram of an electronic device for data processing provided in an embodiment of this application; as shown... Figure 4 As shown, the electronic device also includes a host unit 4-1.

[0048] In this embodiment, host terminal 4-1 is used to launch a tensor type kernel so that kernel queue module 2-11 obtains a tensor type kernel task, and launch a vector type kernel so that kernel queue module 2-11 obtains a vector type kernel task.

[0049] It should be noted that kernel tasks are generated by the kernel started by host-side 4-1, and the kernel types include tensor type and vector type. Here, host-side 4-1 starts a tensor type kernel, so that kernel queue module 2-11 receives tensor type kernel tasks; host-side 4-1 starts a vector type kernel, so that kernel queue module 2-11 receives vector type kernel tasks.

[0050] It is understandable that the host 4-1 starts independent (i.e., non-fused and non-combined) tensor type kernels and vector type kernels to obtain tensor type kernel tasks and vector type kernel tasks, thus providing a source for tensor type kernel tasks and vector type kernel tasks; and, the host 4-1 starts different types of kernels, realizing the control of different types of kernels.

[0051] In this embodiment, the host terminal 4-1 is further configured to start a tensor type kernel based on the data dependency relationship between the tensor type kernel and the vector type kernel, so that the kernel queue module 2-11 obtains the tensor type kernel task, and so that the kernel queue module 2-11 starts the vector type kernel and obtains the vector type kernel task.

[0052] It should be noted that in the data dependency relationship between tensor type kernels and vector type kernels, the kernel that is depended upon takes precedence over the kernel that has a dependency. Furthermore, in this data dependency relationship, when two kernels of the same type are independent of each other, the data dependency relationship between these two kernels of the same type corresponds to the dependency order of two kernels of other types that they depend on. That is, for two kernels of the same type (referred to as the first kernel and the second kernel, the first kernel being, for example, a vector type kernel V1, and the second kernel being, for example, a vector type kernel V0), if one kernel (i.e., the first kernel, such as V1) depends on another kernel of another type (e.g., a tensor type kernel T1), which in turn depends on another kernel of another type (e.g., a tensor type kernel T0) that the other kernel (i.e., the second kernel, such as V0) depends on, i.e., V1 depends on T1, V0 depends on T0, and T1 depends on T0, then the data dependency relationship between the first kernel and the second kernel is determined to be that the second kernel takes precedence over the first kernel, i.e., V0 takes precedence over V1.

[0053] In some embodiments, the host 4-1 determines an event for marking the data dependencies of multiple tensor type kernels and multiple vector type kernels to be launched through the startup interface, launches multiple tensor type kernels and multiple vector type kernels in different data streams, and sends the event for marking the data dependencies of multiple tensor type kernels and multiple vector type kernels to be launched to the kernel queue module 2-11.

[0054] In this embodiment of the application, the host terminal 4-1 is also used to start the combined kernel so that the kernel queue module 2-11 obtains kernel tasks of tensor type and kernel tasks of vector type.

[0055] It should be noted that tensor type kernel tasks and vector type kernel tasks can also be obtained by starting a composite kernel on the host side via 4-1. A composite kernel refers to the result of combining a tensor type kernel and a vector type kernel; this combination could be a loop-like combination, etc. A composite kernel is obtained by logically associating at least one tensor type kernel and at least one vector type kernel, each with a single logical function. When starting a composite kernel, the at least one tensor type kernel and the at least one vector type kernel within the composite kernel are still started independently.

[0056] It is understandable that by combining kernels, the number of fused kernels can be reduced, thereby reducing the cost of acquiring fused kernels, improving data processing efficiency, and also improving the reusability of different types of kernels and the parallelism of computing units.

[0057] In this embodiment, the kernel queue module 2-11 is further configured to store fusion-type kernel tasks in the fusion kernel task queue; the kernel scheduler 2-12 is further configured to schedule the fusion kernel tasks in the fusion kernel task queue to the tensor computation unit and the vector computation unit; the kernel scheduler 2-12 is further configured to determine the tensor kernel tasks to be scheduled from the tensor kernel task queue and the vector kernel tasks to be scheduled from the vector kernel task queue when no fusion kernel tasks are scheduled.

[0058] It should be noted that the kernel queue module 2-11 also includes a fusion kernel task queue, which is used to store fusion type kernel tasks. Thus, after obtaining a fusion type kernel task, the kernel queue module 2-11 stores the fusion type kernel task in the fusion kernel task queue. The fusion type kernel task is obtained by starting a fusion kernel. The fusion kernel refers to the fusion result of tensor type kernel and vector type kernel. That is, the fusion kernel is obtained by fusing at least one independent, single-function tensor type kernel and at least one vector type kernel into one kernel. The fusion result corresponds to, for example, using the output of one type as the input of another type. Here, when scheduling kernel tasks in the tensor kernel task queue and / or vector kernel task queue, kernel scheduler 2-12 restricts the scheduling of fusion-type kernel tasks in the fusion kernel task queue to a prohibited operation. When kernel tasks in the tensor kernel task queue and / or vector kernel task queue are not scheduled, fusion-type kernel tasks in the fusion kernel task queue are scheduled; these fusion-type kernel tasks to be scheduled are referred to as fusion kernel tasks. Kernel scheduler 2-12 schedules fusion kernel tasks to tensor computation unit 2-13 and vector computation unit 2-14 for execution. Furthermore, kernel scheduler 2-12 schedules fusion-type kernel tasks in the fusion kernel task queue according to their enqueue order.

[0059] It should also be noted that when scheduling kernel tasks in the fused kernel task queue, kernel scheduler 2-12 will restrict the scheduling of kernel tasks in the tensor kernel task queue and / or vector kernel task queue to be prohibited; while when kernel tasks in the fused kernel task queue are not scheduled, the scheduling of kernel tasks in the tensor kernel task queue and / or vector kernel task queue will be performed.

[0060] Understandably, by restricting the scheduling timing of kernel tasks in the tensor kernel task queue, vector kernel task queue, and fusion kernel task queue, the orderly scheduling of kernel tasks can be ensured, reducing scheduling conflicts. Furthermore, by storing the fusion-type kernel tasks corresponding to the fusion kernel's startup operation in the fusion kernel task queue, compatibility handling for the fusion kernel is achieved.

[0061] In this embodiment, the kernel scheduler 2-12 is further configured to send a second scheduling instruction for the fused kernel task in the fused kernel task queue to the instruction queue module 3-1; the instruction queue module 3-1 is further configured to, in response to the second scheduling instruction, store the execution instructions of the fused kernel task in the fused kernel instruction queue, and set the working mode of the instruction queue to the fused kernel mode. In the fused kernel mode, the execution instructions of the fused kernel task in the fused kernel instruction queue are sent to the tensor calculation unit 2-13, the vector calculation unit 2-14, and the loading storage unit 3-3; the tensor calculation unit 2-13 is further configured to, together with the vector calculation unit 2-14 and the loading storage unit 3-3, execute the execution instructions of the fused kernel task.

[0062] It should be noted that the instruction queue module 3-1 also includes a fused kernel instruction queue, which represents the instruction queue for fused kernel tasks. When the kernel scheduler 2-12 determines which fused kernel task to schedule in the fused kernel task queue, it sends a second scheduling instruction for the fused kernel task to the instruction queue module 3-1. This second scheduling instruction indicates a signal to schedule the fused kernel task to the tensor computation unit 2-13 and the vector computation unit 2-14. Upon receiving this second scheduling instruction, the instruction queue module 3-1, in response, generates an execution instruction for executing the fused kernel task (i.e., the fused kernel task execution instruction), stores the fused kernel task execution instruction in the fused kernel instruction queue, and sets the instruction queue's operating mode to fused kernel mode. Since the fusion kernel mode refers to the mode of issuing execution instructions for fusion-type kernel tasks to tensor computation unit 2-13 and vector computation unit 2-14, in the fusion kernel mode, the instruction queue module 3-1 can issue the execution instructions of fusion kernel tasks in the fusion kernel instruction queue to tensor computation unit 2-13, vector computation unit 2-14 and loading storage unit 3-3; thus, the execution instructions of fusion kernel tasks are executed simultaneously in tensor computation unit 2-13, vector computation unit 2-14 and loading storage unit 3-3 to realize the execution of fusion kernel tasks.

[0063] Understandably, the kernel tasks of the fusion type corresponding to the startup operation of the fusion kernel can be executed through the fusion kernel task queue and the fusion kernel instruction queue, thus achieving compatibility handling for the fusion kernel.

[0064] In this embodiment, the host terminal 4-1 is further configured to launch tensor type kernels, vector type kernels, combined kernels, and fused type kernels to different data streams; the kernel queue module 2-11 is further configured to obtain tensor type kernel tasks, vector type kernel tasks, and fused type kernel tasks from different data streams.

[0065] It should be noted that when the host-side module 4-1 starts tensor-type kernels, vector-type kernels, composite kernels, and fusion-type kernels, it can start these kernels into multiple different data streams (for example, starting tensor-type kernels into the first data stream, vector-type kernels into the second data stream, and composite and fusion-type kernels into the third data stream, where the first, second, and third data streams are all different). Therefore, the kernel queue module 2-11, based on the type of kernel task, stores the tensor-type, vector-type, and fusion-type kernel tasks obtained from different data streams into the corresponding type of kernel task queue.

[0066] Understandably, since tensor-type kernels, vector-type kernels, composite kernels, and fusion-type kernels can be launched to different data streams, the flexibility of launching different types of kernels can be improved; and by launching tensor-type kernels, vector-type kernels, composite kernels, and fusion-type kernels to different data streams, the transmission efficiency of kernel tasks can be improved.

[0067] In this embodiment of the application, the kernel scheduler 2-12 is further configured to determine the dependent kernel task from the tensor kernel task and the vector kernel task when a dependency relationship is determined between the tensor kernel task and the vector kernel task based on the first dependency relationship and the second dependency relationship, and schedule the dependent kernel task to the target computing unit.

[0068] It should be noted that the target computation unit is either the tensor computation unit or the vector computation unit, corresponding to the kernel task it depends on; that is, the target computation unit and the kernel task they depend on are of the same type. In tensor kernel tasks and vector kernel tasks with dependencies, if the tensor kernel task depends on the vector kernel task, then the vector kernel task is the kernel task it depends on, and the target computation unit is vector computation unit 2-14; if the vector kernel task depends on the tensor kernel task, then the tensor kernel task is the kernel task it depends on, and the target computation unit is tensor computation unit 2-13.

[0069] Understandably, when there is a dependency between tensor kernel tasks and vector kernel tasks, kernel scheduler 2-12 executes the dependent kernel task first, which can ensure the correctness of the task execution order.

[0070] See Figure 5 , Figure 5 This is a flowchart of a data processing method for an electronic device provided in an embodiment of this application; the following is in conjunction with... Figure 5 The steps shown illustrate the data processing method for this electronic device.

[0071] Step 101: The kernel queue module stores tensor type kernel tasks and their first dependencies in the tensor kernel task queue, and vector type kernel tasks and their second dependencies in the vector kernel task queue.

[0072] It should be noted that the first dependency relationship indicates the dependency of a tensor-type kernel task on a vector-type kernel task, while the second dependency relationship indicates the dependency of a vector-type kernel task on a tensor-type kernel task.

[0073] Step 102: The kernel scheduler determines the tensor kernel tasks to be scheduled from the tensor kernel task queue and the vector kernel tasks to be scheduled from the vector kernel task queue.

[0074] Step 103: When it is determined that there is no dependency between the tensor kernel task and the vector kernel task based on the first dependency relationship and the second dependency relationship, the kernel scheduler schedules the tensor kernel task to the tensor computation unit and the vector kernel task to the vector computation unit.

[0075] Step 104: The tensor computation unit executes the tensor kernel task, and the vector computation unit executes the vector kernel task while the tensor computation unit executes the tensor kernel task.

[0076] It should be noted that the implementation process of steps 101 to 104 is the same as the process by which the kernel queue module 2-11, kernel scheduler 2-12, tensor calculation unit 2-13 and vector calculation unit 2-14 in the chip 2-1 provided in this application coordinate to schedule tensor kernel tasks and vector kernel tasks that have no dependency relationship in parallel. This application embodiment will not repeat the description here.

[0077] In this embodiment of the application, when chip 2-1 further includes an instruction queue module 3-1 and a loading storage unit 3-3, step 103, in which the kernel scheduler schedules tensor kernel tasks to tensor computation units and vector kernel tasks to vector computation units, includes: the kernel scheduler sending first scheduling instructions for tensor kernel tasks and vector kernel tasks to the instruction queue module; the instruction queue module responds to the first scheduling instructions by storing execution instructions for tensor kernel tasks in the tensor kernel instruction queue and execution instructions for vector kernel tasks in the vector kernel instruction queue, and setting the instruction queue working mode to independent kernel mode. In independent kernel mode, the execution instructions for tensor kernel tasks in the tensor kernel instruction queue are sent to the tensor computation unit and the loading storage unit, and the execution instructions for vector kernel tasks in the vector kernel instruction queue are sent to the vector computation unit and the loading storage unit.

[0078] Accordingly, in step 104, the tensor computation unit executes the tensor kernel task, and the vector computation unit executes the vector kernel task while the tensor computation unit executes the tensor kernel task, including: the tensor computation unit and the loading storage unit execute the execution instructions of the tensor kernel task; while the tensor computation unit executes the execution instructions of the tensor kernel task, the vector computation unit and the loading storage unit execute the execution instructions of the vector kernel task.

[0079] In this embodiment of the application, when the electronic device also includes a host terminal 4-1, a process of obtaining kernel tasks is included before step 101; that is, before the kernel queue module stores tensor type kernel tasks and the first dependency relationship in the tensor kernel task queue, and stores vector type kernel tasks and the second dependency relationship in the vector kernel task queue, the data processing method of the electronic device further includes: the host terminal starts a tensor type kernel so that the kernel queue module obtains tensor type kernel tasks, and starts a vector type kernel so that the kernel queue module obtains vector type kernel tasks.

[0080] In this embodiment of the application, the host starts a tensor type kernel so that the kernel queue module can obtain tensor type kernel tasks, and starts a vector type kernel so that the kernel queue module can obtain vector type kernel tasks. This includes: the host starts a tensor type kernel so that the kernel queue module can obtain tensor type kernel tasks, and starts a vector type kernel so that the kernel queue module can obtain vector type kernel tasks, based on the data dependency relationship between the tensor type kernel and the vector type kernel.

[0081] In this embodiment of the application, before step 101, another process for obtaining kernel tasks is included; that is, before the kernel queue module stores tensor type kernel tasks and the first dependency in the tensor kernel task queue, and stores vector type kernel tasks and the second dependency in the vector kernel task queue, the data processing method of the electronic device further includes: the host in the electronic device starts a combined kernel so that the kernel queue module obtains tensor type kernel tasks and vector type kernel tasks, wherein the combined kernel refers to the combination result of tensor type kernel and vector type kernel.

[0082] In this embodiment of the application, the data processing method of the electronic device further includes: the kernel queue module stores fusion-type kernel tasks in the fusion kernel task queue, wherein the fusion-type kernel tasks are obtained by starting the fusion kernel, and the fusion kernel refers to the fusion result of the tensor type kernel and the vector type kernel; the kernel scheduler determines the fusion kernel task to be scheduled from the fusion kernel task queue and schedules the fusion kernel task to the tensor computing unit and the vector computing unit.

[0083] Accordingly, in step 102 of this application embodiment, the kernel scheduler determines the tensor kernel task to be scheduled from the tensor kernel task queue and the vector kernel task to be scheduled from the vector kernel task queue, including: when no fused kernel task is scheduled, the kernel scheduler determines the tensor kernel task to be scheduled from the tensor kernel task queue and the vector kernel task to be scheduled from the vector kernel task queue.

[0084] In this embodiment, the kernel scheduler schedules the fused kernel task to the tensor computation unit and the vector computation unit, including: the kernel scheduler sending a second scheduling instruction for the fused kernel task to the instruction queue module in the chip; the instruction queue module responding to the second scheduling instruction storing the execution instructions of the fused kernel task in the fused kernel instruction queue and setting the instruction queue working mode to fused kernel mode; in fused kernel mode, the execution instructions of the fused kernel task in the fused kernel instruction queue are sent to the tensor computation unit, the vector computation unit, and the loading memory unit in the chip; the tensor computation unit, the vector computation unit, and the loading memory unit execute the execution instructions of the fused kernel task.

[0085] In this embodiment of the application, the data processing method of the electronic device further includes: the host in the electronic device starts tensor type kernels, vector type kernels, combined kernels and fusion type kernels to different data streams; the kernel queue module obtains tensor type kernel tasks, vector type kernel tasks and fusion type kernel tasks from different data streams.

[0086] In this embodiment of the application, after step 102, another process for scheduling kernel tasks is included; that is, after the kernel scheduler determines the tensor kernel task to be scheduled from the tensor kernel task queue and the vector kernel task to be scheduled from the vector kernel task queue, the data processing method of the electronic device further includes: when it is determined that there is a dependency relationship between the tensor kernel task and the vector kernel task based on the first dependency relationship and the second dependency relationship, the kernel scheduler determines the dependent kernel task from the tensor kernel task and the vector kernel task; the dependent kernel task is scheduled to the target computing unit, wherein the type of the target computing unit is the same as the type of the dependent kernel task.

[0087] It should be noted that the data processing method of the electronic device provided in this application embodiment is the same as the processing process performed by the kernel queue module 2-11, kernel scheduler 2-12, tensor calculation unit 2-13, vector calculation unit 2-14, instruction queue module 3-1, load storage unit 3-3 and host terminal 4-1 in the chip 2-1 provided in this application embodiment, and will not be described again in this application embodiment.

[0088] The following describes an exemplary application of the embodiments of this application in a practical application scenario. This exemplary application describes the process of scheduling tensor kernel tasks and vector kernel tasks that have no dependencies to tensor computation units and vector computation units in parallel, obtaining a combined kernel by combining tensor kernels and vector kernels to achieve parallel scheduling, and kernel-related processing that is compatible with tensor type and vector type fusion type.

[0089] See Figure 6 , Figure 6 This is an exemplary kernel task queue diagram provided in an embodiment of this application; as shown... Figure 6 As shown, the fusion kernel task queue 6-1 is used to cache fusion-type kernel tasks of tensor and vector types. In fusion-type kernel task queue 6-1, fusion-type kernel tasks are executed sequentially. The tensor kernel task queue 6-2 is used to cache tensor-type kernel tasks and their dependencies on vector-type kernel tasks. In tensor kernel task queue 6-2, tensor-type kernel tasks are executed sequentially. The vector kernel task queue 6-3 is used to cache vector-type kernel tasks and their dependencies on tensor-type kernel tasks. In vector kernel task queue 6-3, vector-type kernel tasks are executed sequentially.

[0090] It should be noted that the kernel task of the tensor type obtained by the host (host) when starting the tensor type kernel enters... Figure 6 Tensor kernel task queue 6-2 in the host-side startup vector type kernel enters the vector type kernel task queue. Figure 6 In the vector kernel task queue 6-3, the fusion-type kernel task obtained by starting a fusion-type kernel on the host side enters... Figure 6 The fusion kernel task queue is described in section 6-1. On the host side, tensor-type kernels, vector-type kernels, and fusion-type kernels can be launched to different data streams. On the device side, based on the type of kernel task, tensor-type kernel tasks, vector-type kernel tasks, and fusion-type kernel tasks from different data streams are cached. Figure 6 In the kernel task queue of the corresponding type.

[0091] In this embodiment, when scheduling fusion-type kernel tasks in fusion kernel task queue 6-1, the kernel scheduler prohibits scheduling kernel tasks in tensor kernel task queue 6-2 and vector kernel task queue 6-3. When not scheduling fusion-type kernel tasks in fusion kernel task queue 6-1, the kernel scheduler can schedule tensor-type kernel tasks in tensor kernel task queue 6-2 and vector-type kernel tasks in vector kernel task queue 6-3 in parallel. The kernel scheduler achieves parallel scheduling based on the dependencies between tensor kernel task queue 6-2 and vector kernel task queue 6-3 (i.e., the dependency of tensor-type kernel tasks on vector-type kernel tasks, and the dependency of vector-type kernel tasks on tensor-type kernel tasks, referred to as the first dependency and the second dependency).

[0092] based on Figure 6 The scheduling method of kernel tasks in the kernel task queue can determine two scheduling modes. One scheduling mode is where the kernel scheduler schedules fusion-type kernel tasks in the fusion kernel task queue to execute the fusion-type kernel tasks on the tensor computation unit and the vector computation unit; this is called the fusion kernel mode. The other scheduling mode is where the kernel scheduler schedules tensor-type kernel tasks in the tensor kernel task queue (6-2) and vector-type kernel tasks in the vector kernel task queue (6-3) in parallel to execute the tensor-type kernel tasks and vector-type kernel tasks on the tensor computation unit and the vector computation unit in parallel; this is called the independent kernel mode.

[0093] See Figure 7 , Figure 7 This is a schematic diagram of the instruction queue corresponding to the scheduling mode provided in the embodiments of this application; as shown... Figure 7 As shown, instruction queue 7-1 corresponds to two scheduling modes: standalone kernel mode 7-21 and fused kernel mode 7-22. Standalone kernel mode 7-21 corresponds to tensor kernel instruction queue 7-31 and vector kernel instruction queue 7-32. Tensor kernel instruction queue 7-31 stores execution instructions for tensor-type kernel tasks, and vector kernel instruction queue 7-32 stores execution instructions for vector-type kernel tasks. Fused kernel mode 7-22 corresponds to fused kernel instruction queue 7-33, which stores execution instructions for fused-type kernel tasks.

[0094] It should be noted that, in order to achieve parallelization of tensor computation units and vector computation units, by starting the kernels of basic tensor types and basic vector types, tensor type kernel tasks and vector type kernel tasks that do not have data dependencies can be executed in parallel on the tensor computation units and vector computation units.

[0095] In addition, in order to achieve parallelization of tensor computation units and vector computation units, a combined kernel for basic tensor type kernels and basic vector type kernels is proposed. By starting this combined kernel, tensor type kernel tasks and vector type kernel tasks that have no dependencies can be executed in parallel on tensor computation units and vector computation units.

[0096] See Figure 8 , Figure 8 This is a schematic diagram illustrating an exemplary parallel execution of a kernel task provided in an embodiment of this application; as shown... Figure 8 As shown, tensor computation unit 8-1 first executes the tensor type kernel task 8-21, and then executes the tensor type kernel task 8-22 after tensor type kernel task 8-21 is completed. Vector computation unit 8-3 executes the vector type kernel task 8-41 in parallel while tensor computation unit 8-1 executes tensor type kernel task 8-22; vector type kernel task 8-41 depends on tensor type kernel task 8-21. Tensor computation unit 8-1 executes tensor type kernel task 8-23 after tensor type kernel task 8-22 is completed; vector computation unit 8-3 executes the vector type kernel task 8-42 in parallel while tensor computation unit 8-1 executes tensor type kernel task 8-23; vector type kernel task 8-42 depends on tensor kernel task 8-22. This execution continues in this manner. After the tensor type kernel task 8-2n (where n is an integer) is completed, the vector type kernel task 8-4n is executed by the vector type kernel task 8-3. The vector type kernel task 8-4n depends on the tensor type kernel task 8-2n.

[0097] It should also be noted that the fusion type kernel is still obtained by fusing tensor type kernel and vector type kernel, which is compatible with the kernel development method in the embodiments of this application.

[0098] It's important to note that in AI chips, the kernel scheduler schedules kernel tasks from three kernel task queues, corresponding to two scheduling methods. Besides parallelizing tensor and vector computation units using fused kernels, two other parallelization methods can also be employed. One method combines tensor and vector kernels to create a combined kernel; this combined kernel, for example, starts from i=0 (where i is an integer variable), looping through the process, first launching a matrix multiplication kernel (a tensor kernel), then an activation function kernel (a vector kernel), with the activation function kernel depending on the matrix multiplication kernel, until `process_num` (a positive integer constant) times are launched. The other method involves launching independent tensor and vector kernels.

[0099] See Figure 9 , Figure 9 This is an exemplary illustration of parallelized computing units provided in the embodiments of this application. Figure 1 ; The following is through Figure 9 This explains the process of utilizing computing units through kernel parallelization.

[0100] like Figure 9 As shown, first, the host starts the combined kernel. For example, in each loop 9-1, a tensor type kernel and a vector type kernel are started, resulting in tensor type kernel tasks corresponding to the tensor type kernel and vector type kernel tasks corresponding to the vector type kernel.

[0101] Secondly, after the host sends the tensor type kernel tasks and the vector type kernel tasks to the device, they are cached in the kernel task queue of the kernel scheduler 9-2. Among them, the tensor type kernel tasks are cached in the tensor kernel task queue 9-21, and the vector type kernel tasks that the tensor type kernel tasks depend on are marked. The vector type kernel tasks are cached in the vector kernel task queue 9-22, and the tensor type kernel tasks that the vector type kernel tasks depend on are marked. It is easy to see that the kernel scheduler 9-2 also includes the fused kernel task queue 9-23.

[0102] Then, kernel scheduler 9-2, based on resource availability (e.g., when computing unit 9-3 is idle), schedules tensor-type kernel tasks and vector-type kernel tasks in parallel. This stores the execution instructions for tensor-type kernel tasks in tensor kernel instruction queue 9-31 within the instruction queue of computing unit 9-3, and stores the execution instructions for vector-type kernel tasks in vector kernel instruction queue 9-32 within the instruction queue of computing unit 9-3. The instruction queues are configured to operate in independent kernel mode. It is easy to see that the instruction queue also includes a fused kernel instruction queue. In independent kernel mode, tensor kernel instruction queue 9-31 and vector kernel instruction queue 9-32 are independent. Therefore, while executing instructions for tensor-type kernel tasks are being emitted from tensor kernel instruction queue 9-31, execution instructions for vector-type kernel tasks can also be emitted from vector kernel instruction queue 9-32.

[0103] Finally, the execution instructions for tensor-type kernel tasks run in tensor computation unit 9-33 and load memory unit 9-34 in computation unit 9-3, while the execution instructions for vector-type kernel tasks run in vector computation unit 9-35 and load memory unit 9-34 in computation unit 9-3, so as to achieve parallelization of tensor computation unit 9-33 and vector computation unit 9-35.

[0104] based on Figure 9See Figure 10 , Figure 10 This is an exemplary illustration of parallelized computing units provided in the embodiments of this application. Figure 2 ; The following is through Figure 10 This explains the process of utilizing computing units through parallelization of independent kernels.

[0105] like Figure 10 As shown, firstly, the host starts multiple independent tensor type kernels and vector type kernels, resulting in tensor type kernel tasks corresponding to tensor type kernels and vector type kernel tasks corresponding to vector type kernels.

[0106] It should be noted that when the host starts multiple independent tensor and vector type kernels, these kernels are launched into different data streams, and data dependencies between them are marked using events. This reduces the likelihood of blocking between tensor and vector type kernel tasks. See also... Figure 11 , Figure 11 This is an exemplary data dependency diagram provided in an embodiment of this application; as shown below. Figure 11 As shown, the vector type kernel 11-1 depends on the tensor type kernel 11-3, the vector type kernel 11-2 depends on the tensor type kernel 11-4, the tensor type kernel 11-3 depends on the vector type kernel 11-5, and the vector type kernel 11-5 depends on the tensor type kernel 11-6.

[0107] based on Figure 11 See Figure 12 , Figure 12 This is a schematic diagram of kernel task processing provided in an embodiment of this application; as shown... Figure 12As shown, by starting the tensor type kernel 11-6, tensor type kernel task 12-1 can be obtained and stored in the tensor kernel task queue 12-71. By starting the vector type kernel 11-5, vector type kernel task 12-2 can be obtained and stored in the vector kernel task queue 12-72, and the vector type kernel task 12-2 is marked as dependent on the tensor type kernel task 12-1. By starting the tensor type kernel 11-4, tensor type kernel task 12-3 can be obtained and stored in the tensor kernel task queue 12-71. By starting the vector type kernel 11-2, vector type kernel task 12-4 can be obtained. Vector type kernel task 12-4 is stored in the vector kernel task queue 12-72, and it is marked as dependent on the tensor type kernel task 12-3. By starting the tensor type kernel 11-3, tensor type kernel task 12-5 can be obtained. Tensor type kernel task 12-5 is stored in the tensor kernel task queue 12-71, and it is marked as dependent on the vector type kernel task 12-2. By starting the vector type kernel 11-1, vector type kernel task 12-6 can be obtained. Vector type kernel task 12-6 is stored in the vector kernel task queue 12-72, and it is marked as dependent on the tensor type kernel task 12-5. Here, when tensor type kernel 11-6, vector type kernel 11-5, tensor type kernel 11-4, tensor type kernel 11-3, vector type kernel 11-2, and vector type kernel 11-1 are started in different data streams, the data dependencies between tensor type kernel 11-6, vector type kernel 11-5, tensor type kernel 11-4, tensor type kernel 11-3, vector type kernel 11-2, and vector type kernel 11-1 are marked by events, so as to store the corresponding kernel tasks in tensor kernel task queue 12-71 and vector kernel task queue 12-72, and mark the dependencies between each kernel task.

[0108] See also Figure 12Based on the tensor kernel task queue 12-71 and the vector kernel task queue 12-72, the tensor type kernel task 12-1 is executed first in the tensor computation unit 12-81. Since the vector type kernel task 12-2 depends on the tensor type kernel task 12-1, and the tensor type kernel task 12-1 has been completed, and the vector type kernel task 12-2 and the tensor type kernel task 12-3 have no dependency relationship, when the tensor type kernel task 12-3 is executed in the tensor computation unit 12-81, the vector type kernel task 12-2 can be executed in parallel in the vector computation unit 12-82. Tensor-type kernel task 12-5 depends on vector-type kernel task 12-2, which has already been executed. Vector-type kernel task 12-4 depends on tensor-type kernel task 12-3, which has also been executed. Tensor-type kernel task 12-5 and vector-type kernel task 12-4 do not have any dependency relationship. Therefore, when tensor-type kernel task 12-5 is executed in tensor computation unit 12-81, vector-type kernel task 12-4 can be executed in parallel in vector computation unit 12-82. Finally, vector-type kernel task 12-6 is executed in vector computation unit 12-82.

[0109] based on Figure 11 See Figure 13 , Figure 13 This is a schematic diagram illustrating another kernel task processing method provided in an embodiment of this application; as shown... Figure 13 As shown, with Figure 12 The difference lies in the data dependency between the vector type kernel 11-5 (which does not use event-marked kernels) and the vector type kernel 11-2 (where kernel 11-5 precedes kernel 11-2). This causes the execution of the vector type kernel task 12-4 to be blocked by the tensor type kernel task 12-3, which it depends on. Consequently, when the vector computation unit 12-82 is idle, the tensor computation unit 12-81 executes the tensor type kernel tasks 12-1 and 12-3 sequentially; when the tensor computation unit 12-81 is idle, the vector computation unit 12-82 executes the vector type kernel tasks 12-4 and 12-2 sequentially; when the vector computation unit 12-82 is idle, the tensor computation unit 12-81 executes the tensor type kernel task 12-5; and when the tensor computation unit 12-81 is idle, the vector computation unit 12-82 executes the vector type kernel task 12-6, thus affecting the parallelism of the computation units.

[0110] In the embodiments of this application, due to Figure 10 The process of storing, scheduling, and executing kernel tasks of tensor type and vector type in the system is similar to... Figure 9 The process of storing, scheduling, and executing kernel tasks of tensor type and vector type is the same, and will not be described again in the embodiments of this application.

[0111] based on Figure 9 See Figure 14 , Figure 14 This is an exemplary illustration of parallelized computing units provided in the embodiments of this application. Figure 3 ; The following is through Figure 14 This explains the process of utilizing computing units through kernel parallelization.

[0112] like Figure 14 As shown, firstly, the host starts the fusion kernel and obtains the fusion type kernel task.

[0113] Secondly, after the host sends the fusion-type kernel task to the device, it is cached in the fusion kernel task queue 9-23 of the kernel scheduler 9-2.

[0114] Then, kernel scheduler 9-2 schedules fusion-type kernel tasks in fusion kernel task queue 9-23 to store the execution instructions of fusion-type kernel tasks in fusion kernel instruction queue 14-1 in the instruction queue of computing unit 9-3, and configures the instruction queue operating mode to fusion kernel mode. In fusion kernel mode, the execution instructions of fusion-type kernel tasks are emitted from fusion kernel task queue 9-23 to tensor computing unit 9-33, vector computing unit 9-35 and load storage unit 9-34 in computing unit 9-3.

[0115] Finally, the execution instructions of the fusion type kernel task run in the tensor computation unit 9-33, the vector computation unit 9-35, and the load storage unit 9-34 in the computation unit 9-3 to achieve parallelization of the tensor computation unit 9-33 and the vector computation unit 9-35.

[0116] Understandably, by launching multiple independent tensor-type kernels and vector-type kernels, or by launching combined kernels and recording dependencies in the tensor kernel task queue and vector kernel task queue, it becomes possible to schedule the tensor-type kernel tasks and vector-type kernel tasks to be scheduled to the tensor computation unit and vector computation unit in parallel when it is determined, based on these dependencies, that the tensor-type kernel tasks and vector-type kernel tasks to be scheduled do not have dependencies. This achieves parallel utilization of the tensor computation unit and vector computation unit. Furthermore, by combining independent tensor-type kernels and vector-type kernels to obtain combined kernels, the number of fused kernels required can be reduced, improving computational efficiency.

[0117] This application provides a computer program product, which includes computer-executable instructions or a computer program stored in a computer-readable storage medium. An electronic device reads the computer-executable instructions or computer program from the computer-readable storage medium and executes the computer-executable instructions or computer program, causing the electronic device to perform the data processing method for the electronic device described in this application.

[0118] This application provides a computer-readable storage medium storing computer-executable instructions or a computer program. When the computer-executable instructions or the computer program are executed, they will cause the execution of the data processing method of the electronic device provided in this application. For example, ... Figure 5 The data processing method of the electronic device is shown.

[0119] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be a variety of devices including one or any combination of the above-mentioned memories.

[0120] In some embodiments, computer-executable instructions may take the form of programs, software, software modules, scripts, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as stand-alone programs or as modules, components, subroutines, or other units suitable for use in a computing environment.

[0121] As an example, computer-executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored as part of a file that holds other programs or data, for example, in multiple co-located files (e.g., files that store one or more modules, subroutines, or code sections).

[0122] As an example, computer-executable instructions may be deployed to execute on a single electronic device (in which case, this single electronic device is an electronic device including the chip provided in the embodiments of this application), or to execute on multiple electronic devices located at one location (in which case, the multiple electronic devices located at one location are electronic devices including the chip provided in the embodiments of this application), or to execute on multiple electronic devices distributed across multiple locations and interconnected via a communication network (in which case, the multiple electronic devices distributed across multiple locations and interconnected via a communication network are electronic devices including the chip provided in the embodiments of this application).

[0123] It is understood that the embodiments of this application involve various types of kernel-related data. When the embodiments of this application are applied to specific products or technologies, user permission or consent is required. The collection, use and processing of related data must comply with relevant laws, regulations and standards, and conform to the principles of legality, legitimacy and necessity. It does not involve obtaining data types prohibited or restricted by laws and regulations, and will not hinder the normal operation of the target website.

[0124] In summary, the embodiments of this application, within the chip of an electronic device, utilize a kernel queue module to store not only tensor-type kernel tasks in the tensor kernel task queue but also the first dependency relationship between tensor-type kernel tasks and vector-type kernel tasks, and in the vector kernel task queue, not only vector-type kernel tasks but also the second dependency relationship between vector-type kernel tasks and tensor-type kernel tasks. This enables the kernel scheduler, when scheduling kernel tasks from the tensor and vector kernel task queues, to determine whether a dependency exists between the tensor kernel task and the vector kernel task to be scheduled based on the first and second dependencies. When no dependency exists, the tensor and vector kernel tasks are scheduled in parallel to the tensor computation unit and the vector computation unit, respectively, allowing for their parallel execution. This enables the vector computation unit to execute vector kernel tasks while the tensor computation unit executes tensor kernel tasks, achieving parallel execution of different types of kernel tasks on different types of computation units. Therefore, this improves the utilization rate of the tensor and vector computation units and enhances the computational efficiency of kernel tasks. In addition, by combining independent tensor type kernels and vector type kernels to obtain a combined kernel, the number of fused kernels can be reduced, thus improving computational efficiency.

[0125] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.

Claims

1. An electronic device, characterized in that, The electronic device includes a chip, which includes a kernel queue module, a kernel scheduler, a tensor computation unit, and a vector computation unit. The kernel queue module is used to store tensor type kernel tasks and a first dependency relationship in a tensor kernel task queue, and to store vector type kernel tasks and a second dependency relationship in a vector kernel task queue, wherein the first dependency relationship represents the dependency of the tensor type kernel task on the vector type kernel task, and the second dependency relationship represents the dependency of the vector type kernel task on the tensor type kernel task. The kernel scheduler is configured to determine tensor kernel tasks to be scheduled from the tensor kernel task queue and vector kernel tasks to be scheduled from the vector kernel task queue. When it is determined based on the first dependency relationship and the second dependency relationship that there is no dependency relationship between the tensor kernel task and the vector kernel task, the tensor kernel task is scheduled to the tensor computing unit and the vector kernel task is scheduled to the vector computing unit. The tensor computation unit is used to execute the tensor kernel task; The vector computation unit is used to execute the vector kernel task when the tensor computation unit executes the tensor kernel task.

2. The electronic device according to claim 1, characterized in that, The chip also includes an instruction queue module and a load storage unit; The kernel scheduler is also used to send a first scheduling instruction for the tensor kernel task and the vector kernel task to the instruction queue module when there is no dependency between the tensor kernel task and the vector kernel task. The instruction queue module is configured to, in response to the first scheduling instruction, store the execution instructions of the tensor kernel task in the tensor kernel instruction queue and store the execution instructions of the vector kernel task in the vector kernel instruction queue, and set the instruction queue working mode to independent kernel mode. In the independent kernel mode, the execution instructions of the tensor kernel task in the tensor kernel instruction queue are sent to the tensor computing unit and the loading storage unit, and the execution instructions of the vector kernel task in the vector kernel instruction queue are sent to the vector computing unit and the loading storage unit. The tensor computation unit is used to execute the execution instructions of the tensor kernel task in conjunction with the loading and storage unit; The vector computation unit is used to execute the execution instructions of the vector kernel task together with the loading and storage unit when the tensor computation unit executes the execution instructions of the tensor kernel task.

3. The electronic device according to claim 1 or 2, characterized in that, The electronic device also includes a host unit; The host is used to start the tensor type kernel so that the kernel queue module obtains the tensor type kernel task, and to start the vector type kernel so that the kernel queue module obtains the vector type kernel task.

4. The electronic device according to claim 3, characterized in that, The host terminal is further configured to, based on the data dependency relationship between the tensor type kernel and the vector type kernel, start the tensor type kernel so that the kernel queue module obtains the tensor type kernel task, and start the vector type kernel so that the kernel queue module obtains the vector type kernel task.

5. The electronic device according to claim 1 or 2, characterized in that, The host terminal in the electronic device is also used to start a combined kernel to obtain a kernel task of the tensor type and a kernel task of the vector type, wherein the combined kernel refers to the combination result of the kernel of the tensor type and the kernel of the vector type.

6. The electronic device according to claim 1 or 2, characterized in that, The kernel queue module is also used to store fusion-type kernel tasks in the fusion kernel task queue, wherein the fusion-type kernel task is obtained by starting the fusion kernel, and the fusion kernel refers to the fusion result of the tensor type kernel and the vector type kernel; The kernel scheduler is further configured to determine the fusion kernel task to be scheduled from the fusion kernel task queue, and schedule the fusion kernel task to the tensor computation unit and the vector computation unit; The kernel scheduler is further configured to, when the fused kernel task is not scheduled, determine the tensor kernel task to be scheduled from the tensor kernel task queue and the vector kernel task to be scheduled from the vector kernel task queue.

7. The electronic device according to claim 6, characterized in that, The kernel scheduler is also used to send a second scheduling instruction for the fused kernel task to the instruction queue module in the chip; The instruction queue module is further configured to, in response to the second scheduling instruction, store the execution instructions of the fusion kernel task in the fusion kernel instruction queue, and set the instruction queue working mode to fusion kernel mode; in the fusion kernel mode, send the execution instructions of the fusion kernel task in the fusion kernel instruction queue to the tensor calculation unit, the vector calculation unit, and the loading storage unit in the chip; The tensor computation unit is also used, together with the vector computation unit and the loading and storage unit, to execute the execution instructions of the fusion kernel task.

8. The electronic device according to claim 6, characterized in that, The host terminal in the electronic device is also used to launch the tensor type kernel, the vector type kernel, the composite kernel, and the fusion type kernel to different data streams; The kernel queue module is also used to obtain the tensor type kernel task, the vector type kernel task, and the fusion type kernel task from the different data streams.

9. The electronic device according to claim 1 or 2, characterized in that, The kernel scheduler is further configured to, when determining that there is a dependency between the tensor kernel task and the vector kernel task based on the first dependency and the second dependency, determine the dependent kernel task from the tensor kernel task and the vector kernel task, and schedule the dependent kernel task to a target computing unit, wherein the type of the target computing unit is the same as the type of the dependent kernel task.

10. A data processing method for an electronic device, characterized in that, The electronic device includes a chip, the chip including a kernel queue module, a kernel scheduler, a tensor computation unit, and a vector computation unit; the method includes: The kernel queue module stores tensor-type kernel tasks and a first dependency in the tensor kernel task queue, and vector-type kernel tasks and a second dependency in the vector kernel task queue. The first dependency indicates that the tensor-type kernel task depends on the vector-type kernel task, and the second dependency indicates that the vector-type kernel task depends on the tensor-type kernel task. The kernel scheduler determines the tensor kernel tasks to be scheduled from the tensor kernel task queue and the vector kernel tasks to be scheduled from the vector kernel task queue. When it is determined, based on the first dependency relationship and the second dependency relationship, that there is no dependency relationship between the tensor kernel task and the vector kernel task, the kernel scheduler schedules the tensor kernel task to the tensor computation unit and the vector kernel task to the vector computation unit. The tensor computation unit executes the tensor kernel task; The vector computation unit executes the vector kernel task while the tensor computation unit executes the tensor kernel task.

Images