Tensor graph configuration method and tensor graph configuration system

Abstract

A tensor graph configuration method includes: providing a tensor graph comprising multiple operation tasks via a graphical user interface (GUI); determining a subgraph in the tensor graph according to instructions input via the GUI, wherein at least one operation task is located within the subgraph; determining a tiling configuration of the subgraph according to multiple target tensor sizes, wherein the multiple target tensor sizes are set by a user via the GUI for the last operation task in the subgraph; and generating a tiled tensor graph according to the tiling configuration, wherein the tiled tensor graph includes at least multiple operation subtasks segmented by tiling the corresponding operation tasks in the subgraph.

Description

[Technical Field]

[0001] This invention generally relates to a tensor graph tiling technique, and more particularly, to a tensor graph configuration method and tensor graph configuration system implemented through a graphical user interface (GUI). [Previous Technology]

[0002] With the development of technology, convolutional neural networks (CNNs) are considered one of the most outstanding neural networks to have achieved significant success in machine learning, for example, they are commonly used in image recognition, image classification, speech recognition, natural language processing, and video classification. Due to the large datasets, intensive computing power, and higher memory storage requirements, CNN architectures have become increasingly complex and difficult to achieve better performance. Therefore, tensor graphs have been introduced to illustrate the tasks of neural networks. However, if the tensor size is too large, it will not fit into the cache of the tensor processing unit, leading to an increased demand for dynamic random-access memory (DRAM). [Summary of the Invention]

[0003] The following summary is illustrative only and is not intended to be limiting in any way. That is, the following overview is provided to introduce the concepts, key points, benefits, and advantages of the novel and non-obvious techniques described herein. Selected embodiments are further described in the detailed description below. Therefore, the following summary is neither intended to identify the essential features of the claimed subject matter nor to define the scope of the claimed subject matter.

[0004] In one embodiment of this disclosure, a tensor graph configuration method is disclosed. The tensor graph configuration method includes providing a tensor graph comprising multiple operation tasks through a graphical user interface (GUI); determining a subgraph within the tensor graph based on instructions input through the GUI, wherein the subgraph contains at least one operation task; determining a tiling configuration of the subgraph based on multiple target tensor sizes, wherein the multiple target tensor sizes are set by the user through the GUI for the last operation task in the subgraph; and generating a tiled tensor graph based on the tiling configuration, wherein the tiled tensor graph includes at least multiple operation subtasks segmented from the corresponding operation tasks in the subgraph through tiling.

[0005] In some embodiments, the method further includes: configuring wake-up signals and wait signals for establishing a pipeline mechanism for the tiled tensor through the GUI to generate a tiled and pipelined tensor.

[0006] In some embodiments, the step of determining the subgraph in the tensor graph according to the instruction input through the GUI includes: generating the range of the tensor graph according to the instruction input through the GUI; and determining the subgraph located within the range.

[0007] In some embodiments, the step of determining the subgraph in the tensor graph according to the instruction input through the GUI includes: selecting a first operation task of the tensor graph through the GUI; selecting a second operation task of the tensor graph through the GUI; and determining the subgraph according to the first operation task and the second operation task.

[0008] In some embodiments, the step of determining the subgraph based on the first operation task and the second operation task includes: determining a first task set, wherein the first task set includes all forward-reachable operation tasks obtained from the first operation task; determining a second task set, wherein the second task set includes all backward-reachable operation tasks obtained from the second operation task; and generating the subgraph by crossing the first task set and the second task set.

[0009] In some embodiments, the method further includes: adjusting the multiple target tensor sizes of the subgraph through the GUI; and dividing at least one operation task in the subgraph into multiple operation subtasks according to the multiple target tensor sizes.

[0010] In some embodiments, the multiple target tensor sizes are adjusted by dragging the division points displayed on the GUI or by entering the values ​​of the multiple target tensor sizes into the window displayed on the GUI.

[0011] In some embodiments, the method further includes: generating a path through the GUI for establishing a pipeline mechanism for the plurality of operational subtasks; and assigning a wake-up signal and a wait signal to the tiled tensor based on at least one data flow edge intersecting the path.

[0012] In some embodiments, the source operation subtask of the at least one data flow edge and the destination operation subtask of the data flow edge are two adjacent operation subtasks, the source operation subtask is executed by a first hardware device, and the destination operation subtask is executed by a different second hardware device.

[0013] In some embodiments, the wake-up signal is configured to indicate that the destination operation subtask should be woken up only after the source operation subtask has been completed, and the wait signal is configured to indicate that the execution of the destination operation subtask needs to wait for the completion of the source operation subtask.

[0014] In another embodiment of this disclosure, a tensor graph configuration system is disclosed. The tensor graph configuration system includes memory, a graphical user interface (GUI) device, and a processor. The memory is configured to store tensor graph data. The GUI device is configured to provide a graphical user interface. The processor is coupled to the memory and the GUI device and is configured to: provide a tensor graph via the GUI, wherein the tensor graph includes a plurality of operation tasks; determine a subgraph in the tensor graph according to instructions input via the GUI, wherein the subgraph includes at least one of the plurality of operation tasks; determine a tiling configuration of the subgraph according to a plurality of target tensor sizes, wherein the plurality of target tensor sizes are set via the GUI for the last operation task in the subgraph; and generate a tiled tensor graph according to the tiling configuration, wherein the tiled tensor graph includes a plurality of operation subtasks segmented by tiling the corresponding operation tasks in the subgraph.

[0015] In some embodiments, the processor is also configured to perform the following operations: configure wake-up and wait signals for the tiled tensor graph through the GUI to establish a pipeline mechanism to generate a tiled and pipelined tensor graph.

[0016] In some embodiments, the operation of determining the subgraph in the tensor graph according to the instruction input through the GUI includes: generating the range of the tensor graph according to the instruction input through the GUI; and determining the subgraph within the range.

[0017] In some embodiments, the operation of determining the subgraph in the tensor graph according to the instruction input through the GUI includes: selecting a first operation task of the tensor graph through the GUI; selecting a second operation task of the tensor graph through the GUI; and determining the subgraph according to the first operation task and the second operation task.

[0018] In some embodiments, the operation of determining the subgraph based on the first operation task and the second operation task includes: determining a first task set, wherein the first task set includes all forward-reachable operation tasks obtained from the first operation task; determining a second task set, wherein the second task set includes all backward-reachable operation tasks obtained from the second operation task; and generating the subgraph by crossing the first task set and the second task set.

[0019] In some embodiments, the processor is further configured to perform the following operations: adjust the size of the plurality of target tensors of the subgraph through the GUI; and divide at least one operation task in the subgraph into a plurality of operation subtasks according to the plurality of target tensor sizes.

[0020] In some embodiments, the multiple target tensor sizes are adjusted by dragging the division points displayed on the GUI or by entering the values ​​of the multiple target tensor sizes into the window displayed on the GUI.

[0021] In some embodiments, the processor is further configured to perform the following operations: generate a path through the GUI for establishing a pipeline mechanism for the plurality of operational subtasks; and assign a wake-up signal and a wait signal to the tiled tensor based on at least one data flow edge intersecting the path.

[0022] In some embodiments, the source operation subtask of the at least one data flow edge and the destination operation subtask of the data flow edge are two adjacent operation subtasks, the source operation subtask is executed by a first hardware device, and the destination operation subtask is executed by a different second hardware device.

[0023] In some embodiments, the wake-up signal is configured to indicate that the destination operation subtask should be woken up only after the source operation subtask has been completed, and the wait signal is configured to indicate that the execution of the destination operation subtask needs to wait for the completion of the source operation subtask.

[0024] This disclosure aims to provide a tensor graph configuration method and system that can intuitively adjust the tiling of multiple operation tasks in a tensor graph. The tensor graph configuration method and system use a GUI to generate a tiled tensor graph from the tensor graph based on instructions input through the GUI. Therefore, the tensor graph can be adjusted more intuitively, and the processing efficiency of operation tasks is improved.

[0025] The present invention is provided by way of example and is not intended to be limiting. Other embodiments and advantages are described in the following detailed description. The invention is defined by the claims.

Implementation Method

[0027] The following description illustrates preferred embodiments of the present invention. These embodiments are merely illustrative of the technical features of the invention and are not intended to limit the scope of the invention. Certain terms are used throughout this specification and the claims to refer to specific components. Those skilled in the art will understand that manufacturers may use different terms to refer to the same components. This specification and the claims do not distinguish components by differences in name, but by differences in function. The scope of the invention should be determined with reference to the appended claims. The terms "comprising" and "including" used in the following description and claims are open-ended terms and should be interpreted as "comprising, but not limited to...". Furthermore, the term "coupled" means an indirect or direct electrical connection. Therefore, if a device is described as coupled to another device, it means that the device can be directly electrically connected to the other device, or indirectly electrically connected to the other device through other devices or connection means. The terms "basically" or "roughly" used in this text refer to a technical problem that can be solved and the desired technical effect achieved by someone with ordinary knowledge in the relevant technical field, within an acceptable range. For example, "roughly equal to" refers to a method that, without affecting the correctness of the result, is acceptable to someone with ordinary knowledge in the relevant technical field, with a certain margin of error compared to "exactly equal to".

[0028] Machine code running on AI (artificial intelligence) accelerators within mobile devices (e.g., smartphones) typically needs to be compiled on a computer (e.g., a compiler). The compilation process involves translating the raw code written by the developer into machine code that the hardware of the AI ​​accelerator can execute. The compilation process may include optimization steps to ensure that the machine code can run efficiently on the AI ​​accelerator. After compilation and optimization, the generated machine code (also known as an executable file or binary file) is deployed to the AI ​​accelerator on the mobile device for execution. This disclosure relates to optimization processes to improve the execution efficiency of the machine code generated by the compilation process on the AI ​​accelerator, for example, by making efficient use of cache, reducing DRAM usage requirements, and / or shortening the execution cycle, etc. In some implementations, users need to specify tiling through a programming language. However, due to the extreme complexity of deep neural networks, it is difficult for users to manually specify tiling in a text-based environment. In this embodiment of the invention, this disclosure introduces a tensor graph configuration system / method that provides a user-friendly and intuitive technique for configuring tiling and pipeline mechanisms through a graphical user interface (GUI). By displaying the architecture of a deep neural network (e.g., a tensor graph) through the GUI, users can intuitively select the parts to be tiled and how to tile them. Specifically, by tiling the data flow shown in the tiled tensor graph, this embodiment provides a method that allows users to insert or configure wake-up and wait signals or operations through the GUI to create a pipeline mechanism. The proposed implementation allows users to specify where and how to tile through the GUI, and also allows users to specify where to insert wake-up and wait signals through the GUI. Understandably, a pipeline mechanism in a tensor graph refers to a computational optimization strategy that improves efficiency by allowing different operational tasks to be executed concurrently while data is ready (wake-up) and data is waiting (wait). This mechanism allows the computation process to proceed like a factory assembly line, with overlapping stages rather than being executed sequentially one after another.

[0029] FIG1 illustrates a block diagram of a tensor graph configuration system 100 according to an embodiment of the present invention. The tensor graph configuration system 100 can provide an intuitive method for adjusting the configuration of a tensor graph. The tensor graph configuration system 100 includes a memory 10, a graphical user interface device 11, and a processor 12. The memory 10 is used to store or store tensor graph data or source code; for example, the tensor graph can be obtained by the processor 12 based on the tensor graph data. The graphical user interface device 11 can be used to provide a graphical user interface. For example, the graphical user interface device 11 can be a computer touch screen capable of providing interactive operation for a user. The processor 12 is coupled to the memory 10 and the graphical user interface device 11. The processor 12 is configured to adjust the tensor graph through the graphical user interface, for example, in conjunction with the memory 10 and the graphical user interface device 11. Here, the tensor graph can include multiple operation tasks. In other words, the tensor graph can be viewed as a topological diagram of the data processing flow of multiple operation tasks. In the tensor graph configuration system 100, processor 12 is configured to provide tensor graphs via a graphical user interface (GUI) based on data stored in memory (e.g., the source code of the tensor graph), wherein the tensor graph includes multiple operation tasks, i.e., the tensor graph to be adjusted is displayed on a screen via the GUI. Processor 12 is then configured to receive instructions input via the GUI (e.g., user input) and determine subgraphs within the tensor graph based on the instructions input via the GUI. Processor 12 can then tile each operation task within the subgraph according to multiple target tensor sizes set for one or more final operation tasks within the subgraph. For example, the multiple target tensor sizes for the final operation tasks within the subgraph are set via the GUI (e.g., by the user), and the final operation tasks within the subgraph are determined to be divided into multiple operation sub-tasks according to the multiple target tensor sizes, wherein the original tensor size of the final operation task is the sum of the multiple target tensor sizes set for that final operation task. Then, a pre-defined backward shape derivation algorithm (designed to derive appropriate tensor sizes and operation (OP) properties in a reverse manner) starts from the last operation task within the subgraph and moves towards the initial operation task within the subgraph, deriving the correct tensor sizes and OP properties for the remaining operation tasks in the subgraph. Therefore, the tiling configuration of all operation tasks within the subgraph can be determined based on multiple target tensor sizes specified by the user for the last operation task in the subgraph via the GUI.Furthermore, processor 12 is configured to generate a tiled tensor graph via a GUI based on a tiled configuration, wherein each operation task within a subgraph is tiled in the tiled tensor graph, i.e., each operation task within a subgraph is divided into multiple operation subtasks. In embodiments, the term "OP" is used interchangeably to refer to an operation task or an operation subtask. In embodiments, multiple operation subtasks corresponding to an operation task are collectively configured to perform the same function as the original operation task. The tiled tensor graph includes at least multiple operation subtasks after each operation task within a subgraph has been tiled. The tiled (updated) tensor graph is saved to memory 10. Furthermore, processor 12 is configured to insert or configure wake-up and wait signals for the tiled tensor graph (particularly for two corresponding OPs adjacent in the data flow) to establish a pipeline mechanism via a graphical user interface. In embodiments, the two corresponding OPs are executed by different hardware devices. In one example, the different hardware devices can be specified by the user via a GUI. In another example, the different hardware devices can be determined based on the function that the respective OP intends to perform. In a tensor graph, operations implemented by different hardware devices can be displayed in different ways, for example, by using different background colors, thus enabling users to easily build pipelines through a GUI. The details of performing tensor graph configuration through the tensor graph configuration system 100 will be described below.

[0030] Figure 2A shows the original tensor graph provided by the tensor graph configuration system 100 via a GUI. Figure 2B shows the tiled tensor graph after operation tasks TA and TB have been tiled. Figure 2C shows the tiled and pipelined tensor graph after operation tasks TA and TB have been tiled and pipelined. As previously mentioned, a tensor graph can be viewed as a topological diagram of the data processing flow of multiple operation tasks. In one example, the operations within a subgraph can be performed by different hardware devices (also referred to as different "hardware engines"). For example, a first operation task can be performed by a first hardware device. A second operation task can be performed by a second hardware device. The first hardware device and the second hardware device are different. For example, in Figure 2A, operation task TA could be the process of converting Bayer data into three primary color (RGB) data. Operation task TB could be a two-dimensional (2D) convolutional image processing task. Therefore, when the machine code obtained from the tensor graph shown in Figure 2A is executed on a tensor processing unit (e.g., an AI accelerator), operation task TA can be executed by the first hardware device after receiving input data. Then, after operation task TA completes, operation task TB can be executed by the second hardware device.

[0031] In Figure 2B, operation tasks TA and TB in Figure 2A are tiled by processor 12. For example, operation task TA can be divided into operation subtasks TA1 and TA2. Operation task TB can be divided into operation subtasks TB1 and TB2. Therefore, when the machine code obtained from the tiled tensor graph shown in Figure 2B is executed on the tensor processing unit (e.g., an AI accelerator), after operation subtask TA1 receives input data, the first hardware device executes operation subtask TA1. Then, after operation subtask TA1 is completed, the first hardware device executes operation subtask TA2. Then, after all operation subtasks corresponding to operation task TA are completed, the second hardware device executes operation subtask TB1. Then, after operation subtask TB1 is completed, the second hardware device executes operation subtask TB2. In Figure 2B, since the branches of the original data flow in Figure 2A are tiled, the tensor processing unit's demand for external memory can be reduced.

[0032] In Figure 2C, the branches of the original data flow in Figure 2A are flattened and pipelined by processor 12. Compared with Figure 2B, as shown in Figure 2C, a first wake-up signal WK1 can be applied / configured for operation subtask TA1, wherein the first wake-up signal WK1 is used to indicate that operation subtask TB1 should be woken up only after operation subtask TA1 is completed. A first wait signal WT1 can be applied / configured for operation subtask TB1, wherein the first wait signal WT1 is used to indicate that the execution of operation subtask TB1 should wait for the completion of operation subtask TA1. A second wake-up signal WK2 can be applied / configured for operation subtask TA2, wherein the second wake-up signal WK2 is used to indicate that operation subtask TB2 should be woken up only after operation subtask TA2 is completed. A second wait signal WT2 can be applied / configured for operation subtask TB2, wherein the second wait signal WT2 is used to indicate that the execution of operation subtask TB2 should wait for the completion of operation subtask TA2. Therefore, when the machine code obtained from the tiled and pipelined tensor graph shown in Figure 2C is executed on a tensor processing unit (e.g., an AI accelerator), operation subtask TA1 is executed by the first hardware device. After operation subtask TA1 is completed, operation subtask TA2 and operation subtask TB1 can be executed simultaneously by different hardware devices through the coordination of wake-up and wait signals. After operation subtask TA2 and operation subtask TB1 are completed, operation subtask TB2 is executed by the second hardware device through the coordination of wake-up signal WK2 and wait signal WT2. Therefore, since the operation tasks TA and TB in Figure 2A are tiled and piped by the processor 12, where the original operation tasks are tiled into smaller operation subtasks (i.e., the output tensor size of the operation subtasks is smaller than the output tensor size of the original operation task), and some operation subtasks (e.g., operation subtasks TA2 and TB1) can be executed in parallel (pipelined) by different hardware devices, the DRAM usage requirement can be reduced and the processing efficiency improved (e.g., execution time reduced) when the machine code obtained from the updated / adjusted (tiled and piped) tensor graph is executed by the tensor processing unit (e.g., an AI accelerator). Such smaller datasets can be accommodated in the cache of the tensor processing unit, thereby reducing the need for external DRAM.

[0033] In another embodiment, the operational tasks within the subgraph can also be performed by the same hardware device, and further, the pipelined process shown in FIG2C can be omitted. Similarly, since the operational tasks TA and TB in FIG2A are tiled into smaller subtasks (e.g., with smaller output tensors), the need for external DRAM is reduced when the tensor processing unit (e.g., an AI accelerator) executes machine code from the tiled tensor graph. For ease of illustration and clarity, the present invention is illustrated by examples including different hardware devices, but the invention is not limited thereto. Details of applying the tiling and pipelined mechanisms to the operational tasks of the tensor graph through a graphical user interface are described below.

[0034] Figure 3 is a schematic diagram of determining a subgraph within a tensor graph through a graphical user interface (GUI) of the tensor graph configuration system 100. In Figure 3, multiple operation tasks T1 to T7 are introduced. These operation tasks T1 to T7 can be displayed through a graphical user interface (GUI). The processor 12 can generate a range R of the tensor graph through the GUI. Here, the range R can be defined by the user for selecting at least one operation task. After generating the range R, the processor 12 can determine a subgraph based on the range R. For example, the tensor graph includes operation tasks T1 to T7. The subgraph includes a subset of operation tasks T1 to T7, such as including operation tasks T3, T4, T5, and T6. Alternatively, the subgraph can be determined by any reasonable selection method. In this embodiment, the subgraph can be defined as the set of all operation tasks located within the range R, where the range R is defined by the user interacting directly with the tensor graph through the GUI, for example, by selecting a desired area.

[0035] Figure 4 is a schematic diagram of determining a subgraph by selecting a first operation task T2 and a second operation task T7 of a tensor graph through a graphical user interface of the tensor graph configuration system 100. In Figure 4, multiple operation tasks T1 to T7 are introduced. These operation tasks T1 to T7 can be displayed on the graphical user interface. Then, the processor 12 can determine the first operation task T2 in the tensor graph through the graphical user interface. The first operation task T2 can be selected by the user through the graphical user interface (GUI). Similarly, the processor 12 can determine the second operation task T7 in the tensor graph through the graphical user interface. The second operation task T7 can also be selected by the user through the graphical user interface (GUI). Therefore, the processor 12 can detect a first task set containing all forward-reachable operation tasks starting from the first operation task T2. For example, when the first operation task T2 is determined, the first task set can be represented as {T2, T3, T4, T5, T6, T7}. Furthermore, the processor 12 can detect a second task set containing all backward-reachable operation tasks starting from the second operation task T7. For example, when the second operation task T7 is determined, the second task set can be represented as {T7, T6, T5, T4, T3, T2, T1}. Then, the processor 12 can determine a subgraph by interleaving the first task set {T2, T3, T4, T5, T6, T7} and the second task set {T7, T6, T5, T4, T3, T2, T1}. Therefore, the subgraph can be determined to contain operation tasks {T2, T3, T4, T5, T6, T7}. Here, the first operation task T2 can be considered as the beginning / initial operation task in the subgraph. The second operation task T7 can be considered as the end / final operation task in the subgraph.

[0036] Figure 5 is a schematic diagram of the graphical user interface of the tensor graph configuration system 100. As previously described, the tensor graph TG1 can be displayed on window W1 of the graphical user interface. In this embodiment, the tiling configuration for all operation tasks in the subgraph can be determined based on multiple target tensor dimensions for the last operation task in the subgraph, wherein these multiple target tensor dimensions can be defined by the user through the graphical user interface (GUI). Here, the multiple target tensor dimensions for the last operation task in the subgraph can be adjusted by dragging the division point SP displayed on the graphical user interface or by inputting the values ​​of the target tensor dimensions into windows W2 and W3 displayed on the graphical user interface. Window W2 can be used to input the target tensor dimensions on the horizontal axis. Window W3 can be used to input the target tensor dimensions on the vertical axis. Windows W2 and W3 can also be used to display the current tiling size of the subgraph of tensor graph TG1. Window W4, as shown in Figure 5, can be used to display a tiling example of the last operation task. Button B1 can be used to perform an equal splitting process on the subgraph of tensor graph TG1. Button B2 can be used to introduce new splitting points for the final operation task in the subgraph. Button B3 is the confirm button. Button B4 is the cancel button. After determining multiple target tensor sizes for the final operation task in the subgraph, processor 12 can determine the tiling configuration of the final operation task in the subgraph. As mentioned earlier, the tiling configurations of other operation tasks in the subgraph can also be determined, for example, based on a preset backward shape derivation algorithm. Therefore, processor 12 can divide each branch of the subgraph into multiple tiling branches based on the multiple target tensor sizes of the final operation task in the subgraph or the tiling configuration of all operation tasks in the subgraph. Details will be explained later.

[0037] Figure 6 is a schematic diagram of the updated or tiled tensor graph of the tensor graph configuration system 100 after adjustment via a graphical user interface (GUI). In Figure 6, the branches of the subgraph of tensor graph TG1 are divided into four branches. These four branches can be executed one after another. After the processor 12 divides the branches of the subgraph of tensor graph TG1, the updated or tiled tensor graph TG2 can be displayed on window W5 through the graphical user interface. Furthermore, since each branch requires less storage space for computation, the storage requirements of the OPs of tiling tensor graph TG2 can be reduced. Cache jitter can also be reduced.

[0038] Figure 7 is a schematic diagram of creating a pipeline mechanism for a tiled tensor graph TG2 through a graphical user interface (GUI) of the tensor graph configuration system 100. After generating the tiled tensor graph TG2, wake-up signals and wait signals can be applied to the tiled tensor graph TG2 through the GUI to create a pipeline mechanism, as shown below. The processor 12 can generate a line (or path) L through the GUI to segment multiple operations of the tiled tensor graph TG2. The line L can be defined by the user through the GUI. The processor 12 can then identify at least one data flow edge in the updated tensor graph based on the line L, for example, at least one data flow edge intersects with the line L in the updated tensor graph. The processor 12 can then assign wake-up signals and wait signals to the tiled tensor graph TG2 based on at least one data flow edge. For example, for the data flow edge D1 shown in Figure 7, the processor 12 can assign a wake-up signal after the source operation of the data flow edge D1. As shown in Figure 7, the source OP of data flow edge D1 is operation subtask STK. Processor 12 can allocate a wait signal before the destination OP of data flow edge D1. As shown in Figure 7, the destination OP of data flow edge D1 is operation subtask DTK. Specifically, the source OP (e.g., operation subtask STK) and the destination OP (e.g., operation subtask DTK) are two adjacent OPs associated with data flow edge D1. The source OP is executed by one hardware device. The destination OP is executed by another hardware device. In this embodiment, a wake-up signal is used to indicate that the destination OP (e.g., operation subtask DTK) is woken up only after the source OP (e.g., operation subtask STK) has completed, and a wait signal is used to indicate that the execution of the destination OP (e.g., operation subtask DTK) should wait for the completion of the source OP (e.g., operation subtask TA1). Therefore, in the embodiments proposed in this invention, a graphical user interface (GUI) is provided to allow users to drag a line L to segment the tiled tensor graph TG2. The line L can be a curve or a straight line, so the wake-up signal and the wait signal can be easily allocated to the tiled tensor graph TG2 to create a pipeline mechanism. Because a pipeline mechanism can be introduced, processing efficiency can be improved.

[0039] Figure 8 is a schematic diagram of the tensor graph configuration method executed by the tensor graph configuration system 100. The tensor graph configuration method includes steps S801 to S804. Any reasonable technical modifications fall within the scope of this disclosure. Steps S801 to S804 are as follows.

[0040] Step S801: Provide tensor graph TG1 through GUI, which includes multiple operation tasks.

[0041] Step S802: Determine a subgraph of tensor graph TG1 based on instructions input through the GUI, wherein the subgraph includes at least one of the plurality of operation tasks.

[0042] Step S803: Determine the tiling configuration of the subgraph based on multiple target tensor sizes, wherein the multiple target tensor sizes are set by the user through the GUI for the end operation task in the subgraph.

[0043] Step S804: Generate a tiled tensor map according to the tiled configuration, wherein the tiled tensor map includes multiple operation subtasks divided by tiling the corresponding operation tasks in the subgraph.

[0044] The details of steps S801 to S804 have already been described above. Therefore, they are omitted here. In the tensor graph configuration system 100, since the tiling mechanism and pipeline mechanism of the tensor graph can be configured using a graphical user interface, the compiler optimization process can be easily implemented. Therefore, the tensor graph can be adjusted intuitively.

[0045] In summary, this invention discloses a tensor graph configuration method and a tensor graph configuration system, which provide a solution for intuitively adjusting tensor graphs using a graphical user interface. For example, the tiling mechanism of the tensor graph can be configured by dragging split points or by directly inputting the target tiling size through the graphical user interface. Furthermore, the pipeline mechanism of the tensor graph can be configured by inserting wake-up signals and wait signals according to the path dragged by the user. Therefore, the tensor graph can be intuitively adjusted, and the adjusted tensor graph can reduce the use of external memory while maintaining high processing efficiency.

[0046] The use of ordinal terms such as “first,” “second,” “third” to modify the elements of a patent application in the scope of the patent application does not in itself indicate any priority, priority, or order of one element of a patent application relative to another element of a patent application, or the temporal order of performing method actions, but is only used as a marker to distinguish one element of a patent application with the same name from another element element with the same name.

[0047] Although the embodiments and advantages of the present invention have been described in detail, it should be understood that various changes, substitutions, and modifications can be made to the present invention without departing from the spirit and scope defined by the claims. For example, new embodiments can be derived by combining several parts of different embodiments. The described embodiments are for illustrative purposes only and are not intended to limit the present invention. The scope of protection of the present invention shall be determined by the appended claims. Those skilled in the art can make some modifications and refinements without departing from the spirit and scope of the present invention. The above description is only a preferred embodiment of the present invention, and all equivalent changes and modifications made in accordance with the claims of the present invention should be included within the scope of the present invention. [Simplified Explanation of the Diagram]

[0026] The accompanying drawings (in which the same numerals denote the same components) illustrate embodiments of the present invention. The drawings are included to provide a further understanding of embodiments of the present disclosure, and are incorporated in and constitute a part of the present disclosure. The drawings illustrate implementations of embodiments of the present disclosure and, together with the description, serve to explain the principles of the embodiments of the present disclosure. It is understood that the drawings are not necessarily drawn to scale, as some components may be shown out of scale to clearly illustrate the concepts of the embodiments of the present disclosure. FIG1 shows a schematic block diagram of a tensor graph configuration system according to an embodiment of the present invention. FIG2A shows an original tensor graph through a graphical user interface of the tensor graph configuration system of FIG1. ​​FIG2B shows a tiled tensor graph of the tensor graph configuration system of FIG1 (the tensor graph after the operation task is tiled). FIG2C shows a tiled and pipelined tensor graph of the tensor graph configuration system of FIG1 (the tensor graph after the operation task is tiled and pipelined). FIG3 is a schematic diagram of determining subgraphs within a tensor graph through a graphical user interface of the tensor graph configuration system of FIG1. Figure 4 is a schematic diagram of selecting the first and second operation tasks of a tensor graph to determine a subgraph through the graphical user interface of the tensor graph configuration system in Figure 1. Figure 5 is a schematic diagram of the graphical user interface of the tensor graph configuration system in Figure 1. Figure 6 is a schematic diagram of a tiled tensor graph (the tensor graph after adjustment through the graphical user interface) of the tensor graph configuration system in Figure 1. Figure 7 is a schematic diagram of creating a pipeline mechanism for a tiled tensor graph through the graphical user interface of the tensor graph configuration system in Figure 1. Figure 8 is a schematic diagram of executing the tensor graph configuration method of the tensor graph configuration system in Figure 1. In the following detailed description, numerous specific details are set forth for illustrative purposes so that those skilled in the art will have a more thorough understanding of the embodiments of the invention. However, it will be apparent that one or more embodiments may be practiced without these specific details, and different embodiments or different features disclosed in different embodiments may be combined as needed, and should not be limited to the embodiments illustrated in the accompanying drawings.

Claims

1. A tensor graph configuration method, executed by a processor, the method comprising: A tensor graph is provided through a graphical user interface (GUI), wherein the tensor graph includes multiple operation tasks; a subgraph in the tensor graph is determined based on instructions received through the GUI, wherein the subgraph includes at least one of the multiple operation tasks; a tiling configuration of the subgraph is determined based on multiple target tensor sizes, wherein the multiple target tensor sizes are set through the GUI for the last operation task in the subgraph; and a tiled tensor graph is generated based on the tiling configuration, wherein the tiled tensor graph includes multiple operation subtasks segmented by tiling the corresponding operation tasks in the subgraph.

2. The method as described in request item 1, wherein, The method also includes configuring wake-up and wait signals for the tiled tensor graph through the GUI to establish a pipeline mechanism, so as to generate a tiled and pipelined tensor graph.

3. The method as described in request item 1, wherein, The step of determining the subgraph in the tensor graph according to the instructions input through the GUI includes: generating the range of the tensor graph according to the instructions input through the GUI; and determining the subgraph located within the range.

4. The method as described in request item 1, wherein, The steps of determining the subgraph in the tensor graph according to the instructions input through the GUI include: selecting a first operation task of the tensor graph through the GUI; selecting a second operation task of the tensor graph through the GUI; and determining the subgraph based on the first operation task and the second operation task.

5. The method as described in request item 4, wherein, The steps of determining the subgraph based on the first operation task and the second operation task include: determining a first task set, wherein the first task set includes all forward-reachable operation tasks obtained from the first operation task; determining a second task set, wherein the second task set includes all backward-reachable operation tasks obtained from the second operation task; and generating the subgraph by crossing the first task set and the second task set.

6. The method as described in request item 1, wherein, The method further includes: adjusting the size of the plurality of target tensors of the subgraph through the GUI; and dividing at least one operation task within the subgraph into a plurality of operation subtasks according to the size of the plurality of target tensors.

7. The method as described in claim 6, wherein, The dimensions of these target tensors are adjusted by dragging the division points displayed on the GUI or by entering the values ​​of these target tensors into the window displayed on the GUI.

8. The method as described in request item 1, wherein, The method further includes: generating a path through the GUI for establishing a pipeline mechanism for the plurality of operational subtasks; and assigning wake-up signals and wait signals to the tiled tensor graph based on at least one data flow edge intersecting the path.

9. The method as described in claim 8, wherein, The source operation subtask of the at least one data flow edge and the destination operation subtask of the data flow edge are two adjacent operation subtasks, the source operation subtask is executed by a first hardware device, and the destination operation subtask is executed by a different second hardware device.

10. The method as described in claim 9, wherein, The wake-up signal is configured to indicate that the destination operation subtask should be woken up only after the source operation subtask has completed, and the wait signal is configured to indicate that the execution of the destination operation subtask must wait for the completion of the source operation subtask.

11. A tensor graph configuration system, comprising: The memory is configured to store tensor graph data; A graphical user interface device is configured to provide a graphical user interface (GUI). The processor, coupled to the memory and the GUI device, is configured to perform the following operations: providing a tensor graph via the GUI, wherein the tensor graph includes a plurality of operational tasks; determining a subgraph in the tensor graph based on instructions received via the GUI, wherein the subgraph includes at least one of the plurality of operational tasks; determining a tiling configuration of the subgraph based on a plurality of target tensor sizes, wherein the plurality of target tensor sizes are set via the GUI for the last operational task in the subgraph; and generating a tiled tensor graph based on the tiling configuration, wherein the tiled tensor graph includes a plurality of operational subtasks segmented by tiling the corresponding operational tasks in the subgraph.

12. The system as described in claim 11, wherein, The processor is also configured to perform the following operations: Configure wake-up and wait signals for the tiled tensor through the GUI to establish a pipeline mechanism to generate a tiled and pipelined tensor.

13. The system as described in claim 11, wherein, The operation of determining the subgraph in the tensor graph according to the instruction input through the GUI includes: generating the range of the tensor graph according to the instruction input through the GUI; and determining the subgraph within the range.

14. The system as described in claim 11, wherein, The operation of determining the subgraph in the tensor graph according to the instructions input through the GUI includes: selecting a first operation task of the tensor graph through the GUI; selecting a second operation task of the tensor graph through the GUI; and determining the subgraph based on the first operation task and the second operation task.

15. The system as described in claim 14, wherein, The operations for determining the subgraph based on the first operation task and the second operation task include: determining a first task set, wherein the first task set includes all forward-reachable operation tasks obtained from the first operation task; determining a second task set, wherein the second task set includes all backward-reachable operation tasks obtained from the second operation task; and generating the subgraph by crossing the first task set and the second task set.

16. The system as described in claim 11, wherein, The processor is also configured to perform the following operations: adjust the size of the plurality of target tensors in the subgraph via the GUI; and divide at least one operation task in the subgraph into a plurality of operation subtasks based on the size of the plurality of target tensors.

17. The system as described in claim 16, wherein, The dimensions of these target tensors are adjusted by dragging the division points displayed on the GUI or by entering the values ​​of these target tensors into the window displayed on the GUI.

18. The system as described in claim 11, wherein, The processor is also configured to perform the following operations: generate a path through the GUI for establishing a pipeline mechanism for the plurality of operational subtasks; and assign wake-up and wait signals to the tiled tensor graph based on at least one data flow edge intersecting the path.

19. The system as described in claim 18, wherein, The source operation subtask of the at least one data flow edge and the destination operation subtask of the data flow edge are two adjacent operation subtasks, the source operation subtask is executed by a first hardware device, and the destination operation subtask is executed by a different second hardware device.

20. The system as described in claim 19, wherein, The wake-up signal is configured to indicate that the destination operation subtask should be woken up only after the source operation subtask has completed, and the wait signal is configured to indicate that the execution of the destination operation subtask must wait for the completion of the source operation subtask.

Images