Task execution apparatus, method, electronic device, and storage medium

Abstract

The present disclosure provides a task execution device, relates to the technical field of artificial intelligence, and particularly relates to the technical field of chips and the technical field of integrated circuits. The device comprises a cache unit and a processor configured to: acquire an operator information set for a target task, wherein the operator information set comprises a plurality of operator information, and the plurality of operator information respectively represents a plurality of operators to be executed in sequence; determine at least one continuous operator group from the plurality of operators, wherein the continuous operator group comprises at least two continuous operators, and the scale of the output tensor of any operator in the continuous operator group is greater than or equal to a preset scale threshold; fuse the at least one continuous operator group respectively to obtain at least one fused operator; in response to determining that to-be-processed data corresponding to the fused operator is stored to the cache unit, load the to-be-processed data from the cache unit; and execute the target task by using the to-be-processed data. The present disclosure also provides a task execution method, device and medium.

Description

Task execution apparatus, method, electronic device and storage medium Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, and more particularly to the field of chip technology and integrated circuit technology. More specifically, this disclosure provides a task execution apparatus, method, electronic device, and storage medium. Background Technology

[0002] With the development of artificial intelligence technology, the amount of data involved in some operators of deep learning models is getting larger and larger. Summary of the Invention

[0003] This disclosure provides a data processing apparatus, method, device, and storage medium.

[0004] According to one aspect of this disclosure, a task execution apparatus is provided, the apparatus comprising: a cache unit; and a processor configured to: acquire an operator information set for a target task, wherein the operator information set includes a plurality of operator information, the plurality of operator information representing a plurality of operators to be executed sequentially; determine at least one consecutive operator group from the plurality of operators, wherein the consecutive operator group includes at least two consecutive operators, and the scale of the output tensor of any operator in the consecutive operator group is greater than or equal to a preset scale threshold; fuse the at least one consecutive operator group to obtain at least one fused operator; in response to determining that data to be processed corresponding to the fused operator is stored in the cache unit, load the data to be processed from the cache unit; and execute the target task using the data to be processed.

[0005] According to another aspect of this disclosure, a task execution method is provided, the method comprising: acquiring an operator information set for a target task, wherein the operator information set includes multiple operator information, each of the multiple operator information representing multiple operators to be executed sequentially; determining at least one consecutive operator group from the multiple operators, wherein the consecutive operator group includes at least two consecutive operators, and the scale of the output tensor of any operator in the consecutive operator group is greater than or equal to a preset scale threshold; fusing the at least one consecutive operator group to obtain at least one fused operator; loading the data to be processed from the cache unit in response to determining that the data to be processed corresponding to the fused operator is stored in a cache unit; and executing the target task using the data to be processed.

[0006] According to another aspect of this disclosure, an integrated circuit chip is provided, including the task execution device provided in this disclosure.

[0007] According to another aspect of this disclosure, an electronic device is provided, including the integrated circuit chip provided in this disclosure.

[0008] According to another aspect of this disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform a method provided according to this disclosure.

[0009] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions for causing a computer to perform the methods provided according to this disclosure.

[0010] According to another aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the method provided according to this disclosure.

[0011] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0012] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0013] Figure 1 is a schematic block diagram of a task execution apparatus according to an embodiment of the present disclosure;

[0014] Figure 2A is a schematic diagram of a plurality of operators according to an embodiment of the present disclosure;

[0015] Figure 2B is a schematic diagram of a fusion operator according to an embodiment of the present disclosure;

[0016] Figure 3 is a schematic block diagram of an integrated circuit chip according to an embodiment of the present disclosure;

[0017] Figure 4 is a schematic block diagram of an integrated circuit chip according to an embodiment of the present disclosure;

[0018] Figure 5 is a flowchart of a task execution method according to an embodiment of the present disclosure; and

[0019] Figure 6 is a block diagram of an electronic device to which a task execution method can be applied according to an embodiment of the present disclosure. Detailed Implementation

[0020] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0021] Artificial Intelligence (AI) chips can have a multi-level storage architecture, which can move data from the Global Memory (GM) unit to the Local Memory (LM) unit for data processing in the local storage unit.

[0022] In some embodiments, a deep learning model may include multiple operators. The size of the input tensor of an operator can be represented as [n, c, h, w]. Taking the correspondence between the input tensor and an image as an example, n can be the batch size, c can be the number of channels, h can be the height of the image, and w can be the width of the image. For example, if the image is an RGB image, the value of n can be 1, 2, 4, 6, 8, 24, 32, etc., and the value of c can be 3.

[0023] During model inference, the size (shape) of the operator's input tensor can be large. The number of batches of this input tensor can be large. The data volume of the input tensor may exceed the capacity of the Level 3 cache (L3 cache) of an AI chip. Therefore, it is difficult to store the entire input tensor in the L3 cache. The input tensor can be divided into K input sub-tensors, where K is an integer greater than 1 and less than or equal to n. The k-th input sub-tensor can be stored in the L3 cache. The operator uses the k-th input sub-tensor for data processing to obtain the k-th first output sub-tensor. The k-th first output sub-tensor can be stored in the L3 cache. Next, the k-th input sub-tensor and the k-th first output sub-tensor can be written to the global storage unit, and the (k+1)-th input sub-tensor can be loaded from the global storage unit into the L3 cache. k can be an integer greater than or equal to 1 and less than K. After the operator performs multiple data processing operations using K input subtensors, its execution is complete, resulting in K first output subtensors stored in the global storage unit. Next, subsequent operators can be run. These subsequent operators can use the K first output subtensors as input. Therefore, the subsequent operators can load the K first output subtensors multiple times from the global storage unit into the L3 cache.

[0024] Therefore, during the process of the operator processing K input subtensors, multiple data loading and storage operations occur between the global storage unit and the three-level cache. Extensive data loading and storage can increase the time cost of model execution, leading to a decrease in chip performance.

[0025] To improve the performance of artificial intelligence chips, this disclosure provides a task execution device, which will be described below.

[0026] Figure 1 is a schematic block diagram of a task execution apparatus according to an embodiment of the present disclosure.

[0027] As shown in Figure 1, the device 100 may include a cache unit 110 and a processor 120.

[0028] Cache unit 110 can be the aforementioned three-level cache.

[0029] Processor 120 can be configured to acquire a set of operator information for the target task.

[0030] In this embodiment of the disclosure, the target task can be an image processing task. In this image processing task, multiple convolution operators can perform multiple convolutions on the image.

[0031] In embodiments of this disclosure, the operator information set may include multiple operator information sets. Each multiple operator information set represents a multiple operator to be executed sequentially. For example, the multiple operators may include multiple convolution operators to be executed sequentially.

[0032] The processor 120 can also be configured to determine at least one group of consecutive operators from a plurality of operators. For example, a group of consecutive operators may include at least two consecutive operators. The size of the output tensor of any operator in a group of consecutive operators may be greater than a preset size threshold. The number of operators in different groups of consecutive operators may be different.

[0033] The processor 120 can also be configured to fuse at least one group of consecutive operators to obtain at least one fused operator. For example, in the at least one group of consecutive operators, taking a first group of consecutive operators comprising three consecutive operators and a second group of consecutive operators comprising four consecutive operators as an example, the three consecutive operators of the first group of consecutive operators can be fused to obtain the first fused operator. Similarly, the four consecutive operators of the second group of consecutive operators can be fused to obtain the second fused operator.

[0034] The processor 120 can also be configured to load the data to be processed from the cache unit in response to determining that the data to be processed corresponding to the fusion operator is stored in the cache unit. For example, in response to determining that the data to be processed corresponding to the first fusion operator is stored in the L3 cache unit, the data to be processed can be loaded from the L3 cache unit.

[0035] Processor 120 can also be configured to perform a target task using data to be processed. For example, the data to be processed can be input into a fusion operator to obtain the corresponding processing result. The data to be processed can be a part of an input tensor. The fusion operator can be run multiple times, processing a part of the input tensor each time to obtain multiple processing results. Multiple processing results can be used as output tensors corresponding to the input tensors.

[0036] By fusing at least two consecutive operators through the embodiments of this disclosure, the intermediate result can be stored in the cache unit after the preceding operator has finished running, and can be directly used by the subsequent operator. This saves the time and resources required to write the intermediate result to global storage, as well as the time and resources required to load the intermediate result from the global storage unit to the three-level cache unit, which helps to significantly improve the data processing efficiency and performance of artificial intelligence chips.

[0037] It is understood that the processing apparatus of this disclosure has been described above, and the processor of this disclosure will be further described below with reference to Figures 2A to 2B.

[0038] Figure 2A is a schematic diagram of a plurality of operators according to an embodiment of the present disclosure.

[0039] In some embodiments, the processor may be configured to acquire a set of operator information for a target task. For example, the set of operator information may include multiple operator information. Three of the multiple operator information represent operators OP210, OP220, and OP230, which are to be executed sequentially as shown in FIG2A.

[0040] In this embodiment of the disclosure, operator information may include an operator network diagram. The operator network diagram may indicate multiple modules of an operator. For example, operator OP210 may include a convolution module 211, a batch normalization module 212, and a rectified linear unit (ReLU) module 213. Operator OP220 may include a convolution transpose module 221, an add module 222, a batch normalization module 223, and a rectified linear unit (ReLU) module 224. Operator OP230 may include a convolution transpose module 231, an add module 232, and a logistic regression module 233. It can be understood that operators OP210, OP220, and OP230 can be considered as three consecutive operators.

[0041] In this embodiment of the disclosure, the operator network graph can also indicate the relationships between multiple modules. For example, the output of the convolution module 211 can be used as the input of the batch normalization module 212. The output of the batch normalization module 212 can be used as the input of the linear rectification module 213.

[0042] In embodiments of this disclosure, multiple operators may include convolution operators and convolution transpose operators. For example, operator OP210 includes a convolution module 211 and can serve as a convolution operator. Operator OP220 includes a convolution transpose module 221 and can serve as a convolution transpose operator. Operator OP230 includes a convolution transpose module 231 and can also serve as a convolution transpose operator.

[0043] In some embodiments, the processor may be configured to determine at least one group of consecutive operators from a plurality of operators. This will be further explained below with reference to FIG2A.

[0044] In this embodiment of the disclosure, the processor can be configured to: in response to determining that the size of the output tensor of the current operator is greater than or equal to a preset size threshold, determine a first number of subsequent consecutive operators whose output tensor sizes are greater than or equal to the preset size threshold among a plurality of subsequent consecutive operators of the current operator. For example, if the size of the output tensor of operator OP210 is greater than the preset size threshold, it can be determined whether the sizes of the output tensors of operators OP220 and OP230 are each greater than or equal to the preset size threshold. If the sizes of the output tensors of operators OP220 and OP230 are both greater than the preset size threshold and the sizes of the subsequent operators of operator OP230 are less than the preset size threshold, the first number can be determined to be 2.

[0045] In this embodiment, the processor can be configured to: determine a second number based on a first number and the current operator. In response to determining that the second number is greater than or equal to a preset number threshold, the current operator and the first number of subsequent consecutive operators are grouped as a continuous operator group. For example, the preset number threshold can be 3. As mentioned above, the first number is 2, and the number of current operators is 1. Therefore, the second number can be 3, equal to the preset number threshold. Operators OP210, OP220, and OP230 can be grouped as a continuous operator group. Through this embodiment, when the number of large-scale continuous operators exceeds the preset number threshold, grouping multiple large-scale continuous operators as a continuous operator group can efficiently determine the continuous operator group. Therefore, the fusion of multiple operators in the continuous operator group can effectively improve memory access efficiency during model operation.

[0046] In some embodiments, the processor may also be configured to fuse at least one group of consecutive operators to obtain at least one fusion operator. This will be described below with reference to Figure 2B.

[0047] Figure 2B is a schematic diagram of a fusion operator according to an embodiment of the present disclosure.

[0048] As shown in Figure 2B, operators OP210, OP220 and OP230 shown in Figure 2A can be merged to obtain the fused operator OP2123.

[0049] In this embodiment of the disclosure, the processor can also be configured to: determine the largest tensor operator among at least two consecutive operators, whose output tensor is the largest. In this embodiment of the disclosure, the output tensor of the operator is split according to the original batch number to obtain the original batch number of second output sub-tensors. The operator with the largest second output sub-tensor among at least two consecutive operators is determined as the largest tensor operator. For example, the batch number corresponding to the second output sub-tensor can be 1. If the data size of the second output sub-tensor of operator OP210 is 30 megabytes (MByte, MB), and the data size of the second output sub-tensors of operators OP220 and OP230 is less than 30 megabytes, operator OP210 can be used as the largest tensor operator.

[0050] In this embodiment of the disclosure, the processor may also be configured to determine the target batch number based on the output tensor of the maximum tensor operator and the storage capacity of the cache unit.

[0051] For example, the processor can also be configured to: take the modulus of the storage capacity of the cache unit relative to the output tensor of the maximum tensor operator to obtain a modulus. Use this modulus as the target batch number. The storage capacity of the cache unit can be 64 megabytes. Taking the modulus of the storage capacity of the cache unit (64 megabytes) relative to the data size of the second output sub-tensor of the maximum tensor operator (30 megabytes) yields a modulus of 2. Therefore, the target batch number can be 2. Through the embodiments of this disclosure, by determining the target batch number based on the data size of the output tensor of the maximum tensor operator and the capacity of the cache unit, the stability of the task execution device can be improved, thereby improving the performance of the related chip.

[0052] For example, the processor can also be configured to determine the target number of executions of the fusion operator based on the target batch number and the original batch number of the fusion operator. The original batch number can be 24. By using the original batch number as the dividend and the target batch number as the divisor, a division operation can be performed, yielding a result of 12. This result can be used as the target number of executions.

[0053] In some embodiments, the processor may also be configured to load data to be processed from the cache unit in response to determining that data to be processed corresponding to a fusion operator is stored in the cache unit. The data to be processed may include the target batch number of fusion operator data.

[0054] In this embodiment of the disclosure, in response to determining that a target batch number of fusion operator data corresponding to the fusion operator is stored in the cache unit, the target batch number of fusion operator data is loaded from the cache unit. For example, the target number of executions can be M, and the target batch number can be I. I can be an integer greater than or equal to 1. During the m-th execution of the M executions of the fusion operator, after the I fusion operator data corresponding to the m-th execution of the fusion operator is stored in the cache unit, the I fusion operator data corresponding to the M-th execution of the fusion operator can be loaded from the cache unit. m can be an integer greater than or equal to 1 and less than or equal to M, and M is an integer greater than 1. In the case where M = 12 and I = 2, taking m = 1 as an example, after the two fusion operator data related to the first execution of the fusion operator OP2123 are stored in the L3 cache, these two fusion operator data can be loaded from the L3 cache.

[0055] In some embodiments, the processor may also be configured to perform a target task using the data to be processed.

[0056] In this embodiment of the disclosure, the processor can also be configured to execute a target task using fusion operator data according to the target batch size. For example, the target task can be executed using I fusion operator data. When m=1, the target task can be executed using two fusion operator data. After completing the target number of executions (e.g., 12 times), M execution results can be obtained, and the M execution results can be concatenated into the output tensor of fusion operator OP2123. Next, the subsequent operators of fusion operator OP2123 can be executed.

[0057] It is understandable that fusing at least one group of consecutive operators yields at least one fusion operator. Therefore, a deep learning model can include at least one fusion operator and at least one original operator. After both the fusion operator and the original operator have been executed, the output tensor of the deep learning model is obtained. Next, another input tensor can be used to perform the corresponding target task.

[0058] In other embodiments of this disclosure, in response to determining that the second number is less than a preset number threshold, the current operator and the first number of subsequent consecutive operators are not considered as a group of consecutive operators. For example, taking a preset number threshold of 4 as an example, if the current operator is operator OP210, the second number is 3, which is less than the preset number threshold. In this case, operators OP210 to OP230 may not be considered as a group of consecutive operators. It can be determined whether the size of the output tensor of the operators following operator OP230 is greater than or equal to a preset size threshold.

[0059] In other embodiments of this disclosure, if the data size of the output tensor of the maximum tensor operator is greater than the storage capacity of the cache unit, the target batch number can be determined based on the output tensor of the maximum tensor operator and the storage capacity of the global storage unit.

[0060] It is understood that the task execution device of this disclosure has been described above, and the integrated circuit chip including the task execution device will be described below.

[0061] Figure 3 is a schematic block diagram of an integrated circuit chip according to an embodiment of the present disclosure.

[0062] As shown in Figure 3, the integrated circuit chip 30 may include a task execution device 300. The task execution device 300 may be the device 100 described above.

[0063] It is understood that the integrated circuit chip disclosed herein has been described above, and the following will describe electronic devices that include integrated circuit chips.

[0064] Figure 4 is a schematic block diagram of an integrated circuit chip according to an embodiment of the present disclosure.

[0065] As shown in Figure 4, the electronic device 4000 may include an integrated circuit chip 40. The integrated circuit chip 40 may be the integrated circuit chip 30 described above.

[0066] It is understood that the above description of the electronic device including the integrated circuit chip of this disclosure has been provided, and the task execution method of this disclosure will be described below.

[0067] Figure 5 is a flowchart of a task execution method according to an embodiment of the present disclosure.

[0068] As shown in Figure 5, the method 500 may include operations S510 to S550.

[0069] In operation S510, a set of operator information for the target task is obtained.

[0070] In this embodiment of the disclosure, K is an integer greater than 1.

[0071] In operation S520, at least one group of consecutive operators is determined from a plurality of operators.

[0072] In this embodiment of the disclosure, the continuous operator group includes at least two continuous operators, and the size of the output tensor of any operator in the continuous operator group is greater than or equal to a preset size threshold.

[0073] In operation S530, at least one group of consecutive operators are fused to obtain at least one fusion operator.

[0074] In operation S540, in response to determining that the data to be processed corresponding to the fusion operator is stored in the cache unit, the data to be processed is loaded from the cache unit.

[0075] When operating the S550, the target task is executed using the data to be processed.

[0076] It is understandable that the processor 120 described above can be used to execute method 500.

[0077] In some embodiments, fusing at least one group of consecutive operators to obtain at least one fused operator may include: determining the largest tensor operator among at least two consecutive operators, based on the output tensor of the largest tensor operator and the storage capacity of the cache unit; determining the target number of batches based on the target number of batches and the original number of batches of the fused operator; and determining the target number of executions of the fused operator based on the target number of batches and the original number of batches of the fused operator.

[0078] In some embodiments, the data to be processed includes a target batch number of fusion operator data. Loading the data to be processed from the cache unit in response to determining that the data to be processed corresponding to the fusion operator is stored in the cache unit may include: loading the target batch number of fusion operator data from the cache unit in response to determining that the data to be processed corresponding to the fusion operator is stored in the cache unit.

[0079] In some embodiments, performing a target task using the data to be processed may include: performing the target task using fusion operator data according to the target batch number.

[0080] In some embodiments, determining the target batch number based on the output tensor of the maximum tensor operator and the storage capacity of the cache unit may include: taking the modulus of the storage capacity of the cache unit relative to the output tensor of the maximum tensor operator to obtain a modulus. The modulus is then used as the target batch number.

[0081] In some embodiments, the target number of executions is M, and the target number of batches is I. Executing the target task using the fusion operator data, according to the target number of batches, may include: loading I fusion operator data corresponding to the m-th execution of the fusion operator from a cache unit. For example, m is an integer greater than or equal to 1 and less than or equal to M, where M is an integer greater than 1. The target task is then executed using the I fusion operator data, where I is an integer greater than or equal to 1.

[0082] In some embodiments, operator information includes an operator network graph.

[0083] In some embodiments, the plurality of operators include a convolution operator and a convolution transpose operator.

[0084] The collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0085] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0086] Figure 6 illustrates a schematic block diagram of an example electronic device 600 that can be used to implement embodiments of the present disclosure. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0087] As shown in Figure 6, device 600 includes a computing unit 601, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 602 or a computer program loaded from storage unit 608 into random access memory (RAM) 603. RAM 603 may also store various programs and data required for the operation of device 600. The computing unit 601, ROM 602, and RAM 603 are interconnected via bus 604. Input / output (I / O) interface 605 is also connected to bus 604.

[0088] Multiple components in device 600 are connected to I / O interface 605, including: input unit 606, such as keyboard, mouse, etc.; output unit 607, such as various types of monitors, speakers, etc.; storage unit 608, such as disk, optical disk, etc.; and communication unit 609, such as network card, modem, wireless transceiver, etc. Communication unit 609 allows device 600 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0089] The computing unit 601 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 601 performs the various methods and processes described above, such as task execution methods. For example, in some embodiments, the task execution method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 608. In some embodiments, part or all of the computer program may be loaded and / or installed on device 600 via ROM 602 and / or communication unit 609. When the computer program is loaded into RAM 603 and executed by the computing unit 601, one or more steps of the task execution method described above may be performed. Alternatively, in other embodiments, the computing unit 601 may be configured to perform the task execution method by any other suitable means (e.g., by means of firmware).

[0090] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0091] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0092] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0093] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) monitor or an LCD (liquid crystal display)) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0094] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0095] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A task execution device, comprising: Cache unit; The processor is configured to: acquire a set of operator information for a target task, wherein the set of operator information includes multiple operator information, each representing multiple operators to be executed sequentially; determine at least one group of consecutive operators from the multiple operators, wherein the group of consecutive operators includes at least two consecutive operators, where the input of the latter module in two adjacent modules of the consecutive operators is the output of the former module, and the size of the output tensor of any operator in the group of consecutive operators is greater than or equal to a preset size threshold; fuse the at least one group of consecutive operators to obtain at least one fused operator, wherein the at least one group of consecutive operators corresponds one-to-one with the at least one fused operator; and, in response to determining that the data to be processed corresponding to the fused operator is stored in the cache unit, load data from the cache unit. The data to be processed; and using the data to be processed to execute the target task, wherein determining at least one group of consecutive operators from the plurality of operators includes: in response to determining that the size of the output tensor of the current operator in the plurality of operators is greater than or equal to a preset size threshold, determining a first number of subsequent consecutive operators whose output tensor size is greater than or equal to the preset size threshold from the plurality of subsequent consecutive operators of the current operator; determining a second number based on the first number and the current operator; in response to determining that the second number is greater than or equal to a preset number threshold, taking the current operator and the first number of subsequent consecutive operators as the group of consecutive operators, wherein using the data to be processed to execute the target task includes: running the fusion operator multiple times, wherein the intermediate results of the fusion operator are stored in the cache unit.

2. The apparatus according to claim 1, wherein, The processor is further configured to: determine the largest tensor operator among the at least two consecutive operators with the largest output tensor; and determine the target batch number based on the output tensor of the largest tensor operator and the storage capacity of the cache unit. And based on the target batch number and the original batch number of the fusion operator, the target number of executions of the fusion operator is determined.

3. The apparatus according to claim 2, wherein, The data to be processed includes a target batch number of fusion operator data, and the processor is further configured to: in response to determining that a target batch number of fusion operator data corresponding to the fusion operator is stored in the cache unit, load the target batch number of fusion operator data from the cache unit; and execute the target task using the fusion operator data according to the target batch number.

4. The apparatus according to claim 2 or 3, wherein, The processor is further configured to: take the storage capacity of the cache unit modulo the output tensor of the maximum tensor operator to obtain a modulus; and use the modulus as the target batch number.

5. The apparatus according to claim 4, wherein, The target execution count is M, the target batch size is I, and the processor is further configured to: load I fusion operator data corresponding to the m-th execution of the fusion operator from the cache unit, where m is an integer greater than or equal to 1 and less than or equal to M, and M is an integer greater than 1; and execute the target task using the I fusion operator data, where I is an integer greater than or equal to 1.

6. The apparatus according to claim 1, wherein, The operator information includes an operator network graph, which is used to indicate the relationships between multiple modules of the operator.

7. The apparatus according to claim 1, wherein, The plurality of operators include the convolution operator and the convolution transpose operator.

8. An integrated circuit chip, comprising a task execution device according to any one of claims 1 to 7.

9. An electronic device comprising the integrated circuit chip according to claim 8.

10. A task execution method, comprising: A set of operator information for the target task is obtained, wherein the set of operator information includes multiple operator information, each representing multiple operators to be executed sequentially; at least one group of consecutive operators is determined from the multiple operators, wherein the group of consecutive operators includes at least two consecutive operators, where the input of the latter module in two adjacent modules of the consecutive operators is the output of the former module, and the size of the output tensor of any operator in the group of consecutive operators is greater than or equal to a preset size threshold; the at least one group of consecutive operators is fused to obtain at least one fused operator, wherein the at least one group of consecutive operators corresponds one-to-one with the at least one fused operator; in response to determining that the data to be processed corresponding to the fused operator is stored in a cache unit, the data to be processed is loaded from the cache unit. According to; and using the data to be processed, to perform the target task, the step of determining at least one group of consecutive operators from the plurality of operators includes: in response to determining that the size of the output tensor of the current operator in the plurality of operators is greater than or equal to a preset size threshold, determining a first number of subsequent consecutive operators whose output tensor size is greater than or equal to the preset size threshold from the plurality of subsequent consecutive operators of the current operator; determining a second number based on the first number and the current operator; in response to determining that the second number is greater than or equal to a preset number threshold, taking the current operator and the first number of subsequent consecutive operators as the group of consecutive operators, the step of using the data to be processed to perform the target task includes: running the fusion operator multiple times, the intermediate results of the fusion operator being stored in the cache unit.

11. The method according to claim 10, wherein, The step of fusing the at least one group of consecutive operators to obtain at least one fused operator includes: determining the largest tensor operator among the at least two consecutive operators with the largest output tensor; determining the target batch number based on the output tensor of the largest tensor operator and the storage capacity of the cache unit; and determining the target number of executions of the fused operator based on the target batch number and the original batch number of the fused operator.

12. The method according to claim 11, wherein, The data to be processed includes a target batch number of fusion operator data. The step of loading the data to be processed from the cache unit in response to determining that the data to be processed corresponding to the fusion operator is stored in the cache unit includes: loading the target batch number of fusion operator data from the cache unit in response to determining that the data to be processed corresponding to the fusion operator is stored in the cache unit; wherein, the step of executing the target task using the data to be processed includes: executing the target task according to the target batch number, using the fusion operator data.

13. The method according to claim 11 or 12, wherein, The step of determining the target batch number based on the output tensor of the maximum tensor operator and the storage capacity of the cache unit includes: taking the modulus of the storage capacity of the cache unit relative to the output tensor of the maximum tensor operator to obtain a modulus; and using the modulus as the target batch number.

14. The method according to claim 12, wherein, The target number of executions is M, and the target number of batches is I. The step of executing the target task using the fusion operator data according to the target number of batches includes: loading I fusion operator data corresponding to the m-th execution of the fusion operator from the cache unit, where m is an integer greater than or equal to 1 and less than or equal to M, and M is an integer greater than 1; and executing the target task using the I fusion operator data, where I is an integer greater than or equal to 1.

15. The method according to claim 10, wherein, The operator information includes an operator network graph, which is used to indicate the relationships between multiple modules of the operator.

16. The method of claim 10, wherein, The plurality of operators include the convolution operator and the convolution transpose operator.

17. An electronic device comprising: At least one processor; And a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 10 to 16.

18. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 10 to 16.

19. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 10 to 16.

Images