Method and computing device for artificial intelligence computing optimization

Abstract

The present invention provides a method and computing device for artificial intelligence computing optimization, the method comprising: the main processor calculates the amount of memory occupied by control information, the control information comprising the control information of the first operator and the control information of the second operator; the main processor determines that the amount of memory occupied by the control information is greater than the memory amount of a single tightly coupled cache, then the main processor copies the control information of the first operator to the first tightly coupled cache of the slave processor, so that the slave processor enables the intelligent computing engine to perform operations using the control information. According to the technical solution of the present invention, it is possible to reduce the memory space occupied by data, reduce communication time, and improve computing efficiency and hardware resource utilization.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an artificial intelligence computing optimization method and computing device. Background Art

[0002] With the development of artificial intelligence (AI) technology, especially the increasing complexity of deep learning models, the demand for AI computing has also increased dramatically. AI model training usually requires processing large data sets and a large amount of computing resources. Many AI applications have high requirements for real-time computing and require efficient communication technology to support data transmission and synchronization. In data flow artificial intelligence chips (AI chips), AI computing is implemented through engines. The data flow computing process mainly consists of direct memory access (DMA) data transfer and a small amount of process control. DMA control information and process control information are stored in the tightly coupled cache (TCM). The capacity of TCM is limited. The storage of full information will lead to a waste of TCM storage space, while increasing the memory copy and communication time cost between the master and slave processors. The computing and communication resources of the entire chip are not fully utilized.

[0003] Therefore, a technical solution is needed that can reduce the memory space occupied by data, reduce communication time, and improve computing efficiency and hardware resource utilization. Summary of the invention

[0004] The present invention aims to provide a method and computing device for artificial intelligence computing optimization, which can reduce the memory space occupied by data, reduce communication time, and improve computing efficiency and hardware resource utilization.

[0005] According to one aspect of the present invention, a method for optimizing artificial intelligence computing is provided. The method is applied to a data stream artificial intelligence chip. The computing process of the method includes at least two operators. The components involved in the computing process include a master processor, a slave processor, a system memory, and an intelligent computing engine. The intelligent computing engine includes a static random access memory and an intelligent computing core. The slave processor includes a tightly coupled cache and a mailbox. The method includes:

[0006] The main processor calculates the amount of memory occupied by control information, where the control information includes control information of the first operator and control information of the second operator;

[0007] The master processor determines that the memory occupied by the control information is greater than the memory of a single tightly coupled cache, then the master processor copies the control information of the first operator to the first tightly coupled cache of the slave processor, so that the slave processor enables the intelligent computing engine to perform operations using the control information,

[0008] According to some embodiments, the master processor copies control information of the second operator to the second tightly coupled cache of the slave processor while the intelligent computing core is computing.

[0009] According to some embodiments, the master processor copies the control information of the first operator to the first tightly coupled cache of the slave processor, including:

[0010] The main processor prepares the input data of the first operator and stores it in the system memory;

[0011] The master processor notifies the slave processor through the first mailbox channel of a message that the control information of the first operator and the input data of the first operator are ready, so that the input data of the first operator and the control information of the first operator are ready.

[0012] According to some embodiments, after the input data of the first operator and the control information of the first operator are ready, the method further includes:

[0013] The slave processor reads control information of the first operator in the first tightly coupled cache and enables the intelligent computing engine;

[0014] The slave processor reads the input data of the first operator into the static random access memory of the intelligent computing engine through a direct memory access read channel;

[0015] The intelligent computing core in the intelligent computing engine starts to execute the calculation of the first operator according to the input data of the first operator.

[0016] According to some embodiments, after the first operator calculation is completed, the slave processor reads the control information of the direct memory access write channel;

[0017] The slave processor moves the output data of the first operator from the static random access memory to the system memory according to the control information of the direct memory access write channel;

[0018] The slave processor sends a calculation completion signal of the first operator to the master processor through the first mailbox channel.

[0019] According to some embodiments, after receiving the calculation completion signal of the first operator, the master processor notifies the slave processor through the second mailbox channel of a message that the control information of the second operator and the input data of the second operator are ready, so that the input data of the second operator and the control information of the second operator are ready;

[0020] The slave processor reads the control information of the second operator in the second tightly coupled cache and enables the intelligent computing engine;

[0021] The slave processor reads the input data of the second operator into the static random access memory of the intelligent computing engine through a direct memory access read channel;

[0022] The intelligent computing core in the intelligent computing engine starts to execute the calculation of the second operator according to the input data of the second operator.

[0023] According to some embodiments, if the main processor determines that the amount of memory occupied by the control information is less than or equal to the amount of memory of a single tightly coupled cache, the control information of at least two operators of the main processor is copied to an idle tightly coupled cache.

[0024] According to some embodiments, control information of at least two operators interacts with the slave processor via at least two channels of the mailbox, respectively.

[0025] According to another aspect of the present invention, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, the method as described above is implemented.

[0026] According to another aspect of the present invention, there is provided a computing device, comprising:

[0027] Processor; and

[0028] A memory stores a computer program, and when the computer program is executed by the processor, the method described in any one of the above items is implemented.

[0029] According to an embodiment of the present invention, the main processor copies the control information of the first operator to the first cache of the slave processor. After the slave processor is notified of the message that the first operator is ready, the slave processor reads the control information in the first cache and enables the computing engine. The direct memory access read channel moves the input data to the static random access memory, and the intelligent computing core starts the calculation of the first operator. At the same time, in the above process, the main processor copies the control information of the second operator to the second cache of the slave processor, waits for the calculation of the first operator to be completed, and then the slave processor starts the calculation of the second operator until all calculations are completed. The present invention reduces the memory space occupied by data through message compression, reduces the communication time between the main processor and the slave processor by using ping-pong cache, and improves computing efficiency and hardware resource utilization.

[0030] According to some embodiments, template specialization is performed during compilation, and data structures of different sizes are generated according to different operator types to achieve message compression. By compressing data, the amount of data required for transmission can be significantly reduced, thereby reducing bandwidth consumption. Smaller data amounts mean shorter transmission time, thereby speeding up data transmission. Compressed data takes up less storage space, reducing the transmission time of data in the network, thereby improving the response speed and performance of the entire system.

[0031] According to some embodiments, by alternating the use of two buffers, the ping-pong cache can achieve seamless data processing, and when one buffer is being read, the other buffer can simultaneously receive new data. The ping-pong cache can avoid pauses in the data processing process, thereby improving the throughput of the system and simplifying the data synchronization problem, because each buffer can independently perform write and read operations, reducing the complexity of synchronization. The multi-channel mailbox channel is responsible for message transmission between the master and slave processors, avoiding the occupancy of a single mailbox channel. The combination of the ping-pong cache and the multi-channel mailbox channel improves the efficiency of data transmission.

[0032] According to some embodiments, ping-pong cache can effectively manage data flow and ensure the continuity and smoothness of data processing. Message compression reduces the amount of transmitted data. Ping-pong cache ensures efficient data processing. The combination of the two can minimize bandwidth usage and improve real-time performance.

[0033] According to some embodiments, the combination of message compression and ping-pong cache can better optimize the utilization of system resources. The ping-pong cache mechanism can provide a certain degree of fault tolerance. Even when a buffer fails, another buffer can continue to work, thereby enhancing the robustness of the system.

[0034] It is to be understood that the foregoing general description and the following detailed description are exemplary only and are not restrictive of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for describing the embodiments are briefly introduced below.

[0036] Figure 1 A flow chart of a method for artificial intelligence computing optimization according to an example embodiment is shown.

[0037] Figure 2 A schematic diagram of data transmission according to an example embodiment is shown.

[0038] Figure 3 A schematic diagram of a computing device according to an example embodiment is shown. DETAILED DESCRIPTION

[0039] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that the present invention will be comprehensive and complete and fully convey the concepts of the example embodiments to those skilled in the art. The same reference numerals in the figures represent the same or similar parts, and thus their repeated description will be omitted.

[0040] In addition, the described features, structures or characteristics may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present invention. However, those skilled in the art will appreciate that the technical solution of the present invention can be practiced without one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other cases, known methods, devices, implementations or operations are not shown or described in detail to avoid blurring various aspects of the present invention.

[0041] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0042] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0043] It should be understood that although the terms first, second, third, etc. may be used herein to describe various components, these components should not be limited by these terms. These terms are used to distinguish one component from another component. Therefore, the first component discussed below can be referred to as the second component without departing from the teachings of the present inventive concept. As used herein, the term "and / or" includes any one of the associated listed items and all combinations of one or more.

[0044] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the present invention are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0045] Those skilled in the art will appreciate that the drawings are merely schematic diagrams of example embodiments, and the modules or processes in the drawings are not necessarily necessary for implementing the present invention, and therefore cannot be used to limit the protection scope of the present invention.

[0046] At present, artificial intelligence technology (AI) has been widely used in all walks of life. With the continuous evolution of AI algorithms, the demand for computing power of AI models continues to increase, and memory performance cannot keep up with the rapid increase in computing power of computing units. Since the amount of data required for calculations, such as weights, feature maps, and intermediate calculation results, is very large, it puts tremendous pressure on the entire system, especially the transmission bandwidth between the chip and external storage.

[0047] The power consumption caused by the data transfer function inside the chip is very high, which has exceeded the power consumption required for the same amount of floating-point operations. Every additional data transfer will increase the power consumption of the chip by a large proportion.

[0048] To this end, the present invention proposes an artificial intelligence computing optimization method, which can reduce the memory space occupied by data, reduce communication time, and improve computing efficiency and hardware resource utilization.

[0049] According to the embodiment of the present invention, the volume of communication messages between the master and slave processors is compressed, the time for copying control information is shortened, and the occupation of the control information in the cache is reduced, so that a fixed-size cache can store more AI operator control information. The multi-channel resources of the MAILBOX mailbox and the multiple cache resources of the slave processor are fully utilized. The ping-pong cache solution fully improves the parallelism of the ENGINE engine calculation and control information copying, while improving the utilization of chip hardware resources and shortening the calculation time of AI reasoning.

[0050] Exemplary embodiments of the present invention are described below with reference to the accompanying drawings.

[0051] Figure 1 A flow chart of a method for artificial intelligence computing optimization according to an example embodiment is shown.

[0052] See also Figure 1 The method is applied to a data stream artificial intelligence chip, and the calculation process of the method includes at least two operators. The method of the present invention adopts TCM ping-pong cache and multi-channel MAILBOX channels. While the first operator is calculating, the control information and input information of the second operator will be copied to another idle tightly coupled cache of the slave processor (see Figure 2) to realize ping-pong cache. The present invention uses message compression to effectively reduce the occupancy of TCM memory and the number of communications between the master and slave CPUs. At the same time, it proposes the use of multiple MAILBOX channels and two TCM ping-pong caches to improve the utilization of hardware resources, so that the calculation and data handling of ENGINE AI CORE can be fully parallelized, the execution time can be shortened, and the AI ​​calculation efficiency can be improved.

[0053] The components involved in the computing process include a main processor, a slave processor, a system memory, and an intelligent computing engine. The intelligent computing engine includes a static random access memory and an intelligent computing core. The slave processor includes a tightly coupled cache and a mailbox (see Figure 3 ).

[0054] In S101, the main processor calculates the amount of memory occupied by control information, where the control information includes control information of a first operator and control information of a second operator.

[0055] According to some embodiments, the control information comprises control information of the first operator and control information of the second operator.

[0056] According to some embodiments, the control information needs to be compressed. The method of compressing the message is to generate data structures of different sizes by specializing the template during compilation according to different operator types.

[0057] According to some embodiments, the present invention uses C++ template metaprogramming to perform template specialization at compile time, and generates data structures of different sizes according to different operator types, thereby achieving the effect of message compression, so as to achieve the purpose of reducing the memory space occupied by the data structure. In C++, template specialization is a mechanism that allows developers to provide different implementations for specific types of templates. When defining a template class or template function, special versions can be provided for certain specific data types. These special versions are called "specializations".

[0058] The following example briefly illustrates the second method. The basic code is as follows:

[0059] structCtrlInfo{

[0060] uint32_top_num;

[0061] boolfunc_en;

[0062] };

[0063] structDmaDesc{

[0064] boolvalid;

[0065] uint64_taddr;

[0066] uint32_tlen;

[0067] };

[0068] enumclassOpType

[0069] {

[0070] OP1=0,

[0071] OP2,

[0072] OP3,

[0073] };

[0074] template <optypemode>

[0075] structParam;

[0076] template<>

[0077] structParam <optype::op1>

[0078] {

[0079] CtrlInfoctrl_info;

[0080] DmaDescinput;

[0081] DmaDescoutput;

[0082] };

[0083] template<>

[0084] structParam <optype::op2>

[0085] {

[0086] CtrlInfoctrl_info;

[0087] DmaDescinput;

[0088] DmaDescoutput;

[0089] DmaDesccoeff;

[0090] };

[0091] template<>

[0092] structParam <optype::op3>

[0093] {

[0094] CtrlInfoctrl_info;

[0095] DmaDescinput;

[0096] DmaDescoutput;

[0097] DmaDesccoeff;

[0098] DmaDescbias;

[0099] };

[0100] Based on the above encoding, different AI operators (OP) can generate DMA (direct memory access) control information and process control information on demand. Among the above operators, the parameter size of OP3 operator is the largest, which is 104 bytes, the parameter size of OP1 is 56 bytes, and the parameter size of OP2 is 80 bytes.

[0101] If the existing scheme uses full information parameters for transmission, the parameter size of the three operators OP1-OP3 is 104*3 = 312 bytes. After message compression according to this scheme, the parameter size becomes 56 + 80 + 104 = 240 bytes. The purpose of reducing the memory space occupied by the data structure is achieved.

[0102] According to some embodiments, message compression can also be achieved through DMA control information compression technology that accelerates AI reasoning. Obtain the full version of DMA control information, which includes valid DMA control information and invalid DMA control information; determine the bit in the header according to the full version of DMA control information, and each bit in the header indicates a DMA control information; select the valid DMA control information required by the target operator from the full version of DMA control information according to the bit in the header; send a data packet to the TCM (cache) corresponding to the real-time processor CPU, and the data packet includes a header and valid DMA control information. Message compression reduces the read bandwidth requirements for DDR (system memory) and the write bandwidth requirements for TCM, and saves data transmission time and TCM space, thereby reducing the storage space occupied by different data structures.

[0103] In S103, the main processor determines that the amount of memory occupied by the control information is greater than the amount of memory of a single tightly coupled cache, then the main processor copies the control information of the first operator to the first tightly coupled cache of the slave processor, so that the slave processor enables the intelligent computing engine to perform operations using the control information, and the main processor copies the control information of the second operator to the second tightly coupled cache of the slave processor while the intelligent computing core is calculating.

[0104] If the main processor determines that the memory occupied by the control information is greater than the memory of a single tightly coupled cache, the main processor copies the control information of the first operator to the first tightly coupled cache of the slave processor, so that the intelligent computing core uses the control information to perform operations.

[0105] The master processor determines that the memory occupied by the control information is less than or equal to the memory of a single tightly coupled cache, and the control information of at least two operators of the master processor is copied to an idle tightly coupled cache. The control information of at least two operators interacts with the slave processor through at least two channels of the mailbox respectively.

[0106] The main processor prepares the input data of the first operator and stores it in the system memory; the main processor notifies the slave processor through the first mailbox channel of the control information of the first operator and the message that the input data of the first operator is ready, so that the input data of the first operator and the control information of the first operator are ready.

[0107] The slave processor reads control information of the first operator in the first tightly coupled cache and enables the intelligent computing engine; the slave processor reads input data of the first operator into the static random access memory of the intelligent computing engine through a direct memory access read channel; the intelligent computing core in the intelligent computing engine starts to execute calculations of the first operator according to the input data of the first operator.

[0108] After the calculation of the first operator is completed, the slave processor reads the control information of the direct memory access write channel; the slave processor moves the output data of the first operator from the static random access memory to the system memory according to the control information of the direct memory access write channel; the slave processor sends a calculation completion signal of the first operator to the master processor through the first mailbox channel.

[0109] According to some embodiments, the main processor prepares the input data, stores it on the DDR, calculates the memory size (tcm_size) occupied by the control information, and if tcm_size is greater than the memory size of a single TCM, the subsequent calculation uses the ping-pong cache. If tcm_size is less than or equal to the memory size of a single TCM, the main processor copies the control information directly to the idle TCM, notifies the slave processor through the MAILBOX channel that the input and control information are ready, reads the control information in the TCM from the slave processor and enables the ENGINE, and reads the DMA control information, reads the input data to the SRAM (static random access memory) of the ENGINE through RDMA (direct memory access read channel), and the AI ​​CORE (intelligent computing core) starts to execute the operator calculation. In the case of multi-threading, up to 2 operator calculations are supported at the same time, and CH0 (first mailbox channel) and CH1 (second mailbox channel) of MAILBOX are used to interact with the slave processor for messages, and the slave processor will process the messages according to the order in which they arrive.

[0110] The main processor copies the DMA control information and flow control information of the first AI operator OP1 to the first cache TCM0 of the slave processor, and then notifies the slave processor through CH0 of MAILBOX that the input data and control information data of the first (group) AI operator are ready.

[0111] DMA control information is mainly responsible for controlling the necessary configuration information for DMA data transfer, that is, the structure DmaDesc written in the above step S101 code, which is used to configure the DMA source address, destination address and data length. The process control information is mainly used to control the process from the processor, such as determining whether repeated data transfer is required, determining whether the data requires special processing, etc. DMA control information and process control information jointly participate in the calculation of AI operators. Process control is responsible for judging some necessary conditions, and DMA is responsible for transferring data. For example, when repeated data transfer is required in the control information, the same data will be transferred multiple times through DMA control information.

[0112] The DMA control information is read from the processor and then configured into the DMA controller. After the DMA controller controls the RDMA channel to move the read data to the ENGINE's SRAM, the AI ​​CORE starts to execute the OP1 calculation.

[0113] While the first operator is calculating, the master processor copies the control information of the second operator and the input information of the second operator to the second tightly coupled cache of the slave processor, and waits for the first operator to be settled.

[0114] After receiving the calculation completion signal of the first operator, the main processor notifies the slave processor through the second mailbox channel of the message that the control information of the second operator and the input data of the second operator are ready, so that the input data of the second operator and the control information of the second operator are ready, waiting to start the calculation of the second operator.

[0115] After the first operator completes calculation, the slave processor reads the control information of the direct memory access write channel and moves the calculation result of the first operator from the static random access memory to the system memory; the slave processor sends a calculation completion signal of the first operator to the master processor through the first mailbox channel. The master processor notifies the slave processor of a message that the input data of the second operator and the control information of the second operator are ready through the second mailbox channel, and starts to calculate the second operator.

[0116] The main processor waits for the OP1 calculation completion signal. After the ENGINE calculation is completed, the slave processor will read the WDMA (direct memory access write channel) control information. The DMA moves the output result from the SRAM to the DDR, and then sends the OP1 calculation completion message through the CH0 of the slave processor to notify the main processor.

[0117] After OP1 calculation is completed, the master processor notifies the slave processor through CH1 of MAILBOX that the input data and control information data of OP2 are ready.

[0118] The slave processor reads the control information in the second cache TCM1 and enables ENGINE, and reads the DMA control information. It reads the input data into the SRAM of ENGINE through RDMA. AI CORE starts to perform the calculation of OP2, and repeats the above steps until all operators complete the calculation. In the above process, the DMA control information data structure is generated by specializing the template compilation period to achieve the purpose of message compression. The optimization is mainly reflected in the process of the main processor copying the DMA control information and process control information of the AI ​​operator to the TCM cache of the slave processor. After the message compression is implemented, the size of the data structure is compressed, which reduces the occupancy of TCM and the time for the main processor to copy TCM.

[0119] Through the TCM ping-pong method, that is, while OP1 is calculating, the main processor is used to copy the DMA control information and process control information of OP2 to another TCM cache, thereby realizing the parallelization of ENGINE calculation and control information copying. That is, the time of the process is changed from the original serial time to the larger value of the two times, while improving the utilization of hardware resources.

[0120] Figure 3 A schematic diagram of a computing device according to an example embodiment is shown.

[0121] In the data flow AI chip, AI computing is implemented through AI ENGINE. The data flow computing process mainly consists of DMA data transfer and a small amount of process control.

[0122] The main components involved in the computing process include: Master CPU, Slave CPU, system memory DDR, engine ENGINE and direct memory access controller (DMA controller). ENGINE includes static random access memory (SRAM) and intelligent computing core (AI CORE), and the slave processor includes two caches TCM0 and TCM1, each TCM is 32KB in size, and MAILBOX for heterogeneous core communication, MAILBOX contains 4 communication channels, CH0-CH3.

[0123] The calculation process has been described in detail above and will not be repeated here.

[0124] The data structure pseudo code of the DMA control information and process control information involved in the above process is described as follows:

[0125] / / Process control information structure

[0126] structCtrlInfo{

[0127] uint32_top_num;

[0128] boolfunc_en; ...

[0129] };

[0130] structDmaDesc{

[0131] boolvalid;

[0132] uint64_taddr;

[0133] uint32_tlen; ...

[0134] };

[0135] / / DMA control information structure

[0136] structDmaInfos{

[0137] DmaDescinput;

[0138] DmaDescoutput;

[0139] DmaDesccoeff;

[0140] DmaDescbias;

[0141] CtrlInfoctrl_info; ...

[0142] };

[0143] It should be added that, according to different optimization methods, the above process may include multiple operator calculations. In this case, TCM stores multiple sets of DMA control information and process control information.

[0144] According to some embodiments, the DMA control information in the AI ​​calculation process in the prior art is a complete set, that is, the DMA control information includes all AI calculation situations, and for some AI operators, at least 2 DMA information are required to complete the calculation. The DMA control information and process control information are stored in the TCM. The capacity of the TCM is limited, and the storage of the full amount of information will lead to a waste of TCM storage space, while increasing the memory copy and communication time costs of the Master-CPU and Slave-CPU. For example, the full amount of control information may include DMA description information such as input, output, coeff, bias, etc., but the simplest copy operator may only need two register descriptions of input and output. The full amount of control information will take up a large storage space, and for the copy operator, most of the DMA descriptions are invalid data.

[0145] In the prior art, only one TCM is used in each AI computing process. Even if the process contains multiple operators, when the size of DMA control information and process control information exceeds the size of a single TCM, TCM0 and TCM1 are used in turn, that is, in a serial relationship. At the same time, MAILBOX has 4 channels. In the case of serial communication, at most one MAILBOX channel is used at a time. Therefore, the computing and communication resources of the entire chip are not fully utilized.

[0146] The present invention compresses the volume of the communication message from Master-CPU to Slave-CPU, shortens the time of copying control information from Master-CPU to TCM cache of Slave-CPU, reduces the occupation of TCM cache by control information, enables TCM cache of fixed size to store more AI operator control information, and makes full use of the multi-channel resources of MAILBOX and multiple TCM cache resources of Slave-CPU. The implementation of TCM ping-pong BUFFER scheme fully improves the parallel rate of ENGINE calculation and control information copy, and improves the utilization rate of hardware resources on SOC chip, shortening the calculation time of AI reasoning. The scheme has different degrees of improvement effect on different neural networks. In the measured resetnet50 network, message compression can reduce TCM occupation by more than 50%, and the reasoning performance is improved by about 25%.

[0147] It should be noted that, in the specific implementation process, the computing device of the present invention may also include other components necessary for normal operation. In addition, those skilled in the art can understand that the above-mentioned device may also only include components necessary for implementing the embodiments of the present specification, and does not necessarily include all the components shown in the figure.

[0148] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of the above method when the program is executed by a processor. The computer-readable storage medium may include, but is not limited to, any type of disk, including a floppy disk, an optical disk, a DVD, a CD-ROM, a micro drive, and a magneto-optical disk, a ROM, a RAM, an EPROM, an EEPROM, a DRAM, a VRAM, a flash memory device, a magnetic card or an optical card, a nanosystem (including a molecular memory IC), a network storage device, a cloud storage device, or any type of medium or device suitable for storing instructions and / or data.

[0149] An embodiment of the present invention further provides a computer program product, which includes a computer program. The computer program is operable to enable a computer to execute part or all of the steps of any one of the methods described in the above method embodiments.

[0150] Those skilled in the art can clearly understand that the technical solution of the present invention can be implemented with the help of software and / or hardware. "Unit" and "module" in this specification refer to software and / or hardware that can independently complete or cooperate with other components to complete specific functions, where the hardware can be, for example, a field programmable gate array, an integrated circuit, etc.

[0151] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should know that the present invention is not limited by the described action sequence, because according to the present invention, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

[0152] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0153] In the several embodiments provided by the present invention, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some service interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0154] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0155] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0156] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the whole or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a memory and includes several instructions for enabling a computer device (which can be a personal computer, server or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present invention.

[0157] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0158] The exemplary embodiments of the present invention are specifically shown and described above. It should be understood that the present invention is not limited to the detailed structures, configurations or implementations described herein; on the contrary, the present invention is intended to cover various modifications and equivalent configurations included in the spirit and scope of the appended clauses. < / optypemode>

Claims

1. A method for optimizing artificial intelligence computing, the method is applied to a data stream artificial intelligence chip, the computing process of the method includes at least two operators, the components involved in the computing process include a master processor, a slave processor, a system memory, and an intelligent computing engine, the intelligent computing engine includes a static random access memory and an intelligent computing core, the slave processor includes a tightly coupled cache and a mailbox, and the method includes: The main processor calculates the amount of memory occupied by the control information of the first operator and the control information of the second operator; The master processor determines that the memory occupied by the control information is greater than the memory of a single tightly coupled cache, and the master processor copies the control information of the first operator to the first tightly coupled cache of the slave processor, so that the slave processor enables the intelligent computing engine to perform operations using the control information; The master processor copies the control information of the first operator to the first tightly coupled cache of the slave processor, including: the master processor prepares the input data of the first operator and stores it in the system memory; the master processor notifies the slave processor through the first mailbox channel of the message that the control information of the first operator and the input data of the first operator are ready, so that the input data of the first operator and the control information of the first operator are ready.

2. The method according to claim 1, characterized in that The master processor copies the control information of the second operator to the second tightly coupled cache of the slave processor while the intelligent computing core is computing.

3. The method according to claim 1, characterized in that After the input data of the first operator and the control information of the first operator are ready, the method further includes: The slave processor reads control information of the first operator in the first tightly coupled cache and enables the intelligent computing engine; The slave processor reads the input data of the first operator into the static random access memory of the intelligent computing engine through a direct memory access read channel; The intelligent computing core in the intelligent computing engine starts to execute the calculation of the first operator according to the input data of the first operator.

4. The method according to claim 3, characterized in that Also includes: After the first operator completes calculation, the slave processor reads control information of a direct memory access write channel; The slave processor moves the output data of the first operator from the static random access memory to the system memory according to the control information of the direct memory access write channel; The slave processor sends a calculation completion signal of the first operator to the master processor through the first mailbox channel.

5. The method according to claim 4, characterized in that Also includes: After receiving the calculation completion signal of the first operator, the master processor notifies the slave processor through the second mailbox channel of a message that the control information of the second operator and the input data of the second operator are ready, so that the input data of the second operator and the control information of the second operator are ready; The slave processor reads control information of the second operator in the second tightly coupled cache and enables the intelligent computing engine; The slave processor reads the input data of the second operator into the static random access memory of the intelligent computing engine through a direct memory access read channel; The intelligent computing core in the intelligent computing engine starts to execute the calculation of the second operator according to the input data of the second operator.

6. The method according to claim 1, characterized in that Also includes: The main processor determines that the memory occupied by the control information is less than or equal to the memory amount of a single tightly coupled cache, and then the main processor copies the control information of at least two operators to an idle tightly coupled cache.

7. The method according to claim 6, characterized in that include: The control information of at least two operators interacts with the slave processor through at least two channels of the mailbox respectively.

8. A computer program product, characterized in that The method comprises a computer program, which implements the method according to any one of claims 1 to 7 when being executed by a processor.

9. A computing device, characterized in that include: processor; as well as A memory storing a computer program, wherein when the computer program is executed by the processor, the method according to any one of claims 1 to 7 is implemented.

Images