Zero-copy AI chip communication system, method, device, medium and product

Abstract

The invention relates to the technical field of chips, and discloses a zero-copy AI chip communication system, method, device, medium and product wherein the system comprises at least two AI chips of the same structure wherein the AI chip comprises: a computing core module comprising a core sub-module and a draft book storage sub-module; the direct memory access engine module is used for receiving and executing a direct memory access instruction; the internal network operation center module is used for routing the communication data to the target AI chip; and the inter-chip communication controller module is used for controlling communication with the target AI chip through the communication link so as to transmit communication data to the target AI chip. By designing the AI chips and addressing the AI chips in a unified manner, when communication transmission among the chips is carried out, data is directly transmitted from the draft book storage of one AI chip to the draft book storage of another AI chip, a low-efficiency transmission mode depending on multiple copies in a traditional data transmission mode is avoided, and the data transmission efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the field of chip technology, and in particular to a zero-copy AI chip communication system, method, device, medium and product. Background Art

[0002] In the process of large model training, a large number of artificial intelligence chips are needed for parallel computing. The computing processes of different artificial intelligence chips are often different, and a large amount of communication between AI chips is required. In process-level parallel optimization, the main optimization idea is to divide the overall task so that each process performs an independent task. During the execution of a task and after the task is completed, a single process needs to exchange or reduce data between processes. It can be seen that communication between processes is a key factor affecting the parallel efficiency of the entire program. Reducing the communication overhead between processes is the key to improving overall computing efficiency.

[0003] Current AI chip-to-chip communication is accompanied by multiple data copying processes, including copying from internal storage to the main memory HBM in the chip, copying from the main memory HBM of one chip to the main memory HBM of another chip, and copying from the main memory HBM of another chip to the internal storage of another chip. When the amount of data is large, multiple data copies will take up a lot of transmission time, affecting the efficiency of inter-chip communication. Summary of the invention

[0004] In view of this, the present invention provides a zero-copy AI chip communication system, method, device, medium and product to solve the problems of long transmission time and low communication efficiency caused by multiple data copies in traditional AI chip communication.

[0005] In a first aspect, the present invention provides a zero-copy AI chip communication system, the system comprising at least two AI chips with the same structure, wherein the address space of each AI chip is continuous and unified, and any two AI chips are connected through a communication link, and the AI ​​chip comprises:

[0006] The computing core module includes a core submodule and a scratchpad storage submodule; the core submodule is used to send a direct memory access instruction to the direct memory access engine module, and the direct memory access instruction is used to read communication data from the scratchpad storage submodule and send it to the target AI chip through the internal network operation center module; the scratchpad storage submodule is used to store communication data;

[0007] A direct memory access engine module, used for receiving and executing direct memory access instructions;

[0008] An internal network operations center module for routing communication data to the target AI chip;

[0009] The inter-chip communication controller module is used to control communication with the target AI chip through a communication link to transmit communication data to the target AI chip.

[0010] The zero-copy AI chip communication system provided by the embodiment of the present invention, through unified addressing and structural design of the AI ​​chip, the read and write requests issued by the direct memory access engine module in the AI ​​chip can be routed to the internal network operation center module in the target chip through the communication link, so that when communicating between chips, data is directly transmitted from the scratch pad storage of one chip to the scratch pad storage of another chip, avoiding the step of CPU acting as an intermediary to copy data in the traditional data transmission method, thereby significantly improving the efficiency of data transmission.

[0011] In an optional implementation, the computing core module further includes:

[0012] The protocol engine submodule is used to perform protocol calculations based on communication data to obtain calculation results.

[0013] The zero-copy AI chip communication system provided by the embodiment of the present invention integrates protocol computing into the AI ​​chip to efficiently execute the protocol computing. At the same time, the operands required for the protocol computing are pre-stored in the draft storage submodule, thus saving the time of multiple calls from external storage, thereby improving the computing efficiency of the entire protocol computing engine.

[0014] In a second aspect, the present invention provides a zero-copy AI chip communication method, which is applied to the zero-copy AI chip communication system of the first aspect, and the method is executed by a computing core module, and the method includes:

[0015] storing the communication data in the first chip in the first scratchpad storage submodule, and sending a direct memory access instruction to the first direct memory access engine module;

[0016] Based on the direct memory access instruction, the communication data is transferred to the first internal network operation center module;

[0017] receiving communication data routed by the first internal network operation center module; the communication data is routed by the first internal network operation center module to the first inter-chip communication controller module and then sent to the second internal network operation center module in the second chip through the communication link;

[0018] The communication data is stored in the second scratch pad storage submodule of the second chip.

[0019] The zero-copy AI chip communication method provided by the embodiment of the present invention achieves zero copy by reducing the number of copies of data transmission, thereby avoiding multiple copies of data between internal storage and main memory HBM in the traditional method. Data is directly transmitted from the scratchpad storage submodule of the first chip to the scratchpad storage of the second chip through the direct memory access engine module, the internal network operation center module and the inter-chip communication controller module, reducing the delay and overhead of data transmission.

[0020] In an optional embodiment, the method further includes:

[0021] A protocol calculation is performed based on the communication data stored in the second draft book storage submodule to obtain a calculation result.

[0022] The zero-copy AI chip communication method provided by the embodiment of the present invention directly performs protocol calculation on the stored communication data in the second draft book storage submodule, which can make full use of the computing power of the AI ​​chip and realize computing acceleration. It avoids the intermediate storage and conversion links of data during transmission, reduces the delay and error in the calculation process, and improves the accuracy and efficiency of protocol calculation.

[0023] In an optional implementation, performing a protocol calculation based on the communication data stored in the second draft book storage submodule to obtain a calculation result includes:

[0024] When the protocol instruction sent by the second core submodule in the second chip is detected, the communication data stored in the second draft book storage submodule is read, and the protocol calculation is performed to obtain the calculation result.

[0025] The zero-copy AI chip communication method provided by the embodiment of the present invention can detect the protocol instructions in real time and read the data in the draft storage submodule, so that the system can quickly respond to computing needs, reduce waiting time, and improve overall computing efficiency. By asynchronously processing the protocol instructions, the system can process different protocol calculations according to the operation type of the protocol instructions, such as summing, finding the minimum value, finding the maximum value, etc.

[0026] In an optional implementation, performing protocol calculation based on the communication data stored in the second draft book module further includes:

[0027] The calculation results are written into the second draft book storage submodule for storage.

[0028] The zero-copy AI chip communication method provided in an embodiment of the present invention directly stores the calculation results back to the scratch book storage sub-module. If the current calculation results are needed for subsequent calculations, they can be quickly accessed by storing them in the scratch book storage sub-module. Especially for scenarios where multiple reduction calculations need to be performed continuously, storing the intermediate results in the scratch book storage sub-module can support seamless continuous calculations.

[0029] In an optional implementation, after writing the calculation result into the second draft book module for storage, the method further includes:

[0030] A target direct memory access instruction is sent to the second direct memory access engine module of the second chip; the target direct memory access instruction is used to instruct the second direct memory access engine module to send the calculation result to the target AI chip.

[0031] The zero-copy AI chip communication method provided by an embodiment of the present invention sends a direct memory access instruction to the second direct memory access engine module, i.e., the second direct memory access engine module, to directly transfer data between the memory and the peripheral or between different memory areas, thereby significantly improving the speed and efficiency of data transmission, and passing the calculation results of the reduced calculation to the target AI chip.

[0032] In a third aspect, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor are communicatively connected to each other, computer instructions are stored in the memory, and the processor executes the zero-copy AI chip communication method of the above-mentioned second aspect or any corresponding embodiment thereof by executing the computer instructions.

[0033] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the zero-copy AI chip communication method of the above-mentioned second aspect or any corresponding embodiment thereof.

[0034] In a fifth aspect, the present invention provides a computer program product, comprising computer instructions, wherein the computer instructions are used to enable a computer to execute the zero-copy AI chip communication method of the above-mentioned second aspect or any corresponding embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0036] Figure 1 It is a flowchart of the current inter-chip communication transmission method;

[0037] Figure 2 is a schematic diagram of the structure of a zero-copy AI chip according to an embodiment of the present invention;

[0038] Figure 3 is a schematic diagram of the structure of a computing core module according to an embodiment of the present invention;

[0039] Figure 4 is a flowchart of a zero-copy AI chip communication method according to an embodiment of the present invention;

[0040] Figure 5 is a schematic diagram of a zero-copy AI chip communication method according to an embodiment of the present invention;

[0041] Figure 6 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0043] In large-scale model training scenarios, it is very common and widespread to use AI chips for parallel computing. However, the computing progress of each AI chip often differs, and frequent and large amounts of data communication must be carried out between them.

[0044] When performing process-level parallel optimization, the core strategy is to finely divide the overall task to ensure that each process can independently and efficiently execute the assigned task. During the execution of a task by a single process and after the task is completed, data exchange or protocol operations between processes are required. It can be seen that the efficiency of inter-process communication has become a key factor restricting the parallel performance of the entire program. Therefore, reducing the overhead of inter-process communication plays a vital role in improving the overall computing efficiency.

[0045] Currently, the more common communication processes between AI chips are as follows: Figure 1 As shown, first, the computing core in the AI ​​chip 0 will temporarily store the data that needs to be communicated in its own internal memory. Then, the data will be copied from the internal memory and placed in the main memory HBM (High Bandwidth Memory) of the AI ​​chip 0 for subsequent transmission operations.

[0046] Next, AI chip 0 will actively initiate an inter-chip communication request with AI chip 1. The data previously stored in the main memory HBM of AI chip 0 will be transferred to the main memory of AI chip 1 directly connected to chip 0 through the chip-to-chip link.

[0047] After the data is successfully transferred to the main memory HBM of AI chip 1, AI chip 1 will copy the data from the main memory and store it in its own internal memory. In this way, the computing core of AI chip 1 can directly access the data and start the corresponding computing tasks.

[0048] In summary, there are three data copies in the above inter-chip communication scheme, namely: 1. Copy from the internal storage in chip 0 to the main memory HBM in chip 0; 2. Copy from the main memory HBM in chip 0 to the main memory HBM in chip 1; 3. Copy from the main memory HBM in chip 1 to the internal storage in chip 1.

[0049] However, the amount of data communicated between AI chips is very large, and the three copies of data take up a lot of transmission time, affecting the efficiency of inter-chip communication.

[0050] Based on this, an embodiment of the present invention provides a zero-copy AI chip communication system, which realizes zero-copy high-performance network computing transmission. The system includes at least two AI chips with the same structure, wherein the address space of each AI chip is continuous and unified, and any two AI chips are connected through a communication link, and the AI ​​chip includes:

[0051] The computing core module includes a core submodule and a scratchpad storage submodule; the core submodule is used to send a direct memory access instruction to the direct memory access engine module, and the direct memory access instruction is used to read communication data from the scratchpad storage submodule and send it to the target AI chip through the internal network operation center module; the scratchpad storage submodule is used to store communication data;

[0052] A direct memory access engine module, used for receiving and executing direct memory access instructions;

[0053] An internal network operations center module for routing communication data to the target AI chip;

[0054] The inter-chip communication controller module is used to control communication with the target AI chip through a communication link to transmit communication data to the target AI chip.

[0055] Specifically, if Figure 2As shown in the figure, the structures of the various AI chips that make up the zero-copy communication system are the same, and are composed of a computing core module, a direct memory access (DMA) engine module, an internal network operation center (NOC) module, and an inter-chip communication controller (C2C controller) module. Among them, the direct memory access engine module (DMA engine) is responsible for executing the DMA instructions sent by the core and performing DMA operations. Compared with the traditional CPU copy method, the DMA engine can directly transfer data between memory and peripherals or different memory areas; the internal NOC is responsible for routing read and write requests within the chip; and the C2C controller is responsible for connecting the inter-chip communication link.

[0056] The computing core module is responsible for performing calculations for inter-chip communication of the AI ​​chip. It also includes a core sub-module and a scratchpad storage sub-module. The core sub-module is responsible for initiating and managing calculations; the scratchpad storage sub-module serves as an on-chip SRAM, providing high-speed access to the core and protocol engine.

[0057] In some optional implementations, the computing core module further includes:

[0058] The protocol engine submodule is used to perform protocol calculations based on communication data to obtain calculation results.

[0059] The protocol engine submodule is as follows Figure 3 As shown, it is an asynchronous SIMD component that executes reduction instructions asynchronously, supports multiple data types such as FP32 / FP16 / BF16 / FP8E5M2 / FP8E4M3 / INT8, and supports multiple reduction operation types such as reduce_sum / reduce_min / reduce_max.

[0060] According to an embodiment of the present invention, a zero-copy AI chip communication method embodiment is also provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in an order different from that shown here.

[0061] In this embodiment, a zero-copy AI chip communication method is provided, which can be used in the above-mentioned zero-copy AI chip communication system. Figure 4 is a flowchart of a zero-copy AI chip communication method according to an embodiment of the present invention, such as Figure 4 As shown, the process includes the following steps:

[0062] Step S401, storing the communication data in the first chip in the first scratch pad storage submodule, and sending a direct memory access instruction to the first direct memory access engine module.

[0063] Specifically, within the computing core module of the first chip, the core submodule generates or determines the data required for inter-chip communication according to the AI ​​computing task to be performed.

[0064] Furthermore, once the communication data is generated, the core submodule writes the data into the scratchpad storage submodule. The scratchpad storage submodule is usually a high-speed on-chip SRAM (static random access memory) that provides the computing core with fast access to storage, with lower access latency than the main memory HBM.

[0065] Furthermore, the core submodule constructs direct memory access (DMA) instructions according to the communication requirements. These instructions optionally include information such as the source address (the address of the data in the scratchpad storage submodule), the target address, and the data size. The core submodule sends the prepared DMA instructions to the direct memory access engine module through the internal communication mechanism.

[0066] Step S402: Based on the direct memory access instruction, the communication data is transferred to the first internal network operation center module.

[0067] Specifically, when the direct memory access engine (DMA) module in the first chip receives a direct memory access instruction from the computing core module, the DMA module starts preparing for data transmission according to the received instruction.

[0068] Furthermore, the DMA module directly reads the communication data from the scratchpad storage submodule. Since the scratchpad storage submodule is usually a high-speed on-chip SRAM, the data reading speed is very fast and almost no additional delay is introduced. The DMA engine then moves this data to the internal network operation center (NOC) module.

[0069] Further, after the data is moved to the NOC module and is ready for routing, the data will be temporarily stored in the buffer of the NOC module, waiting for further instructions from the chip-to-chip communication controller (C2C controller).

[0070] Step S403, receiving communication data routed by the first internal network operation center module.

[0071] Specifically, the communication data is first routed by the first internal network operation center module to the first inter-chip communication controller module, and then sent to the second internal network operation center module in the second chip through the communication link;

[0072] Further, when the first internal network operation center (NOC) module completes the data routing preparation, the first chip-to-chip communication controller (C2C controller) module will start preparing the communication link, and then the first chip-to-chip communication controller module sends the communication data to the second chip through the communication link.

[0073] Further, in the second chip, a second internal network operation center (NOC) module receives data from the first chip.

[0074] Step S404, storing the communication data in the second scratch pad storage submodule of the second chip.

[0075] Specifically, after the second internal network operation center (NOC) module receives the communication data, after unpacking and verification, the data will be routed to the scratchpad storage submodule in the computing core module of the second chip. The scratchpad storage submodule, as an on-chip SRAM, provides high-speed data access capabilities, allowing the computing core of the second chip to quickly process the data.

[0076] In summary, the zero-copy AI chip communication method is as follows: Figure 5 As shown, any two directly connected first and second chips in the system are connected through link communication. Through unified addressing, the read and write requests issued by the DMA engine in the first chip can be routed to the NOC in the second chip through C2Clink, that is, the DMA engine in the first chip can access the main memory HBM or the scratchpad storage of the computing core in the second chip.

[0077] The entire inter-chip communication process is as follows: the computing core in the first chip stores the data to be communicated in the scratchpad storage, and then sends a direct memory access instruction to the DMA engine module. This instruction is intended to tell the DMA engine to read the data from the scratchpad storage submodule and prepare to send it to the target AI chip.

[0078] After receiving the instruction, the DMA engine will directly move the data from the scratchpad storage submodule to the internal network operation center module without the intervention of the CPU and the additional data copying steps. After the internal network operation center module of the first chip receives the data, it routes the data to the C2C controller. Subsequently, the C2C controller of the first chip sends the data to the NOC module of the second chip through the link link. Finally, the NOC module of the second chip routes the data to the second scratchpad storage submodule of the computing core of the second chip for storage.

[0079] In some optional embodiments, the method further comprises:

[0080] A protocol calculation is performed based on the communication data stored in the second draft book storage submodule to obtain a calculation result.

[0081] Specifically, the protocol calculation can be implemented through a variety of algorithms or methods, such as summation, maximum value and minimum value, by specially setting up a protocol calculation engine in the computing core module to perform the above-mentioned protocol calculation through the communication data stored in the second draft book storage submodule.

[0082] In some optional implementations, performing a protocol calculation based on the communication data stored in the second draft book storage submodule to obtain a calculation result includes:

[0083] When the protocol instruction sent by the second core submodule in the second chip is detected, the communication data stored in the second draft book storage submodule is read, and the protocol calculation is performed to obtain the calculation result.

[0084] When the second core submodule in the second chip needs to perform protocol calculation, it will send a protocol instruction to the protocol calculation engine. After receiving the instruction, the protocol calculation engine will parse the content of the instruction, disassemble it into a series of read requests and write requests, and read the communication data of the corresponding address position stored in the second draft book storage submodule according to these read and write requests. And read the corresponding communication data stored in the second draft book storage submodule according to the instruction.

[0085] After reading the operands required to perform the reduction calculation, the reduction calculation engine will perform the reduction calculation according to the parsed operation type to obtain the calculation result.

[0086] In some optional implementations, performing protocol calculation based on the communication data stored in the second draft book module further includes:

[0087] The calculation results are written into the second draft book storage submodule for storage.

[0088] After completing the reduced calculation, the reduced calculation engine will write the calculation result to the scratchbook storage submodule by sending a write request to the scratchbook storage submodule, and the write request also includes the data and storage address of the calculation result. Storing the calculation result in the scratchbook storage submodule enables the computing core of the second chip to quickly access these results for subsequent processing or calculation.

[0089] In some optional implementations, after writing the calculation result into the second draft book module for storage, the method further includes:

[0090] A target direct memory access instruction is sent to the second direct memory access engine module of the second chip; the target direct memory access instruction is used to instruct the second direct memory access engine module to send the calculation result to the target AI chip.

[0091] When the second core submodule or protocol computing engine in the second chip completes the calculation and needs to send the result to other target AI chips, it constructs a target direct memory access (DMA) instruction. The second core submodule or protocol computing engine sends this target DMA instruction to the second direct memory access engine module of the second chip through the internal communication mechanism. After receiving the target DMA instruction, the second direct memory access engine module reads the calculation result from the second draft book storage submodule and prepares to send it to the target AI chip.

[0092] The embodiment of the present invention also provides a computer device, see Figure 6 , Figure 6 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Figure 6 As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system).

[0093] Optionally, the processor 10 may also be the AI ​​chip involved in the present application. When the processor 10 is an AI chip, the computer device may include multiple AI chips with the same structure, wherein the address space of each of the AI ​​chips is continuous and unified, and any two of the AI ​​chips are connected through a communication link, thereby realizing the relevant functions involved in the embodiments of the present application.

[0094] Figure 6 A processor 10 is taken as an example.

[0095] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0096] The memory 20 stores instructions executable by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.

[0097] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0098] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0099] The computer device also includes an input device 30 and an output device 40. The processor 10, the memory 20, the input device 30 and the output device 40 may be connected via a bus or other means. Figure 6 The example of connecting through bus is taken in the following.

[0100] The input device 30 can receive input digital or character information, and generate key signal input related to the user settings and function control of the computer device, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, an indicator bar, one or more mouse buttons, a trackball, a joystick, etc. The output device 40 may include a display device, an auxiliary lighting device (e.g., an LED) and a tactile feedback device (e.g., a vibration motor), etc. The above-mentioned display device includes but is not limited to a liquid crystal display, a light emitting diode, a display and a plasma display. In some optional embodiments, the display device can be a touch screen.

[0101] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0102] A part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of the computer program instruction in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc., and accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.

[0103] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A zero-copy AI chip communication system, characterized in that: The system comprises at least two AI chips with the same structure, wherein the address space of each AI chip is continuous and unified, and any two AI chips are connected via a communication link, and the AI ​​chip comprises: A computing core module, including a core submodule and a scratch pad storage submodule; the core submodule is used to send a direct memory access instruction to the direct memory access engine module, the direct memory access instruction is used to read communication data from the scratch pad storage submodule and send it to the target AI chip through the internal network operation center module; the scratch pad storage submodule is used to store the communication data; The direct memory access engine module is used to receive and execute the direct memory access instruction; The internal network operation center module is used to route the communication data to the target AI chip; The inter-chip communication controller module is used to control communication with the target AI chip through a communication link to transmit the communication data to the target AI chip.

2. The system according to claim 1, characterized in that The computing core module also includes: The protocol engine submodule is used to perform protocol calculation according to the communication data to obtain a calculation result.

3. A zero-copy AI chip communication method, characterized in that: Applied to the zero-copy AI chip communication system of claim 1 or 2, the method is performed by a computing core module, and the method includes: storing the communication data in the first chip in the first scratchpad storage submodule, and sending a direct memory access instruction to the first direct memory access engine module; Based on the direct memory access instruction, the communication data is transferred to a first internal network operation center module; receiving the communication data routed by the first internal network operation center module; the communication data is routed by the first internal network operation center module to the first inter-chip communication controller module and then sent to the second internal network operation center module in the second chip through the communication link; The communication data is stored in a second scratch pad storage submodule of the second chip.

4. The method according to claim 3, characterized in that The method further comprises: A protocol calculation is performed based on the communication data stored in the second draft book storage submodule to obtain a calculation result.

5. The method according to claim 4, characterized in that The performing of protocol calculation based on the communication data stored in the second draft book storage submodule to obtain a calculation result includes: When a protocol instruction sent by the second core submodule in the second chip is detected, the communication data stored in the second draft book storage submodule is read, and protocol calculation is performed to obtain the calculation result.

6. The method according to claim 5, characterized in that The performing protocol calculation based on the communication data stored in the second draft book module further includes: The calculation result is written into the second draft book storage submodule for storage.

7. The method according to claim 6, characterized in that After writing the calculation result into the second draft book module for storage, the method further includes: Send a target direct memory access instruction to the second direct memory access engine module of the second chip; the target direct memory access instruction is used to instruct the second direct memory access engine module to send the calculation result to the target AI chip.

8. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the zero-copy AI chip communication method according to any one of claims 3 to 7 by executing the computer instructions.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the zero-copy AI chip communication method according to any one of claims 3 to 7.

10. A computer program product, characterized in that The invention comprises computer instructions, wherein the computer instructions are used to cause a computer to execute the zero-copy AI chip communication method according to any one of claims 3 to 7.

Images