Instruction synchronization method and artificial intelligence chip

Abstract

The application provides an instruction synchronization method and an artificial intelligence chip, and relates to the technical field of the artificial intelligence chip. The method comprises the following steps: after a processing core sends a calculation instruction, a barrier instruction with a memory address is sent, so that the calculation instruction and the barrier instruction are in order into a cache, so as to ensure that the cache receives the calculation instruction and implements the calculation. In this way, a large number of memory barrier operations are reduced, so that the processing time delay of the memory instruction is reduced. For a plurality of processing cores bound to the same target barrier identifier, the number of synchronized instructions of the plurality of processing cores is recorded in real time. When the number of synchronized instructions is equal to the expected synchronization number, a synchronization success message is returned to the plurality of processing cores, so as to realize the instruction synchronization of the plurality of processing cores. The process does not need each processing core to loop read the atomic accumulation result, so that the occupation of bandwidth resources such as bus bandwidth and direct interconnection bandwidth is greatly reduced, so that the instruction synchronization cost is reduced and the efficiency of the whole system is improved.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of artificial intelligence chip, and in particular to an instruction synchronization method and an artificial intelligence chip. BACKGROUND

[0002] In an artificial intelligence chip, a cache usually adopts a relaxed memory consistency model to improve efficiency; wherein the memory consistency model improves efficiency through read and write of different addresses in disorder and same addresses in order.

[0003] Under the related art, in near-memory computing, a plurality of processing cores respectively execute a plurality of computing instructions to accumulate respective operands in a last level cache (LLC). Since each processing core cannot know the progress of the computing instructions executed by other processing cores, a memory fence operation, an atomic add operation and a loop read operation are added to ensure that the computing instructions of all processing cores have been sent to the LLC and implemented, thereby realizing synchronization between the computing instructions of the plurality of processing cores.

[0004] However, the memory fence instruction, the atomic add operation and the loop read operation will cause large synchronization overhead, and the loop read operation will frequently occupy bandwidth, thereby affecting the efficiency of the entire system. SUMMARY

[0005] Embodiments of the present application provide an instruction synchronization method and an artificial intelligence chip, which are used to reduce instruction synchronization overhead and thereby improve the efficiency of the entire system.

[0006] In one aspect, the present application provides an instruction synchronization method applied to a cache, which comprises:

[0007] receiving a first barrier instruction sent by a first processing core, the first barrier instruction carrying a target memory address, a target barrier identifier and an expected synchronization quantity; wherein the first barrier instruction is sent by the first processing core after sending a first computing instruction carrying the target memory address;

[0008] when there is a target synchronization record in the barrier engine that matches the target barrier identifier and the expected synchronization quantity, updating the number of synchronized instructions in the target synchronization record, the number of synchronized instructions representing the number of received computing instructions carrying the target memory address;

[0009] returning a synchronization success message to the first processing core and at least one second processing core bound with the target barrier identification when the expected synchronization number is equal to the synchronized instruction number.

[0010] Optionally, further comprising:

[0011] creating a first synchronization record in the barrier engine based on the target barrier identification and the expected synchronization number when there is no target synchronization record in the barrier engine matching the target barrier identification and the expected synchronization number.

[0012] Optionally, the returning a synchronization success message to the first processing core and at least one second processing core bound with the target barrier identification when the expected synchronization number is equal to the synchronized instruction number comprises:

[0013] sending a synchronization success message carrying the target barrier identification to a first distribution center to make the first distribution center forward the synchronization success message to the first processing core and each of the second processing cores when the expected synchronization number is equal to the synchronized instruction number.

[0014] Optionally, the cache and the first distribution center are located in a same artificial intelligence chip; the synchronization success message is transferred between the cache and the first distribution center through a barrier channel; a bandwidth of the barrier channel is less than a bus bandwidth of the artificial intelligence chip.

[0015] Optionally, the first distribution center and the first processing core are located in a same artificial intelligence chip.

[0016] the synchronization success message is transferred between the first distribution center and the first processing core through the barrier channel.

[0017] Optionally, the first distribution center is located in a first artificial intelligence chip and the first processing core is located in a second artificial intelligence chip.

[0018] the making the first distribution center forward the synchronization success message to the first processing core comprises:

[0019] the making the first distribution center forward the synchronization success message to the first processing core comprises:

[0020] Optionally, the synchronization success message is transferred between the first distribution center and the second distribution center through a barrier channel.

[0021] The bandwidth of the barrier passage is less than the direct interconnection bandwidth between the first artificial intelligence chip and the second artificial intelligence chip.

[0022] Optionally, after returning the synchronization success message to the first processing core and at least one second processing core bound with the target barrier identifier, the method further comprises:

[0023] In the barrier engine, the target synchronization record is marked as invalid.

[0024] Optionally, after returning the synchronization success message to the first processing core and at least one second processing core bound with the target barrier identifier, the method further comprises:

[0025] Receiving a first read request sent by the first processing core and carrying the target memory address;

[0026] Reading a target calculation result from the target memory address; the target calculation result is a calculation result of the first calculation instruction and a second calculation instruction sent by the at least one second processing core; the second calculation instruction carries the target memory address;

[0027] Returning the target calculation result to the first processing core.

[0028] In an aspect, an embodiment of the present application provides an artificial intelligence chip, comprising:

[0029] A processing core, a cache and a main memory;

[0030] The cache is configured to execute the instruction synchronization method.

[0031] In an aspect, an embodiment of the present application provides a computer device, comprising a memory, an artificial intelligence chip and a computer program stored in the memory and capable of running on the artificial intelligence chip, wherein the artificial intelligence chip executes the computer program to implement the steps of the instruction synchronization method.

[0032] In an embodiment of the present application, first, the processing core sends a calculation instruction and then sends a barrier instruction with the same memory address, so that the calculation instruction and the barrier instruction are in order into the cache, to ensure that the cache receives the calculation instruction and implements the calculation; in this way, a large number of memory barrier operations are reduced, and the subsequent memory access instructions of the blocked processing core are correspondingly reduced, thereby reducing the processing delay of the memory access instructions and improving the processing efficiency of the processing core.

[0033] Secondly, for multiple processing cores bound to the same target barrier identifier, the number of synchronized instructions of the multiple processing cores is recorded in real time. When the number of synchronized instructions is equal to the expected synchronization number, a synchronization success message is returned to the multiple processing cores, so as to realize instruction synchronization of the multiple processing cores; the process does not require each processing core to loop read the atomic accumulation result, so that the occupation of bandwidth resources such as bus bandwidth and direct interconnection bandwidth is greatly reduced, thereby reducing instruction synchronization overhead and improving the efficiency of the entire system. BRIEF DESCRIPTION OF DRAWINGS

[0034] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0035] Figure 1 A structural schematic diagram of an artificial intelligence chip is provided for the embodiments of the present application.

[0036] Figure 2 A structural schematic diagram of an artificial intelligence chip is provided for the embodiments of the present application.

[0037] Figure 3 A structural schematic diagram of an artificial intelligence chip is provided for the embodiments of the present application.

[0038] Figure 4 A flowchart of an instruction synchronization method is provided for the embodiments of the present application.

[0039] Figure 5 A flowchart of an instruction synchronization method is provided for the embodiments of the present application.

[0040] Figure 6 A flowchart of a data reading method is provided for the embodiments of the present application.

[0041] Figure 7 A structural schematic diagram of a computer device is provided for the embodiments of the present application. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solutions and beneficial effects of the present application clearer, the present application will be further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0043] The terms "first", "second", etc. in this application are only for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the technical features indicated. In addition, in the description of the embodiments of the present application, the meaning of "multiple" is two or more, unless otherwise specified.

[0044] Under the related art, in near-memory computing, the synchronization between instructions is implemented through a memory fence operation, an atomic accumulation operation and a loop read operation. For example, it is assumed that processing core 1 and processing core 2 respectively execute an arithmetic logic unit (ALU) instruction to accumulate an operation number in a cache; at this time, the instruction synchronization is performed in the following manner:

[0045] After processing core 1 issues an ALU instruction of a first memory address, the cache accumulates the operation number (such as 2) carried by the ALU instruction to the first memory address; processing core 1 performs a memory fence operation after issuing the ALU instruction, which is used to block the subsequent memory access instruction of processing core 1 to the cache before receiving a response message of the ALU instruction, so as to ensure that the operation number of processing core 1 is successfully accumulated to the first memory address in the cache. Processing core 1 then performs an atomic accumulation operation, that is, a plus one operation is performed at a second memory address agreed by processing core 1 and processing core 2, to obtain an atomic accumulation result; the atomic accumulation result represents the number of synchronized instructions.

[0046] Similarly, after processing core 2 issues an ALU instruction of the first memory address, the cache accumulates the operation number (such as 3) carried by the ALU instruction to the first memory address; processing core 2 performs a memory fence operation and an atomic accumulation operation after issuing the ALU instruction.

[0047] After processing core 1 and processing core 2 issue the ALU instruction, they continuously loop read the second memory address until the atomic accumulation result read from the second memory address is 2, and then read the accumulation result (at this time, the accumulation result is 5) from the first memory address, so as to ensure that the calculation instruction of processing core 1 and processing core 2 has been sent to the cache and the calculation is implemented, that is, the instruction synchronization of multiple processing cores is implemented.

[0048] However, in the above scheme, at least the following problems exist:

[0049] I. The ALU instruction issued by the processing core may write data in different caches, so the next instruction of the processing core is blocked before the response message of the ALU instruction is received by the processing core by executing the memory fence operation, which causes the next instruction of the processing core to be blocked, thereby causing the time delay of the memory access instruction to increase, and further causing the processing efficiency of the processing core to decrease.

[0050] Secondly, the processing core needs to read the second memory address in a loop, and the loop reading will frequently occupy the bus bandwidth of the artificial intelligence chip, which will affect the memory access request of other processing cores; especially in the cross-chip reading scenario, a large number of direct connection resources (such as direct interconnection bandwidth) between artificial intelligence chips need to be occupied, which not only affects the efficiency of the whole system, but also causes resource waste.

[0051] Therefore, the present application provides an instruction synchronization method, which ensures that the cache receives the calculation instruction by adding a barrier instruction, without the need to block the subsequent instructions of the processing core through the memory barrier operation; the barrier engine records the number of synchronized instructions, and returns a synchronization success message when the number of synchronized instructions is equal to the expected synchronization number, without the need for loop reading atomic accumulation result, which greatly reduces the instruction synchronization overhead, thereby improving the efficiency of the whole system.

[0052] The specific flow of the instruction synchronization method is described below. First, the structure diagram of an artificial intelligence chip to which the instruction synchronization method of the present application is applicable is introduced.

[0053] Reference Figure 1 It is a structure diagram of an artificial intelligence chip to which the embodiment of the present application is applicable, which includes at least a processing core 101, a cache 102, a main memory 103, a scheduling unit 104, and a distribution center 105.

[0054] In the present application, the artificial intelligence chip 100 can include one or more processing cores 101; it can also include one or more caches 102, which are not specifically limited in the present application.

[0055] The processing core 101 has the ability of artificial intelligence related calculation, such as matrix multiplication, convolution operation, etc., which are common calculation operations in artificial intelligence models.

[0056] The cache 102 is a small and fast storage area between the processing core 101 and the main memory 103, which is used to temporarily store data or instructions that the processing core 101 or other hardware devices may frequently access in the near future, with the purpose of speeding up access and reducing access delay. The main memory 103 is the next level of the cache 102, which generally has larger capacity but slower speed compared to the cache 102. In actual application, data can be read from the main memory 103 to the cache 102, or data in the cache 102 can be synchronized to the main memory 103.

[0057] The cache 102 includes multiple levels of cache, and the request fusion method of the cache in the present application is applicable to LLC in the multi-level cache, and is also applicable to caches of other levels, which are not specifically limited in the present application.

[0058] The scheduling unit 104 is configured to schedule the computing instructions of the plurality of processing cores 101 that need to be synchronized into the cache 102, and return the computing results of the computing instructions of the plurality of processing cores 101 to the plurality of processing cores 101. The scheduling unit 104 and the cache 102 communicate data through a bus bandwidth.

[0059] The distribution center 105 is configured to forward the synchronization success message returned by the cache 102 to the plurality of processing cores 101. The cache 102 and the distribution center 105 communicate the synchronization success message through a barrier fabric; the bandwidth of the barrier fabric is less than the bus bandwidth of the artificial intelligence chip.

[0060] In some embodiments, referring to Figure 2 When the plurality of processing cores 101 that need to be synchronized are located in different artificial intelligence chips 100, the computing instructions that need to be synchronized and the computing results obtained accordingly are communicated between the different artificial intelligence chips 100 through the respective scheduling units 104. For example, the first artificial intelligence chip includes a first scheduling unit; the second artificial intelligence chip includes a second scheduling unit; the first scheduling unit and the second scheduling unit communicate the computing instructions that need to be synchronized and the computing results obtained accordingly through a direct interconnection bandwidth.

[0061] In addition, the returned synchronization success message is communicated between the different artificial intelligence chips 100 through the respective distribution centers 105. For example, the first artificial intelligence chip includes a first distribution center; the second artificial intelligence chip includes a second distribution center; the first distribution center and the second distribution center communicate the synchronization success message through a barrier fabric. The bandwidth of the barrier fabric is less than the direct interconnection bandwidth.

[0062] In some embodiments, referring to Figure 3 The cache 102 includes a selector 301, a barrier engine 302, and a hit-miss check 303.

[0063] The cache 102 can receive a plurality of computing instructions sent by one processing core 101, or receive computing instructions sent by a plurality of processing cores 101 respectively. The plurality of processing cores 101 can be located in the same artificial intelligence chip, or can be located in different artificial intelligence chips.

[0064] The first processing core and the second processing core described in the present application are an example of the processing core 101. The first processing core and the second processing core can refer to the same processing core, or can refer to two independent processing cores.

[0065] Specifically, the selector 301 receives a first computing instruction sent by the first processing core, the first computing instruction carrying a target memory address. The selector 301 issues the first computing instruction to the hit check unit 303.

[0066] The hit check unit 303 reads the tag information of each cache line from the tag memory; then performs hit check on the target memory address based on the read tag information, and obtains a check result.

[0067] When the target memory address matches the tag information of a cache line, the check result is that the first computing instruction hits a cache line, and then the operand carried by the first computing instruction is accumulated to the cache line.

[0068] When there is no tag information matching the target memory address, the check result is that the first computing instruction does not hit a cache line, and therefore a cache line is allocated for the first computing instruction, and the memory address of the cache line is written into the tag memory; then the operand carried by the first computing instruction is accumulated to the cache line.

[0069] After the selector 301 receives the first computing instruction sent by the first processing core, the selector 301 receives a first barrier instruction sent by the first processing core; the first barrier instruction carries a target memory address, a target barrier identifier and an expected synchronization quantity.

[0070] The first processing core issues the first computing instruction first, and then issues the first barrier instruction; and the source address (the first processing core) and the destination address (i.e. the target memory address) corresponding to the first computing instruction and the first barrier instruction are the same, so as to ensure that the first computing instruction and the first barrier instruction enter the cache 102 in order, thereby ensuring that the cache 102 receives the first computing instruction and implements the computation; based on this, the first processing core no longer needs to perform a memory barrier operation, so as not to block the subsequent memory access instructions of the first processing core, thereby greatly reducing the processing delay of the memory access instructions and improving the processing efficiency of the processing core.

[0071] After the selector 301 receives the first barrier instruction sent by the first processing core, the selector 301 issues the first barrier instruction to the barrier engine 302. The barrier engine 302 can store a plurality of synchronization records, each synchronization record including at least the following elements: a valid bit (barrier valid), a barrier identifier (barrier id), an expected synchronization quantity (expect counter), and a synchronized instruction quantity (reach cnt).

[0072] The barrier engine 302 scans the plurality of synchronization records based on the target barrier identifier and the expected synchronization quantity carried by the first barrier instruction; when there is a target synchronization record matching the target barrier identifier and the expected synchronization quantity and being valid, the synchronized instruction quantity in the target synchronization record is updated.

[0073] When the expected synchronization number is equal to the synchronized instruction number, a synchronization success message is returned to the dispatch center 105; the dispatch center 105 dispatches the synchronization success message to the first processing core and at least one second processing core bound with the target barrier identifier.

[0074] After the first processing core receives the synchronization success message, a first read request carrying the target memory address is sent to the scheduling unit 104, and the scheduling unit 104 issues the first read request to the cache 102.

[0075] In the cache 102, the selector 301 issues the first computing instruction to the hit check unit 303 for hit check, and after the cache line corresponding to the target memory address is hit, the target computing result is read from the target memory address, the target computing result being the computing result of the first computing instruction and the second computing instruction respectively sent by the at least one second processing core. The cache 102 returns the target computing result to the scheduling unit 104, and the scheduling unit 104 returns the target computing result to the first processing core.

[0076] Similarly, after the second processing core receives the synchronization success message, the target computing result can also be read in the same way, which is not described here.

[0077] In the embodiment of the application, for a plurality of processing cores bound with the same target barrier identifier, the barrier engine 302 records the synchronized instruction number of the plurality of processing cores in real time. When the synchronized instruction number is equal to the expected synchronization number, a synchronization success message is returned to the plurality of processing cores to realize instruction synchronization of the plurality of processing cores. This process does not require each processing core to loop read the atomic accumulation result, which greatly reduces the occupation of bandwidth resources such as bus bandwidth and direct interconnection bandwidth, thereby reducing the instruction synchronization overhead and improving the efficiency of the entire system.

[0078] In addition to the above structure, the artificial intelligence chip 100 in the application can also include other structures, which are not limited in the application.

[0079] The artificial intelligence chip 100 can be a graphics processing unit (GPU), a general-purpose graphics processing unit (GPGPU), a domain specific architecture (DSA), etc.

[0080] The following will be based on the architecture diagram of the artificial intelligence chip shown in FIG. 1, and specifically introduce the flow of the instruction synchronization method provided, please refer to FIG. 2. Figures 1-3 FIG. 2 is a flowchart of an instruction synchronization method according to an embodiment of the application.Figure 4 The method is executed by a cache in an artificial intelligence chip, and the method comprises the following steps:

[0081] In step 401, a first barrier instruction sent by a first processing core is received.

[0082] Specifically, the first processing core first sends a first calculation instruction carrying a target memory address, and then sends a first barrier instruction for the same memory address (i.e., the target memory address).

[0083] Correspondingly, the cache first receives the first calculation instruction carrying the target memory address and issues the first calculation instruction to perform a hit check, then receives the first barrier instruction carrying the target memory address, a target barrier identifier and an expected synchronization quantity, and then issues the first barrier instruction to a barrier engine.

[0084] In this application, the core identifier of the first processing core and the core identifier of each of the at least one second processing core are bound to the same target barrier identifier.

[0085] In some cases, the processing core can run one or more kernel functions, each of which can issue a calculation instruction; therefore, when the calculation instructions sent by multiple kernel functions need to be synchronized, the target identifier of each kernel function is bound to the same target barrier identifier; the target identifier of each kernel function comprises a core identifier of the processing core where the kernel function is located and a function identifier of the kernel function.

[0086] In step 402, when there is a target synchronization record in the barrier engine that matches the target barrier identifier and the expected synchronization quantity, the number of synchronized instructions in the target synchronization record is updated.

[0087] Specifically, the barrier engine can store multiple synchronization records, and each synchronization record comprises at least the following elements: a valid bit, a barrier identifier, an expected synchronization quantity, and a number of synchronized instructions; wherein the meaning represented by the valid bit is different when the value of the valid bit is different; for example, when the valid bit is 1, it means that the synchronization record is valid; when the valid bit is 0, it means that the synchronization record is invalid.

[0088] Based on the target barrier identifier and the expected synchronization quantity carried by the first barrier instruction, multiple synchronization records in the barrier engine are scanned.

[0089] When there is a target synchronization record matching the target barrier identifier and the expected synchronization number and being valid, the value of the synchronized instruction number in the target synchronization record is greater than 0; that is, the cache has received a second computing instruction and a second barrier instruction carrying a target memory address before receiving the first computing instruction sent by the first processing core. At this time, the synchronized instruction number in the target synchronization record is directly updated, and the specific updating manner can be performing an add-one operation on the value of the synchronized instruction number; the synchronized instruction number represents the number of received computing instructions carrying the target memory address.

[0090] When there is no target synchronization record matching the target barrier identifier and the expected synchronization number and being valid, a first synchronization record is created in the barrier engine based on the target barrier identifier and the expected synchronization number.

[0091] In a specific implementation, an initial synchronization record is added in the barrier engine, and then the target barrier identifier and the expected synchronization number are added to the initial synchronization record, while the synchronized instruction number in the initial synchronization record is set to 1 and the initial synchronization record is marked as valid, to obtain the first synchronization record.

[0092] In step 403, when the expected synchronization number is equal to the synchronized instruction number, a synchronization success message is returned to the first processing core and at least one second processing core bound with the target barrier identifier.

[0093] Specifically, when the expected synchronization number is equal to the synchronized instruction number, it indicates that the computing instructions of all processing cores have been sent, and therefore, a synchronization success message is returned to the first processing core and at least one second processing core bound with the target barrier identifier; meanwhile, the target synchronization record in the barrier engine is marked as invalid.

[0094] In the embodiment of the application, first, the processing core sends a computing instruction and then sends a barrier instruction with the same memory address, so that the computing instruction and the barrier instruction can be sequentially entered into the cache based on the same address ordering of the cache, and then the cache has received the computing instruction and implemented the computation when processing the barrier instruction, that is, the cache receives the computing instruction and implements the computation through the barrier instruction; in this way, a large number of memory barrier operations are reduced, and the subsequent memory access instructions of the blocked processing core are correspondingly reduced, thereby reducing the processing delay of the memory access instruction and improving the processing efficiency of the processing core.

[0095] Secondly, for multiple processing cores bound to the same target barrier identifier, the number of synchronized instructions of the multiple processing cores is recorded in real time. When the number of synchronized instructions is equal to the expected synchronization number, a synchronization success message is returned to the multiple processing cores to realize instruction synchronization of the multiple processing cores; this process does not require each processing core to loop read the atomic accumulation result, thus greatly reducing the occupation of bandwidth resources such as bus bandwidth and direct interconnection bandwidth, thereby reducing instruction synchronization overhead and improving the efficiency of the entire system.

[0096] In some implementations, when the expected synchronization number is equal to the number of synchronized instructions, a synchronization success message carrying the target barrier identifier is sent to the first distribution center, so that the first distribution center forwards the synchronization success message to the first processing core and each second processing core.

[0097] Specifically, the first distribution center receives the synchronization success message carrying the target barrier identifier, then obtains the core identifier of the first processing core bound to the target barrier identifier, and the core identifier of each second processing core bound to the target barrier identifier. Based on the core identifier of the first processing core, the synchronization success message is forwarded to the first processing core; based on the core identifier of each second processing core, the synchronization success message is forwarded to each second processing core.

[0098] In some cases, if the target barrier identifier is bound to the core identifier of the first processing core and the function identifier of at least one kernel function in the first processing core, based on the core identifier of the first processing core, the synchronization success message is forwarded to the first processing core; then the first processing core distributes the synchronization success message to the at least one kernel function according to the respective function identifiers of the at least one kernel function.

[0099] Similarly, if the target barrier identifier is bound to the core identifier of the second processing core and the function identifier of at least one kernel function in the second processing core, based on the core identifier of the second processing core, the synchronization success message is forwarded to the second processing core; then the second processing core distributes the synchronization success message to the at least one kernel function according to the respective function identifiers of the at least one kernel function.

[0100] In some embodiments, the cache and the first distribution center are located in the same artificial intelligence chip; the cache and the first distribution center pass the synchronization success message through a barrier channel; the bandwidth of the barrier channel is less than the bus bandwidth of the artificial intelligence chip.

[0101] Specifically, since the bus bandwidth of the artificial intelligence chip is usually large, and the data amount of the synchronization success message is small, when the bus bandwidth is used to pass the synchronization success message, bandwidth waste is easily caused; based on this, the application additionally sets a barrier channel to pass the synchronization success message between the cache and the first distribution center.

[0102] Since the bandwidth of the barrier channel is small, it is more suitable for transmitting small amount of data synchronization success messages, thereby reducing the waste of bandwidth resources; and when transmitting the synchronization success message through the barrier channel, the synchronization success message no longer occupies the bus bandwidth, and these released bus bandwidths can be allocated to other memory access requests, thereby improving the utilization rate of bandwidth resources and further improving the overall efficiency of the system.

[0103] In some embodiments, the first distribution center and the first processing core are located in the same artificial intelligence chip; the first distribution center and the first processing core transmit the synchronization success message through the barrier channel.

[0104] Specifically, when the first distribution center and the first processing core are located in the same artificial intelligence chip, the first distribution center directly returns the synchronization success message to the first processing core through the barrier channel.

[0105] Similarly, when the first distribution center and the second processing core are located in the same artificial intelligence chip, the first distribution center directly returns the synchronization success message to the second processing core through the barrier channel.

[0106] For example, referring to Figure 5 , it is assumed that GPU0 includes: a first processing core 501, a second processing core 502, a first scheduling unit 503, a first distribution center 504, and a first cache 505.

[0107] The first processing core 501 issues an ALU instruction 1 carrying a memory address 1, and the first scheduling unit 503 schedules the ALU instruction 1 to the first cache 505; the first cache 505 accumulates the operand (such as an operand of 2) carried by the ALU instruction 1 to the memory address 1.

[0108] The first processing core 501 issues a barrier instruction 1 carrying a memory address 1, a barrier identifier 1, and an expected synchronization quantity (such as 2). The first scheduling unit 503 schedules the barrier instruction 1 to the first cache 505. The first cache 505 queries the barrier engine based on the barrier identifier 1 and the expected synchronization quantity, and does not query a matching synchronization record, so that a synchronization record 1 corresponding to the barrier instruction 1 is established in the barrier engine. The synchronization record 1 includes: a valid bit, the memory address 1, the barrier identifier 1, the expected synchronization quantity, and a synchronized instruction quantity, wherein the valid bit is 1, the expected synchronization quantity is 2, and the synchronized instruction quantity is 1.

[0109] The second processing core 502 issues an ALU instruction 2 carrying a memory address 1, and the first scheduling unit 503 schedules the ALU instruction 2 to the first cache 505; the first cache 505 accumulates the operand (such as an operand of 3) carried by the ALU instruction 2 to the memory address 1.

[0110] The second processing core 502 issues a barrier instruction 2, which carries the memory address 1, the barrier identifier 1, and the expected synchronization quantity (for example, 2). The first scheduling unit 503 schedules the barrier instruction 2 to the first cache 505. The first cache 505 queries the barrier engine based on the barrier identifier 1 and the expected synchronization quantity. Since the matching synchronization record 1 is queried, the synchronized instruction quantity in the synchronization record 1 is updated to 2.

[0111] At this time, the expected synchronization quantity is equal to the synchronized instruction quantity, and therefore the first cache 505 sends a synchronization success message carrying the barrier identifier 1 to the first distribution center 504 through the barrier channel. The first distribution center 504 determines the first processing core 501 and the second processing core 502 bound to the barrier identifier 1, and then forwards the synchronization success message to the first processing core 501 and the second processing core 502 through the barrier channel.

[0112] In some embodiments, the first distribution center is located in a first artificial intelligence chip, and the first processing core is located in a second artificial intelligence chip; that is, when the first distribution center and the first processing core are located in different artificial intelligence chips, the first distribution center forwards the synchronization success message to the first processing core through a second distribution center, and the second distribution center is located in the second artificial intelligence chip.

[0113] Specifically, when the first distribution center and the first processing core are located in different artificial intelligence chips, the first distribution center cannot directly send the synchronization success message to the first processing core, but needs to forward the synchronization success message through the second distribution center.

[0114] Similarly, when the first distribution center and the second processing core are located in different artificial intelligence chips, the first distribution center cannot directly send the synchronization success message to the second processing core, but needs to forward the synchronization success message through the second distribution center.

[0115] The first distribution center and the second distribution center pass the synchronization success message through the barrier channel; the bandwidth of the barrier channel is smaller than the direct interconnection bandwidth between the first artificial intelligence chip and the second artificial intelligence chip.

[0116] For example, referring to Figure 6 , it is assumed that GPU0 includes the first processing core 501, the second processing core 502, the first scheduling unit 503, the first distribution center 504, and the first cache 505. GPU1 includes the third processing core 601, the fourth processing core 602, the second scheduling unit 603, the second distribution center 604, and the second cache 605.

[0117] The first processing core 501 issues an ALU instruction 1 carrying a memory address 1, and the first scheduling unit 503 schedules the ALU instruction 1 to the first cache 505; the first cache 505 accumulates an operand (such as an operand 2) carried by the ALU instruction 1 to the memory address 1.

[0118] The first processing core 501 issues a barrier instruction 1 carrying a memory address 1, a barrier identifier 1, and an expected synchronization quantity (such as 3). The first scheduling unit 503 schedules the barrier instruction 1 to the first cache 505. The first cache 505 queries the barrier engine based on the barrier identifier 1 and the expected synchronization quantity, and no matching synchronization record is found, so a synchronization record 1 corresponding to the barrier instruction 1 is established in the barrier engine. The synchronization record 1 includes: a valid bit, the memory address 1, the barrier identifier 1, the expected synchronization quantity, and a synchronized instruction quantity, where the valid bit is 1, the expected synchronization quantity is 3, and the synchronized instruction quantity is 1.

[0119] The second processing core 502 issues an ALU instruction 2 carrying the memory address 1, and the first scheduling unit 503 schedules the ALU instruction 2 to the first cache 505; the first cache 505 accumulates an operand (such as an operand 3) carried by the ALU instruction 2 to the memory address 1.

[0120] The second processing core 502 issues a barrier instruction 2 carrying the memory address 1, the barrier identifier 1, and the expected synchronization quantity (such as 3). The first scheduling unit 503 schedules the barrier instruction 2 to the first cache 505. The first cache 505 queries the barrier engine based on the barrier identifier 1 and the expected synchronization quantity. Since the matching synchronization record 1 is found, the synchronized instruction quantity in the synchronization record 1 is updated to 2.

[0121] The third processing core 601 issues an ALU instruction 3 carrying the memory address 1; the second scheduling unit 603 forwards the ALU instruction 3 to the first scheduling unit 503, and the first scheduling unit 503 schedules the ALU instruction 3 to the first cache 505; the first cache 505 accumulates an operand (such as an operand 4) carried by the ALU instruction 3 to the memory address 1.

[0122] The third processing core 601 issues a barrier instruction 3 carrying the memory address 1, the barrier identifier 1, and the expected synchronization quantity (such as 3). The second scheduling unit 603 forwards the barrier instruction 3 to the first scheduling unit 503, and the first scheduling unit 503 schedules the barrier instruction 3 to the first cache 505. The first cache 505 queries the barrier engine based on the barrier identifier 1 and the expected synchronization quantity, and since the matching synchronization record 1 is found, the synchronized instruction quantity in the synchronization record 1 is updated to 3.

[0123] At this time, the expected synchronization number is equal to the synchronized instruction number, and thus the first cache 505 sends a synchronization success message carrying the barrier identifier 1 to the first distribution center 504 through the barrier channel. The first distribution center 504 forwards the synchronization success message to the first processing core 501 and the second processing core 502 through the barrier channel.

[0124] Meanwhile, the first distribution center 504 forwards the synchronization success message to the second distribution center 604 through the barrier channel, and the second distribution center 604 forwards the synchronization success message to the third processing core 601 through the barrier channel.

[0125] In the embodiment of the present application, the bandwidth of the barrier channel is small, and thus the barrier channel is more suitable for transmitting the small amount of data synchronization success message, thereby reducing the waste of bandwidth resources. When the synchronization success message is transmitted through the barrier channel, the occupation of the direct interconnection bandwidth between chips can be reduced, and the released direct interconnection bandwidth can be allocated to other memory access requests, thereby improving the utilization rate of the direct interconnection bandwidth and the overall efficiency of the system.

[0126] In some embodiments, after returning the synchronization success message to the first processing core and the at least one second processing core, a first read request carrying a target memory address sent by the first processing core is received; then a target calculation result is read from the target memory address; and the target calculation result is returned to the first processing core.

[0127] Specifically, the target calculation result is the calculation result of the first calculation instruction and the second calculation instruction sent by the at least one second processing core; that is, the calculation result of the calculation instruction of all processing cores has been sent to the cache and implemented. Each second calculation instruction carries a target memory address.

[0128] Similarly, when a first read request carrying a target memory address sent by the second processing core is received, a target calculation result is read from the target memory address; and the target calculation result is returned to the first processing core.

[0129] For example, referring to Figure 5 The first processing core 501 sends a read request 1 carrying a memory address 1, and the first scheduling unit 503 schedules the read request 1 to the first cache 505. The first cache 505 reads an accumulated result from the memory address 1, where the accumulated result is equal to 5; and then returns the accumulated result to the first scheduling unit 503, which returns the accumulated result to the first processing core 501. The second processing core 502 can read the accumulated result in the same way, which is not described here.

[0130] For another example, referring to Figure 6The third processing core 601 issues a read request 2 carrying memory address 1. The second scheduling unit 603 forwards the read request 2 to the first scheduling unit 503, which schedules the read request 2 to the first cache 505. The first cache 505 reads the accumulated result from memory address 1, where the accumulated result equals 9; then it returns the accumulated result to the first scheduling unit 503, which forwards the accumulated result to the second scheduling unit 603, which then returns the accumulated result to the third processing core 601.

[0131] In this embodiment, after the processing core issues a calculation instruction, it then issues a barrier instruction at the same memory address to ensure that the cache receives the calculation instruction and performs the calculation. The barrier engine records in real time the number of synchronized instructions of multiple processing cores bound to the same target barrier identifier; when the number of synchronized instructions is equal to the expected number of synchronized instructions, a synchronization success message is returned to multiple processing cores to ensure that the calculation instructions of all processing cores have been sent to the cache and performed, thereby achieving instruction synchronization and ensuring the accuracy of the calculation results read subsequently.

[0132] Based on the same technical concept, embodiments of this application provide a computer device, such as... Figure 7 As shown, it includes at least one artificial intelligence chip 100 and a memory 701 connected to at least one artificial intelligence chip 100. In this embodiment, the specific connection medium between the artificial intelligence chip 100 and the memory 701 is not limited. Figure 7 Taking the connection between the AI ​​chip 100 and the memory 701 via a bus as an example, the bus can be divided into address bus, data bus, control bus, etc.

[0133] In this embodiment of the application, the memory 701 stores instructions that can be executed by at least one artificial intelligence chip 100. By executing the instructions stored in the memory 701, at least one artificial intelligence chip 100 can perform the steps of the above-mentioned instruction synchronization method.

[0134] Among them, the artificial intelligence chip 100 is the control center of the computer device. It can connect to various parts of the computer device through various interfaces and lines, and realize instruction synchronization by running or executing instructions stored in memory 701 and calling data stored in memory 701.

[0135] Optionally, the artificial intelligence chip 100 can include one or more processing units, and the artificial intelligence chip 100 can integrate an application processor and a modem processor, where the application processor mainly processes an operating system, a user interface, and an application program, and the modem processor mainly processes wireless communication. It can be understood that the above-mentioned modem processor can also not be integrated into the artificial intelligence chip 100. In some embodiments, the artificial intelligence chip 100 and the memory 701 can be implemented on the same chip, and in some embodiments, they can also be implemented on separate chips respectively.

[0136] The artificial intelligence chip 100 can be a general processor, such as a central processing unit (CPU), a digital signal processor, an application specific integrated circuit (ASIC), a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, which can implement or execute the methods, steps and logic block diagrams disclosed in the embodiments of the present application. The general processor can be a microprocessor or any conventional processor. The steps of the method disclosed in combination with the embodiments of the present application can be directly embodied as execution completed by a hardware processor, or executed by a combination of hardware and software modules in the processor.

[0137] The memory 701 as a non-volatile computer readable storage medium can be used to store non-volatile software programs, non-volatile computer executable programs and modules. The memory 701 can include at least one type of storage medium, such as flash memory, hard disk, multimedia card, card type memory, random access memory (RAM), static random access memory (SRAM), programmable read-only memory (PROM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), magnetic storage, magnetic disk, optical disk, etc. The memory 701 is any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer device, but is not limited to this. The memory 701 in the embodiments of the present application can also be a circuit or other any device capable of realizing a storage function, used to store program instructions and / or data.

[0138] Based on the same inventive concept, the embodiment of the present application provides a computer readable storage medium, which stores a computer program executable by a computer device, and when the computer program is executed on the computer device, the computer device executes the steps of the instruction synchronization method.

[0139] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, or a computer program product. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, etc.) containing computer-usable program code.

[0140] The present application is described with reference to the flowcharts and / or block diagrams of the method, device (system) and computer program product according to the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer device or other programmable data processing device produce the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 means for performing the function specified by one or more blocks

[0141] These computer program instructions can also be stored in a computer readable memory that can direct the computer device or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce a product including instruction means, which implements the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 means for performing the function specified by one or more blocks

[0142] These computer program instructions can also be loaded into the computer device or other programmable data processing device, so that a series of operation steps are performed on the computer device or other programmable device to produce a processing implemented by the computer device, so that the instructions executed on the computer device or other programmable device provide the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 means for performing the function specified by one or more blocks

[0143] While the preferred embodiments of the application have been described, additional variations and modifications can be made to these embodiments by those skilled in the art once they have the benefit of the present disclosure without departing from the spirit and scope of the application. Accordingly, it is intended that the appended claims include all such modifications and variations as fall within the scope of the present application.

[0144] It is apparent that those skilled in the art can make various changes and modifications to the application without departing from the spirit and scope of the application. It is therefore intended that the present application cover all such changes and modifications that are within its scope.

Claims

1. A method of instruction synchronization, the method comprising: Applied to cache, comprising: Receiving a first barrier instruction sent by a first processing core, the first barrier instruction carrying a target memory address, a target barrier identifier and an expected synchronization quantity; wherein the first barrier instruction is sent by the first processing core after sending a first calculation instruction carrying the target memory address; the first processing core sends the first calculation instruction first, and then sends the first barrier instruction; When there is a target synchronization record in the barrier engine that matches the target barrier identifier and the expected synchronization quantity, updating the synchronized instruction quantity in the target synchronization record, the synchronized instruction quantity representing the number of received calculation instructions carrying the target memory address; When the expected synchronization quantity is equal to the synchronized instruction quantity, returning a synchronization success message to the first processing core and at least one second processing core bound with the target barrier identifier.

2. The method of claim 1, wherein, Further comprising: When there is no target synchronization record in the barrier engine that matches the target barrier identifier and the expected synchronization quantity, creating a first synchronization record in the barrier engine based on the target barrier identifier and the expected synchronization quantity.

3. The method of claim 1, wherein, The when the expected synchronization quantity is equal to the synchronized instruction quantity, returning a synchronization success message to the first processing core and at least one second processing core bound with the target barrier identifier, comprising: When the expected synchronization quantity is equal to the synchronized instruction quantity, sending a synchronization success message carrying the target barrier identifier to a first distribution center, so that the first distribution center forwards the synchronization success message to the first processing core and each second processing core.

4. The method of claim 3, wherein, The cache and the first distribution center are located in the same artificial intelligence chip; the cache and the first distribution center transmit the synchronization success message through a barrier channel; the bandwidth of the barrier channel is less than the bus bandwidth of the artificial intelligence chip.

5. The method of claim 4, wherein, The first distribution center and the first processing core are located in the same artificial intelligence chip; The first distribution center and the first processing core transmit the synchronization success message through the barrier channel.

6. The method of claim 3, wherein, The first distribution center is located in a first artificial intelligence chip, and the first processing core is located in a second artificial intelligence chip; The so that the first distribution center forwards the synchronization success message to the first processing core, comprising: So that the first distribution center forwards the synchronization success message to the first processing core through a second distribution center, the second distribution center being located in the second artificial intelligence chip.

7. The method of claim 6, wherein, The first distribution center and the second distribution center transmit the synchronization success message through a barrier channel; The bandwidth of the barrier channel is less than the direct interconnection bandwidth between the first artificial intelligence chip and the second artificial intelligence chip.

8. The method of any one of claims 1 to 7, wherein, After the return of the synchronization success message to the first processing core and at least one second processing core bound with the target barrier identifier, further comprising: Marking the target synchronization record as invalid in the barrier engine.

9. The method of any one of claims 1 to 7, wherein, The returning the synchronization success message to the first processing core and at least one second processing core bound with the target barrier identifier further comprises: receiving a first read request carrying the target memory address sent by the first processing core; reading a target calculation result from the target memory address; the target calculation result is a calculation result of the first calculation instruction and a second calculation instruction sent by the at least one second processing core; the second calculation instruction carries the target memory address; returning the target calculation result to the first processing core.

10. An artificial intelligence chip, characterized by, comprise: a processing core, a cache and a main memory; the cache is configured to execute the method in any one of claims 1 to 9.

Images