A method and apparatus for implementing a reduction algorithm

Abstract

This invention discloses a method and apparatus for implementing a reduction algorithm, relating to the field of artificial intelligence technology. The method includes: upon receiving a memory access request from an interface, determining whether the memory access request is a reduction read request; if it is a reduction read request, splitting the memory access request into multiple requests based on address and suspending them in a suspending buffer; upon receiving a response with a reduction flag, performing a reduction operation in the suspending buffer; and upon receiving responses from multiple storage controllers, returning the reduced result as a single response to the interface. This invention efficiently implements a reduction algorithm for multi-core AI processors with minimal logical overhead, avoiding numerous repetitive memory access operations and maximizing data utilization.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method and apparatus for implementing a reduction algorithm. Background Technology

[0002] A deep learning model is a type of machine learning model whose training is based on learning data representations rather than learning specific tasks. In other words, deep learning, or machine learning, is the ability of a computer to learn without being explicitly programmed to perform certain functions. Therefore, machine learning allows programmers to initially program algorithms that can be used to predict data responses, rather than explicitly programming for every response to every possible situation the computer might encounter.

[0003] In the implementation of the parallel training algorithm for deep learning models, each processor sends its trained model parameters ∆W to the parameter server and receives new model parameters returned by the parameter server to perform new training.

[0004] When training the same model, an artificial intelligence processor has multiple processing elements (PEs) training their own images in parallel, generating their own model parameters ∆W, and storing them in the local memory of the corresponding processing element.

[0005] When an AI processor needs to send model parameters ∆W to a parameter server, multiple ∆W values ​​need to be reduced to one. To merge multiple model parameters ∆W into one model parameter ∆W, the reduction unit needs to perform multiple pairwise operations on data from different memory locations, write the operations into memory, and finally send the calculation results stored in memory to the parameter server.

[0006] However, the execution of existing reduction algorithms inevitably involves a large number of memory read and write operations, which are inefficient and require a lot of energy. As the computing power of AI processors increases, the memory access bottleneck becomes more and more obvious.

[0007] Therefore, it is essential to implement a reasonable and efficient reduction algorithm, both for improving memory access efficiency and reducing energy consumption. Summary of the Invention

[0008] The purpose of this invention is to provide a method and apparatus for implementing a reduction algorithm. It uses relatively low logical cost to efficiently implement a reduction algorithm for multi-core artificial intelligence processors, avoids a large number of repetitive memory access operations, and maximizes data utilization.

[0009] According to a first aspect of the present invention, a method for implementing a reduction algorithm is proposed, comprising:

[0010] When a memory access request is received from the interface, determine whether the memory access request is a reduced read request; if it is a reduced read request, suspend the memory access request to the suspend buffer.

[0011] When a reduction response is received from the on-chip network of the AI ​​processor, a reduction operation is performed in the dangling buffer.

[0012] When multiple responses from storage controllers are received, the reduced result is returned as a single response to the interface.

[0013] Furthermore, before receiving the memory access request from the interface:

[0014] A specification component is placed at the common area between the external interface of the artificial intelligence processor and the various processing components.

[0015] The protocol components are electrically connected to the interface of the AI ​​processor and the on-chip network of the AI ​​processor.

[0016] Furthermore, suspend memory access requests before the suspend buffer:

[0017] Determine whether the memory access request contains multiple reduced read requests. If so, split the memory access request into multiple reduced read requests by address; otherwise, do not perform the operation.

[0018] Furthermore, after splitting the memory access request into multiple reduction read requests based on address:

[0019] The extracted packets are sent to different storage controllers on the on-chip network, and the reduced read requests are suspended to the suspend buffer.

[0020] Furthermore, after the extracted packets are sent to different storage controllers on the on-chip network:

[0021] The protocol component receives the response packet from the storage controller sent by the on-chip network, checks the dangling buffer to see if the corresponding entry already has a value, and if so, the response packet is a protocol response.

[0022] Furthermore, the specific procedures for reducing the suspension buffer include:

[0023] The dangling buffer checks whether the response packet from the memory controller sent by the on-chip network already has a value. If it does, the value is read and added using a single instruction multiple data stream.

[0024] The summation result is sent to the second-level single instruction multiple data stream unit for addition, multiplication, and addition operations.

[0025] Furthermore, the reduced result is returned to the interface as a response, specifically:

[0026] The data packets, after passing through the second-level single instruction multiple data stream component, are returned to the interface.

[0027] According to a second aspect of the present invention, an apparatus for implementing a reduction algorithm is provided, comprising:

[0028] Judgment module: When a memory access request is received from the interface, it is used to determine whether the memory access request is a read request.

[0029] The unpacking module is used to unpack memory access requests into several memory access requests.

[0030] The suspending module: will suspend memory access requests for reduced read requests to the suspending buffer;

[0031] The reduction calculation module is used to perform reduction operations in the dangling buffer when a reduction response is received from the on-chip network of the AI ​​processor.

[0032] The response module is used to return the reduced result as a single response to the interface when multiple responses are received from the storage controllers.

[0033] According to a third aspect of the present invention, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method steps of any of the first aspects described above.

[0034] According to a fourth aspect of the present invention, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the method steps of any one of the first aspects described above.

[0035] The beneficial effects of this invention are as follows:

[0036] This invention provides a method and apparatus for implementing a reduction algorithm. It efficiently implements a reduction algorithm for multi-core artificial intelligence processors with relatively low logical cost, avoids a large number of repetitive memory access operations, and maximizes data utilization. Attached Figure Description

[0037] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and, together with the description, serve to explain the principles of the invention. In these drawings, similar reference numerals are used to denote similar elements. The drawings described below are some embodiments of the invention, but not all embodiments. Other drawings will be readily available to those skilled in the art based on these drawings without any inventive effort.

[0038] Figure 1 This is a flowchart illustrating a method for implementing a reduction algorithm according to an embodiment of the present invention;

[0039] Figure 2 This is a schematic diagram illustrating the logical processing of a method for implementing a reduction algorithm according to an embodiment of the present invention;

[0040] Figure 3 This is a schematic diagram of the module connections of an apparatus for implementing a reduction algorithm according to an embodiment of the present invention. Detailed Implementation

[0041] To more clearly illustrate the technical solutions in the embodiments of the present invention and the prior art, the specific implementation methods of the present invention will be described below with reference to the accompanying drawings. Obviously, the accompanying drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings and other implementation methods can be obtained based on these drawings without any creative effort. Furthermore, the design orientation only indicates the relative positional relationship between the components, not the absolute positional relationship. Example

[0042] like Figure 1 , 2 As shown, according to a first aspect of the present invention, a method for implementing a reduction algorithm is proposed, comprising:

[0043] S101: When a memory access request is received from the interface, determine whether the memory access request is a reduced read request; if it is a reduced read request, suspend the memory access request to the suspended buffer.

[0044] In this embodiment of the invention, it should be noted that the execution subject of step S101 is a specification component, which is located at the common part between the external interface of the artificial intelligence processor and each processing component.

[0045] When the AI ​​processor receives a memory access request at its interface, the request will inevitably be received by the specification component before reaching the processing component (PE).

[0046] When a protocol component receives a memory access request from the interface, it determines whether the memory access request is a protocol read request; if it is a protocol read request, it suspends the memory access request in the protocol component's suspend buffer.

[0047] Of course, before receiving the memory access request from the interface, a protocol component is placed in the common area between the AI ​​processor's external interface and each processing component. The protocol component can be placed in the interface component of the AI ​​processor's external interface, in the on-chip network component, or in other suitable locations.

[0048] Furthermore, the protocol components are electrically connected to the interface of the AI ​​processor and the on-chip network of the AI ​​processor.

[0049] In this embodiment of the invention, the interface component of the artificial intelligence processor's external interface needs to support the control core in configuring reduction read-related descriptors for the interface component. The descriptor needs to include information such as the starting address and length of each storage controller. The interface component extracts a corresponding number of reduction read requests based on the length in the descriptor. The header of each reduction read request no longer contains {control information + memory access address}, but rather {control information + memory controller 0 memory access address + memory controller 1 memory access address + ... + memory controller n memory access address}. Other processing procedures for reduction read requests in the interface component are no different from those for ordinary read requests.

[0050] It should be noted that these tasks can only be initiated by the interface component; the specification component itself cannot be the initiator. In other words, the interface component must send the memory access request to the specification component.

[0051] In this embodiment of the invention, before suspending a memory access request to the suspending buffer, it is necessary to determine whether the memory access request contains multiple reduced read requests. If so, the memory access request is split into n reduced read requests according to the address; otherwise, the splitting operation is not performed.

[0052] Of course, after splitting the memory access request into multiple reduction read requests by address, the split packets are sent to different memory controllers on the on-chip network, and the reduction read requests are suspended to the suspend buffer.

[0053] S102: When a reduction response is received from the on-chip network of the AI ​​processor, a reduction operation is performed in the dangling buffer;

[0054] In this embodiment of the invention, after the received reduction read request is suspended in the suspension buffer, and multiple reduction read requests extracted from the memory access request are sent to the on-chip network of the artificial intelligence processor, after multiple processing units (PEs) of the artificial intelligence processor send the reduction response, the reduction response packet and the reduction read request need to be reduced in the suspension buffer.

[0055] In this embodiment of the invention, after the extracted packets are sent to different storage controllers of the on-chip network: the protocol component obtains the response packets from the storage controllers sent by the on-chip network, queries the dangling buffer to see if the corresponding entry already has a value. If there is a value, it means that the protocol read request corresponding to the response packet has been dangling and can be used for protocol operation. The response packet can then be regarded as a protocol response.

[0056] When performing reduction operations in the dangling buffer, the specific steps include: querying the dangling buffer to see if the response packet from the memory controller sent by the on-chip network already has a value; if it does, reading it out and performing single instruction multiple data stream addition; and sending the addition result to the second-level single instruction multiple data stream unit for addition-multiplication-addition operations.

[0057] The multiplication in the multiply-add operation is actually multiplying the addition result by 1 / n, where n is the number of reduced read requests that the memory access request is broken down into, which means that the data only accounts for one-nth of the total data.

[0058] Conversely, if the response packet from the on-chip network to the storage controller in the dangling buffer does not contain a value, the data is stored and the reduction operation is performed only after the corresponding reduction read request arrives.

[0059] It is important to note that Single Instruction Multiple Data (SIMD) can copy multiple operands and perform reduction operations on multiple reduced data.

[0060] The reduction algorithm includes finding the maximum value, summation, etc. Its characteristics are that the amount of data involved in the operation is large and the operation is simple. Therefore, the reduction operation can be completed by setting a reduction component in the common part between the external interface of the artificial intelligence processor and each processing component.

[0061] S103: When multiple responses from storage controllers are received, the reduced result is returned as a single response to the interface.

[0062] In this embodiment of the invention, the reduced result is returned to the interface as a response, specifically as follows:

[0063] The data packets processed by the second-level single instruction multiple data stream (SMILE) component are returned to the interface. This means that the data packets resulting from the addition, multiplication, and addition operations performed by the second-level SMILE component are packaged together and then sent out as a response packet to the memory access request received from the interface.

[0064] This invention provides a method for implementing a reduction algorithm. It efficiently implements a reduction algorithm for multi-core artificial intelligence processors with relatively low logical cost, avoids a large number of repetitive memory access operations, and maximizes data utilization. Example

[0065] like Figure 3 As shown, according to a second aspect of the present invention, an apparatus for implementing a reduction algorithm is provided, comprising:

[0066] Judgment module: When a memory access request is received from the interface, it is used to determine whether the memory access request is a read request.

[0067] The unpacking module is used to unpack memory access requests into several memory access requests.

[0068] The suspending module: will suspend memory access requests for reduced read requests to the suspending buffer;

[0069] The reduction calculation module is used to perform reduction operations in the dangling buffer when a reduction response is received from the on-chip network of the AI ​​processor.

[0070] The response module is used to return the reduced result as a single response to the interface when multiple responses are received from the storage controllers.

[0071] It is understood that the apparatus provided in the embodiments of the present invention is applicable to the method described in Embodiment 1, and the specific functions of each module can be referred to the above method flow, which will not be repeated here. Example

[0072] According to a third aspect of the present invention, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method steps described in Embodiment 1 above.

[0073] The electronic device is used to implement the method described in Embodiment 1. The electronic device may include: at least one central processing unit, at least one network interface, a control interface, a memory, and at least one communication bus.

[0074] The communication bus is used to enable communication and information exchange between the various components.

[0075] The network interface may include a standard wired interface or a wireless interface (such as a Wi-Fi interface).

[0076] The control interface is used to output control operations according to instructions.

[0077] The central processing unit (CPU) may include one or more processing cores. The CPU connects to various parts of the terminal via various interfaces and lines, and executes instructions, programs, code sets, or instruction sets stored in memory, as well as accessing data stored in memory, to perform various functions of the terminal and process data according to the method described in Embodiment 1.

[0078] The memory may include random access memory (RAM) or read-only memory. Optionally, the memory may include non-transitory computer-readable storage medium. The memory can be used to store instructions, programs, code, code sets, or instruction sets. The memory may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the methods described in Embodiment 1 above, etc.; the data storage area may store data involved in the various method embodiments above, etc.

[0079] A fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in Embodiment 1 above. The computer-readable storage medium may include, but is not limited to, any type of disk, including floppy disks, optical disks, DVDs, CD-ROMs, microdrives, as well as magneto-optical disks, ROMs, RAMs, EPROMs, EEPROMs, DRAMs, VRAMs, flash memory devices, magnetic cards or optical cards, nanosystems (including molecular memory ICs), or any type of medium or device suitable for storing instructions and / or data.

[0080] This invention provides a method and apparatus for implementing a reduction algorithm. It efficiently implements a reduction algorithm for multi-core artificial intelligence processors with relatively low logical cost, avoids a large number of repetitive memory access operations, and maximizes data utilization.

[0081] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0082] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0083] In the several embodiments provided in this application, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some service interface; the indirect coupling or communication connection between devices or units may be electrical or other forms.

[0084] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0085] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0086] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0087] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, which may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc.

[0088] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element.

[0089] The foregoing description is merely an exemplary embodiment of this disclosure and should not be construed as limiting the scope of this disclosure. Any equivalent changes and modifications made in accordance with the teachings of this disclosure shall still fall within the scope of this disclosure. Those skilled in the art will readily conceive of embodiments of this disclosure upon considering the specification and practicing the disclosure herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not described herein. The specification and embodiments are to be considered exemplary only, and the scope and spirit of this disclosure are defined by the claims.

Claims

1. A method for implementing a reduction algorithm, characterized in that, include: When receiving a memory access request from the interface, determine whether the memory access request is a read request. If it is a read request, the memory access request is suspended to the suspend buffer; When a reduction response is received from the on-chip network of the AI ​​processor, a reduction operation is performed in the dangling buffer. When multiple responses from storage controllers are received, the reduced result is returned as a single response to the interface. Hang memory access requests before the hanging buffer: Determine if the memory access request contains multiple reduced read requests. If so, split the memory access request into multiple reduced read requests by address; otherwise, do not perform the operation. After splitting the memory access request into multiple reduction read requests by address: the split packets are sent to different memory controllers on the on-chip network, and the reduction read requests are suspended to the suspend buffer; When performing reduction operations in the dangling buffer, the specific steps include: querying the dangling buffer to see if the response packet from the memory controller sent by the on-chip network already has a value; if it does, reading it out and performing single instruction multiple data stream addition. The summation result is sent to the second-level single instruction multiple data stream unit for addition, multiplication, and addition operations; The reduced result is returned to the interface as a response, specifically: The data packets, after passing through the second-level single instruction multiple data stream component, are returned to the interface.

2. The method for implementing the reduction algorithm according to claim 1, characterized in that, Before receiving the memory access request from the interface: A specification component is placed at the common area between the external interface of the artificial intelligence processor and the various processing components. The protocol components are electrically connected to the interface of the AI ​​processor and the on-chip network of the AI ​​processor.

3. The method for implementing the reduction algorithm according to claim 2, characterized in that, After the extracted packets are sent to different storage controllers on the on-chip network: The protocol component receives the response packet from the storage controller sent by the on-chip network, checks the dangling buffer to see if the corresponding entry already has a value, and if so, the response packet is a protocol response.

4. An apparatus for implementing a reduction algorithm, characterized in that, include: Judgment module: When a memory access request is received from the interface, it is used to determine whether the memory access request is a read request. The unpacking module is used to unpack memory access requests into several memory access requests. The suspending module: will suspend memory access requests for reduced read requests to the suspending buffer; The reduction calculation module is used to perform reduction operations in the dangling buffer when a reduction response is received from the on-chip network of the AI ​​processor. The response module is used to return the reduced result as a single response to the interface when multiple responses are received from the storage controllers. Hang memory access requests before the hanging buffer: Determine if the memory access request contains multiple reduced read requests. If so, split the memory access request into multiple reduced read requests by address; otherwise, do not perform the operation. After splitting the memory access request into multiple reduction read requests by address: the split packets are sent to different memory controllers on the on-chip network, and the reduction read requests are suspended to the suspend buffer; When performing reduction operations in the dangling buffer, the specific steps include: querying the dangling buffer to see if the response packet from the memory controller sent by the on-chip network already has a value; if it does, reading it out and performing single instruction multiple data stream addition. The summation result is sent to the second-level single instruction multiple data stream unit for addition, multiplication, and addition operations; The reduced result is returned to the interface as a response, specifically: The data packets, after passing through the second-level single instruction multiple data stream component, are returned to the interface.

5. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of a method for implementing a reduction algorithm as described in any one of claims 1 to 3.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of a method for implementing a reduction algorithm as described in any one of claims 1 to 3.

Images