Accumulation system and method

Abstract

To provide systems and methods for performing multiply / accumulate operations.SOLUTION: In a series 300 of shared memories, three convolution cores perform steps of: receiving data for processing; performing a multiplication operation on the received data to generate multiplied data; performing an addition operation on the multiplied data to generate a result; storing at least a portion of the least significant bits of the result in a first region of an accumulation buffer of a convolution core; and storing at least a portion of the remaining bits of the result in a shared memory separate from the convolution core.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present disclosure relates to systems and methods for performing multiply / accumulate operations. [Background technology]

[0002] Various types of systems perform multiplication and accumulation operations. For example, neural networks and matrix multiplication systems may perform one or more multiplication or accumulation operations. These multiplication and accumulation operations may be applied to a variety of mathematical problems that lend themselves to computational solutions. Some of these computational solutions may include accumulation buffers that support combining multiple groups of data together.

[0003] The systems and methods discussed herein provide an improved approach for performing multiply / accumulate operations.

[0004] Non-limiting and non-exhaustive embodiments of the present disclosure are described with reference to the following figures, in which like reference numerals refer to like parts throughout the various views unless otherwise specified. [Brief explanation of the drawings]

[0005] [Figure 1] An embodiment of a system for performing multiply / accumulate operations is described. [Figure 2] An embodiment of an accumulation buffer will now be described. [Figure 3] An embodiment of a family of convolution cores and associated shared memories is described. [Figure 4] FIG. 1 is a flow diagram illustrating an embodiment of a method for performing a multiply / accumulate operation. [Figure 5] 1 illustrates an exemplary block diagram of a computing device. DETAILED DESCRIPTION OF THE INVENTION

[0006] In the following disclosure, references are made to the accompanying drawings, which form a part hereof, and which show, by way of illustration, specific implementations in which the disclosure may be practiced. It is understood that other implementations may be utilized and structural changes may be made without departing from the scope of the present disclosure. References in the specification to "one embodiment," "an embodiment," "an exemplary embodiment," etc., indicate that the described embodiment may include a particular feature, structure, or feature, but that not all embodiments necessarily include the particular feature, structure, or feature. Moreover, such phrases do not necessarily refer to the same embodiment. Furthermore, when a particular feature, structure, or feature is described in connection with one embodiment, it is relied upon to affect such feature, structure, or feature in connection with other embodiments, whether or not explicitly described.

[0007] Implementations of the systems, devices, and methods disclosed herein may include or utilize special purpose or general purpose computers, including computer hardware such as one or more processors and system memory, as discussed herein. Implementations within the scope of the present disclosure may also include physical and other computer-readable media for carrying or storing computer-executable instructions and / or data structures. Such computer-readable media may be any available media that can be accessed by a general purpose or special purpose computer system. Computer-readable media that store computer-executable instructions are computer storage media (devices). Computer-readable media that carry computer-executable instructions are transmission media. Thus, by way of example, and not limitation, implementations of the disclosure may include at least two distinctly different types of computer-readable media: computer storage media (devices) and transmission media.

[0008] Computer storage media (devices) include RAM, ROM, EEPROM, CD-ROM, solid-state drives (“SSD”) (e.g., RAM-based), flash memory, phase-change memory (“PCM”), other types of memory, other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store desired program code means in the form of computer-executable instructions or data structures and that can be accessed by a general-purpose or special-purpose computer.

[0009] Implementations of the devices, systems, and methods disclosed herein may communicate over a computer network. A "network" is defined as one or more data links that enable the transport of electronic data between computer systems and / or modules and / or other electronic devices. When information is transferred or provided to a computer over a network or another communications connection (either hardwired, wireless, or a combination of hardwired or wireless), the computer properly views the connection as a transmission medium. Transmission media may include networks and / or data links that may be used to transport desired program code means in the form of computer-executable instructions or data structures and that may be accessed by a general-purpose or special-purpose computer. Combinations of the above may also be included within the scope of computer-readable media.

[0010] Computer-executable instructions include, for example, instructions and data that, when executed by a processor, cause a general-purpose computer, a special-purpose computer, or a special-purpose processing device to perform a certain function or group of functions. Computer-executable instructions may be, for example, binaries, intermediate format instructions such as assembly language, or even source code. While the subject matter has been described in language specific to structural mechanisms and / or methodological acts, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the mechanisms or acts described herein. Rather, the described mechanisms and acts are disclosed as example forms for implementing the claims.

[0011] Those skilled in the art will appreciate that the disclosure may be practiced in networked computing environments with many types of computer system configurations, including personal computers, desktop computers, laptop computers, message processors, handheld devices, multiprocessor systems, multiprocessor-based or programmable appliances, network PCs, microcomputers, mainframe computers, cellular phones, PDAs, tablets, pagers, routers, switches, and various storage devices. The disclosure may also be practiced in distributed system environments where tasks are performed by both local and remote computer systems that are linked through a network (either by wired data links, wireless data links, or a combination of wired and wireless data links). In a distributed system environment, program modules may be located in both local and remote memory storage devices.

[0012] Additionally, where appropriate, the functions described herein may be implemented in one or more of hardware, software, firmware, digital components, or analog components. For example, one or more application-specific integrated circuits (ASICs) may be programmed to execute one or more of the systems and procedures described herein. Certain terms are used throughout the description and claims to refer to particular system components. As one skilled in the art will understand, components may be referred to by different names. This document does not intend to distinguish between components that differ in name but not function.

[0013] At least some embodiments of the disclosure are directed to computer program products including such logic (e.g., in the form of software) stored on any computer-usable medium that, when executed on one or more data processing devices, causes the devices to operate as described herein.

[0014] The systems and methods discussed herein are useful in a variety of computing environments and procedures, such as machine learning environments, neural networks, and matrix multiplication procedures. As described herein, the systems and methods may reduce the power used by a device (such as a processing device or a storage device) and may eliminate the need for memory storage space during operation of the system.

[0015] FIG. 1 illustrates an embodiment of a system 100 for performing multiply / accumulate operations. System 100 may be referred to herein as a "convolution core." In some embodiments, system 100 may be implemented in any device, system, or environment requiring long accumulation sequences. In some embodiments, system 100 performs a multiplication operation on two 8-bit numbers (also referred to herein as "data") using multiplier 102. The output of multiplier 102 is a 16-bit number, which is communicated to adder 104. Adder 104 operates in conjunction with accumulation buffer 106, as shown in FIG. 1. In some embodiments, accumulation buffer 106 is a 32-bit accumulation buffer.

[0016] The accumulation buffer 106 may act as a buffer and may store multiple data entries over time. For example, the accumulation buffer 106 may accumulate incoming data up to a particular number of entries, such as 32 entries. In a particular example, data 1 may be stored in entry 4 of the accumulation buffer 106, data 2 may be stored in entry 17 of the accumulation buffer 106, and so on. In some embodiments, there is no particular mapping of data to entries in the accumulation buffer. In particular implementations, the accumulated data is interleaved within the accumulation buffer 106. In some embodiments, each entry in the accumulation buffer 106 is a channel of a neural network. A particular convolutional layer of a neural network may have any number of channels.

[0017] In some implementations, a single accumulation buffer 106 may support data associated with any number of channels. This may allow data to be fetched faster because data from multiple channels may be fetched from the accumulation buffer 106 simultaneously (instead of a slower serial fetch of the data). For example, data may be fetched at one time and accumulated in the accumulation buffer 106. The data may then be accessed or applied to multiple channels (from the accumulation buffer 106) simultaneously (or at different times without having to re-fetch the data).

[0018] FIG. 2 illustrates an embodiment of accumulation buffer 200. As shown in FIG. 2, accumulation buffer 200 has three portions, referred to as "regions." These regions of accumulation buffer 200 include region A 202, region B 204, and region C 206. In some embodiments, each region contains a portion of the bits of a particular entry. For example, if a total of 32 bits are available in accumulation buffer 200, region A 202 may contain the least significant 16 bits, region B 204 may contain the next most significant 8 bits, and region C 206 may contain the most significant 8 bits. In other embodiments, any number of bits may be available in accumulation buffer 200, and any number of bits may be provided in each region. Additionally, alternative embodiments may separate accumulation buffer 200 into any number of different regions.

[0019] In some implementations, 2 15A significant number of additions, such as 10 additions, are required to fill a 32-bit buffer. In some embodiments, the neural network may not have enough steps or activity to fill a 32-bit buffer (or it may take a long time to fill a 32-bit buffer). In some instances, the buffer begins to fill region A 202. When region A 202 is full, a carryover command 208 (or activity) is generated indicating that data will begin to accumulate in region B 204. Similarly, when region B 204 is full, a carryover command 210 (or activity) is generated indicating that data will begin to accumulate in region C 206.

[0020] First, accumulation buffer 200 does not need to access region B 204 or region C 206 until region A 202 is filled. In some embodiments, if region B 204 or region C 206 is not utilized, the clock (e.g., clock signal) for each unused region is disabled to reduce power consumption. In some implementations, if there is no data activity but the clock remains enabled, significant power may be required to generate an unnecessary clock. Therefore, disabling the clock when a particular region is not being used saves power.

[0021] If region B 204 or region C 206 is needed to store data (e.g., based on carryover 208 or 210), the clock for the appropriate region 204, 206 is enabled to allow data activity in that region. For example, if carryover 208 is detected, the clock for region B 204 is enabled to allow data storage in region B 204. However, the clock for region C 206 remains disabled until carryover 210 is detected. Thus, the systems and methods described herein may reduce power consumption by disabling the clocks for all regions that are not actively processing data.

[0022] FIG. 3 illustrates an embodiment of a series 300 of convolution cores and associated shared memory. In the example of FIG. 3, three convolution cores are shown: a first convolution core 302, a second convolution core 304, and a third convolution core 306. In particular implementations, any number of convolution cores may be used in the systems and methods discussed herein. Each convolution core 302, 304, 306 includes a multiplier 308, an adder 310, and an accumulation buffer 312. In some embodiments, each convolution core 302, 304, 306 is coupled to a shared memory 314. As shown in FIG. 3, the shared memory 314 may include any number of memory segments (or memory portions) 316, 318, 320, and 322 for storing various types of data. In some embodiments, each memory segment 316, 318, 320, and 322 is associated with one of the convolution cores 302, 304, 306.

[0023] In some embodiments, the accumulation buffer 312 includes regions A and B as discussed herein with respect to FIG. 2 . Thus, data associated with regions A and B is stored within each convolution core 302, 304, 306 to enable fast access to the data. In some embodiments, data associated with region C is stored in a shared memory 314, which may be external to each convolution core 302, 304, 306. In the example of FIG. 3 , the most commonly used data is likely stored within regions A and B within each convolution core 302, 304, 306. Data in region C, which may be used less frequently, is stored in a separate shared memory 314. In some embodiments, this organization of regions A, B, and C reduces the amount of memory required within each convolution core 302, 304, 306. This reduction in memory size reduces the size of the silicon on which the convolution cores 302, 304, 306 are implemented. In some embodiments, this reduction in silicon size may reduce the power consumption of the systems and methods described herein.

[0024] In the example of Figure 3, shared memory 314 is updated when accumulation buffer 312 has a carryover to region C (similar to carryover 210 discussed with respect to Figure 2). In some embodiments, carryover to region C does not occur frequently, so shared memory 314 may not be a high performance memory component.

[0025] In some implementations, a FIFO (first-in, first-out) buffer (not shown) may be disposed between each convolution core 302, 304, 306 and the shared memory 314. The FIFO buffer receives data storage requests sent to the shared memory 314 and buffers the requests for processing in order.

[0026] In other embodiments, instead of a FIFO buffer, the system and method may use a sequencer (not shown) that runs on the accumulation buffer 312 of each convolution core 302, 304, 306. The sequencer may provide each convolution core 302, 304, 306 with a specific period (e.g., a time slot) during which that convolution core may output a carryover signal to the shared memory 314. Because each convolution core 302, 304, 306 sends its carryover signal at a different time, a FIFO buffer is not necessary. In some embodiments, data storage requests and other data are sent from each convolution core 302, 304, 306 to the shared memory 314 during that convolution core's designated period as managed by the sequencer.

[0027] 3 illustrates an example environment using multiple convolution cores 302, 304, 306, similar systems and methods may be used in other types of hardware and software processing structures and systems. In some embodiments, the convolution cores 302, 304, 306 may be similar to the convolution cores shown in FIG.

[0028] FIG. 4 is a flow diagram illustrating an embodiment of a method 400 for performing a multiplication / accumulation operation. First, the method 400 receives data (or a data processing request) (402). The method performs a multiplication operation on the received data to generate multiplied data (404). The method 400 then performs an addition operation on the multiplied data to generate a result (406). At least a portion of the least significant bits of the result are accumulated within the convolution core (408). The method also accumulates at least a portion of the most significant bits of the result in a shared memory located outside the convolution core (410). In some embodiments, accumulating at least a portion of the most significant bits of the result in a shared memory located outside the convolution core (410) is performed in response to determining that an accumulation buffer of the convolution core has generated a carryover command.

[0029] 4 is discussed with respect to one or more convolution cores. In other embodiments, the method 400 may be used in any type of hardware, software, processing architecture, data processing system, and the like.

[0030] 5 illustrates an exemplary block diagram of a computing device 500. The computing device 500 may be used to perform various procedures, such as those discussed herein. For example, the computing device 500 may implement any of the functions or methods of the computing devices and systems discussed herein. The computing device 500 may perform various functions as discussed herein and may execute one or more application programs, such as the application programs or functions described herein. The computing device 500 may be any of a wide variety of computing devices, such as a desktop computer, a notebook computer, a server computer, a handheld computer, a tablet computer, and a wearable device.

[0031] Computing device 500 includes one or more processors 502, one or more memory devices 504, one or more interfaces 506, one or more mass storage devices 508, one or more input / output (I / O) devices 510, and a display device 530, all coupled to a bus 512. Processor 502 includes one or more processors or controllers that execute instructions stored in memory device(s) 504 and / or mass storage device(s) 508. Processor 502 may also include various types of computer-readable media, such as cache memory.

[0032] The memory device 504 includes a variety of computer-readable media, such as volatile memory (e.g., random access memory (RAM) 514) and / or non-volatile memory (e.g., read-only memory (ROM) 516). The memory device 504 may also include re-writable ROM, such as flash memory.

[0033] The mass storage device 508 includes various computer-readable media such as magnetic tape, magnetic disks, optical disks, and solid-state memory (e.g., flash memory). As shown in Figure 5, a particular mass storage device is a hard disk drive 524. Various drives may also be included within the mass storage device 508 to enable reading from and / or writing to various computer-readable media. The mass storage device 508 includes removable media 526 and / or non-removable media.

[0034] The I / O devices 510 include a variety of devices that allow data and / or other information to be input to or retrieved from the computing device 500. Exemplary I / O devices 510 include cursor control devices, keyboards, keypads, microphones, monitors or other display devices, speakers, printers, network interface cards, modems, and the like.

[0035] Display device 530 includes any type of device capable of displaying information to one or more users of computing device 500. Examples of display device 530 include monitors, display terminals, video projection devices, and the like.

[0036] The interface 506 includes various interfaces that allow the computing device 500 to interact with other systems, devices, or computing environments. The exemplary interface 506 may include any number of different network interfaces 520, such as interfaces to a local area network (LAN), a wide area network (WAN), a wireless network, and the Internet. Other interfaces include a user interface 518 and a peripheral device interface 522. The interface 506 may also include one or more user interface elements 518. The interface 506 may also include one or more peripheral interfaces, such as interfaces for a printer, a pointing device (such as a mouse, trackpad, or any suitable user interface known or later discovered by those skilled in the art), a keyboard, etc.

[0037] The bus 512 allows the processor 502, memory device 504, interface 506, mass storage device 508, and I / O device 510 to communicate with each other and with other devices or components coupled to the bus 512. The bus 512 represents one or more of several types of bus structures, such as a system bus, a PCI bus, an IEEE bus, and a USB bus.

[0038] For purposes of explanation, programs and other executable program components are illustrated herein as separate blocks, with the understanding that such programs and components may reside at various times in different storage components of computing device 500 and be executed by processor 502. Alternatively, the systems and procedures described herein may be implemented in hardware or a combination of hardware, software, and / or firmware. For example, one or more application-specific integrated circuits (ASICs) may be programmed to perform one or more of the systems and procedures described herein.

[0039] While various embodiments of the present disclosure have been described herein, it should be understood that they are presented by way of example only, and not limitation. It will be apparent to those skilled in the art that various changes in form and detail can be made therein without departing from the spirit and scope of the present disclosure. Thus, the breadth and scope of the present disclosure should not be limited by any of the exemplary embodiments described, but should be defined only in accordance with the following claims and their equivalents. The description herein has been presented for purposes of illustration and description. It is not intended to be exhaustive or to limit the disclosure to the precise form disclosed. Many modifications and variations are possible in light of the teachings disclosed. Furthermore, it should be noted that any or all of the alternative implementations discussed herein may be used in any combination desired to form additional composite implementations of the disclosure.

Claims

1. receiving data for processing; performing a multiplication operation on the received data to generate multiplied data; performing an addition operation on the multiplied data to generate a result; accumulating a first portion of the result including the least significant bits of the result in a first area of ​​an accumulation buffer of a first convolution core; storing a remainder of the result, including a most significant bit of the result, in a shared memory, the shared memory being separate from the first convolution core. A method comprising:

2. The method of claim 1 , wherein the accumulation buffer includes a plurality of entries.

3. The method of claim 2 , wherein each entry in the accumulation buffer is associated with a channel of a neural network.

4. The method of claim 3 , further comprising fetching data from multiple channels simultaneously in the accumulation buffer.

5. 2. The method of claim 1 , wherein accumulating the residual portion of the results in the shared memory is performed in response to determining that the accumulation buffer of the first convolution core has generated a carryover command.

6. 2. The method of claim 1, wherein storing the remaining portion of the result in the shared memory comprises forwarding the data request to a first-in-first-out buffer.

7. 2. The method of claim 1 , wherein storing the remaining portion of the results in the shared memory comprises allocating a specific time period to the first convolution core, wherein the first convolution core may transfer data during the specific time period.

8. a first multiplier configured to generate first multiplication data; a first adder configured to generate a first result based on the first multiplication data; a first accumulation buffer configured to accumulate a first portion of the first result, the first portion including the least significant bit of the first result; a first convolution core comprising: a shared memory coupled to the first convolution core and configured to store a remainder of the first result, the remainder including a most significant bit of the first result; 1. An apparatus comprising:

9. a second multiplier configured to generate second multiplication data; a second adder configured to generate a second result based on the second multiplication data; a second accumulation buffer configured to accumulate a first portion of the second result, the first portion including the least significant bit of the second result; and a second convolution core including: the shared memory is further coupled to the second convolution core and configured to store a remainder of the second result, including a most significant bit of the second result.

9. The apparatus of claim 8.

10. 9. The apparatus of claim 8, wherein the first accumulation buffer includes a first region and a second region, and when no data is accumulated in the second region, a clock associated with the second region is disabled.

11. 11. The apparatus of claim 10, wherein the clock associated with the second domain is enabled in response to receiving data for storage within the second domain.

12. 11. The apparatus of claim 10, wherein the clock associated with the second domain is enabled in response to receiving a carryover command from the first domain.

13. 10. The apparatus of claim 9, further comprising a first-in-first-out buffer coupled between the shared memory and each of the first convolution core and the second convolution core.

14. 10. The apparatus of claim 9, wherein the shared memory includes a plurality of memory segments, each memory segment associated with the first convolution core or the second convolution core.

Images