Memory device with micro-bumps to emit data for machine learning operations

Abstract

The invention relates to a memory device with micro-bumps to emit data for machine learning operations. A system including a memory device with micro-bumps is disclosed. The non-volatile memory device stores data for a machine learning operation. The non-volatile memory device includes a set of micro-bumps. The set of micro-bumps transmits the data for the machine learning operation from the non-volatile memory device to another set of micro-bumps of a machine learning processing device that performs the machine learning operation.

Description

[0001] Information about divisional applications

[0002] This application is a divisional application of the Chinese invention patent application with the invention name “Memory device with microbumps for transmitting data for machine learning operations”, application number 202011402551.2, and application date December 2, 2020. Technical Field

[0003] Embodiments of the present disclosure relate generally to memory subsystems, and more particularly, to memory devices having microbumps to transmit data for machine learning operations. Background Art

[0004] The memory subsystem may include one or more memory components that store data. The memory components may be, for example, non-volatile memory components and volatile memory components. In general, the host system may utilize the memory subsystem to store data at the memory components and retrieve data from the memory components. Summary of the invention

[0005] In one aspect, the present disclosure relates to a system comprising: a non-volatile memory device that stores data for a machine learning operation, the non-volatile memory device comprising a plurality of micro-bumps, wherein: the plurality of micro-bumps transmit the data for the machine learning operation from the non-volatile memory device to another plurality of micro-bumps of another device associated with the machine learning operation.

[0006] In another aspect, the present disclosure relates to a system comprising: a volatile memory device that stores data for a machine learning operation, the volatile memory device comprising a plurality of microbumps, wherein: the plurality of microbumps transmit the data for the machine learning operation from the volatile memory device to another plurality of microbumps of another device associated with the machine learning operation.

[0007] In yet another aspect, the present disclosure relates to a method comprising: receiving a command to transmit data for a machine learning operation; determining, by a processing device, a set of microbumps of a memory device that will be used to transmit the data to a machine learning processing device associated with the machine learning operation; and transmitting the data from the memory device to the machine learning processing device via the set of microbumps. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] The present disclosure will be more fully understood from the detailed description given below and the accompanying drawings of various embodiments of the present disclosure. However, the accompanying drawings should not be considered to limit the present disclosure to specific embodiments, but are only for explanation and understanding.

[0009] Figure 1 An example computing system including a memory subsystem according to some embodiments of the present disclosure is described.

[0010] Figure 2A Side and bottom views of an example memory device including microbumps according to some embodiments of the present disclosure are illustrated.

[0011] Figure 2B An example computing environment including a memory device having microbumps and a machine learning processing device having microbumps according to some embodiments of the present disclosure is described.

[0012] Figure 3 An example computing environment including a first memory device having microbumps, a machine learning processing device having microbumps, and a second memory device having input and output (I / O) pins according to some embodiments of the present disclosure is described.

[0013] Figure 4A An example of performing machine learning operations according to some embodiments of the present disclosure is described.

[0014] Figure 4B Another example of performing a machine learning operation according to some embodiments of the present disclosure is described.

[0015] Figure 5 is a flowchart of an example method of transmitting data for machine learning operations according to some embodiments of the present disclosure.

[0016] Fig. 6A An example computing environment including a memory device having microbumps and a machine learning processing device having microbumps according to some embodiments of the present disclosure is described.

[0017] Figure 6B An example of performing machine learning operations using a memory device including microbumps according to some embodiments of the present disclosure is described.

[0018] Fig. 7A Side and bottom views of an example memory device including a single die and microbumps are illustrated according to some embodiments of the present disclosure.

[0019] Figure 7B An example memory device including multiple dies and microbumps according to some embodiments of the present disclosure is described.

[0020] Figure 8 is a flowchart of another example method of transmitting data for machine learning operations according to some embodiments of the present disclosure.

[0021] Fig. 9is a flow chart of an example method of transmitting data using different sets of microbumps on a memory device according to some embodiments of the present disclosure.

[0022] Fig. 10A An example of selecting microbumps in a memory device for transmitting data according to some embodiments of the present disclosure is described.

[0023] Fig. 10B Other examples of selecting microbumps in a memory device for transmitting data according to some embodiments of the present disclosure are described.

[0024] Fig. 10C Other examples of selecting microbumps in a memory device for transmitting data according to some embodiments of the present disclosure are described.

[0025] Fig.11 is a flow chart of another example method of transmitting data using different sets of microbumps on a memory device according to some embodiments of the present disclosure.

[0026] Fig.12 is a block diagram of an example computer system in which embodiments of the present disclosure may operate. DETAILED DESCRIPTION

[0027] Aspects of the present disclosure are directed to a memory device having microbumps to transmit data for machine learning operations. The memory subsystem may be a memory device, a memory module, or a hybrid of a memory device and a memory module. Figure 1 Examples of storage devices and memory modules are described. In general, a host system may utilize a memory subsystem that includes one or more memory components (e.g., a memory device that stores data). The host system may provide data for storage at the memory subsystem and may request retrieval of data from the memory subsystem.

[0028] In addition, the host system can perform machine learning operations that utilize a machine learning model to process data (e.g., image data). For example, the machine learning model can be used to classify the data or make other inferences or decisions based on the processing of the data using the machine learning model. A machine learning model refers to a model artifact that is created by a training process and can be composed of a single-level linear or nonlinear operation (e.g., a support vector machine [SVM]) or multiple levels of nonlinear operations such as a neural network (e.g., a deep neural network, a spiking neural network, a recursive neural network, etc.). As an example, a deep neural network model can have multiple layers and can be trained by adjusting the weights of the neural network according to a back-propagation learning algorithm, etc.

[0029] The memory subsystem may store data to be processed by the machine learning operation (e.g., image data to be applied to the machine learning model), as well as any other data associated with the machine learning model. The host system may further utilize a machine learning processor (e.g., a neural network processor or a neural network accelerator), which will perform the machine learning operation based on the data and the machine learning model stored in the memory device of the memory subsystem. For example, the data and the machine learning model may be retrieved from the memory device and provided to the machine learning processor. For certain machine learning operations, there may be repeated transmissions of intermediate data of the machine learning operation (e.g., intermediate data generated by different layers of the machine learning model) between the machine learning processor and the memory device of the memory subsystem.

[0030] Conventional memory subsystems may use input and output (I / O) pins to transmit data for machine learning operations between a memory device and a machine learning processor (and / or a host system). The transmission of data associated with machine learning operations between a memory device (and / or a memory subsystem) and a machine learning processor (and / or a host system) may require the transmission of large amounts of data at high speeds. However, the limited number of I / O pins of conventional memory subsystems may not be able to meet the bandwidth and speed required for data transmission for machine learning operations, thereby causing significant delays in performing machine learning operations.

[0031] Aspects of the present disclosure address the above and other deficiencies by having a memory subsystem include one or more memory devices each having a microbump to transmit data for performing machine learning operations. For example, multiple memory devices may store different types of data for machine learning operations, and any one or more of the memory devices may have microbumps that interface with a machine learning processor (and / or a host system). Microbumps are interconnect components made of tiny metal bumps that can carry power or data signals. Because the size of the microbumps is small, each of the memory devices may have a greater number of microbumps than I / O pins and may therefore transmit much more data for machine learning operations during a particular time period.

[0032] In addition, aspects of the present disclosure address the above and other deficiencies by enabling the memory subsystem to selectively use microbumps of a memory device depending on operating conditions when performing machine learning operations. For example, when rapid transmission of data for machine learning operations is not a top priority, the memory subsystem may change the number of microbumps used to transmit data between the memory device and the machine learning processor (and / or host system) in accordance with changes in operating conditions. The memory subsystem may also select specific microbumps above other microbumps based on their corresponding positions in the memory device in order to improve the signal integrity of the data being transmitted via the microbumps.

[0033] Advantages of the present disclosure include, but are not limited to, improving the performance of machine learning operations. For example, the rate at which data is transferred between a memory subsystem and a machine learning processor (and / or a host system) can be significantly increased by using microbumps. Thus, the performance of a memory device or memory subsystem in performing machine learning operations through a machine learning processor (and / or a host system) can be improved because a single machine learning operation is performed using less time, thereby facilitating the memory device or memory subsystem to perform additional machine learning operations. Additional advantages may include saving processing resources required when performing machine learning operations. For example, as the number of microbumps transmitting any data for a machine learning operation is adjusted, any processing resources consumed by loading data into the microbumps can be reduced.

[0034] Figure 1 An example computing system 100 is illustrated that includes a memory subsystem 110 according to some embodiments of the present disclosure. Memory subsystem 110 may include media such as one or more volatile memory devices (e.g., memory device 140), one or more non-volatile memory devices (e.g., memory device 130), or a combination of the like.

[0035] The memory subsystem 110 may be a storage device, a memory module, or a mixture of storage devices and memory modules. Examples of storage devices include solid-state drives (SSDs), flash drives, universal serial bus (USB) flash drives, embedded multimedia controller (eMMC) drives, universal flash storage devices (UFS) drives, and hard disk drives (HDDs). Examples of memory modules include dual in-line memory modules (DIMMs), small outline DIMMs (SO-DIMMs), and non-volatile dual in-line memory modules (NVDIMMs).

[0036] The computing system 100 may be a computing device such as a desktop computer, a laptop computer, a network server, a mobile device, a vehicle (e.g., an airplane, a drone, a train, a car, or other transportation), a device with Internet of Things (IoT) capabilities, an embedded computer (e.g., an embedded computer included in a vehicle, industrial equipment, or a networked commercial device), or such a computing device that includes a memory and a processing device.

[0037] The computing system 100 may include a host system 120 coupled to one or more memory subsystems 110. In some embodiments, the host system 120 is coupled to memory subsystems 110 of different types. Figure 1An example of a host system 120 coupled to one memory subsystem 110 is illustrated. As used herein, "coupled to" generally refers to a connection between components, which may be an indirect communication connection or a direct communication connection (e.g., without intervening components), whether wired or wireless, including connections such as electrical connections, optical connections, magnetic connections, etc.

[0038] The host system 120 may include a processor chipset and a software stack executed by the processor chipset. The processor chipset may include one or more cores, one or more caches, a memory controller (e.g., an NVDIMM controller), and a storage protocol controller (e.g., a PCIe controller, a SATA controller). The host system 120 uses the memory subsystem 110, for example, to write data to the memory subsystem 110 and read data from the memory subsystem 110.

[0039] The host system 120 may be coupled to the memory subsystem 110 via a physical host interface. Examples of the physical host interface include, but are not limited to, a Serial Advanced Technology Attachment (SATA) interface, a Peripheral Component Interconnect Express (PCIe) interface, a Universal Serial Bus (USB) interface, a Fibre Channel, a Serial Attached SCSI (SAS), a Dual In-line Memory Module (DIMM) interface (e.g., a DIMM socket interface supporting a Double Data Rate (DDR)), etc. The physical host interface may be used to transmit data between the host system 120 and the memory subsystem 110. When the memory subsystem 110 is coupled to the host system 120 through a PCIe interface, the host system 120 may further utilize an NVM Express (NVMe) interface to access memory components (e.g., the memory device 130). The physical host interface may provide an interface for transferring control, address, data, and other signals between the memory subsystem 110 and the host system 120.

[0040] The memory devices may include any combination of different types of non-volatile memory devices and / or volatile memory devices. Volatile memory devices (e.g., memory device 140) may be, but are not limited to, random access memory (RAM), such as dynamic random access memory (DRAM) and synchronous dynamic random access memory (SDRAM).

[0041] Some examples of non-volatile memory devices (e.g., memory device 130) include "NAND" type flash memory and write-in-place memory, such as three-dimensional cross-point ("3D cross-point") memory. Cross-point arrays of non-volatile memory can be combined with stackable cross-grid data access arrays to perform bit storage based on changes in bulk resistance. In addition, in contrast to many flash-based memories, cross-point non-volatile memory can perform write-in-place operations, where non-volatile memory cells can be programmed without pre-erasing the non-volatile memory cells.

[0042] Each of the memory devices 130 may include one or more memory cell arrays. One type of memory cell, such as a single-level cell (SLC), may store one bit per cell. Other types of memory cells, such as multi-level cells (MLC), triple-level cells (TLC), and quad-level cells (QLC), may store multiple bits per cell. In some embodiments, each of the memory devices 130 may include one or more memory cell arrays, such as SLC, MLC, TLC, QLC, or any combination of these. In some embodiments, a particular memory device may include an SLC portion of memory cells, as well as an MLC portion, a TLC portion, or a QLC portion. The memory cells of the memory devices 130 may be grouped into pages, which may refer to a logical unit of a memory device used to store data. For some types of memory (e.g., NAND), pages may be grouped to form blocks.

[0043] Although non-volatile memory components such as 3D cross-point type and NAND type flash memory are described, the memory device 130 can be based on any other type of non-volatile memory, such as read-only memory (ROM), phase change memory (PCM), self-selected memory, other chalcogenide-based memory, ferroelectric random access memory (FeRAM), magnetic random access memory (MRAM), "NOR" flash memory, and electrically erasable programmable read-only memory (EEPROM).

[0044] The memory subsystem controller 115 (for simplicity, controller 115) can communicate with the memory device 130 to perform operations, such as reading data, writing data, or erasing data at the memory device 130, as well as other such operations. The memory subsystem controller 115 can include hardware, such as one or more integrated circuits and / or discrete components, buffer memory, or a combination thereof. The hardware can include digital circuitry with dedicated (i.e., hard-coded) logic to perform the operations described herein. The memory subsystem controller 115 can be a microcontroller, dedicated logic circuitry (e.g., a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), etc.), or other suitable processor.

[0045] The memory subsystem controller 115 may include a processor (processing device) 117 configured to execute instructions stored in a local memory 119. In the example shown, the local memory 119 of the memory subsystem controller 115 includes an embedded memory configured to store instructions for executing various processes, operations, logic flows, and routines that control the operation of the memory subsystem 110, including handling communications between the memory subsystem 110 and the host system 120.

[0046] In some embodiments, local memory 119 may include memory registers that store memory pointers, fetched data, etc. Local memory 119 may also include read-only memory (ROM) for storing microcode. Figure 1 The example memory subsystem 110 in FIG. 1 has been illustrated as including a memory subsystem controller 115, but in another embodiment of the present disclosure, the memory subsystem 110 may not include a memory subsystem controller 115 and may instead rely on external control (e.g., provided by an external host or by a processor or controller separate from the memory subsystem).

[0047] In general, the memory subsystem controller 115 may receive commands or operations from the host system 120, and may convert the commands or operations into instructions or appropriate commands to achieve the desired access to the memory device 130. The memory subsystem controller 115 may be responsible for other operations, such as wear leveling operations, garbage collection operations, error detection and error correction code (ECC) operations, encryption operations, cache operations, and address translation between logical addresses (e.g., logical block addresses (LBA), name space) and physical addresses (e.g., physical block addresses) associated with the memory device 130. The memory subsystem controller 115 may additionally include a host interface circuit system to communicate with the host system 120 via a physical host interface. The host interface circuit system may convert commands received from the host system into command instructions to access the memory device 130, and convert responses associated with the memory device 130 into information for the host system 120.

[0048] The memory subsystem 110 may also include additional circuitry or components not illustrated. In some embodiments, the memory subsystem 110 may include a cache or buffer (e.g., DRAM) and address circuitry (e.g., row decoder and column decoder) that may receive addresses from the memory subsystem controller 115 and decode the addresses to access the memory device 130.

[0049] In some embodiments, the memory device 130 includes a local media controller 135 that operates in conjunction with the memory subsystem controller 115 to perform operations on one or more memory cells of the memory device 130. An external controller (e.g., the memory subsystem controller 115) may externally manage the memory device 130 (e.g., perform media management operations on the memory device 130). In some embodiments, the memory device 130 may be a managed memory device, which is a raw memory device combined with a local controller (e.g., the local controller 135) to perform media management within the same memory device package. An example of a managed memory device is a managed NAND (MNAND) device.

[0050] The memory subsystem 110 includes a machine learning operation management component 113, which can use a microbump set of a memory device to transmit data for a machine learning operation. Depending on changes in machine learning operating conditions, the machine learning operation management component 113 can use different microbump sets of a memory device to transmit data for a machine learning operation. In some embodiments, the memory subsystem controller 115 includes at least a portion of the machine learning operation management component 113. For example, the memory subsystem controller 115 may include a processor 117 (processing device) configured to execute instructions stored in a local memory 119 for performing the operations described herein. In some embodiments, the machine learning operation management component 113 is part of the host system 110, an application, or an operating system. Further details about the operation of the machine learning operation management component 113 are described below.

[0051] Figure 2A Side view 200 and bottom view 230 of an example memory device including microbumps according to some embodiments of the present disclosure are illustrated. Figure 2A The memory device may correspond to Figure 1 Memory device 130 or 140.

[0052] like Figure 2A As shown in the side view 200 of FIG. 1 , the memory device may include a logic component 210 and an interface component 220. The logic component 210 may correspond to a memory cell array and / or digital logic for a processing device (e.g., a local media controller, such as Figure 1 The logic component 210 may include one or more bare chips. Fig. 7A and 7B Further details regarding the structure of logic component 210 are described.

[0053] The interface component 220 may correspond to the microbump array. The interface component 220 supports power and signals (e.g., data for machine learning operations) for the corresponding logic component 210. The interface component 220 may include Figure 2A The microbump array illustrated in the bottom view 230 of FIG. 240. The memory device may include thousands (e.g., about five thousand) microbumps 240. Thus, the memory device can transmit a large amount of data at one time (e.g., in one clock cycle) through the microbumps 240. As shown in the bottom view 230, each microbump 240 can be arranged adjacent to each other in a manner that the spacing between each pair of microbumps is equal. This will be discussed below with respect to Fig. 7A Further details regarding the microbumps are described in detail.

[0054] In one embodiment, the memory device may correspond to a non-volatile memory device that stores data for machine learning operations. In addition, such non-volatile memory devices may use microbumps 240 to transmit data to another device associated with the machine learning operation. That is, in one embodiment, the non-volatile memory device may be coupled to another device for machine learning operations. For example, microbumps 240 may transmit data signals for machine learning operations to another microbump set included in another device. Therefore, each of the microbumps 240 is coupled to each microbump in another microbump set. In one embodiment, the non-volatile memory device may be coupled to another memory device (e.g., a volatile memory including microbumps). In another embodiment, the non-volatile memory device may be coupled to a machine learning processing device via two microbump sets.

[0055] The machine learning processing device may perform machine learning operations in conjunction with at least a non-volatile memory device to perform classification or support other inferences or decisions based on processing of data, such as image data. The machine learning operations may involve processing input data (e.g., image data) according to a machine learning model using a set of weight values ​​for one or more levels of non-linear (or linear) operations. To perform such machine learning operations, the machine learning processing device may request data (e.g., input data, data for the machine learning model, and data for the set of weight values) from the memory device. Figure 4A and 4B Details of performing machine learning operations in conjunction with one or more memory devices and a machine learning processing device are described.

[0056] In another embodiment, the memory device may correspond to a volatile memory device storing data for machine learning operations. Similar to the case of a non-volatile memory device, the volatile memory device may communicate with another device via a microbump array 240. The volatile memory device may also include a logic component 210 and a microbump 240. The logic component 210 may correspond to a memory cell array and a processing device packaged in one or more dies. The microbump 240 of the volatile memory device may be coupled to another microbump array set in another device. In one embodiment, the volatile memory device may be coupled to another memory device (e.g., a non-volatile memory with microbumps). In another embodiment, the volatile memory device may be coupled to a machine learning processing device with microbumps. Each of the microbumps 240 may be coupled to each microbump of another device.

[0057] Figure 2B An example computing environment 250 is illustrated that includes a memory device 270 having microbumps 275 and a machine learning processing device 280 having microbumps 285 in accordance with some embodiments of the present disclosure. In one implementation, the memory device 270 and the machine learning processing device 280 may be disposed on an interface layer or interposer 290.

[0058] The memory device 270 may correspond to Figure 2A Memory device 270 may thus include microbumps 275. Machine learning processing device 280 may be disposed adjacent to memory device 270. In one embodiment, machine learning processing device 280 may also include microbumps 285. Machine learning processing device 280 may perform machine learning operations in conjunction with at least memory device 270. Figure 4A and 4B Details of performing machine learning operations in conjunction with one or more memory devices and a machine learning processing device are described.

[0059] Each of the microbumps 275 of the memory device 270 may be coupled to a corresponding one of the microbumps 285 of the machine learning processing device 280 via the interface layer 290. Thus, the number of microbumps 275 may be the same as the number of microbumps 285 that transmit data for machine learning operations. The interface layer 290 may be made of silicon or any other conductive material. To connect a pair of microbumps (e.g., one microbump from the microbump 275 of the memory device 270 and the other microbump from the microbump 285 of the machine learning processing device 280), the interface layer 290 may include wires or traces interconnecting the pair of microbumps. Thus, the interface layer 290 supports data transfer between the memory device 270 and the machine learning processing device 280.

[0060] In another embodiment, the interface layer 290 may include a top metal layer 291 and a bottom metal layer 293. Between the two metal layers 291 and 293, the interface layer 290 may include a plurality of through silicon vias (TSVs) that vertically interconnect the two metal layers 291 and 293. In one embodiment, there may be one TSV for each microbump 275 or 285. In another embodiment, one TSV may interconnect a plurality of microbumps 275 with a corresponding plurality of microbumps 285. Thus, the TSV 295 may support the transmission of signals from 270 and / or 280 to another device connected to the bottom metal layer 293.

[0061] Figure 3 An example computing environment 300 is illustrated that includes a first memory device 310 having microbumps, a machine learning processing device 320 having microbumps, and a second memory device 340 having input and output (I / O) pins 345 according to some embodiments of the present disclosure.

[0062] like Figure 3 As shown in FIG. 1 , the memory device 310 and the machine learning processing device 320 are disposed on the first interface layer 330. The first interface layer 330 may correspond to Figure 2B Thus, the first interface layer 330 can interconnect the memory device 310 and the machine learning processing device 320. Figure 2B The first interface layer 330 may also include top and bottom metal layers and through silicon vias (not illustrated for simplicity). The first interface layer 330 may further couple the memory device 310 and the machine learning processing device 320 with an I / O pin (e.g., I / O pin 345) of another device (e.g., another memory device 340) via solder balls 335 and a second interface layer or interposer 350.

[0063] The solder ball 335 may transmit a signal between the first interface layer 330 and the second interface layer 350. The solder ball 335 may be at least ten times larger (in diameter) than the microbump of the memory device 310 or the machine learning processing device 320. In another embodiment, any coupling device other than a solder ball (e.g., a pin) may be implemented alternatively. The second interface layer 350 may transmit a signal between the solder ball 335 to the I / O pin 345 of another memory device 340. The second interface layer 350 may be made of silicon, copper, or any other conductive material.

[0064] In one embodiment, memory device 310 may correspond to a non-volatile memory device and another memory device 340 may correspond to a volatile memory device. In another embodiment, memory device 310 may correspond to a volatile memory device and another memory device 340 may correspond to a non-volatile memory device. Another memory device 340 may include I / O pins 345 for interfacing with other devices via second interface layer 350. For example, I / O pins 345 may be eight or sixteen pins for transmitting input and output data signals. Such I / O pins may present a bottleneck problem (sometimes referred to as an I / O bottleneck problem) to processes involving the transfer of large amounts of data in a short amount of time.

[0065] Figure 4A An example 400 of performing machine learning operations according to some embodiments of the present disclosure is illustrated. The machine learning operations are performed for classification or other inferences or decisions based on processing of data, such as image data. The machine learning processing device 407 can perform the machine learning operations in conjunction with the memory devices 403 and 405.

[0066] Machine learning operations involve processing input data, such as image data, according to model data using appropriate weight data. As an example, a machine learning operation may correspond to processing input data (e.g., image data) using model data or a machine learning model to classify or identify an object or subject of the input data. Thus, the input data for a machine learning operation may correspond to data to be processed by the machine learning operation for classification or other inference or decision making based on the processing of the data. For example, the input data may be image data including pixel bit values ​​associated with an image.

[0067] The model data may correspond to a machine learning model or model artifact, which is created by a training process and may be composed of a single level of linear or nonlinear operations (e.g., support vector machine [SVM]) or multiple levels of nonlinear operations such as neural networks (e.g., deep neural networks, recurrent neural networks, convolutional neural networks, etc.). The machine learning model is trained by adjusting the weights of the neural network according to a back-propagation learning algorithm, etc.

[0068] As an example, a deep neural network model may include multiple layers, such as an input layer for receiving input data, an output layer for generating predictions, and a hidden layer between the input layer and the output layer for performing machine learning operations (e.g., multiply-accumulate operations) on the input data to generate predictions. Each layer may include or be represented by multiple neurons or nodes. Each node may be assigned a numerical value and coupled to one or more nodes in the next layer by an edge having an assigned weight value. The weight data may correspond to a set of weight values ​​for each layer or all layers of the model data. The weight value may be a numerical value, such as any decimal value between the integers zero and one.

[0069] In the example 400 of performing a machine learning operation, the machine learning processing device 407 can communicate with a processing device, such as the machine learning operation management component 113, to retrieve data required to perform the machine learning operation from the memory devices 403 and 405. In another embodiment, the machine learning processing device 407 can communicate with a corresponding processing device (e.g., Figure 1 The machine learning processing device 407 may be part of the host system or separate from the host system. In one embodiment, the memory device 403 may correspond to a non-volatile memory device having microbumps. The microbumps may enable the non-volatile memory device to interface with the processing device, the memory device 405, and / or the machine learning processing device 407. The non-volatile memory device 403 may store model data and weight data. On the other hand, the memory device 405 may correspond to a volatile memory device having input and output (I / O) pins to interface with the processing device, the memory device 403, and / or the machine learning processing device 407. The volatile memory device 405 may store input data. In one embodiment, when a picture is acquired by a camera of the host system, the processing device may store image data input data for machine learning operations in the volatile memory device 405.

[0070] Using the deep neural network model example described above, the machine learning processing device 407 may send a request for model data associated with the deep neural network model. In one embodiment, the machine learning processing device 407 may provide the request to a processing device, such as the machine learning operation management component 113. In another embodiment, the machine learning processing device 407 may transmit the request to a processing device of a non-volatile memory device with microbumps 403, such as a local media controller 135. In some embodiments, the machine learning processing device 407 may provide the request to a processing device of a volatile memory device with I / O pins 405, such as a local media controller 135. In response, the processing device may determine whether the model data is stored on the non-volatile memory device with microbumps 403 or the volatile memory device with I / O pins 405.

[0071] At operation 410, once the processing device determines that the model data is stored in the non-volatile memory device with microbumps 403, the processing device may retrieve the model data from the non-volatile memory device 403. The processing device may determine the microbumps of the non-volatile memory device 403 for transmitting the model data to the machine learning processing device 407. Then, the processing device may transmit the retrieved model data to the machine learning processing device 407. In another embodiment, the processing device may cause the non-volatile memory device with microbumps 403 to transmit the model data directly to the machine learning processing device 407. Thus, the machine learning processing device 407 may directly retrieve the model data from the non-volatile memory device 403 and bypass the processing device (e.g., the machine learning operation management component 113). In response, the machine learning processing device 407 may implement a deep neural network model from the model data for the machine learning operation.

[0072] At operation 420, simultaneously with operation 410 or after operation 410, the machine learning processing device 407 may transmit another request for input data to be processed by the implemented deep neural network model. In one embodiment, the machine learning processing device 407 provides the request to a processing device (e.g., machine learning operations management component 113). In another embodiment, the machine learning processing device 407 may communicate with the volatile memory device 405 (e.g., Figure 1 The processing device may determine that the input data is stored in the volatile memory device 405. The processing device may cause the volatile memory device 405 to transmit the input data directly to the machine learning processing device 407. Thus, the machine learning processing device 407 may access the input data directly from the volatile memory device 405 through the I / O pins. In another embodiment, the processing device may retrieve the input data from the volatile memory device 405 and provide it to the machine learning processing device 407.

[0073] In another embodiment, depending on operating conditions, such as processing speed, power supply, or noise level in the memory subsystem, the processing device may manage the two memory devices 403 and 405 to provide any data stored in the memory device 405 with I / O pins (i.e., volatile memory device 405) from the memory device 403 with microbumps (i.e., non-volatile memory device 403) or the memory device 405 with I / O pins (i.e., volatile memory device 405) to the machine learning processing device 407. That is, the processing device may determine whether to enable input data stored in the volatile memory device 405 to be transmitted to the machine learning processing device 407 directly using the I / O pins or indirectly through the microbumps of the non-volatile memory device 403.

[0074] For example, in the case where the processing device determines that using the I / O pins of the volatile memory device 405 is slower or less efficient in terms of processing resource consumption (or using the microbumps of the non-volatile memory device 403 is faster or more efficient), at operation 430, the processing device may retrieve input data from the volatile memory device 405 having the I / O pins and store the input data to the non-volatile memory device 403 having the microbumps. Subsequently, at operation 440, the processing device may cause the non-volatile memory device 403 having the microbumps to provide the input data to the machine learning processing device 407. On the other hand, in the case where the processing device determines that using the I / O pins of the volatile memory device 405 is faster or more efficient in terms of processing resource consumption (or using the microbumps of the non-volatile memory device 403 is slower or less efficient), the processing device may perform operation 420 (i.e., transmit data from the volatile memory device 405 having the I / O pins to the machine learning processing device 407).

[0075] To continue with the example of the deep neural network model, one or more nodes on the current layer may be coupled to one or more nodes in the next layer via corresponding edges. Each edge is assigned a weight value that may correspond to a numerical value between zero and one. Therefore, as the machine learning processing device 407 advances through each layer of the deep neural network model (e.g., from the input layer to the hidden layer, from the hidden layer to another hidden layer, and from the hidden layer to the output layer), the machine learning processing device 407 may request weight data or a set of weight values ​​corresponding to the current layer associated with the machine learning operation. The value of the node in the next layer corresponds to the result of a multiplication-accumulation operation. For example, for each node on the current layer that is coupled to a node in the next layer, the product of the value assigned to the node on the current layer and the weight assigned to the corresponding edge that couples the node of the current layer to the corresponding node on the next layer is calculated (i.e., multiplication) and then added (i.e., accumulation).

[0076] Thus, the machine learning processing device 407 may send a request for weight data to the processing device. In one embodiment, the machine learning processing device 407 may provide the request only once, while in another embodiment, the machine learning processing device 407 may request weight data each time the machine learning processing device 407 performs machine learning operations on different layers of the deep neural network model. As described above, once the processing device receives the request from the machine learning processing device 407, the processing device may determine where the weight data is stored. In response to determining that the weight data is stored in the non-volatile memory device 403 with microbumps, the processing device may provide the weight data to the machine learning processing device 407 in a manner similar to that described above with respect to operation 410.

[0077] In some embodiments, the machine learning processing device 407 may request the processing device to store any intermediate data, such as the values ​​of nodes in the intermediate layers, resulting from the multiplication-accumulation operations of the machine learning processing device 407 in a memory device with microbumps (e.g., non-volatile memory device 403) for faster access by the machine learning processing device 407. In other embodiments, the machine learning processing device 407 may store such intermediate data in the internal memory of the machine learning processing device 407.

[0078] After processing the input data (e.g., image data of an animal) based on the model (i.e., performing a multiply-accumulate operation on the input data), the machine learning processing device 407 may generate output data. For example, the output data may include a classification or category of the input data (e.g., type of animal species - "cat") and / or a probability (e.g., 0.97) that the input data belongs to the category. In one embodiment, the machine learning processing device 407 may request the processing device to store the output data in a memory device (e.g., non-volatile memory device 403) having microbumps for faster access by the host system. In another embodiment, the machine learning processing device 407 may store the output data in an internal memory of the machine learning processing device 407.

[0079] Figure 4B Another example 450 of performing machine learning operations according to some embodiments of the present disclosure is illustrated. Figure 4A In the example 400 illustrated in FIG. 4 , machine learning operations may be performed in conjunction with memory devices 453 and 455 and machine learning processing device 457 .

[0080] Similar to the above Figure 4A In the manner described, in the instance 450 of performing a machine learning operation, the machine learning processing device 457 can communicate with the processing device to retrieve data required to perform the machine learning operation from the memory devices 453 and 455. The processing device can correspond to the local media controller 135 of the machine learning operation management component 113 or the corresponding memory device (i.e., the non-volatile memory device 453 or the volatile memory device 455).

[0081] During a machine learning operation, the machine learning processing device 407 may provide a request to the processing device to access data (e.g., model data and / or weight data) stored at the non-volatile memory device with I / O pins 453. In one embodiment, the processing device may determine whether the data is stored in the non-volatile memory device with I / O pins 453 or the volatile memory device with microbumps 455. Once the processing device determines that the requested data is stored in the non-volatile memory device with I / O pins 453, at operation 460, the processing device may cause the data to be transmitted to the volatile memory device with microbumps 455. In one embodiment, the processing device may retrieve the data from the non-volatile memory device with I / O pins 453 and transmit the data to the volatile memory device 455 for a write operation.

[0082] Thus, at operation 470, the machine learning processing device 457 may access the requested data from the volatile memory device 455 having microbumps. In this way, the machine learning processing device 457 may access the data required for the machine learning operation at a much faster rate through the microbumps of the volatile memory device 455 than using the I / O pins of the non-volatile memory device 453. In one embodiment, the processing device may retrieve the requested data from the volatile memory device 455 and transmit the retrieved data to the machine learning processing device 457.

[0083] In another implementation, the processing device may cause the volatile memory device with microbumps 455 to transmit data directly to the machine learning processing device 457. Thus, the machine learning processing device 457 may access data directly from the volatile memory device 455. Thus, when the non-volatile memory device 453 has I / O pins as an interface component, the processing device may route any data requested by the machine learning processing device 457 to the volatile memory device with microbumps for faster processing of machine learning operations.

[0084] On the other hand, in the case where the machine learning processing device 457 requests data stored in the volatile memory device 455 having microbumps, the data may be similar to Figure 4A The method in operation 410 is directly provided to the machine learning processing device 457.

[0085] Figure 5is a flow chart of an example method 500 of transmitting data for machine learning operations according to some embodiments of the present disclosure. The method 500 may be performed by processing logic, which may include hardware (e.g., a processing device, a circuit system, dedicated logic, programmable logic, microcode, hardware of a device, an integrated circuit, etc.), software (e.g., instructions running or executed on a processing device), or a combination thereof. In some embodiments, the method 500 is performed by Figure 1 The machine learning operation management component 113 of the embodiment of the present invention is executed. Although shown in a specific order or sequence, the order of the process can be modified unless otherwise specified. Therefore, the described embodiments should be understood as examples only, and the described processes can be performed in a different order, and some processes can be performed in parallel. In addition, one or more processes can be omitted in various embodiments. Therefore, not all processes are required in every embodiment. Other process flows are possible.

[0086] At operation 510, the processing device receives a command to transmit data for a machine learning operation. In one embodiment, the processing device may receive the command from a host system. The command may include information about the data being requested. For example, the command may include a data address indicating where the data is stored (e.g., in which memory device and / or which memory cell of the memory device). The command may also include a type of data, such as input data, model data, or weight data for a machine learning operation as described above.

[0087] At operation 520, the processing device determines a set of microbumps of the memory device to transmit data to the machine learning processing device associated with the machine learning operation. As an example, the processing device may identify or select one or more microbumps of the memory device to transmit the requested data. Fig.11 Details about selecting which microbumps to use for transmission are described. The microbumps are capable of transmitting data stored in a memory device. That is, the microbumps can support data transmission from the memory device to the machine learning processing device. As described above, the machine learning processing device can perform machine learning operations using the transmitted data.

[0088] At operation 530, the processing device transmits data from the memory device to the machine learning processing device via the set of microbumps. In one embodiment, the machine learning processing device may also include microbumps. Therefore, the processing device may transmit data from the microbumps of the memory device to the microbumps of the machine learning processing device. Each microbump of the memory device may be coupled to a corresponding one of the microbumps of the machine learning processing device. To transmit the data, the processing device may use a pre-generated mapping table to map the microbumps of the memory device to the microbumps of the machine learning processing device. Therefore, before transmitting the data, the processing device may inform the machine learning processing device which microbump of the memory device will carry the data and / or which corresponding microbump of the machine learning processing device will receive the data.

[0089] In a further embodiment, the processing device may determine whether the data for the machine learning operation is stored on a memory device (i.e., a memory device with microbumps) or another memory device (i.e., a memory device without microbumps). In one embodiment, the processing device may determine where the data is stored from the received command. For example, the processing device may determine the storage location of the data from the data information contained in the command. In another embodiment, the data address of the data may be stored in a local memory of the memory subsystem controller. In such cases, the processing device may access the data address from the local memory and determine whether the data is stored on a memory device with microbumps or on a memory device without microbumps. In response to determining that the data is stored on a memory device with microbumps, the processing device may determine the microbumps used to transmit the data.

[0090] On the other hand, in the case where the processing device determines that the data is stored on a memory device without microbumps, the processing device may retrieve the data from the memory device without microbumps. Such memory devices may have input and output (I / O) pins instead of microbumps. The I / O pins may be coupled to (the microbumps of) the memory device. After retrieving the data from the memory device with I / O pins, the processing device may store the data to the memory device with microbumps.

[0091] For example, a memory device with microbumps may correspond to a volatile memory device and a memory device without microbumps may correspond to a non-volatile memory device. The volatile memory device may store input data and the non-volatile memory device may store model data and weight data for machine learning operations. In the case where the requested data is data not stored on the volatile memory device (e.g., model data and / or weight data), the processing device may perform a read operation on a non-volatile memory device with an I / O pin to access the data. The processing device may then perform a write operation on the volatile memory device with microbumps to store the read data. In another embodiment, the processing device may cause the non-volatile memory device with an I / O pin to transmit data to the volatile memory device with microbumps. In one embodiment, the processing device may request a local media controller of the non-volatile memory device to read data from one or more memory cells of the non-volatile memory device and provide the data to the volatile memory device with microbumps.

[0092] In some other implementations, in response to determining that the data is stored on a memory device that does not have microbumps (i.e., instead has I / O pins), the processing device may cause the memory device with I / O pins to transmit the data to the machine learning processing device. In one implementation, the processing device may request a local media controller of a memory device with I / O pins to transmit the data directly to the machine learning processing device. Such a memory device may have I / O pins coupled to the machine learning processing device. Figure 3 Describes details about how a memory device with I / O pins can be coupled with a machine learning processing device with microbumps.

[0093] As another example, a memory device with a microbump may correspond to a nonvolatile memory device and a memory device without a microbump may correspond to a volatile memory device. Similar to the above example, a volatile memory device without a microbump may store input data and a nonvolatile memory device with a microbump may store model data and weight data for machine learning operations. In the case where the requested data is data (e.g., input data) not stored on a nonvolatile memory device with a microbump, the processing device may perform a read operation on a volatile memory device without a microbump (i.e., alternatively, with an I / O pin) to access the data and perform a write operation on a nonvolatile memory device with a microbump to store the data. By transmitting data for machine learning operations to a memory device with a microbump, the time required to transmit the data can be reduced. Therefore, the processing device can save processing resources for accessing data for machine learning operations. In another embodiment, the processing device may cause a volatile memory device with an I / O pin to transmit data to a nonvolatile memory device with a microbump in a manner similar to that described above.

[0094] Fig. 6A An example computing environment 600 including memory devices 610 and 630 having microbumps 615 and 635 , respectively, and a machine learning processing device 620 having microbumps 625 is illustrated in accordance with some embodiments of the present disclosure.

[0095] In one embodiment, memory device 610 may correspond to a non-volatile memory device and memory device 630 may correspond to a volatile memory device. In another embodiment, memory device 610 may correspond to a volatile memory device and memory device 630 may correspond to a non-volatile memory device. When machine learning processing device 620 performs a machine learning operation, each of memory device 610 and memory device 630 may store data for the operation. Therefore, microbump 615 and microbump 635 may transmit corresponding data to machine learning processing device 620 via microbump 625 of machine learning processing device 620.

[0096] As illustrated, two memory devices 610 and 630 and a machine learning processing device 620 may be disposed on an interface layer or interposer 640. The interface layer 640 may correspond to Figure 2B The interface layer 640 may interconnect the three devices 610, 620, and 630. The interface layer 640 may also include a top metal layer and a bottom metal layer and a through silicon via (not illustrated for simplicity) for connecting the top metal layer and the bottom metal layer.

[0097] In one embodiment, the machine learning processing device 620 is disposed between the two memory devices 610 and 630. In this way, the distance between the memory device 610 or 630 and the machine learning processing device 620 can be minimized. However, in other embodiments, the two memory devices 610 and 630 can be disposed next to each other. The machine learning processing device 620 can perform machine learning operations in conjunction with the memory devices 610 and 630. Figure 6B Details of performing machine learning operations in conjunction with a memory device and a machine learning processing device are described.

[0098] The microbumps 625 of the machine learning processing device 620 may include two microbump sets 627 and 629. The first microbump set 627 may be coupled with the microbumps 615 of the memory device 610. Each of the first microbump set 627 may be coupled with the microbumps 615 of the memory device 610. In a similar manner, the second microbump set 629 may be coupled with the microbumps 635 of the memory device 630. Therefore, the number of microbumps in the machine learning processing device 620 may correspond to the total number of microbumps in the memory device 610 and the memory device 630. The number of microbumps 615 of the memory device 610 may be greater than the number of microbumps 635 of the memory device 630 and vice versa.

[0099] Figure 6B An example 650 of performing machine learning operations using memory devices 653 and 655 including microbumps according to some embodiments of the present disclosure is illustrated. The machine learning operations are performed for classification or other inferences or decisions based on processing of data such as image data. The machine learning processing device 657 can perform the machine learning operations in conjunction with the memory devices 653 and 655.

[0100] The machine learning processing device 657 may be part of the host system or separate from the host system. In one embodiment, the memory device 653 may correspond to a non-volatile memory device having microbumps for input and output interface components. The memory device 653 may communicate with the processing device (e.g., Figure 1 The non-volatile memory device 653 may store model data and weight data. On the other hand, the memory device 655 may correspond to a volatile memory device having microbumps to similarly interface with the processing device, the non-volatile memory device 653, and / or the machine learning processing device 657. The volatile memory device 655 may store input data. In one embodiment, when a picture is acquired from a camera of the host system, the processing device may store the image data in the volatile memory device 655 as input data for the machine learning operation.

[0101] In the example 650 of performing a machine learning operation, the machine learning processing device 657 may transmit a request to access model data and input data to a processing device, such as the machine learning operation management component 113. The processing device may determine whether the model data is stored in the non-volatile memory device 653 or in the volatile memory device 655. In response to determining that the model data is stored in the non-volatile memory device 653, at operation 660, the processing device enables the machine learning processing device 657 to access the model data stored in the non-volatile memory device 653 via micro-bumps of the non-volatile memory device 653 and micro-bumps of the machine learning processing device 657.

[0102] After accessing the model data, the machine learning processing device 657 may implement the model data (e.g., a deep neural network model) as part of the machine learning operation. In addition, the processing device may determine that the input data is stored in the volatile memory device 655. Therefore, at operation 670, the processing device may enable the machine learning processing device 657 to access the input data stored in the volatile memory device 655 via the microbumps of the volatile memory device 655 and the microbumps of the machine learning processing device 657. In one embodiment, the microbumps of the machine learning processing device 657 used to communicate with the volatile memory device 655 may be different from the microbumps of the machine learning processing device 657 used to communicate with the non-volatile memory device 653.

[0103] In another embodiment, the machine learning processing device 657 may determine where each data required for the machine learning operation is stored. The machine learning processing device 657 may then communicate with a processing device (e.g., a local media controller (e.g., Figure 1 The machine learning processing device 657 may communicate with the local media controller 135 of the non-volatile memory device 653 to access the corresponding data. For example, in the case of model data, the machine learning processing device 657 may transmit a request for the model data to the processing device of the non-volatile memory device 653 and retrieve the model data directly from the non-volatile memory device 653. In the case of input data, after determining that the input data is stored in the volatile memory device 655, the machine learning processing device 657 may provide a request to the processing device of the volatile memory device 655 and retrieve the input data directly from the volatile memory device 655.

[0104] Once the model data and input data are transmitted to the machine learning processing device 657, the machine learning processing device 657 can process each layer of the deep neural network model in sequence from the input layer to the hidden layer and finally to the output layer. At the same time or subsequently, the machine learning processing device 657 may request weight data from the processing device. In one embodiment, the machine learning processing device 657 may request the weight data once at the beginning of the machine learning operation. In such cases, the weight data may include a set of weight values ​​for each layer of the model. In another embodiment, the machine learning processing device 657 may request the corresponding weight data for each layer. In this case, the weight data may include a set of weight values ​​only for the corresponding layer.

[0105] In a manner similar to that at operation 660, the processing device may enable the machine learning processing device 657 to access the weight data stored in the non-volatile memory device 653 via the microbumps of the non-volatile memory device 653 and the microbumps of the machine learning processing device 657. After processing the input data (e.g., an image of an animal) based on the model data (i.e., performing a multiplication-accumulation operation on the input data), the machine learning processing device 657 may generate output data, which may include a classification or category of the input data (e.g., type of animal species - "cat") and / or a probability that the input data belongs to the category (e.g., 0.97).

[0106] Fig. 7A Side view 700 and bottom view 730 of an example memory device including a single die 710 and microbumps 720 are illustrated in accordance with some embodiments of the present disclosure.

[0107] As shown in side view 700, the memory device may include a die 710 and a microbump array 720. The memory device may correspond to a non-volatile memory device or a volatile memory device. Die 710 may include a device layer and a metal layer (not illustrated for simplicity). The device layer may include an array of memory cells and / or a microbump array for a processing device (e.g., Figure 1 The metal layer of the die 710 can be placed between the device layer and the microbumps 720, thereby interconnecting the two components.

[0108] As illustrated in bottom view 730, die 710 may be covered with microbump array 720A. Microbumps may be made of a material or any mixture selected from a group including but not limited to the following: copper, gold, nickel, palladium, solder, or any other conductive material. Each microbump 720A may be disposed on die 710 in a manner equal to the distance of another microbump. For example, microbump 720A may have a spacing of 10 to 30 microns (μm). Spacing refers to the distance 760 between the center 735 of microbump 720A and the center 735 of adjacent microbump 720B. In addition, microbump 720A may have a diameter 745 in the range of 5μm to 15μm. In one embodiment, all microbumps 720 have the same diameter 745 or substantially the same diameter 745.

[0109] In other embodiments, the machine learning processing device may similarly have a single die and microbumps disposed on the die. The microbumps may have similar characteristics to the microbumps 720 described above.

[0110] Figure 7B An example memory device 750 including a plurality of dies 760 and 780 and microbumps 770 and 790 according to some embodiments of the present disclosure is illustrated. The memory device 750 may correspond to a non-volatile memory device and / or a volatile memory device. In another embodiment, a machine learning processing device may be similarly structured as the memory device 750.

[0111] Dies 760 and 780 may be similar to those described above with respect to Fig. 7A 770 and the like. The die 780 may be structured in the manner described above with respect to the die 210. Thus, each die 760 and 780 may include a device layer and a metal layer (not shown for simplicity of illustration). However, the die 780 may include two metal layers, one metal layer for the top connected to the microbumps 770 and the other metal layer for the bottom connected to the microbumps 790. In addition, the die 780 may include through silicon vias (TSVs) connecting the two metal layers. Thus, the TSVs may enable the upper die 760 to communicate with the lower die 780. As illustrated, the die 760 may be disposed on the top of the die 780 via the microbumps 770. The microbumps 790 may be disposed on the bottom of the die 780 to enable the die 780 to communicate with the interface layer (e.g., a layer to be coupled to other devices) Figure 2B In other embodiments, the memory device 750 may include more than two dies 760 and 780. In such cases, another set of microbumps may be disposed between any two adjacent (ie, any top and bottom) dies.

[0112] Figure 8800 is a flowchart of another example method 800 of transmitting data for machine learning operations according to some embodiments of the present disclosure. The method 800 may be performed by processing logic, which may include hardware (e.g., a processing device, a circuit system, a dedicated logic, a programmable logic, a microcode, hardware of a device, an integrated circuit, etc.), software (e.g., instructions running or executed on a processing device), or a combination thereof. In some embodiments, the method 800 is performed by Figure 1 In some embodiments, the method 800 is performed by Figure 1 The local media controller 135 of the embodiment of the present invention is executed. Although shown in a specific order or sequence, unless otherwise specified, the order of the processes can be modified. Therefore, the illustrated embodiments should be understood as examples only, and the illustrated processes can be performed in a different order, and some processes can be performed in parallel. In addition, one or more processes can be omitted in various embodiments. Therefore, not all processes are required in every embodiment. Other process flows are possible.

[0113] At operation 810, the processing device receives a command to transmit data for machine learning operations to the machine learning processing device. The processing device may receive commands from the machine learning processing device. The machine learning processing device may perform machine learning operations in conjunction with the memory device. The machine learning processing device may perform machine learning operations to classify or support other inferences or decisions based on processing of data, such as image data. Figure 6B Details of performing machine learning operations in conjunction with a memory device are described in detail in . In one embodiment, the machine learning processing device performs machine learning operations using data stored in the memory device.

[0114] At operation 820, the processing device determines whether the data is stored in a memory device (hereinafter, a first memory device) or in another memory device (hereinafter, a second memory device). For example, the first memory device may correspond to a non-volatile memory device and the second memory device may correspond to a volatile memory device. In addition, the first memory device may store model data and weight data and the second memory device may store input data for machine learning operations.

[0115] If at least about Figure 4AAs described, input data for a machine learning operation may correspond to data to be processed by the machine learning operation (e.g., image data including pixel bit values) for classification or other inference or decision making based on processing of the data. Model data may correspond to a machine learning model or model artifact, which is created by a training process and may be composed of a single-level linear or nonlinear operation (e.g., a support vector machine [SVM]) or multiple levels of nonlinear operations such as a neural network (e.g., a deep neural network, a recursive neural network, a convolutional neural network, etc.). The machine learning model is trained by adjusting the weights of the neural network according to a back-propagation learning algorithm, etc. Weight data may correspond to a set of weight values ​​for each layer of the model data. The weight value may be a numeric value, such as any decimal value between the integers zero and one.

[0116] In addition, the first memory device may include an array of microbumps. The second memory device may also have an array of microbumps. In one embodiment, the number of microbumps in the first memory device may be the same as the number of microbumps in the second memory device. However, in other embodiments, the number of microbumps in the first memory device may be greater than the number of microbumps in the second memory device and vice versa.

[0117] In addition, the machine learning processing device may also include a micro-bump array. In such cases, the machine learning processing device may be used with Fig. 6A Thus, the microbump array in the first memory device can be coupled to a portion of the microbumps in the machine learning processing device, and the microbump array in the second memory device can be coupled to a portion of the microbumps in the machine learning processing device. The microbumps can be coupled to each other in a one-to-one relationship.

[0118] At operation 830, the processing device determines a microbump set corresponding to the one of the first memory device or the second memory device storing the data. For example, in response to determining that the data is stored in the first memory device, the processing device may select or identify the microbump set in the first memory device. Fig.11 The details of selecting which microbumps are used to transmit data are described. Similarly, the processing device may determine the set of microbumps in the second memory device in response to determining that the data is stored in the second memory device.

[0119] In one embodiment, in response to determining that data is stored in the first memory device, the processing device may further determine the type of data requested for the machine learning operation (e.g., input data, model data (which defines the machine learning operation), or weight data (which represents one or more numerical values ​​for the machine learning operation)). The processing device may identify the type of data requested from the command received at operation 820 or any metadata associated with the command. The processing device may then determine the number of microbumps used to transmit the data based on the type of data. For example, where the type of data corresponds to model data, the processing device may determine that the number of microbumps used for transmission should correspond to the total number of microbumps in the first memory device. In this way, the processing device uses all available microbumps so that large amounts of data (model data can be up to 64KB or 128KB) can be transmitted from the first memory device to the machine learning processing device as quickly as possible.

[0120] However, in response to determining that the type of data requested corresponds to weight data, the processing device may determine that the number of microbumps used to transmit the data should correspond to a portion of the microbumps in the first memory device. Because the size of the weight data may be relatively small when compared to the model data, the weight data may still be quickly transmitted using a smaller number of microbumps. In this way, the processing device may save processing resources used to transmit data to the machine learning processing device.

[0121] On the other hand, in response to determining that data is stored in the second memory device, the processing device may not determine the type of data. Because the input data is typically of a larger size (e.g., image data may have pixel values ​​of 1200 pixels by 1000 pixels, 1024 pixels by 768 pixels, etc.), the processing device may determine that the data should be transmitted using the entire microbump array in the second memory device.

[0122] At operation 840, the processing device transmits data to the machine learning processing device using the set of microbumps of the corresponding memory device. In one embodiment, the processing device may not utilize all of the microbumps in the corresponding device. In another embodiment, the processing device may cause the corresponding memory device to transmit data to the machine learning processing device using all of the microbumps in the corresponding memory device.

[0123] Fig. 9 900 is a flow chart of an example method 900 for transmitting data using different microbump sets on a memory device according to some embodiments of the present disclosure. The method 900 may be performed by processing logic, which may include hardware (e.g., a processing device, a circuit system, dedicated logic, programmable logic, microcode, hardware of a device, an integrated circuit, etc.), software (e.g., instructions running or executed on a processing device), or a combination thereof. In some embodiments, the method 900 is performed by Figure 1 In some embodiments, the method 900 is performed by Figure 1 The local media controller 135 of the embodiment of the present invention is executed. Although shown in a specific order or sequence, unless otherwise specified, the order of the processes can be modified. Therefore, the illustrated embodiments should be understood as examples only, and the illustrated processes can be performed in a different order, and some processes can be performed in parallel. In addition, one or more processes can be omitted in various embodiments. Therefore, not all processes are required in every embodiment. Other process flows are possible.

[0124] At operation 910, a processing device transmits data for a machine learning operation based on a microbump set of a memory device, wherein the data is stored at the memory device. The processing device may transmit the data from the memory device to a machine learning processing device. In one embodiment, the processing device may correspond to a machine learning operation management component (e.g., Figure 1 In such cases, the processing device may communicate with the memory device (or a local media controller (e.g., Figure 1 In another embodiment, the processing device may correspond to a local media controller of the memory device (e.g., Figure 1 In such cases, the processing device may retrieve the data from the corresponding memory device and transmit the data to the machine learning processing device. In addition, in some embodiments, the memory device may be a non-volatile memory device that stores any data used for machine learning operations (e.g., input data, model data, and output data). In some other embodiments, the memory device may be a volatile memory device that stores similar data used for machine learning operations.

[0125] The machine learning processing device may perform machine learning operations in conjunction with the memory device to classify or support other inferences or decisions based on processing of data such as image data. The machine learning processing device may be part of a host system or separate from the host system. The machine learning operation may involve processing input data (e.g., image data) according to a machine learning model of a weight value set using one or more levels of non-linear (or linear) operations. To perform such machine learning operations, the machine learning processing device may request data (e.g., input data, data for the machine learning model, and data for the weight value set) from the memory device. In response, the memory device may transmit data for the machine learning operation to the machine learning processing device.

[0126] At operation 920, the processing device determines a change in a machine learning operation condition. Examples of the condition may include a state of a power supply associated with the memory device (e.g., power to a memory subsystem, a memory device, or a host system), a temperature associated with the memory device (e.g., the temperature of the memory subsystem, the memory device, or the host system), and a data size associated with the machine learning operation (i.e., the size of the data used for the machine learning operation). Thus, the condition of the machine learning operation may correspond to a condition of an operating environment in which a machine learning operation for any of the memory device, the memory subsystem, or the host system is performed.

[0127] In one embodiment, the processing device may periodically receive information indicating current conditions from the machine learning processing device or the host system. In another embodiment, the processing device may periodically request current conditions from the machine learning processing device or the host system. Upon receiving the condition information, the processing device may determine how much the condition has changed. For example, the processing device may first receive condition information indicating that the power supply associated with the memory device is at 100% capacity (or 10mW). Then, the processing device may later receive information indicating that the power supply is at 80% capacity (or 8mW). As long as the current condition is different from the previous condition, the processing device may determine that the condition of the machine learning operation has changed. In another embodiment, the processing device may determine that the condition has changed when the difference between two consecutive conditions exceeds a threshold amount (e.g., a change of 10%). As another example of condition information, the processing device may periodically receive condition information indicating that the machine learning operation requires a certain size of data (e.g., the size of input data, model data, or weight data) for the current machine learning operation. As another example, the processing device may receive temperature information such as a change from 50F to 75F. Thus, the change in conditions may correspond to a change in power state, a change in operating temperature, and / or a change in the size of data used for machine learning operations.

[0128] At operation 930, in response to determining that the condition of the machine learning operation has changed, the processing device determines a new microbump set of the memory device. The new microbump set is a microbump set that transmits subsequent data for the machine learning operation. The machine learning operation may involve processing a series of data. For example, the machine learning processing device may first request model data to configure itself according to the definition of the machine learning model contained in the model data to perform the machine learning operation. Then, the machine learning processing device may request input data to start processing the input data according to the model data. While processing the input data, the machine learning processing device may intermittently request corresponding weight data for each layer of the machine learning model. Therefore, the data emitted using the previous microbump set may be different from the subsequent data to be emitted using the next microbump set because the machine learning processing device requests different data.

[0129] In another embodiment, the data used for the machine learning operation can be divided into portions of data. For example, the input data can be divided into multiple portions. Thus, a first portion of the input data can be transmitted using a previous microbump set, while a second portion of the input data can be transmitted using a next microbump set. The first and second portions of the input data can have different sizes, and different portions of the input data can be transmitted using different microbump sets based on the different sizes.

[0130] In some embodiments, before determining another microbump set for subsequent data, the processing device may determine whether a threshold condition is met. The threshold condition may be associated with the magnitude of the condition change. For example, the threshold condition may specify that the change is at least 10%. Using the above example, when the power drops from 100% to 80%, the processing device may determine that the magnitude of the power change is 20%. The processing device may further determine that the threshold condition (e.g., 10%) has been met. As another example, when the power changes from 100% to 95%, the processing device may determine that the threshold condition (e.g., 10%) is not met. In such cases, the processing device may prevent determining another microbump set and subsequently use the current microbump set to transmit data (e.g., without updating the microbump set).

[0131] When determining a new microbump set, the processing device may take into account the direction of the change. That is, the processing device may determine whether the number of microbumps to be included in the new set is relatively more or less than the number of microbumps in the current microbump set, depending on the direction of the change. For example, if the power supply has been reduced from 100% to 80%, the processing device may determine that the number of microbumps in the new set should be reduced. The processing device may select a proportionally reduced number of microbumps for the new set. For example, the current microbump set may correspond to the entire microbump set in the memory device. In response to determining that the power supply has dropped from 100% to 80%, the processing device may identify or select 80% of the microbumps of the memory device or 20% fewer microbumps than the current set. At least with respect to Fig.11 The details of which microbumps are selected for transmitting data are described. However, if the direction of the change is an increase in power from 50% to 75%, then the processing device may determine that the number of microbumps for the new set should be increased. Thus, for example, the processing device may select 75% of the microbumps instead of 50% of the microbumps or 25% more microbumps than the current set for transmitting subsequent data.

[0132] At operation 940, the processing device transmits subsequent data using the new microbump set of the memory device. In one embodiment, the processing device may determine whether the new microbump set corresponds to all microbumps of the memory device (i.e., data may be transmitted to each microbump of the machine learning processing device). In response to determining that the new microbump set corresponds to all microbumps of the memory device, the processing device may transmit the subsequent data at one time. In other words, the processing device may transmit the subsequent data via each microbump of the memory device in one clock cycle. However, in response to determining that the new microbump set does not correspond to all microbumps of the memory device, the processing device may divide or split the subsequent data into multiple parts. The processing device may then transmit parts of the subsequent data over a period of time. For example, the processing device may transmit each part of the subsequent data in each clock cycle, thereby transmitting the subsequent data via clock cycles.

[0133] Fig. 10A Examples 1010 and 1020 are described for selecting microbumps in a memory device for transmitting data according to some embodiments of the present disclosure. Examples 1010 and 1020 may be performed by, for example, Figure 1 The processing device of the machine learning operation management component 113 is executed. In some embodiments, instances 1010 and 1020 are executed by, for example, Figure 1 The processing is performed by another processing device of the local media controller 135. The memory device may be a non-volatile or volatile memory device.

[0134] Example 1010 illustrates selected microbumps in a memory device 1015. It should be appreciated that for simplicity of illustration, the memory device 1015 is illustrated as having a relatively small number of microbumps, and that the memory device 1015 may have thousands of microbumps. Each square represents a microbump in the memory device 1015, and the square with a cross "X" represents a selected microbump. Each letter along the vertical side of the memory device 1015 indicates each microbump group. In this example 1010, each row represents a microbump group. However, one skilled in the art should appreciate that any other form of grouping (e.g., group columns instead of group rows) may be applicable to this example 1010.

[0135] The processing device may determine that 50% of the microbumps in the memory device 1015 should be selected as a new microbump set for transmitting data for machine learning operations. For each group (e.g., each of groups A to H), the processing device may select 50% of the microbumps within the corresponding group. For example, the processing device may determine that there are eight microbumps in each group. Therefore, the processing device may determine to select four microbumps from each group. When selecting four microbumps, the processing device may select microbumps that are far away from each other (i.e., every other microbump). As described, for group A, the processing device may identify the first, third, fifth, and seventh microbumps to transmit data. As for the next group in the sequence (i.e., group B), the processing device may determine four microbumps that are far away from each other and are not adjacent to other selected microbumps (if possible). Thus, as described in example 1010, the processing device may select the second, fourth, sixth, and eighth microbumps for the new microbump set. In some embodiments, selecting microbumps that are farther away from each other rather than closer to each other can improve the signal integrity of the data being transmitted via the microbumps. For example, the effect of noise from one microbump will have less impact on another microbump that is being used to transmit data.

[0136] Example 1020 illustrates selected microbumps in memory device 1025. It should be appreciated that for simplicity of illustration, memory device 1025 is illustrated as having a relatively small number of microbumps, and memory device 1025 may have thousands of microbumps. Similar to example 1010, each square represents a microbump in memory device 1025, and squares with crosses "X" represent selected microbumps.

[0137] In example 1020, the processing device may determine that 25% of the microbumps in the memory device 1025 should be selected as a new microbump set for transmitting data for machine learning operations. For each group (e.g., groups A to H), the processing device may select 25% of the microbumps within the corresponding group. For example, the processing device may determine that two microbumps from each group should be selected. When selecting the two microbumps, the processing device may select microbumps with the same number of spaces between each other (i.e., the same number of unselected microbumps). Therefore, for group A, the processing device may identify that the first microbump and the fifth microbump transmit data as described in example 1020. As for the next group in the sequence (i.e., group B), the processing device may determine two microbumps that have the same spacing as the previous group and are not adjacent to other selected microbumps (if possible). Therefore, the processing device may select the third microbump and the seventh microbump for the new microbump set as described in example 1020.

[0138] Fig. 10BOther examples 1030 and 1040 of selecting microbumps in a memory device for transmitting data according to some embodiments of the present disclosure are described. Fig. 10A , instances 1030 and 1040 may be provided by, for example Figure 1 The machine learning operation management component 113 or the processing device of the local media controller 135 is executed. The memory device can be a non-volatile or volatile memory device.

[0139] Example 1030 illustrates selected microbumps in a memory device 1035. It should be appreciated that for simplicity of illustration, the memory device 1035 is illustrated as having a relatively small number of microbumps, and that the memory device 1035 may have thousands or any other number of microbumps. As previously described, each square represents a microbump in the memory device 1035, and the squares with a cross "X" represent selected microbumps.

[0140] The processing device may determine that 50% of the microbumps in the memory device 1035 should be selected as the new microbump set for transmitting data for the machine learning operation. The processing device may determine the number of groups included in the memory device 1035. As illustrated, there are eight groups - groups A to H. Then, the processing device may determine that four of the eight groups (i.e., 50% of the groups) should be selected. When selecting the four groups, the processing device may select groups that are equidistant from each other (i.e., every other group). As illustrated, the processing device may select groups A, C, E, and H. Then, the processing device may identify that all microbumps in each selected group will be included in the new microbump set as marked by X in example 1030.

[0141] Example 1040 illustrates selected microbumps in a memory device 1045. It should be appreciated that for simplicity of illustration, the memory device 1045 is illustrated as having a relatively small number of microbumps, and the memory device 1045 can have any number of microbumps. Similar to example 1030, each square represents a microbump in the memory device 1045, and the square with a cross "X" represents a selected microbump.

[0142] In example 1040, the processing device may determine that 25% of the microbumps in memory device 1045 should be selected as a new microbump set for transmitting data for machine learning operations. As illustrated, the processing device may determine that there are eight groups (groups A to H) in memory device 1045. Then, the processing device may determine that two of the eight groups (i.e., 25% of the groups) should be selected. When selecting the two groups, the processing device may select groups that are far away from each other. As illustrated, the processing device may select groups A and H. Then, the processing device may recognize that all microbumps in each selected group will be included in the new microbump set as marked by X in example 1040.

[0143] Fig. 10C Other examples 1050 and 1060 of selecting microbumps in a memory device for transmitting data according to some embodiments of the present disclosure are described. Fig. 10A , instances 1050 and 1060 may be provided by, for example Figure 1 The machine learning operation management component 113 or the processing device of the local media controller 135 is executed. The memory device can be a non-volatile or volatile memory device.

[0144] Example 1050 illustrates different forms of groups in memory device 1055. It should be understood that for simplicity of illustration, memory device 1055 is illustrated as having a relatively small number of microbumps, and memory device 1055 may have any number of microbumps. Each letter inside the square represents a group of four microbumps. In example 1050, there are a series of sixteen groups (e.g., groups A to M) arranged adjacent to each other. For example, group E is located below group D, so that groups E and D are adjacent to each other. However, it should be understood by those skilled in the art that any other forms of groups (e.g., square or rectangular) with different numbers of microbumps may be possible.

[0145] Example 1060 illustrates selected microbumps in a memory device 1065. As previously described, for simplicity of illustration, the memory device 1065 is illustrated as having a relatively small number of microbumps; however, the memory device 1065 can have any number of microbumps.

[0146] In example 1060, the processing device may determine that 50% of the microbumps in the memory device 1065 should be selected as a new microbump set for transmitting data for machine learning operations. As illustrated, the processing device may determine that there are sixteen groups in the memory device 1065 (e.g., corresponding to groups A to M in example 1050). Then, the processing device may determine that eight of the sixteen groups (i.e., 50% of the groups) should be selected. When selecting the eight groups, the processing device may select groups that are equidistant from each other or far from each other. As illustrated, the processing device may select every other group in the series (e.g., groups A, C, E, I, K, M, and O). Then, the processing device may identify that all microbumps in each selected group will be included in the new microbump set as marked by X in example 1060.

[0147] Fig.111 is a flow chart of another example method 1100 for transmitting data using different microbump sets on a memory device according to some embodiments of the present disclosure. The method 1100 may be performed by processing logic, which may include hardware (e.g., a processing device, a circuit system, dedicated logic, programmable logic, microcode, hardware of a device, an integrated circuit, etc.), software (e.g., instructions running or executed on a processing device), or a combination thereof. In some embodiments, the method 1100 is performed by Figure 1 In some other embodiments, the method 1100 is performed by Figure 1 The local media controller 135 of the embodiment of the present invention is executed. Although shown in a specific order or sequence, unless otherwise specified, the order of the processes can be modified. Therefore, the illustrated embodiments should be understood as examples only, and the illustrated processes can be performed in a different order, and some processes can be performed in parallel. In addition, one or more processes can be omitted in various embodiments. Therefore, not all processes are required in every embodiment. Other process flows are possible.

[0148] At operation 1110, the processing device receives an instruction to change the microbump set in the memory device that transmits data for machine learning operations. The microbumps in the memory device are used to transmit data, such as input data, model data, and weight data for machine learning operations between the memory device and the machine learning processing device. The input data may correspond to data (e.g., image data) to be processed by the machine learning operation. The model data may define the machine learning operation and the weight data represents one or more numerical values ​​for the machine learning operation. When using the microbump set to transmit data for the machine learning operation, the processing device may receive an instruction to change the microbump set (i.e., the number of microbumps) being used to transmit data from the host system or the memory subsystem.

[0149] Such an indication may include information about a change in a condition (e.g., an operating condition for performing a machine learning operation). For example, the indication may describe that the power of a power source (to a corresponding memory device, memory subsystem, or host system) has changed from 100% to 90%. As another example, the indication may specify that the temperature of the operating condition has changed from 50F to 65F. In another embodiment, the indication may specify that the number of microbumps that transmit data for a machine learning operation is to be changed to a specific number of microbumps (e.g., the indication requests that the processing device change the number of microbumps from 100% to 70%).

[0150] At operation 1120, the processing device selects a new set of microbumps based on the orientation of the corresponding microbumps in the memory device according to the indication. In one embodiment, the processing device may determine the groups of microbumps in the memory device. For example, the microbumps may be divided into a plurality of groups based on the corresponding orientations in the memory device. For example, the microbumps may be grouped per row or column or in any other form. Each group may include the same number of microbumps. In addition, each group may be adjacent to each other in the memory device (i.e., arranged in a sequence).

[0151] In a further embodiment, at operation 1120, the processing device may determine to select a total number of microbumps as a new set based on the indication. In one embodiment, the indication may specify a change in the operating condition of the machine learning operation from 100% to 65% of the power supply. Based on the indication, the processing device may determine that the total number of microbumps in the new set is proportional to the change in condition. For example, the processing device may determine that the total number of microbumps in the new set should correspond to 65% of the microbumps in the memory device because the power supply has been reduced to 65%. As another example, in the case where the indication describes that the temperature has changed from 50F to 65F (which corresponds to a 30% increase), the processing device may determine that the total number of microbumps in the new set should be increased by 30% when compared to the number of microbumps in the current set. In another embodiment, the indication may specify a specific number of microbumps. In such cases, the processing device may determine that the total number of microbumps corresponds to the specified number including the indication.

[0152] After determining the groups of microbumps and the number of microbumps to be selected for the new set, the processing device may identify a new microbump set corresponding to the total number of microbumps from the group. In one embodiment, the processing device may determine that a number of microbumps will be selected from each group. For example, the processing device may calculate the number of microbumps in each group based on the total number of microbumps for the new set and the total number of groups in the memory device (i.e., the processing device may divide the total number of microbumps for the new set by the total number of groups in the memory device). The processing device may select microbumps corresponding to the calculated number of microbumps from each group. In this way, the processing device may select the same number of microbumps from each group. When selecting microbumps from each group, the processing device may evenly distribute the microbumps throughout the corresponding group. For example, the processing device may select microbumps that have the same number of microbumps (i.e., unselected microbumps) between each other. In the case where the processing device is selecting 50% of the microbumps in each group, the processing device may select every other microbump. When selecting 25% of the microbumps, the processing device may select every third microbump in each group.

[0153] In another embodiment, the processing device may identify a new set of microbumps based on the selected groups. For example, the processing device may determine the number of groups to be selected based on the total number of microbumps to be selected and the number of microbumps in each group. In the case where one thousand microbumps are to be selected and there are one hundred microbumps in each group, the processing device may determine that ten groups will be selected. Therefore, the processing device may select the determined number of groups. Then, the processing device may identify all of the microbumps in the selected groups as new sets of microbumps. When selecting the determined number of groups, the processing device may select groups that are far away from each other. For example, if the processing device selects ten groups from a total of twenty groups, the processing device may select every other group and therefore select every microbump in every other group.

[0154] At operation 1130, the processing device transmits data for machine learning operations using the new microbump set. For example, the processing device may retrieve data from a corresponding memory device and determine a new microbump set to transmit the data. The processing device may then transmit the data to a machine learning processing device or a host system via the new microbump set.

[0155] Fig.12 An example machine of a computer system 1200 is illustrated, within which a set of instructions for causing the machine to perform any one or more of the methodologies discussed herein may be executed. In some embodiments, the computer system 1200 may correspond to a computer system that includes, is coupled to, or uses a memory subsystem (e.g., Figure 1 The memory subsystem 110 of the controller may be used to execute the operation of the controller (for example, to execute the operating system to execute the corresponding Figure 1 The machine learning operation management component 113 is operated by a host system (e.g., Figure 1 In some embodiments, the machine may be connected (e.g., using a network) to other machines in a LAN, an intranet, an extranet, or the Internet. The machine may operate in the capacity of a server or a client machine in a client-server network environment, or in the capacity of a peer machine in a peer-to-peer (or distributed) network environment, or as a server or a client machine in a cloud computing infrastructure or environment.

[0156] The machine may be a personal computer (PC), a tablet PC, a set-top box (STB), a personal digital assistant (PDA), a cellular phone, a network appliance, a server, a network router, a switch or bridge, an automobile, or any machine capable of executing (sequentially or otherwise) a set of instructions that specify actions to be taken by the machine. In addition, while a single machine is described, the term "machine" should also be construed to include any collection of machines that individually or collectively execute one (or more) sets of instructions to perform any one or more of the methodologies discussed herein.

[0157] The example computer system 1200 includes a processing device 1202, a main memory 1204 (e.g., read-only memory (ROM), flash memory, dynamic random access memory (DRAM) such as synchronous DRAM (SDRAM) or RDRAM, etc.), a static memory 1206 (e.g., flash memory, static random access memory (SRAM), etc.), and a data storage system 1218, which communicate with each other via a bus 1230.

[0158] The processing device 1202 represents one or more general-purpose processing devices, such as a microprocessor, a central processing unit, etc. More specifically, the processing device can be a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, or a processor that implements other instruction sets, or a processor that implements a combination of instruction sets. The processing device 1202 can also be one or more special-purpose processing devices, such as an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital signal processor (DSP), a network processor, etc. The processing device 1202 is configured to execute instructions 1226 for performing the operations and steps discussed herein. The computer system 1200 may additionally include a network interface device 1208 to communicate on a network 1220.

[0159] The data storage system 1218 may include a machine-readable storage medium 1224 (also referred to as a computer-readable medium) on which is stored one or more sets of instructions 1226 or software embodying any one or more of the methods or functions described herein. The instructions 1226 may also reside, completely or at least partially, within the main memory 1204 and / or within the processing device 1202 during execution by the computer system 1200, the main memory 1204 and the processing device 1202 also constituting machine-readable storage media. The machine-readable storage medium 1224, the data storage system 1218, and / or the main memory 1204 may correspond to Figure 1 Memory subsystem 110.

[0160] In one embodiment, instructions 1226 include implementing instructions corresponding to a machine learning operations management component (e.g., Figure 1 The machine-readable storage medium 1224 is a storage medium that is used to store or encode a set of instructions for execution by a machine and causes the machine to perform any one or more of the methods of the present disclosure. Thus, the term "machine-readable storage medium" should be considered to include, but not be limited to, solid-state memory, optical media, and magnetic media.

[0161] Some portions of the previously described detailed description have been presented with respect to algorithms and symbolic representations of operations on data bits within a computer memory. These algorithmic descriptions and representations are the means by which those skilled in the art of data processing can most effectively communicate the substance of their work to other persons skilled in the art. Algorithms are here and generally considered to be self-consistent sequences of operations that lead to desired results. Operations are operations that require physical control of physical quantities. These quantities are usually, but not necessarily, in the form of electrical or magnetic signals that can be stored, combined, compared, and otherwise manipulated. Sometimes, primarily for general reasons, it has proven convenient to refer to these signals as bits, values, elements, symbols, characters, terms, numbers, etc.

[0162] It should be borne in mind, however, that all of these and similar terms are to be associated with the appropriate physical quantities and are merely convenient labels applied to these quantities. The present disclosure may refer to the actions and processes of a computer system or similar electronic computing device that manipulates and transforms data represented as physical (electronic) quantities within the computer system's registers and memories into other data similarly represented as physical quantities within the computer system's memories or registers or other such information storage systems.

[0163] The present disclosure also relates to an apparatus for performing the operations described herein. This apparatus may be specially constructed for the desired purpose, or it may comprise a general purpose computer selectively activated or reconfigured by a computer program stored in the computer. Such a computer program may be stored in a computer readable storage medium, such as, but not limited to, any type of disk (including floppy disks, optical disks, CD-ROMs, and magnetic optical disks), read-only memory (ROM), random access memory (RAM), EPROM, EEPROM, magnetic or optical cards, or any type of medium suitable for storing electronic instructions, each coupled to a computer system bus.

[0164] The algorithms and displays presented herein are not inherently related to any particular computer or other device. Various general purpose systems may be used with programs according to the teachings herein, or it may prove convenient to construct more specialized devices to perform the methods. The structures of a variety of these systems will be presented as set forth in the description below. In addition, the present disclosure is not described with reference to any particular programming language. It should be appreciated that the teachings of the present disclosure as described herein may be implemented using various programming languages.

[0165] The present disclosure may be provided as a computer program product or software, which may include a machine-readable medium having stored thereon instructions that can be used to program a computer system (or other electronic device) to perform a process according to the present disclosure. A machine-readable medium includes any mechanism for storing information in a form readable by a machine (e.g., a computer). In some embodiments, a machine-readable (e.g., computer-readable) medium includes a machine (e.g., computer) readable storage medium, such as a read-only memory ("ROM"), a random access memory ("RAM"), a magnetic disk storage medium, an optical storage medium, a flash memory device, etc.

[0166] In the foregoing description, embodiments of the present disclosure have been described with reference to specific example embodiments thereof. It should be apparent that various modifications may be made to the present disclosure without departing from the broader spirit and scope of embodiments of the present disclosure as set forth in the appended claims. Accordingly, the description and drawings should be viewed in an illustrative rather than a restrictive sense.

Claims

1. A system comprising: A non-volatile memory device storing data for machine learning operations, the non-volatile memory device comprising a plurality of micro-bumps, wherein: The plurality of microbumps transmit the data for the machine learning operation from the nonvolatile memory device to another plurality of microbumps of another device associated with the machine learning operation.

Images