Hybrid memory management system and method combining storage processing and attribute data management

Abstract

The memory module may include a volatile memory (105), a non-volatile memory (110), a non-volatile memory buffer (145), and a memory mapped input / output (MMIO) register set. A hybrid media controller (155) may be configured to read and write data to the volatile memory (105) of the memory module's memory mapped space. The hybrid media controller (155) may also be configured to read and write batch data to the non-volatile memory (110) of the memory mapped space. The hybrid media controller (155) may also be configured to read and write data at random access granularity to the non-volatile memory (110) of the memory mapped space. The hybrid media controller (105) may also be configured to self-index data between the non-volatile memory (110) and the volatile memory (105) of the memory module.

Description

Background Art

[0001] Computing systems have made significant contributions to the progress of modern society and are used in many applications to achieve beneficial results. Numerous devices, such as desktop personal computers (PCs), laptop PCs, tablet PCs, netbooks, smartphones, and servers, have enabled increased efficiency and reduced costs in communications and data analysis across most areas of entertainment, education, business, and science. Many technologies and applications require processing units with high computational intensity and high memory bandwidth, optimized for performance based on large data sets. For example, graphics processing applications involve both structured and feature data. Structured data is characterized by a small data volume and random data access based on a small access granularity. Therefore, volatile memory (such as dynamic random access memory) is typically used to store structured data. Feature data is characterized by a large data volume, less random access, and a larger access granularity. Therefore, non-volatile memory, such as flash memory, is typically used to store feature data. In applications such as graph computing, a memory system with multiple memory channels is required to access data in both non-volatile and non-volatile memories. Summary of the Invention

[0002] The present technology may be best understood by referring to the following description and accompanying drawings, which illustrate embodiments of the present technology for hybrid memory management with both storage processing and attribute data management.

[0003] In one embodiment, a memory module may include volatile memory, non-volatile memory, a non-volatile memory buffer, a memory mapped input / output (MMIO) register set, and a hybrid media controller. The MMIO register set may include: a feature size register configured to store the number of bytes per feature; an index base register configured to store a sampling node location; an index count register configured to store the number of sampling node locations; a target base register configured to store a volatile memory address storing one or more features; an operation register configured to store an operation code; and a status register configured to store a current non-volatile memory read / write controller state. The hybrid media controller may be configured to provide read and write access to the volatile memory, batch read and write access to the non-volatile memory, random read and write access to the non-volatile memory, and self-indexed data movement between the non-volatile memory and the volatile memory using the MMIO register set.

[0004] In one embodiment, a method for accessing data may include reading and writing data to a volatile memory within a memory-mapped space of a memory module. The method may also include reading and writing batches of data to a non-volatile memory within the memory-mapped space of the memory module. The method may also include reading and writing data at random access granularity to the non-volatile memory within the memory-mapped space of the memory module. The method may also include self-indexing data movement between the non-volatile memory and the volatile memory of the memory module.

[0005] This Summary is provided to introduce some concepts in a simplified form that are further described in the Detailed Description below. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] Embodiments of the present technology are illustrated by way of example and not limitation in the accompanying drawings and in which like reference numerals refer to similar elements and in which:

[0007] Figure 1 A memory module in accordance with aspects of the present technology is shown.

[0008] Figure 2A and 2B Illustrated is read / write access to volatile memory of a memory module in accordance with aspects of the present technique.

[0009] Figure 3A and 3B Illustrated is read / write access to volatile memory of a memory module in accordance with aspects of the present technique.

[0010] Figure 4A and 4B Bulk read / write access to non-volatile memory of a memory module is shown in accordance with aspects of the present technology.

[0011] Figure 5A and Figure 5B Illustrated is read / write access to a non-volatile memory of a memory module at random access granularity in accordance with aspects of the present technique.

[0012] Figure 6 Self-indexing data movement between non-volatile memory and volatile memory of a memory module in accordance with aspects of the present technique is shown.

[0013] Figure 7 A method for self-indexed data movement for node sampling and post-sampling feature reading in accordance with aspects of the present technique is shown. DETAILED DESCRIPTION

[0014] Reference will now be made in detail to embodiments of the present technology, examples of which are illustrated in the accompanying drawings. Although the present technology will be described in conjunction with these embodiments, it will be understood that they are not intended to limit the present technology to these embodiments. On the contrary, the present invention is intended to cover alternatives, modifications, and equivalents that may be included within the scope of the present invention as defined by the claims. In addition, in the following detailed description of the present technology, many specific details are set forth in order to provide a thorough understanding of the present technology. However, it will be understood that the present technology can be implemented without these specific details. In other instances, well-known methods, processes, components, and circuits are not described in detail so as not to unnecessarily obscure aspects of the present technology.

[0015] The following embodiments of the present technology are presented in terms of routines, modules, logic blocks, and other symbolic representations of operations on data within one or more electronic devices. Description and representation are the means used by those skilled in the art to most effectively convey the essence of their work to other persons skilled in the art. Routines, modules, logic blocks, and / or the like are herein and generally conceived as a self-consistent sequence of processes or instructions that lead to desired results. These processes are those that include physical manipulations of physical quantities. Typically, although not necessarily, these physical manipulations take the form of electrical or magnetic signals that can be stored, transmitted, compared, and otherwise manipulated in an electronic device. For convenience, and with reference to common usage, with reference to the embodiments of the present technology, these signals are referred to as data, bits, values, elements, symbols, characters, terms, numbers, character strings, etc.

[0016] However, it should be kept in mind that these terms should be interpreted as referring to physical operations and quantities and are merely convenient labels and are further interpreted in light of terminology commonly used in the art. Unless otherwise specifically stated, as will be apparent from the following discussion, it should be understood that throughout the discussion of the present technology, discussions utilizing terms such as "receiving" and the like refer to the actions and processes of electronic devices, such as electronic computing devices, that manipulate and transform data. This data is represented as physical (e.g., electronic) quantities within the logic circuits, registers, memories, etc. of the electronic device and is converted into other data similarly represented as physical quantities within the electronic device.

[0017] In this application, the use of disjunctives is intended to include conjunctions. The use of definite or indefinite articles is not intended to indicate cardinality. Specifically, reference to "the" object or "an" object is intended to also refer to one of a possible plurality of such objects. The use of the terms "comprises," "comprising," "includes," "including," etc. specifies the presence of the elements described, but does not exclude the presence or addition of one or more other elements and / or groups thereof. It should also be understood that although the terms first, second, etc. may be used herein to describe various elements, such elements should not be limited by these terms. These terms are used herein to distinguish one element from another. For example, a first element may be referred to as a second element, and similarly, a second element may be referred to as a first element, without departing from the scope of this embodiment. It should also be understood that when an element is referred to as being "coupled" to another element, the element may be connected to the other element directly or indirectly, or there may be an intermediate element. In contrast, when an element is referred to as being "directly connected" to another element, there are no intermediate elements. It should also be understood that the term "and / or" includes any and all combinations of one or more related elements. Also, it is to be understood that the phraseology and terminology used herein is for the purpose of description and should not be regarded as limiting.

[0018] Computing devices used to perform artificial intelligence, machine learning, deep learning, neural network (NN) processing, graph neural network (GNN) processing and other such applications have a need to access volatile and non-volatile memory. For example, a computing device that performs graph neural network (GNN) processing may need to access volatile memory for graph sampling and graph structure updates. Such a computing device may also need to access non-volatile memory in batches for graph construction. The computing device may also need to access non-volatile memory randomly for graph feature updates. The computing device may also need to move data between non-volatile memory and volatile memory in order to read features after sampling. The present technology provides a unified memory access device and technology, which includes read / write access to volatile memory, batch read / write and random read / write access to non-volatile memory, and data movement between non-volatile memory and volatile memory.

[0019] Figure 11 shows a memory module according to various aspects of the present technology. Memory module 100 may include volatile memory (VM) 105 and non-volatile memory (NVM) 110. In one implementation, volatile memory 105 may be dynamic random access memory (DRAM), and non-volatile memory 110 may be flash memory (FLASH), phase change memory (PCM), or the like. Memory module 100 may also include a memory-mapped input / output (MMIO) register set 115-140, a non-volatile memory buffer 145, an interface 150, and a hybrid media controller 155. The memory-mapped input / output (MMIO) register set 115-140, the non-volatile memory buffer 145, and the volatile memory 105 may comprise a memory-mapped space of the memory module 100. The memory mapped input / output (MMIO) registers 115-140, the non-volatile memory buffer 145, the interface 150, and the hybrid media controller 155 of the memory module 100 can be configured to provide read / write access to the volatile memory 105, bulk read / write access to the non-volatile memory 110, random read / write access with random access granularity to the non-volatile memory 110, and self-indexed data movement between the non-volatile memory 110 and the volatile memory 105.

[0020] Hybrid media controller 155 may include a non-volatile memory read / write controller 160, a volatile memory read / write controller 165, an address generator 170, and a flow controller 175. Memory-mapped input / output (MMIO) registers 115-140 may include, but are not limited to, a feature size register 115, an index base register 120, an index count register 125, a target base register 130, an operation register 135, and a status register 140. Feature size register 115 may be configured to store the number of bytes per feature. Index base register 120 may be configured to store the location of a sampling node. Index count register 125 may be configured to store the number of sampling node locations. Target base register 130 may be configured to store a volatile memory address for storing one or more features. Operation register 135 may be configured to store an operation code. Status register 140 may be configured to store the current memory read / write controller state. Non-volatile memory buffer 145 may be configured to store data used for read and write accesses to non-volatile memory 110. In one implementation, the non-volatile memory buffer 145 may be a set of ping-pong buffers having the size of a block of the non-volatile memory 110. The memory-mapped input / output (MMIO) register bank 115-140 may optionally further include, but is not limited to, a random access data buffer (not shown). The random access data buffer may be configured to store data at a random access granularity for read and write accesses to the volatile memory 105. In another implementation, a given portion of the non-volatile memory buffer 145 may be utilized to store data at a random access granularity for read and write accesses to the volatile memory 105.

[0021] In one implementation, the volatile memory 105, the nonvolatile memory 110, the memory-mapped input / output (MMIO) registers 115-140, the nonvolatile memory buffer 145, the interface 150, and the hybrid media controller 155 of the memory module 100 may be implemented as a system-in-package (SiP). In another implementation, the volatile memory 105, the nonvolatile memory 110, the memory-mapped input / output (MMIO) registers 115-140, the nonvolatile memory buffer 145, the interface 150, and the hybrid media controller 155 of the memory module 100 may be implemented as a peripheral card having a dual in-line memory module (DIMM) form factor and interface. In another implementation, the volatile memory 105, non-volatile memory 110, memory mapped input / output (MMIO) register sets 115-140, non-volatile memory buffer 145, interface 150, and hybrid media controller 155 of the memory module 100 may be implemented as a peripheral card, such as, but not limited to, a Peripheral Component Express (PCIe) card.

[0022] The memory module 100 can be configured to provide unified memory access, including read / write access to volatile memory, batch read / write access to non-volatile memory, random access to non-volatile memory, and self-indexed data movement between non-volatile memory and volatile memory. The memory module 100 can provide read / write access to volatile memory for graph sampling and graph structure updates. The memory module 100 can provide batch read / write access to non-volatile memory for graph construction. The memory module 100 can provide random read / write access to non-volatile memory for graph feature updates. The memory module 100 can also provide data movement between non-volatile memory and volatile memory for feature reading after sampling.

[0023] Will refer to Figure 2A and 2B , 3A and 3B, 4A and 4B, and 5 further describe the operation of the memory module 100. Figure 2A , describes a read access to the volatile memory 105 according to aspects of the present technology. At 210, the read access may include receiving a volatile memory address and a random access read operation into the MMIO register set 115-140 via the interface 150 of the memory module 100. In one embodiment, the host may drive the target register 130 with the volatile memory address and drive the operation register 135 with the random access read command. At 220, in response to the random access read operation in the MMIO register set 115-140, data at random access granularity may be read from the volatile memory 105 at the volatile memory address in the MMIO register 115-140, and the state of the MMIO register set 115-140 may be set by the hybrid media controller 155. The data at random access granularity may be a predetermined amount of data, such as a byte, a word, or a page of data. In one implementation, the volatile memory read / write controller 165 may read the data at random access granularity from the volatile memory 105 and feed the batch of data into a data buffer, such as a dedicated register. The volatile memory read / write controller 165 may also send a signal to the flow control module 175 indicating that random access granularity data is in the buffer. The flow control module 175 may set the status register 140 to a given state to indicate that the bulk data loading process into the buffer has been completed. When the status register 140 is set to a given state indicating that the bulk data loading process has been completed, the host may poll the status register 140 and read the random access granularity data from the buffer via the interface 150.

[0024] See also Figure 2B, describes a write access to the volatile memory 105 according to various aspects of the present technology. At 230, the write access may include receiving a random access write operation to the MMIO register set 115-140 through the interface 150 of the memory module 100, receiving data at random access granularity to a given buffer, and receiving a volatile memory address to the MMIO register set 115-140. The data at the random access granularity may be a predetermined amount of data, such as a byte, a word, or a page of data. The given buffer may be a dedicated register or a given portion of the non-volatile memory buffer 145 that may be reused for each random access read operation. In one implementation, the host may send data at random access granularity to the buffer, drive the target register 130 with the volatile memory address, and drive the operation register 135 with the random access write command. At 240, in response to a random access write operation in the MMIO register sets 115-140, data at random access granularity can be written from the buffer to the non-volatile memory 105 at the volatile memory address within the MMIO register sets 115-140, and the state of the MMIO register sets 115-140 can be set by the hybrid media controller 155. In one implementation, the volatile memory read / write controller 165 can write the data at random access granularity from the buffer to the volatile memory 105. When the data at random access granularity has been written to the volatile memory 105, the volatile memory read / write controller 165 can send a completion signal to the flow control module 175. The flow control module 175 can set the status register 140 to a given state indicating that the random access data write operation is complete. Thus, the MMIO register sets 115-140, the non-volatile memory buffer 145, the volatile memory read / write controller 165, and the flow control module 175 of the memory module 100 can provide random read / write access to the volatile memory. In one implementation, random read / write access to volatile memory can be used for graph sampling and graph structure updates for graph neural network (GNN) applications.

[0025] See also Figure 3A, describes a read access to the volatile memory 150 according to aspects of the present technology. At 310, the read access may include receiving a hybrid media controller disable operation to the MMIO register set 115-140. In one implementation, the host may drive the operation register 135 with a hybrid media controller disable command. The hybrid media controller 155 may be disabled to allow direct read access to the volatile memory 105 (such as RAM). At 320, a volatile memory address and a random access read operation may be received into the volatile memory 105 via the interface 150 of the memory module 100. At 320, in response to receiving the random access read operation by the volatile memory, data of random access granularity may be read from the volatile memory 105 located at the volatile memory address. The data of the random access granularity may be a predetermined amount of data, such as a byte, a word, or a page of data.

[0026] See also Figure 3B , describes write access to volatile memory 105 according to aspects of the present technology. At 340, the write access may include receiving a hybrid media controller disable operation to MMIO registers 115-140. In one implementation, the host may drive the operation register 135 with a hybrid media controller disable command. The hybrid media controller 155 may be disabled to allow direct write access to volatile memory 105 (such as RAM). At 350, a random access write operation, random access granularity data, and a volatile memory address may be received into volatile memory 105 via interface 150 of memory module 100. The random access granularity data may be a predetermined amount of data, such as a byte, a word, or a page of data. At 360, in response to receiving the random access write operation by volatile memory 105, the random access granularity data may be written to non-volatile memory 105 at the volatile memory address. Thus, MMIO registers 115-140 of memory module 100 may provide random read / write access to volatile memory. In one implementation, random read / write access to volatile memory can be used for graph sampling and graph structure updates for graph neural network (GNN) applications.

[0027] See also Figure 4A, describes a bulk read access to the non-volatile memory 110 according to aspects of the present technology. At 410, the bulk read access may include receiving a non-volatile memory address and a bulk read operation into the MMIO register bank 115-140 via the interface 150 of the memory module 100. In one implementation, the host may drive the target register 130 with the non-volatile memory address and the operation register 135 with the bulk read command. At 420, in response to the bulk read operation in the MMIO register bank 115-140, the bulk data may be read from the non-volatile memory 110 at the non-volatile memory address in the MMIO register bank 115-140, and the state of the MMIO register bank 115-140 may be set by the hybrid media controller 155. In one implementation, the non-volatile memory read / write controller 160 may read the bulk data from the non-volatile memory 110 and feed the bulk data into the non-volatile memory buffer 145. The non-volatile memory read / write controller 160 may also send a signal to the flow control module 175 to indicate that the bulk data is in the non-volatile memory buffer 145. The flow control module 175 may set the status register 140 to a given state to indicate that the operation of loading the bulk data into the non-volatile memory buffer 145 has been completed. When the status register 140 is set to the given state indicating that the bulk data loading is complete, the host may poll the status register 140 and read the bulk data from the non-volatile memory buffer 145 through the interface 150.

[0028] See also Figure 4B, describes a batch write access to the non-volatile memory 110 according to aspects of the present technology. At 430, the batch write access may include receiving a batch write operation to the MMIO register sets 115-140 via the interface 150 of the memory module 100, receiving batch data to the non-volatile memory buffer 145, and receiving the non-volatile memory address into the MMIO register sets 115-140. In one implementation, the host may send the batch data to the non-volatile memory buffer 145, drive the target register 130 with the non-volatile memory address, and drive the operation register 135 with the batch write command. At 440, in response to the batch write operation in the MMIO register sets 115-140, the batch data may be written from the non-volatile memory buffer 145 to the non-volatile memory 110 at the non-volatile memory address in the MMIO register sets 115-140, and the state of the MMIO register sets 115-140 may be set by the hybrid media controller 155. In one implementation, the non-volatile memory read / write controller 160 can write data from the non-volatile memory buffer 145 to the non-volatile memory 110. When the batch data has been written to the non-volatile memory 110, the non-volatile memory read / write controller 160 can send a completion signal to the flow control module 175. The flow control module 175 can set the status register 140 to a given state indicating that the batch data write operation is complete. Thus, the MMIO registers 115-140 of the memory module 100, the non-volatile memory buffer 145, the hybrid media controller 155, and the non-volatile memory read / write controller 160 can provide batch read / write access to the non-volatile memory. In one implementation, the batch read / write access to the non-volatile memory can be used for graph construction in graph neural network (GNN) applications.

[0029] See also Figure 5A, describes a read of the non-volatile memory 110 based on random access granularity according to aspects of the present technology. At 510, the read access may include receiving a non-volatile memory address and a random access read operation into the MMIO register bank 115-140 via the interface 150 of the memory module 100. In one implementation, the host may drive the target register 130 with the non-volatile memory address and the operation register 135 with the random access read command. At 520, in response to the random access read operation in the MMIO register 115-140, data at the random access granularity may be read from the non-volatile memory 110 at the non-volatile memory address in the MMIO register bank 115-140, and the state of the MMIO register bank 115-140 may be set by the hybrid media controller 155. The data at the random access granularity may be a predetermined amount of data, such as one byte, one word, or one page of data. In one implementation, the non-volatile memory read / write controller 160 can read random access granularity data from the non-volatile memory 110 and feed the bulk data into a data buffer, such as a dedicated register that can be reused for each random access read operation or a given portion of the non-volatile memory buffer 145. The non-volatile memory read / write controller 160 can also send a signal to the flow control module 175 to indicate that the random access granularity data is in the buffer. The flow control module 175 can set the status register 140 to a given state to indicate that the bulk data has been loaded into the buffer. When the status register 140 is set to the given state indicating that the bulk data load is complete, the host can poll the status register 140 and read the random access granularity data from the buffer through the interface 150.

[0030] See also Figure 5B, describes a write to the non-volatile memory 110 based on random access granularity according to various aspects of the present technology. At 530, the write access may include receiving a random access write operation to the MMIO register group 115-140 through the interface 150 of the memory module 100, receiving data at the random access granularity to a given buffer, and receiving the non-volatile memory address to the MMIO register group 115-140. The data at the random access granularity may be a predetermined amount of data, such as a byte, a word, or a page of data. The given buffer may be a dedicated register or a given portion of the non-volatile memory buffer 145 that may be reused for each random access read operation. In one implementation, the host may send data at the random access granularity to the buffer, drive the target register 130 with the non-volatile memory address, and drive the operation register 135 with the random access write command. At 540, in response to the random access write operation in the MMIO register set 115-140, data at random access granularity may be written from the buffer to the non-volatile memory 110 at the non-volatile memory address within the MMIO register set 115-140, and the state of the MMIO register set 115-140 may be set by the hybrid media controller 155. In one implementation, the non-volatile memory read / write controller 160 may write the data at random access granularity from the buffer to the non-volatile memory 110. When the data at random access granularity has been written to the non-volatile memory 110, the non-volatile memory read / write controller 160 may send a completion signal to the flow control module 175. The flow control module 175 may set the status register 140 to a given state indicating that the random access data write operation is complete. Therefore, the MMIO register groups 115-140, the hybrid media controller 155, the non-volatile memory read / write controller 160, and the flow control module 175 of the memory module 100 can provide random read / write access to the non-volatile memory. In one implementation, the random read / write access to the non-volatile memory can be used for graph sampling and graph structure updates in graph neural network (GNN) applications.

[0031] See also Figure 6, describes a self-indexed data move between non-volatile memory 110 and volatile memory 105 according to aspects of the present technology. At 610, the data move may include receiving a self-indexed data move operation into the MMIO register set 115-140. In one implementation, the host may drive the index base register 120 with the sampling node location, drive the index count register 125 with the sampling node number, drive the target register 130 with the volatile memory address to store the feature, and drive the operation register 135 with the self-indexed data move command. At 620, a volatile memory address may be generated based on the self-indexed data move operation in the MMIO register set 115-140, and the sampling result may be read from the volatile memory 150 located at the generated volatile memory address by the hybrid media controller 155. In one embodiment, the address generator 170 may generate a volatile memory address from the sample node position in the index base register 120, the sample node number in the index count register 124, and the volatile memory address for storing the feature in the target register 130. The volatile memory read / write controller 165 may send the generated volatile memory address to the volatile memory 105. In one implementation, the address generator 170 may calculate a non-volatile memory address from the sample results from the volatile memory 105. At 630, a non-volatile memory address may be calculated from the volatile memory address, attributes may be extracted from the calculated non-volatile memory address, the attributes may be stored in the volatile memory, and the state of the MMIO register groups 115-140 may be set by the hybrid media controller 155. In one implementation, the address generator 170 may generate a non-volatile memory address based on the volatile memory address. The non-volatile memory read / write controller 160 and the volatile memory read / write controller 165 can extract attributes from the non-volatile memory 110 and store them in the volatile memory 105. Thereafter, the flow control module 175 can set the status register 140 to indicate that the self-indexed data movement between the non-volatile memory (NVM) 110 and the volatile memory (VM) 105 is complete. Thus, the MMIO register group 115-140 of the memory module 100, the non-volatile memory read / write controller 160, the volatile memory read / write controller 165, and the address generator 170 can provide data movement between the non-volatile memory and the volatile memory. In one implementation, random read / write access to the volatile memory can be used for feature reading after sampling in a graph neural network (GNN) application.

[0032] See also Figure 7, a method for self-indexing data movement for node sampling and feature reading after sampling according to multiple aspects of the present technology is shown. The method of self-indexing data movement between non-volatile memory and volatile memory can be used for node sampling and attribute extraction. At 710, data movement may include sampling node data in volatile memory. In one implementation, node data can be sampled in a pre-allocated continuous dynamic random access memory 105. Data can be sampled at a specified address plus a count (Index_Base+Index_Count-1) in the non-volatile memory. The base address (Index_Base) in the volatile memory, the count (Index_Count), the target address (Trget_Base) in the non-volatile memory, and the memory operation code (OP) can also be used to drive the index base register 120, the index count register 125, the target base register 130, and the operation register 135 in the MMIO register group 115-140. At 720, a self-indexing address can be generated for the attribute. In one implementation, the address generator 170 may read the base address (Index_Base) and count (Index_Count) in the volatile memory, and the target address (Trget_Base) in the non-volatile memory from the index base register 120, the index count register 125, and the target base register 130, generate a volatile memory address, and send the generated volatile memory address to the volatile memory controller 165. The volatile read / write controller 165 may read the sampling result from the volatile memory 105 located at the generated volatile memory address. The address generator 170 may also calculate the non-volatile memory address from the address of the generated sampling result. At 730, attributes may be extracted from the non-volatile memory using the self-indexed address, and the extracted attributes may be stored in the volatile memory. In one implementation, the non-volatile memory controller 160 may extract the attributes from the non-volatile memory 110, and the volatile memory controller 165 may store the extracted attributes in the volatile memory 105. The status register 140 in the MMIO register set 115 - 140 may then be updated to indicate the completion of the self-indexed data move operation.

[0033] Various aspects of the present technology advantageously provide unified memory access, including read / write access to volatile memory, batch read / write access to non-volatile memory, random access to non-volatile memory, and self-indexing data movement between non-volatile memory and volatile memory. Memory modules according to various aspects of the present technology can be advantageously used in graph neural network applications. The memory module can advantageously reduce energy consumption for input / output access to non-volatile memory and volatile memory. Various aspects of the present technology can advantageously improve memory bandwidth and reduce access latency. Various aspects of the present technology can advantageously reduce host processing overhead for accessing non-volatile memory and volatile memory. Various aspects of the present technology can also advantageously provide better support for small memory attribute sizes.

[0034] The foregoing descriptions of specific embodiments of the present technology have been presented for purposes of illustration and description. It is not intended to be exhaustive or to limit the present technology to the precise form disclosed, and obviously many modifications and variations are possible in light of the above teachings. The embodiments were chosen and described in order to best explain the principles of the present technology and its practical application, thereby enabling others skilled in the art to best utilize the present technology and various embodiments with various modifications suitable for the specific use contemplated. The scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A memory module, comprising: Volatile memory; Non-volatile memory; a non-volatile memory buffer; Memory mapped input / output MMIO register set; interface; as well as a hybrid media controller configured to provide; read and write access to said volatile memory; bulk read and write access to said non-volatile memory; Random exclusive and write access to said non-volatile memory; as well as Self-indexing data movement between the non-volatile memory and the volatile memory; The memory module configured to provide read / write random access to the non-volatile memory includes: The interface is configured to receive random access read operations and non-volatile memory addresses into the MMIO register set; and The hybrid media controller is configured to, in response to the random access read operation in the MMIO register set, read data of random access granularity from the non-volatile memory at the non-volatile memory address in the MMIO register set, feed the data of random access granularity into the non-volatile memory buffer, and set a state of the MMIO register set.

2. The memory module according to claim 1, wherein: The memory module configured to provide read and write access to the volatile memory comprises: The interface is configured to receive volatile memory addresses and random access read operations to the MMIO register set; and The hybrid media controller is configured to read data of random access granularity from the volatile memory at the volatile memory address in the MMIO register set in response to the random access read operation in the MMIO register set and set a state of the MMIO register set.

3. The memory module according to claim 1, wherein The memory module configured to provide read and write access to the volatile memory comprises: The interface is configured to receive random access write operations to the MMIO register set, receive random access granularity data to a buffer, and receive a volatile memory address to the MMIO register set; and The hybrid media controller is configured to, in response to the random access write operation in the MMIO register set, write the data of the random access granularity in the buffer to the volatile memory at the volatile memory address in the MMIO register set and set a state of the MMIO register set.

4. The memory module according to claim 1, wherein: The memory module configured to provide read and write access to the volatile memory comprises: The interface is configured to receive a hybrid media controller disable operation into the MMIO register set; The interface is configured to receive a volatile memory address and a random access read operation to the volatile memory; and The volatile memory is configured to read data of random access granularity at the volatile memory address in response to the random access read operation.

5. The memory module according to claim 1, wherein The memory module configured to provide read and write access to the volatile memory comprises: The interface is configured to receive a hybrid media controller disable operation into the MMIO register set; The interface is configured to receive a random access write operation, random access granularity data, and a volatile memory address to the volatile memory; and The volatile memory is configured to write data of the random access granularity into the volatile memory at the volatile memory address in response to the random access write operation. The memory module according to claim 1 , wherein: The memory module configured to provide read and write access to the non-volatile memory comprises: The interface is configured to receive a non-volatile memory address and a bulk read operation to the MMIO register set; and The hybrid media controller is configured to, in response to the bulk read operation in the MMIO register set, read bulk data from the nonvolatile memory at the nonvolatile memory address in the MMIO register set, feed the bulk data into the nonvolatile memory buffer, and set a state of the MMIO register set.

7. The memory module according to claim 1, wherein: The memory module configured to provide read / write access to the non-volatile memory comprises: The interface is configured to receive a bulk write operation to the MMIO register set, receive bulk data to the non-volatile memory buffer, and receive a non-volatile memory address to the MMIO register set; and The hybrid media controller is configured to, in response to the bulk write operation in the MMIO register set, write the bulk data from the non-volatile memory buffer to the non-volatile memory at the non-volatile memory address in the MMIO register set and set a state of the MMIO register set.

8. The memory module according to claim 1, wherein: The memory module configured to provide read / write access to the non-volatile memory comprises: The interface is configured to receive random access write operations to the MMIO register set, receive random access granularity data to the non-volatile memory buffer, and receive non-volatile memory addresses to the MMIO register set; and The hybrid media controller is configured to, in response to the random access write operation in the MMIO register set, write data at the random access granularity from the non-volatile memory buffer to the non-volatile memory at the non-volatile memory address in the MMIO register set and set a state of the MMIO register set.

9. The memory module according to claim 1, wherein: The memory module configured to perform self-indexing data movement between the non-volatile memory and the volatile memory includes: The interface is configured to receive a self-indexed data move operation into the MMIO register set; and a hybrid media controller configured to generate a volatile memory address based on the self-indexed data move operation in the MMIO register set, read a sample result from the volatile memory at the generated volatile memory address, calculate a non-volatile memory address from the volatile memory address, extract attributes from the calculated non-volatile memory address, store the attributes in a volatile memory, and set a state of the MMIO register set.

10. The memory module according to claim 1, wherein The non-volatile memory buffer includes a plurality of ping-pong buffers, and a size of the ping-pong buffer is the same as a size of a memory block in the non-volatile memory.

11. The memory module according to claim 1, wherein: The memory controller comprises: a non-volatile memory read / write controller, the non-volatile memory read / write controller being connected to the non-volatile memory, the MMIO register set, and the non-volatile memory buffer; a volatile memory read / write controller, the volatile memory read / write controller connecting the volatile memory and the MMIO register group; a flow controller connected to the non-volatile memory read / write controller, the volatile memory read / write controller and the MMIO register set; and An address generator is connected to the volatile memory read / write controller and the MMIO register group.

12. The memory module according to claim 11, wherein: The MMIO register set further includes: A random access data buffer is connected to the volatile memory read / write controller and is configured to store volatile memory read and write data.

13. The memory module according to claim 12, wherein: The random access data buffer includes a given portion of the non-volatile memory buffer.

14. The memory module according to claim 1, wherein: The MMIO register set includes: a feature size register configured to hold the number of bytes of each feature; an index base register configured to store a sampling node location; An index count register configured to store the number of sampling node positions; a target base address register configured to hold a volatile memory address for storing one or more features; an operation register configured to store an operation code; and A status register configured to save the current nonvolatile memory read / write controller state.

15. A method for accessing data, comprising: Read and write data to volatile memory in the memory mapped space within the memory module; reading and writing bulk data into a non-volatile memory in the memory mapping space within the memory module; reading and writing data at random access granularity to the non-volatile memory of the memory mapping space; as well as self-indexing moving data between the non-volatile memory and the volatile memory of the memory module; The step of reading and writing data at random access granularity to the non-volatile memory in the memory mapping space within the memory module includes: Receiving a random access read operation and a non-volatile memory address into an MMIO register set of the memory mapped space; In response to the random access read operation in the MMIO register set, reading, by a hybrid media controller, data of the random access granularity from the nonvolatile memory at the nonvolatile memory address within the MMIO register set into a nonvolatile memory buffer; and Set the read data status in the MMIO register set.

16. The method according to claim 15, wherein Reading and writing data to the volatile memory in the memory mapping space, comprising: Receive volatile memory addresses and random access read operations into a memory mapped input / output (MMIO) register set within a memory mapped space; In response to the random access read operation in the MMIO register set, reading, by a hybrid media controller, random access granularity data from the volatile memory at the volatile memory address in the MMIO register set; and A read data state is set in the MMIO register set by the hybrid media controller.

17. The method according to claim 15, wherein: Reading and writing data to the volatile memory in the memory mapping space, comprising: Receiving random access write operations and volatile memory addresses into an MMIO register set, and receiving random access granularity data into a buffer within the memory mapped space; In response to the random access write operation in the MMIO register set, writing, by a hybrid media controller, data at the random access granularity from the buffer to the volatile memory at the volatile memory address in the MMIO register set, and The hybrid media controller sets a write data status in the MMIO register set.

18. The method according to claim 15, wherein Reading and writing data to the volatile memory in the memory mapping space, comprising: Receive hybrid media controller read disable operation into MMIO register group; receiving a volatile memory address and a random access read operation into the volatile memory; and In response to the random access read operation received by the volatile memory, data of random access granularity is read from the volatile memory at the volatile memory address.

19. The method according to claim 15, wherein Reading and writing data to the volatile memory in the memory mapping space, comprising: receiving a hybrid media controller disable write operation to the MMIO register set; receiving a random access write operation, random access granularity data, and volatile memory into the volatile memory; and In response to the random access write operation received by the volatile memory, data of the random access granularity is written to the volatile memory at the volatile memory address.

20. The method according to claim 15, wherein Reading and writing bulk data to non-volatile memory in the memory mapped space includes: Receiving a non-volatile memory address and a batch read operation into an MMIO register set within the memory mapped space; In response to the bulk read operation in the MMIO register set, reading, by a hybrid media controller, the bulk data from the nonvolatile memory at the nonvolatile memory address in the MMIO register set into a nonvolatile memory buffer; and A read data state is set in the MMIO register set by the hybrid media controller.

21. The method according to claim 15, wherein Reading and writing bulk data to non-volatile memory in the memory mapped space includes: Receiving a batch write operation and a non-volatile memory address into an MMIO register set, and receiving batch data into a non-volatile memory buffer within the memory mapping space; In response to the bulk write operation in the MMIO register set, writing, by a hybrid media controller, the bulk data from the nonvolatile memory buffer to the nonvolatile memory at the nonvolatile memory address within the MMIO register set; and The hybrid media controller sets a write data status in the MMIO register set.

22. The method according to claim 15, wherein Reading and writing data at random access granularity to the non-volatile memory of the memory mapped space within the memory module comprises: Receiving random access write operations and non-volatile memory addresses into an MMIO register set, and receiving data at random access granularity into a non-volatile memory buffer within the memory mapping space; In response to the random access write operation in the MMIO register set, writing, by a hybrid media controller, data at the random access granularity from the nonvolatile memory buffer to the nonvolatile memory at the nonvolatile memory address within the MMIO register set; and A write status is set in the MMIO register set by the hybrid media controller.

23. The method according to claim 15, wherein Self-indexing moving data between the non-volatile memory and the volatile memory of the memory module includes: Receiving a self-indexed data move operation into the MMIO register set of the memory mapped space; generating a volatile memory address based on the self-indexed data move operation in the MMIO register set; reading a sample result from the volatile memory at the generated volatile memory address; calculating a non-volatile memory address from the volatile memory address; extracting attributes from the calculated non-volatile memory address; and storing the attributes in the volatile memory; and A self-indexing data movement status is set in the MMIO register set.

Images