Method of using tensor memory access in compute fast link communications

Abstract

The invention relates to a method of using tensor memory access in compute fast link communication. An example of a compute fast link (CXL) system includes a memory and a tensor access circuit having a memory mapper configured to configure a memory map based on compute fast link (CXL) commands associated with access operations of the memory. The memory mapping includes a particular sequence of CXL instructions to access the memory via a CXL bus.

Description

Technical Field

[0001] The present disclosure relates generally to semiconductor devices and more particularly to memory devices. Background Art

[0002] High-speed memory access and reduced power consumption are some of the features required of semiconductor devices. In recent years, systems that use multi-core processors to execute applications have resulted in faster access modes to memory devices, and also more random access modes. For example, a memory device may repeat a typical access mode (e.g., a dynamic random access memory (DRAM) may sequentially repeat bank activation, read access or write access, and bank precharge during an access operation). An access mode to a memory device is needed to achieve faster access. The efficiency and performance of a computing device may be affected by different memory devices. Therefore, a fast and efficient access mode is needed.

[0003] Tensors, which are often geometric objects related to linear systems, can be used for machine learning and artificial intelligence applications. Tensor processing can include processing of matrix algebra or other linear system analysis. This processing can be intensive and repetitive because, for example, common operands may be used several times in hierarchical processing of tensors. This repetition combined with processing speed may require repeated memory accesses to perform operations for tensor processing. Summary of the invention

[0004] In one aspect, the present disclosure provides an apparatus comprising: a memory; and a tensor access circuit having a memory mapper to configure a memory map based on a compute express link (CXL) command associated with an access operation of the memory, wherein the memory map comprises a specific CXL instruction sequence to access the memory via a CXL bus.

[0005] In another aspect, the present disclosure provides an apparatus comprising: a compute express link (CXL) device having a memory; and a processor coupled to the CXL unit via a CXL bus and including a tensor access circuit, wherein the tensor access circuit is configured to configure a memory map based on a CXL command associated with an access operation of the memory, wherein the memory map includes a specific CXL instruction sequence to access the memory via the CXL bus.

[0006] In another aspect, the present disclosure provides a method comprising: obtaining an access command associated with an access operation of a memory accessible via a compute express link (CXL) bus; retrieving a memory map for the access operation based on the access command; and performing the access operation associated with the memory of a CXL device via the CXL bus based on the memory map. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Figures 1A to 1C are schematic illustrations of respective levels of a Compute Express Link (CXL) computing system configured to implement tensor memory access operations according to embodiments described herein.

[0008] Figure 2 is a schematic illustration of a computing system arranged in accordance with embodiments described herein.

[0009] Figure 3 is an illustration of an access mode of a CXL memory cell according to embodiments described herein.

[0010] Figure 4 is a flow chart of a method of performing tensor access operations using the CXL protocol according to embodiments described herein.

[0011] Figure 5 is a schematic illustration of a computing system having a computing device arranged according to embodiments described herein. DETAILED DESCRIPTION

[0012] Certain details are set forth below to provide a thorough understanding of the embodiments of the present disclosure. However, it will be apparent to those skilled in the art that the embodiments of the present disclosure may be practiced without various of these specific details. In some instances, well-known computing device components, circuits, control signals, timing protocols, computing system components, and software operations are not shown in detail to avoid unnecessarily obscuring the described embodiments of the present disclosure.

[0013] In general, Compute Express Link (CXL) is an open standard, industry-supported cache-coherent interconnect for processors, memory extensions, and accelerators. Essentially, CXL technology maintains memory coherency between the central processing unit (CPU) memory space and the memory on the attached device. This enables resource sharing (or pooling) for higher performance, reduced software stack complexity, and / or reduced overall system cost. A CXL unit (e.g., device, system, chip, component, etc.) is a unit that is capable of communicating using CXL. CXL units typically belong to one of the following three categories or tiers: cache devices / accelerators (e.g., tier 1 CXL units); graphics processing units (GPUs), application-specific integrated circuits (ASICs), and field-programmable gate arrays (FPGAs) equipped with double data rate (DDR) dynamic random access memory (DRAM) or high bandwidth memory (HBM) (e.g., tier 2 CXL units); or memory buffers configured to provide additional bandwidth and capacity to the host processor independent of the host processor's memory (e.g., tier 3 CXL units).

[0014] For level 1 CXL units, accelerators such as intelligent network interface controllers (NICs) (e.g., partitioned global address space (PGAS) NICs, NIC Atomic, etc.) may lack local memory, so these devices may communicate with the host processor's DDR DRAM using CXL. For level 2 CXL units, CXL may be used to make the host processor's memory available locally to the level 2 CXL unit—and the level 2 CXL unit's memory available locally to the host processor. For level 3 CXL units, the host processor may utilize CXL to communicate with additional memory buffers (e.g., memory devices) in order to extend memory bandwidth and / or capacity expansion beyond the host processor's memory.

[0015] The CXL cell may use one or more CXL protocols to receive and execute read and write commands from a host processor, a memory controller, and / or directly from a computing device or network that sends memory commands. The CXL cell may receive a read or write command as a sequence of CXL instructions, where each instruction corresponds to a specific location identified by an address to store data to or retrieve data from. For example, a read command may be processed by the host processor or memory controller as a request to read a specific address of a specific CXL cell. This command may be sent to the CXL cell as an instruction to access the location of the CXL cell. The instruction may include such addressable information (e.g., a row / column of a memory cell and / or a logical address pointing to a row / column of a memory cell) as determined by the host processor or memory controller based on the read command. For example, the location of the CXL cell may be located at a specific physical memory cell in a logical CXL memory partition of the CXL cell. In the example of a memory array as a CXL cell, the host processor or memory controller may perform circuit system operations (e.g., charging a row or column line) to access a specific physical memory cell. Such circuit system operations may be both time-consuming and power-consuming. Similarly, a host processor or memory controller determining which logical storage partition of a CXL unit may contain requested information that may be accessed at a particular access rate may be a time-consuming and power-consuming process for executing a CXL command.

[0016] When accessing a particular memory cell of a CXL cell, a read or write command may not distinguish where the requested information is physically stored in the CXL cell. Furthermore, a host processor and / or memory controller may not send instructions to the CXL cell based on any pattern in which information is stored in the CXL cell. The CXL cell may receive a write command and thereby process a write to the CXL cell without regard to the details of an operation executed or implemented in a processor or computing device. For example, the CXL cell may receive a write command and store information associated with the write command to a particular location in a memory of a CXL cell that has been determined to be available. As described herein, advantageously, the operation executed or implemented in the processor or computing device may include executing a CXL command as defined by the operation executed. For example, a particular CXL access instruction sequence to access a memory cell of a memory array of a CXL cell may include a CXL access instruction sequence defined by an order of operations of the CXL command. For example, a CXL cell may be accessed in a particular pattern that may facilitate tensor operations as described herein. Tensor operations may utilize matrix data, and thus may seek to read and / or write data in a particular pattern (e.g., diagonal, etc.). In the example of diagonal computation of matrix operations, a CXL access instruction sequence may access various memory cells along the diagonal of a memory array according to a diagonal CXL command implemented by a host processor and / or memory controller coupled to the CXL unit including such a memory array. It should be noted that in some examples, the CXL unit may be configured to perform similar access operations associated with processor memory using a similar CXL protocol.

[0017] Figures 1A to 1C Schematic illustrations of respective levels of a Compute Express Link (CXL) computing system 100 , 101 , and 102 , respectively, configured to implement tensor memory access operations according to embodiments described herein. Figures 1A to 1C The respective levels of the CXL computing systems 100, 101, and 102 may include common elements, and the elements are Figures 1A to 1C Therefore, for the sake of brevity, all three Figures 1A to 1C The detailed description of the operations of these specific elements is repeated.

[0018] Go to Figure 1A, Level 1 CXL computing system 100 may include a processor 110 coupled to a CXL unit 130 via a CXL bus 105. Processor 110 may be coupled to dedicated memories 120, 122. Processor 110 may include tensor access circuitry 112 configured to support tensor memory access operations. CXL unit 130 may include hardware for a specific application, such as an intelligent NIC (e.g., PGAS NIC, NIC Atom, etc.) for processing network data. In some examples, CXL unit 130 may include a small cache memory 134 or no local memory at all. Since CXL unit 130 may lack any or sufficient local memory, CXL unit 130 may communicate with dedicated memories 120 and 122 via processor 110 using CXL bus 105. CXL unit 130 may include tensor access circuitry 132 for supporting tensor access operations.

[0019] Go to Figure 1B , a level 2 CXL computing system 101 may include a processor 110 coupled to a CXL unit 140 via a CXL bus 105. The CXL unit 140 may include hardware for specific applications, such as a GPU, ASIC, and FPGA equipped with a small cache 144 and double data rate (DDR) dynamic random access memory (DRAM) or high bandwidth memory (HBM) (memory) 146, 148. In this example, the CXL bus 105 may facilitate pooling of memory of the processor 110 (e.g., memory 120, 122) and memory of the CXL unit 140 (e.g., memory 146, 148). The CXL unit 140 may include a tensor access circuit 142 for supporting tensor access operations.

[0020] Go to Figure 1C , a level 3 CXL computing system 102 may include a processor 110 coupled to a CXL unit 150 via a CXL bus 105. The CXL unit 150 may include memories 154(1)-(4), and the processor 110 may communicate with the CXL unit 150 using the CXL bus 105 to access the memories 154(0)-(3) to extend memory bandwidth and / or capacity expansion beyond the memories 120, 122 of the processor 110. The CXL unit 150 may include tensor access circuitry 152 for supporting tensor access operations.

[0021] Processor 110 and CXL unit 130, CXL unit 140 and / or CXL unit 150 (CXL unit) may communicate various commands back and forth using CXL bus 105, including receiving read and write commands to retrieve or store data at one of memories 120, 122, 134, 144, 146, 148 and 154 (1)-(4), respectively. The CXL unit may receive a read or write command from processor 110 as a sequence of instructions, where each instruction corresponds to a specific location identified by an address to store data to or retrieve data from. For example, a read command may be processed by processor 110 as a request to read a specific address of a specific memory of the CXL unit. This command may be sent to the CXL unit as an instruction to access that location of the CXL unit. The instruction may include such addressable information (e.g., a row / column of a memory cell and / or a logical address pointing to a row / column of a memory cell) as determined by processor 110 based on the read command. For example, the location of one of the memories of the CXL cell may be located at a particular physical memory cell in a logical CXL memory partition of the CXL cell. In the example of a memory array as the memory of the CXL cell, the processor 110 may perform a circuitry operation (e.g., charging a row or column line) to access a particular physical memory cell. Similar operations may be initiated by the CXL cell to access the memories 120, 122 of the processor 110. Such circuitry operations may be both time consuming and power consuming. Similarly, the processor 110 and / or the CXL cell determining which logical memory partition of the target memory may include the requested information that may be accessed at a particular access rate may be a time consuming and power consuming process for executing the access command.

[0022] When accessing a particular memory cell of one of the processor 110 or the CXL cell, a read or write command may not distinguish where the requested information is physically stored in the CXL cell. Moreover, the processor 110 may not send instructions to the CXL cell based on any pattern of storing information in the CXL cell, and vice versa. The CXL cell (or processor 110) may receive a write command and thereby process a write to a memory of the CXL cell (or processor 110) without regard to the details of the operations executed or implemented in the processor 110 or the CXL cell. For example, the CXL cell may receive a write command and store information associated with the write command to a particular location in the memory of the CXL cell that has been determined to be available. As described herein, advantageously, the operations executed or implemented in the processor or computing device may include executing CXL commands as defined by the operations executed. For example, a particular CXL access instruction sequence to access memory cells of a memory array of the CXL cell may include a CXL access instruction sequence defined by an order of operations of the CXL commands. For example, memory of the CXL unit and / or processor 110 may be accessed in a particular pattern that may facilitate tensor operations as described herein. Tensor operations may utilize matrix data, and thus may seek to read and / or write data in a particular pattern (e.g., diagonal, etc.). In the example of a diagonal computation of a matrix operation, a CXL access instruction sequence may access various memory cells along a diagonal of the memory array, according to a diagonal CXL command implemented by the CXL unit and / or processor 110 coupled to a memory including such a memory array.

[0023] Figure 2 is a schematic illustration of a computing system 200 arranged according to embodiments described herein. In some examples, Figure 1A Level 1CXL computing system 100, Figure 1B Tier 2 CXL computing system 101 and / or Figure 1C The Tier 3 CXL computing system 102 may implement the computing system 200 in some examples. The computing system 200 includes a processor 205 coupled to a compute express link (CXL) unit 240a, 240b. The CXL units 240a, 240b may include a cache device / accelerator, an accelerator with a memory, a memory buffer with a memory, or any combination thereof. The CXL units 240a, 240b may implement Figure 1A CXL unit 130, Figure 1B CXL unit 140, Figure 1CThe processor 205 may include a CXL unit 150 or any combination thereof. The processor may implement a memory controller 210 that includes a tensor access circuit 212 having a memory mapper 220 and an address translator 230. The memory controller 210 may be coupled to the CXL units 240a, 240b via CXL interfaces 235a, 235b. The CXL units 240a, 240b may each include a respective tensor access circuit 242a, 242b. Portions of the foregoing discussion will focus on accessing or communicating with the tensor access circuit 212 of the CXL units 240a, 240b. Similarly, inference operations may be performed by the tensor access circuits 242a, 242b of the CXL units 240a, 240b to communicate with and / or access memory coupled to the processor 205. For the sake of brevity, a detailed description of these operations is not included.

[0024] The processor 205 may implement CXL commands received from various data sources or processes executing on the processor 205. For example, the processor 205 may receive a CXL memory access request (e.g., a read or write command) from a process executing on the processor 205. In this case, the memory controller 210 may process the CXL memory access request as implemented by the processor 205 to access the memory of one or more of the CXL units 240a, 240b.

[0025] The processor 205 may be used to implement a memory system of the computing system 200 using the memory controller 210. In some examples, the processor 205 may be a multi-core processor including multiple cores. For example, the multiple cores may be implemented using processing circuits that independently read and execute program instructions. The memory controller 210 may handle communications with a memory system that may be external to the processor 205. For example, the memory controller 210 may provide CXL access commands from the multiple cores of the processor 205 to the CXL units 240a, 240b. The memory controller 210 may provide such access commands via the CXL interfaces 235a, 235b. For example, the CXL interfaces 235a, 235b may provide clock signals, command signals, and / or address signals to any of the CXL units 240a, 240b. When data is written by storing the data in the CXL units 240a, 240b, the memory controller 210 provides instructions to write the data to the CXL units 240a, 240b based on the write command. When reading stored data from the CXL cells 240a, 240b, the memory controller 210 provides instructions based on a read command and receives data from the CXL cells 240a, 240b.

[0026] The memory controller 210 may be implemented using circuitry that controls data flow to the CXL units 240a, 240b. The memory controller 210 may be a separate chip or integrated circuit coupled to or implemented on the processor 205, for example, as a core of the processor 205 to control the memory system of the computing system 200. In some embodiments, the memory controller 210 may be integrated into the processor 205 to be referred to as an integrated memory controller (IMC).

[0027] The memory controller 210 may communicate with multiple CXL units to implement a computing system having the processor 205. For example, the CXL units 240a, 240b may communicate with the memory controller 210 at the same time. Figure 2 The example of depicts two CXL cells 240a, 240b, but it is contemplated that the memory controller 210 may interact with any number of CXL cells. For example, 8 CXL cells may be included and each CXL cell may include a data bus having an 8-bit width, so the memory system implemented by the processor 205 may have a 64-bit width. The CXL cells 240a, 240b may include or be coupled to a memory, such as a cache memory, a dynamic random access memory (DRAM), or a non-volatile random access memory (RAM), such as a ferroelectric RAM (FeRAM), a spin transfer torque RAM (STT-RAM), a phase change RAM (PCRAM), a resistance change RAM (ReRAM), or the like. In various embodiments, such CXL cells may include a memory chip, a memory module, a memory die, a memory card, a memory device, a memory array, and / or a memory cell. Physically, the memory of the CXL cells 240a, 240b may be arranged and disposed as one layer, or may be disposed as a stacked layer. In some embodiments, the memory of the CXL cells 240a, 240b may be arranged as multiple layers stacked on top of each other to form a vertical memory, such as 3D NAND flash memory.

[0028] In some examples where the memory of the CXL units 240a, 240b is implemented using a DRAM or a non-volatile RAM integrated into a single semiconductor chip, the memory of the CXL units 240a, 240b may be mounted on a memory module substrate, a motherboard, or the like. For example, the memory of the CXL units 240a, 240b may be referred to as a memory chip. The memory of the CXL units 240a, 240b may include a memory cell array area and a peripheral circuit area. The memory cell array area includes a memory cell array having a plurality of memory banks, each memory bank including a plurality of word lines, a plurality of bit lines, and a plurality of memory cells arranged at intersections of the plurality of word lines and the plurality of bit lines. The selection of the bit lines may be performed by a plurality of column decoders and the selection of the word lines may be performed by a plurality of row decoders.

[0029] The peripheral circuit area of ​​the memory of the CXL unit 240a, 240b may include a clock terminal, an address terminal, a command terminal, and a data input / output (I / O) terminal (DQ). For example, the data I / O terminal may handle 8-bit data communication. A data input / output (I / O) buffer may be coupled to the data input / output terminal (DQ) for data access, such as read access and write access of the memory. The address terminal may receive an address signal and a memory bank address signal. The memory bank address signal may be used to select a memory bank among a plurality of memory banks. A row address and a column address may be provided as the address signal. The command terminal may include a chip select ( / CS) pin, a row address strobe ( / RAS) pin, a column address strobe ( / CAS) pin, a write enable ( / WE) pin, and / or the like. The command decoder may decode the command signal received from the memory controller 210 via one of the CXL interfaces 235a, 235b at the command terminal to receive various commands including a read command and / or a write command. This command decoder may provide control signals in response to received commands to control the memory cell array region.The clock terminal may be supplied with an external clock signal, such as from one of the CXL interfaces 235a, 235b.

[0030] The memory mapper 220 of the memory controller 210 may provide a memory map to the memory unit to access the memory unit. For example, the memory map may be selected according to a CXL command provided to access the CXL unit 240a and / or the CXL unit 240b. For example, a read or write operation of a process or program implemented on the processor 205 may be a CXL memory access operation that sends a read or write command to the memory controller 210. The memory mapper 220 may retrieve or configure the memory map based on the CXL command associated with the CXL memory access operation. The memory map may include one or more specific memory access instruction sequences to access the memory of the CXL unit (e.g., the CXL unit 240a and / or the CXL unit 240b). For example, the specific memory access instruction sequence may access one or more memory cells of the memory array in a specific pattern, which may be advantageous for performing tensor and / or matrix operations. Each memory access instruction may include an instruction for a corresponding address of one of the plurality of memory cells. For example, the memory map may be provided to CXL unit 240a via an address terminal of CXL interface 235a to an address terminal of CXL unit 240a.

[0031] The memory mapper may generate, provide and / or select a memory map specific to the memory command type. CXL memory commands may include row memory commands or column memory commands, e.g., to access a corresponding row or column of a memory array. CXL commands may also include commands based on operations executed or implemented on processor 205. Tensor operations may include various matrix operations and / or calculations. For example, a tensor operation may include the calculation of the diagonal of a particular matrix or the determinant of a particular matrix; the latter being part of a matrix inverse calculation, for example.

[0032] The memory mapper 220 may provide a memory map so that the memory is accessed as defined by the order of operations of the tensor CXL commands. The tensor CXL commands may include diagonal memory commands, row-column memory commands, or any matrix memory commands. For example, a particular sequence of memory access instructions to access memory cells of a memory array may include a sequence of memory access instructions defined by the order of operations of the tensor CXL commands. In some examples of diagonal computations of matrix operations, as part of the tensor operations performed on the processor 205, the sequence of memory access instructions may access various memory cells along the diagonal of the memory array according to the diagonal memory commands implemented by the processor 205. Thus, when providing a memory map for a memory command, the memory mapper 220 may identify a plurality of memory addresses of one or more CXL units 240a, 240b to access the corresponding memory addresses; and thereby, the plurality of memory addresses are allocated into the memory map according to the order of operations of the memory commands so that the particular sequence of instructions is executed in the order of operations.

[0033] The memory mapper 220 may provide a memory map that may be based on a tensor CXL command for a memory unit of a CXL unit 240a, 240b having a three-dimensional memory array. The memory map may specify that each layer of the three-dimensional memory array is stored according to a corresponding layer of tensor processing. Tensor operations may be performed on a three-dimensional structure; and as described herein, the memory map may be used to efficiently access the memory of the CXL unit 240a, 240b based on the three-dimensional structure of the tensor operation. Tensor operations on a three-dimensional matrix may include operations based on patterns such as vertical column patterns, row patterns, tube patterns, vertical slice patterns, transverse slice patterns, and / or frontal slice patterns. Each pattern may correspond to a two-dimensional vector or plane of a corresponding axis of the three-dimensional operation; and thus, memory may be accessed in a similar manner, such as reading or writing to memory. Advantageously, in some instances, accessing memory in three dimensions may allow for more efficient processing of tensors; for example, in machine learning applications.

[0034] Advantageously, in some instances, for tensor CXL commands, the systems and methods described herein may be used as a memory access scheme when processing tensors or performing tensor operations such as tensor decomposition. Tensor-based processing may compute a number of data layers to be processed, where each subsequent layer is based on a combination of the results of the previous layer. In some applications, the layers may be stored in the memory of the CXL units 240a, 240b so that subsequent processing of the layers may be performed more efficiently. For example, the layers may be written diagonally in a two-dimensional memory array or a three-dimensional memory device. The memory mapper 220 may provide a memory map that is arranged to access such memory cells in a diagonal manner, as specified by the order of operations in the tensor CXL command. In some instances, the tensor CXL command may include processing the results of each previous layer by initially writing the results of each layer as diagonals in the memory cells. Thus, the memory cells may also be accessed to process the diagonal calculations in the manner in which the results of each layer are written according to the tensor CXL command. In this manner, any memory unit or device may be accessed according to a tensor memory access, which may be an access to memory defined by an order of operations for tensor processing, as described herein.

[0035] Although the memory mapper 220 has been described as part of the memory controller 210 in the context of an embodiment of the processor 205, it is contemplated that the memory mapper 220 may be implemented differently in other embodiments as well. Additionally or alternatively, the memory mapper 220 may be implemented in the CXL units 240a, 240b to support accessing the memory of the processor 205. In addition, the memory mapper 220 may be coupled to the processor 205 as a separate circuit such as an application specific integrated circuit (ASIC), a digital signal processor (DSP) implemented as part of a field programmable gate array (FPGA), or a system-on-chip (SoC). As another example, the memory mapper 220 may be coupled to the memory controller 210 implemented by the processor 205 as a series of switches that determine the sequence of instructions to be executed on the CXL units 240a, 240b. The switches may be, for example, multiplexers with selected lines coupled to the memory controller 210.

[0036] The address translator 230 of the memory controller 210 may translate a memory map for a memory unit based on a CXL command used to access the memory of the CXL unit 240a and / or the CXL unit 240b. The memory map may be translated when the operation sequence of the CXL command is different from the operation sequence of a previous CXL command provided to perform a previous memory access operation at the CXL unit 240a and / or the CXL unit 240b. For example, the memory mapper 220 may have previously provided a memory map based on a diagonal command, and subsequently, the memory mapper 220 receives a command from the memory controller 210 to provide a memory map for a columnar CXL command. In such an example, the address translator 230 may identify the address of the memory unit in the memory map for the diagonal memory command. Once identified, the address translator 230 may assign the identified address to a memory map configured for the row-based memory command such that the row-based memory command accesses the identified address in an order of operations defined by the row-based operation rather than a diagonal memory operation that previously accessed a memory cell. Once assigned, the address translator 230 completes the translation of the memory map for the row-based CXL command, and the memory mapper 220 may provide the translated memory map to the CXL unit 240a and / or the CXL unit 240b.

[0037] Although the address translator 230 has been described as part of the memory controller 210 in the context of an embodiment of the processor 205, it is contemplated that the address translator 230 may be implemented differently in other embodiments as well. Additionally or alternatively, the address translator 230 may be implemented in the CXL units 240a, 240b to support accessing the memory of the processor 205. The address translator 230 may be coupled to the processor 205 as a separate circuit such as an ASIC, a digital signal processor (DSP) implemented as part of a field programmable gate array (FPGA), or a system-on-chip (SoC). As another example, the address translator 230 may be coupled to the memory controller 210 implemented by the processor 205 as a series of switches that identify the respective addresses of the CXL units 240a, 240b to translate the identified addresses for different memory mappings. The switches may be, for example, multiplexers having a selected line coupled to the memory controller 210.

[0038] Figure 33 is an illustration of access modes for CXL memory cells according to embodiments described herein. Access mode 300 is an illustration of a CXL memory cell accessed according to a row access mode as part of a tensor CXL access operation. As shown in access mode 300, within a 6x6 memory cell array, row 3 is accessed during the row access mode. Access mode 301 is an illustration of a CXL memory cell accessed according to a column access mode as part of a tensor CXL access operation. As shown in access mode 301, within a 6x6 memory cell array, column 3 is accessed during the column access mode. Access mode 302 is an illustration of a CXL memory cell accessed according to a diagonal access mode as part of a tensor operation. As shown in access mode 302, within a 6x6 memory cell array, a diagonal of memory cells starting at the upper left corner and ending at the lower right corner is accessed during the diagonal access mode. Access pattern 303 is an illustration of memory cells accessed according to a sub-matrix access mode as part of a tensor CXL access operation. As shown in access pattern 303, within a 6x6 array of memory cells, a 3x3 sub-matrix of memory cells (starting at cells (3, 4)) is accessed during the sub-matrix access mode. Different or additional access modes and corresponding access modes may be implemented without departing from the scope of the present disclosure.

[0039] Figure 4 is a flow chart of a method 400 for performing tensor access operations using the CXL protocol according to embodiments described herein. In some examples, the method 400 may be performed by Figures 1A to 1C processor 110, Figure 1A CXL unit 130, Figure 1B CXL unit 140, Figure 1C CXL unit 150, and / or Figure 2 The processor CXL bus 105 and / or any of the CXL units 240a, 240b executes.

[0040] The method 400 may include obtaining an access command associated with an access operation of a memory accessible via a compute express link (CXL) bus at 410. The access command may be a CXL access command associated with a read or write operation of the memory.

[0041] The method 400 may further include retrieving a memory map for the access operation based at least on the access command at 420. In some examples, the memory map includes a specific sequence of memory access instructions for accessing a plurality of memory cells of the memory. In some examples, the memory map is based on an operation order of the access command. In some examples, the memory map may be retrieved by a memory mapper (e.g., Figure 2In some examples, the method 400 further includes retrieving the memory map from a memory coupled to the processor.

[0042] Method 400 may further include performing access operations associated with the memory of the CXL device via the CXL bus based on the memory map at 430. In some examples, method 400 further includes accessing respective addresses of the plurality of memory cells based on the memory map to perform the access operations via the CXL bus.

[0043] Figure 5 is a schematic illustration of a computing system 502 having a computing device 502 arranged according to embodiments described herein. The computing device 502 may operate according to any of the embodiments described herein. The computing device may be a smartphone, a wearable electronic device, a server, a computer, an appliance, a vehicle, or any type of electronic device. The computing system 500 includes a memory system 502, a processor 505, and an I / O interface 570, and a network interface 590 coupled to a network 595. The memory system 502 includes a memory controller 510 having a tensor access circuit 512, the tensor access circuit 512 having a memory mapper 520 and an address translator 530, both of which operate according to the functionality described herein with respect to the memory mapper and the address translator. Figure 5 Similar numbered components include similar Figure 2 For example, a CXL unit 540 with a tensor access circuit 542 may be configured as Figure 2 Processor 505 may include any type of microprocessor, central processing unit (CPU), application specific integrated circuit (ASIC), digital signal processor (DSP) implemented as part of a field programmable gate array (FPGA), system on a chip (SoC), or other hardware that provides processing for computing system 500.

[0044] The memory system 502 also includes a CXL unit 540 and non-transitory hardware-readable media 550, 560 including instructions for memory access and address translation, respectively. The processor 505 may control the memory system 502 using control instructions that indicate when to execute the instructions 550 for CXL unit access and / or the instructions 560 for address translation. Upon receiving such control instructions, the memory mapper 520 may execute the instructions 550 for CXL unit access; and / or the address translator 530 may execute the instructions 560 for address translation. The instructions 550 for CXL unit access may include a program to perform the method 400. The instructions 560 for address translation may include a program to perform the method 400. Communication between the processor 505, the I / O interface 570, and the network interface 590 is provided via a processor internal bus 580. The processor 505 may receive control instructions from the I / O interface 570 or the network interface 590, such as instructions to control the execution of memory access or address translation.

[0045] Bus 580 may include one or more physical buses, communication lines / interfaces, and / or point-to-point connections, such as a peripheral component interconnect (PCI) bus. I / O interface 570 may include various user interfaces, including video and / or audio interfaces for a user, such as a flat-panel display with a microphone. Network interface 590 communicates with other computing devices, such as computing device 500 or a cloud computing server, via network 595. For example, network interface 590 may be a USB interface.

[0046] Although certain features and aspects have been described with respect to exemplary embodiments, it will be appreciated by those skilled in the art that various modifications and additions may be made to the embodiments discussed without departing from the scope of the invention. Although the embodiments described above relate to specific features, the scope of the invention also includes embodiments having different combinations of features and embodiments that do not include all of the above features. For example, the methods and processes described herein may be implemented using hardware components, software components, and / or any combination thereof. In addition, although for ease of description, the various methods and processes described herein may be described with respect to specific structural and / or functional components, the methods provided by the various embodiments are not limited to any specific structural and / or functional architecture and may be implemented in any suitable hardware, firmware, and / or software configuration instead. Similarly, although certain functionality is attributed to certain system components, unless otherwise indicated by the context, this functionality may be distributed throughout various other system components according to several embodiments.

[0047] In addition, although the procedures of the methods and processes described herein are described in a particular order for ease of description, various procedures may be reordered, added, and / or omitted according to various embodiments. The procedures described with respect to one method or process may be incorporated into other described methods or processes; similarly, hardware components described according to a particular structural architecture and / or with respect to one system may be organized in an alternative structural architecture and / or incorporated into other described systems. Therefore, although various embodiments are described as having or not having certain features for ease of description, various components and / or features described herein with respect to specific embodiments may be combined, substituted, added, and / or subtracted from other described embodiments. Therefore, although several exemplary embodiments are described above, it will be understood that the present invention is intended to cover all modifications and equivalents within the scope of the appended claims.

Claims

1. A device, wherein include: Memory; and A tensor access circuit has a memory mapper to configure a memory map based on a compute fast link (CXL) command associated with an access operation of the memory, wherein the memory map includes a specific CXL instruction sequence to access the memory via a CXL bus. 2 . The apparatus of claim 1 , wherein the particular CXL instruction sequence is specific to the CXL command type. 3 . The apparatus of claim 2 , wherein the CXL command type comprises a diagonal memory command, a row-column memory command, or any matrix memory command. 4 . The apparatus of claim 1 , wherein each CXL instruction of the particular sequence of CXL instructions comprises an instruction for a respective address of a memory cell of the memory.

5. The apparatus of claim 1, further comprising a memory controller including the tensor access circuit.

6. The apparatus of claim 5, further comprising a processor including the memory controller, wherein the processor is coupled to the memory via the CXL bus.

7. The apparatus of claim 6, further comprising a CXL device including the memory.

8. The apparatus of claim 1, further comprising a CXL device including the memory mapper.

9. The apparatus of claim 8, further comprising a processor coupled to the CXL device via the CXL bus, wherein a CXL unit is configured to route the particular CXL instruction sequence to the memory via the processor.

10. The apparatus of claim 1, further comprising a CXL interface coupled to the tensor access circuit and configured to communicate with the memory.

11. The apparatus of claim 10, wherein the memory mapper is configured to provide the memory map to the memory via the CXL interface.

12. A device, wherein include: A computing fast link CXL device having a memory; and A processor is coupled to the CXL unit via a CXL bus and includes tensor access circuitry, wherein the tensor access circuitry is configured to configure a memory map based on CXL commands associated with access operations to the memory, wherein the memory map includes a specific CXL instruction sequence to access the memory via the CXL bus.

13. The apparatus of claim 12, wherein the CXL command types include diagonal memory commands, row-column memory commands, or any matrix memory commands.

14. The apparatus of claim 12, wherein each CXL instruction of the particular sequence of CXL instructions comprises an instruction for a respective address of a memory unit of the memory.

15. The apparatus of claim 12, wherein the processor is configured to route the particular CXL instruction sequence to the memory via the CXL bus.

16. A method, wherein include: Obtaining an access command associated with an access operation to a memory accessible via a compute express link (CXL) bus; retrieving a memory map for the access operation based on the access command; and The access operations associated with the memory of a CXL device are performed via the CXL bus based on the memory map.

17. The method of claim 16, wherein the memory map includes a specific sequence of memory access instructions to access a plurality of memory cells of the memory.

18. The method of claim 16, further comprising accessing respective addresses of the plurality of memory cells based on the memory map to perform the access operation via the CXL bus.

19. The method of claim 16, wherein the memory map is based on an operation order of the access commands.

20. The method of claim 16, further comprising retrieving a memory map from a memory coupled to a processor.

Images