Computing devices with distributed object computing memory

Abstract

Disclosed are a compute node (400) for a computer, a computer (600) having distributed memory as main memory assembled from the compute node, and a memory access method (800) for such a computer. The compute node (400) comprises at least one compute core (410) that operates on information organized and stored in a memory object (10). The memory object is subdivided into fragments (12) that can be referenced by at least one compute core. A memory object management unit (420) of the compute node operably connects at least one compute core (410) to the distributed memory in order to access the fragments (12) from each memory location in the distributed memory that can be accessed via their corresponding physical addresses. The memory object management unit (420) calculates the physical address as a function of the fragment offset in the logical address space (11) associated with the memory object containing the referenced fragment and the unique identifier (OID) assigned to the memory object containing the referenced fragment.

Description

Technical Field

[0001] The present invention relates to the field of memory management in computer architectures for high-performance computing (HPC) and supercomputers.

Background Art

[0002] The extremely large number of interconnected computing cores in today's supercomputers causes a significant amount of communication overhead. The coherence of shared memory structures such as shared caches in multi-core computer architectures, and the synchronization of large-scale distributed databases that manage the distributed memory resources of supercomputers, require a large amount of computer resources that cannot be utilized for useful computing. This has caused a performance bottleneck in many of the world's leading hyperscale computers.

[0003] Furthermore, when a machine (supercomputer) is constructed as one or more clusters of computing nodes, the machine can appear to the programmer as either a single system image or a pool of independent but interconnected nodes. When appearing as a single system image, the programmer does not recognize the distinction between physical nodes, so the behavior of the nodes must be consistent regardless of their location. This causes a problem of global synchronization: the information accessed by multiple processes must always be consistent, but this is extremely difficult to achieve in view of the nature of the clusters being physically distributed. When appearing as a pool of independent but interconnected nodes, the programmer must consider the movement and synchronization of data between physical nodes. Both approaches result in extremely high messaging overhead and complex programming. In the context of supercomputing (HPC), up to 97% of the time is used for data synchronization and transfer relative to the calculation of data.

[0004] Dedicated file systems such as "Lustre" are being developed to reduce the speed gap between memory and computing cores by further increasing the speed of memory access in supercomputers. However, memory content managed by file systems is inherently slow to access due to the numerous indirect references involved.

[0005] In Abrahamse et al., “Memory-Disaggregated In-Memory Object Store Framework for Big Data Applications”, Proceedings of the 2022 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW), ed. L.O. Conner, Piscataway: IEEE, 2022, pp. 1228-1234, the use of ThymesisFlow, a software-defined interconnect stack for rack-scale memory distribution, is proposed within a memory-disaggregated in-memory object store framework to overcome local memory constraints in distributed computing environments. The solution proposed by these authors is a software solution built on top of existing hardware and therefore only hides, rather than fundamentally solves, the underlying problems with caching and addressing. [Overview of the project]

[0006] The object of the embodiments of the present invention is to overcome the aforementioned drawbacks of the prior art.

[0007] The above-mentioned objectives are achieved by the apparatus and method according to the present invention.

[0008] In one embodiment, the present invention relates to a compute node suitable for replication and assembly into a multi-node multi-processing computer such as a supercomputer, wherein such a computer has distributed memory or a store as the physical main memory of the computer. The compute node has at least one compute core for performing compute processes on information / data organized into memory objects and stored as memory objects, the at least one compute core referring to this information as a set of one or more fragments contained within the memory object. Each different fragment of the memory object has a different offset in the logical address space associated with the memory object (e.g., per-object address space). Furthermore, each memory object is assigned a unique identifier. The compute node further includes a memory object management unit that operably couples at least one compute core to the distributed memory in order to access the memory object fragments from their respective memory locations in the distributed memory. Each memory location in the distributed memory is accessible via a corresponding physical address. The memory object management unit calculates the physical address corresponding to the memory location in distributed memory of each fragment referenced by at least one computing core, as a function of the fragment offset in the logical address space associated with the memory object containing the fragment, and the unique identifier assigned to the memory object containing the fragment.

[0009] In contrast to persistent data objects such as blobs, which are managed by a file system and located in persistent but slower secondary storage memory (e.g., hard drives), memory objects according to embodiments of the present invention are stored in the computer's main memory, such as random access memory, which is faster primary memory compared to secondary storage media. Memory objects according to the present invention are directly accessible to the computing cores of a computing node, and the computing cores of the computing node interact with data in the computer's main memory through the memory objects. Because the memory objects are accessible in the computer's main memory, a file system is not required to query, discover, or retrieve the contents of the memory objects. Conveniently, memory transactions originating from the computing node are sent directly to the distributed store (i.e., main memory) via a memory object management unit without redirection by software processes of the operating system or application programming interface (API), and the memory object management unit calculates the physical address in computer memory based on the fragment offset and object identifier (OID) information contained in the memory transaction. The calculated physical address is sufficient to retrieve the requested memory object fragment from the distributed store and does not include, for example, further identification information. In embodiments of the present invention, storing different fragments containing memory objects at different locations in a distributed store can follow the principle of block storage, in which data blocks of a predetermined length (i.e., fragments or groups of contiguous fragments - shards) are stored at a physical location in computer memory indicated by a corresponding physical address.

[0010] In embodiments of the present invention, one or more computing cores in each computing node receive and execute instructions from a user program and generate output results, for example at the application level, according to the received instructions. These instructions generally include memory transaction commands (e.g., load and store commands) for data stored in fragments of one or more memory objects in a distributed store. That is, the computing cores of each computing node operate on memory objects at their most basic level, for example, when performing memory load / store operations, but generally cannot query the distributed store to find out what information is stored in the distributed store. The computing cores also perform actual calculations on user data, not just load / store or programmable I / O operations. Output results are also typically stored in memory objects in the distributed store. Computing core instructions relating to memory transactions may result in the dynamic creation of one or more new memory objects in the distributed store / computer memory. As a result, since the distributed store constitutes the only available computer main memory and the contents of retrieved memory objects cannot be transferred or copied to different main memories of the multi-node multiprocessing computer, the computing cores of each computing node must be operablely coupled to the distributed store during runtime. Therefore, a distributed store for compute nodes and memory objects should not be confused with a backend storage system like those used in distributed file systems, for example, but rather can be considered part of the frontend of a large-scale computing system that has multiple parallel-operating compute nodes coupled to and sharing the distributed store.

[0011] The memory objects in the embodiments are primarily objects representing computation input and output data, including application code, and vary depending on the underlying data type. They do not represent file system-related information such as inodes, directories, or files. In embodiments of the present invention, application and user program code can be stored in memory objects within computer memory / distributed stores.

[0012] In embodiments of the present invention, memory objects in a distributed store are referenced through their respective system-wide unique object identifiers. Each compute node's memory object management unit can dynamically generate, allocate, and free / delete unique object identifiers at runtime. This can be achieved by a dedicated hardware unit for the memory object management unit and / or a hardware unit associated with the compute node's task management unit. No application programming interface is required to create and maintain memory objects in distributed computer memory; the process is transparent to the user. Memory objects according to embodiments of the present invention are ephemeral structures: they are online entities in computer main memory that exist during runtime and are generally lost after a user program has finished executing (e.g., released, abandoned, forgotten, freed, erased, or garbage collected). In other words, memory objects exist temporarily in computer memory but are not permanently stored there. This does not preclude the possibility of having software that manages the transfer or copying of the contents of memory objects to a secondary persistent memory device such as a hard drive for offline availability.

[0013] In embodiments of the present invention, each fragment of a memory object is preferably distributed across the physical memory resources constituting the distributed store, for example, uniformly. Each different fragment or group of fragments (e.g., a shard) of a memory object is preferably stored in a corresponding different storage location, for example, a different storage node or memory module of computer memory. Each different fragment or group of fragments of a memory object typically has different discontinuous physical addresses within the physical address space associated with the distributed store / computer main memory. This means that the memory object is not owned by a particular storage node or memory module of the distributed store. The uniform accessibility of each fragment of a memory object has the advantage of improving the uniformity of memory access across multiple compute nodes in a multi-node multiprocessing computer, for example with respect to access latency, and eliminates the need to move the memory object between compute nodes. Thus, the memory object appears local to each of the multiple compute nodes in the multi-node multiprocessing computer. This does not preclude the possibility of further improving memory access latency by prefetching fragments of memory objects that are likely to be accessed or are certain to be accessed locally to an object cache memory unit on a particular compute node, for example (by computing the address of the data in the memory object prior to performing the computation on this data), thereby consolidating groups of related data that are not necessarily contiguous in physical main memory onto the compute node consuming the group of related data. In embodiments of the present invention in which the storage capacity of a distributed store / computer memory, such as a memory module, is jointly located on different compute nodes of a multi-node multi-processing computer, memory objects in the distributed store are typically maintained on multiple compute nodes rather than on a single compute node.

[0014] The list of active object identifiers is preferably maintained by the local compute node where the object identifier was first created and is limited to that local compute node. Object identifiers associated with one compute node may be shared with compute cores of other compute nodes under certain circumstances, such as task migration for workload balancing. Object identifiers are accessible to the compute cores of the compute nodes, but preferably hidden from users. Furthermore, users typically have no influence over the storage location of memory objects and their individual fragments in computer memory / distributed stores, and in particular, users cannot retrieve the contents of memory objects by referring to user names, such as file names.

[0015] In embodiments of the present invention, the computing core of each computing node uses an object-based addressing scheme in which the content of the memory object to be accessed is specified by a unique object identifier (OID) belonging to the memory object and a fragment / address offset in the object space (logical linear address space) associated with the memory object. This addressing scheme is global and applies to each computing core of multiple computing nodes in a multi-node multiprocessing computer. In contrast, the addressing model for distributed computer memory is based on a physical address that indicates the location in distributed memory where each fragment of a memory object is stored. Fixing the object identifier and offset of the data stored in a memory object in distributed computer memory also fixes the corresponding physical address, as the memory object management unit calculates the physical address as a deterministic function of the object identifier and fragment offset. Thus, a static physical address can be obtained, thereby avoiding the problem of tracking the physical address of a memory object fragment when it is possible for the memory object or its fragments to move dynamically between different storage locations in distributed computer memory. This does not preclude the possibility of having redundant backup copies of each data item in the form of backup memory objects, whose physical addresses can be determined by applying function composition, such as the iterative application of a function for physical address calculation implemented in the hardware of the memory object management unit. For example, in embodiments of the present invention, a chain of hash functions can be applied to a combination of an object identifier and a fragment offset to obtain the physical address of a redundant copy of the same data item.

[0016] Furthermore, in embodiments of the present invention, neither a virtual addressing model nor paging is required. By eliminating computational effort spent on virtual address translation, page table lookup, and page table management, the computational resources of a multi-node multiprocessing computer can be used more efficiently. In embodiments of the present invention, the calculation of the physical address is a deterministic function evaluation and becomes a single address translation step that can be computed efficiently. There is no need to translate the object-based addressing scheme to an intermediate address space / model, and the physical address is the final address used to access data in distributed computer memory. Moreover, in embodiments of the present invention, since the physical address of the fragment corresponding to the memory object data is computed deterministically, there are no miss cases for the requested data: only the relevant memory module or storage unit that holds the requested data fragment in distributed computer memory is addressed / receives the memory access request. There is no need to follow a lookup or tree traversal process to find the location of the requested data fragment in object-based distributed computer memory. In other words, in embodiments of the present invention, there are only hit cases for computation core requests to access object-based distributed computer memory.

[0017] An advantage of the embodiments of the present invention is that additional fragments can be written on the fly, eliminating the need for prior allocation requests, thus allowing memory objects to be "sizeless," for example, substantially infinite in size. This greatly simplifies memory management. Furthermore, distributed computer memory can be made easily coherent with respect to access to memory objects present therein by allowing only one task at a time to acquire ownership of memory objects in the distributed computer memory when running locally on one of the compute nodes of a multi-node multiprocessing computer. Ownership can be granted to only one task at a time by revealing the object identifier to that task only. Subsequently, any other task may be given read-only access to the memory object that has been written to or modified. Subsequent modifications to the memory object can be conveniently stored in another, for example, a newly created memory object.

[0018] In embodiments of the present invention, the distributed store as the computer main memory may be a shared global computer memory shared by a plurality of compute nodes of the first embodiment and globally accessible to the plurality of compute nodes of the first embodiment. In embodiments of the present invention, each memory module or storage unit of a compute node or a separate storage node can directly return the requested data, i.e., memory object fragments related to compute core memory operation, to the requester, i.e., the compute core of the compute node issuing the memory access request, via a communication network. This has the advantage of reducing unnecessary data traffic and response data transfer between memory devices of distributed computer memory and / or between different compute nodes of a multi-node multiprocessing computer.

[0019] According to some embodiments of the present invention, distributed computer memory may be organized hierarchically and may include, for example, one or more hierarchical memory subsystems. Parts of the hierarchical memory subsystem may be implemented locally, one at a time, in each of the compute nodes of a multi-node multiprocessing computer, while other parts of the memory subsystem deeper in the hierarchy may be implemented remotely in either remote compute nodes or storage devices / memory modules that are outside the compute nodes but connected to them via a communication network.

[0020] In embodiments of the present invention, multiple separate physical memory resource components can function as a single memory resource, i.e., distributed computer memory.

[0021] According to some embodiments of the present invention, access to memory objects by the memory object management unit is independent of the runtime environment provided by the operating system, that is, no kernel functions of the operating system are invoked.

[0022] In another embodiment, the present invention relates to a memory access method for a multi-node multiprocessing computer having a distributed store as the computer's main memory. The steps include organizing information accessible within a distributed store into memory objects as the underlying storage format, Each memory object is assigned a unique identifier, each memory object is subdivided into multiple fragments, and different fragments of the memory object have different offsets in the logical address space associated with the memory object. The information accessible in the distributed store is then determined by the steps corresponding to the fragments of the memory object. The steps include determining a unique identifier for a memory object that contains fragments referenced by one or more computing cores of a computer, Determining the offset of each of the fragments referenced by the one or more computing cores; For each of the fragments referenced by the one or more computing cores, calculating the physical address of the memory location in the distributed store as a function of the fragment offset and the unique identifier of the memory object corresponding to that fragment; Accessing each of the fragments referenced by the one or more computing cores from their respective memory locations in the distributed store via the corresponding physical address; and including.

[0023] Advantages of embodiments of the present invention are that information can be organized and stored as objects on a distributed store as computer main memory. The information can be stored in object fragments that can advantageously be evenly distributed across the individual store units that make up the distributed store, thus enabling efficient utilization of available memory resources. Further, embodiments of the present invention do not face the problem of memory fragmentation. The random and uniform distribution of the fragments of each memory object across the plurality of store units that make up the distributed object store is beneficial for the integrity of the content stored in the memory object, i.e., serves to protect against malicious attacks.

[0024] Embodiments of the present invention are particularly suitable for programming models known as "functional programming with immutable data structures".

[0025] Advantages of embodiments of the present invention are that the physical address of a memory location in the distributed store (computer main memory) is obtained by a function, i.e., is calculated rather than being looked up in a large table, thereby reducing the latency and variability of information load and store operations.

[0026] A further advantage of the embodiments of the present invention is that the object-based memory (distributed store) is controlled by the memory object management unit of the computing node, making the object storage transparent to the user. In addition, the object storage in the physical memory is safer because the content of an object cannot be overwritten by writing to another object (implicit memory protection).

[0027] In a preferred embodiment of the present invention, the address calculation and data format operations are separated from the calculation operation itself ("separated memory"). By moving the address calculation and format of the data to a separate processing unit from the calculation core, the calculation core can be kept busy with the execution of useful calculations.

[0028] An advantage of the embodiments of the present invention is that information can be accessed in the distributed store with minimal indirection.

[0029] In particular, the embodiments of the present invention can access information on the distributed store without the need for page table and / or operating system calls. This means that the content of memory objects in the distributed computer memory can be accessed without the need for page table and / or operating system calls (e.g., interrupts, exceptions). Each local computing node according to the embodiments of the present invention "knows" exactly where the accessed information is located because it is derived from the OID and location within the object, and thus can directly issue requests, such as network packets, to the distributed memory controller that holds the data. An operating system or a central database is not required to access the computer memory.

[0030] An advantage of embodiments of the present invention is that they are freed from hierarchical cache memory configurations, i.e., they provide distributed computer memory that does not involve hierarchical cache memory configurations. Information access by convenient embodiments of the present invention does not require local memory cache coherence / consistency control and / or synchronization between compute nodes, and is therefore less or nowhere affected by corresponding communication overhead. By combining object-based machine memory with functional programming using immutable data structures, consistency issues with shared memory resources can be avoided.

[0031] An advantage of the embodiments of the present invention is that the contents of memory objects can be easily shared between application processes running on different compute nodes or on the same compute node.

[0032] An advantage of the embodiments of the present invention is that the compute nodes are basic units whose numbers can be easily scaled, thereby making the compute nodes suitable for use in high-performance computers. Furthermore, more efficient software pipelining of compute and memory operations can be obtained.

[0033] In contrast to conventional computer architectures, the computing memory according to embodiments of the present invention is not under the direct control of the CPU, but is distributed across multiple controllers, each managing a smaller portion of the computer memory. This distributed object store is preferably accessible via a network, or at least a portion thereof, where the network address is equal to or equivalent to the computed physical memory address.

[0034] Specific preferred embodiments of the present invention are described in the appended independent and dependent claims. Features from the dependent claims can be combined as appropriate with the features of the independent claims and other dependent claims, not merely as explicitly stated in the claims.

[0035] For the purpose of summarizing the advantages achieved beyond the present invention and the prior art, specific objectives and advantages of the present invention have been described herein. Naturally, it should be understood that not all such objectives or advantages may necessarily be achieved in any particular embodiment of the present invention. Therefore, it will be understood by those skilled in the art that the present invention may be embodied or practiced in such a way that one advantage or set of advantages taught herein is achieved or optimized without necessarily achieving other objectives or advantages that may be taught or suggested herein.

[0036] The above and other aspects of the present invention will become clear and clarified by referring to the embodiments described below.

[0037] The present invention will now be further described with reference to the attached drawings. [Brief explanation of the drawing]

[0038] [Figure 1] This is a diagram illustrating memory objects. [Figure 2] This is a block diagram of a computing node according to the first embodiment of the present invention. [Figure 3] This illustrates the steps performed by the memory object management unit of a compute node during memory address translation according to an embodiment of the present invention. [Figure 4] This is a block diagram of a computing node according to a second embodiment of the present invention. [Figure 5] This illustrates the various stages of a pipeline related to tasks performed by the hardware components of a compute node according to one embodiment of the present invention. [Figure 6] This is a block diagram of a multi-node computer comprising multiple computing nodes according to an embodiment of the present invention. [Figure 7] This is a block diagram of a memory object management unit according to an embodiment of the present invention. [Figure 8]This is a flowchart illustrating a memory access method according to an embodiment of the present invention. [Figure 9] This describes the flow of information requests consumed by a computing core via a computer system comprising two computing nodes according to an embodiment of the present invention. [Figure 10] This shows an exemplary execution flow of an input program using one embodiment of the present invention.

[0039] The drawings are merely schematic and non-limiting. No reference numeral in the claims should be construed as limiting the scope.

[0040] In different drawings, the same reference numeral refers to the same or similar element. [Modes for carrying out the invention]

[0041] The present invention will be described with reference to specific drawings with respect to certain embodiments, but will not be limited solely by the claims.

[0042] It should be noted that the term “comprising” as used in the claims should not be construed as limiting the means enumerated by this term; it does not exclude other elements or steps. Therefore, this term should be construed as specifying that the described features, integers, steps, or components exist as mentioned, but not as excluding the existence or addition of one or more other features, integers, steps, or components, or groups thereof. Accordingly, the scope of the expression “apparatus comprising means A and B” should not be limited to an apparatus consisting solely of components A and B. This means, with respect to the present invention, that A and B are the only essential components of the apparatus.

[0043] Throughout this specification, any reference to “one embodiment” or “embodiment” means that a particular feature, structure, or characteristic described in relation to that embodiment is included in at least one embodiment of the present invention. Therefore, the occurrence of the phrase “in one embodiment” or “in an embodiment” in various parts of this specification does not necessarily all refer to the same embodiment. Furthermore, a particular feature, structure, or characteristic may be combined in any suitable manner in one or more embodiments, as will be apparent to those skilled in the art from this disclosure.

[0044] Similarly, in the description of exemplary embodiments of the present invention, various features of the invention may be summarized in a single embodiment, figure, or description thereof for the purpose of simplifying the disclosure and aiding in the understanding of one or more different progressive embodiments. However, this method of disclosure should not be interpreted as reflecting an intention that the claimed invention requires further features beyond those explicitly described in each claim. Rather, as reflected in the following claims, embodiments of the invention do not possess all the features of a single, aforementioned disclosed embodiment. Therefore, the claims following the detailed description are explicitly incorporated into this detailed description, and each claim stands independently as a separate embodiment of the invention.

[0045] Furthermore, while some embodiments described herein include some features included in other embodiments and some features not included, as will be understood by those skilled in the art, combinations of features of different embodiments are intended to be included within the scope of the present invention and to form different embodiments.

[0046] It should be noted that if a particular term is used when describing a particular feature or aspect of the present invention, it should not be construed as meaning that the term is redefined herein to be limited to any particular feature of the feature or aspect of the present invention to which the term relates.

[0047] Numerous specific details are described in the description provided herein. However, embodiments of the present invention are understood to be able to be carried out without these specific details. In other cases, well-known methods, structures, and techniques are not described in detail so as not to obscure the understanding of this description.

[0048] In the following, the terms non-aggregated storage (memory, store) and distributed storage (memory, store) may be used interchangeably. In particular, the terms distributed store and distributed memory may be used interchangeably. Distributed memory in a computer may also be called distributed computer memory, or in the case of a computing system, distributed system memory.

[0049] In the context of this invention, a compute node is a scalable unit or building block for a larger computing system, such as a supercomputer. Compute nodes are typically duplicated 1000 or more times and assembled into a supercomputer. A compute node includes at least one compute core.

[0050] In the context of the present invention, the compute core refers to the central functional unit of a compute node. The compute core is a node resource that actually executes program code and therefore accepts and operates on instruction code and data. Both the instruction code and data for execution by the core are core inputs stored in computer memory in the form of memory objects. Tasks performed by the components of the compute node are special types of memory objects. Computation tasks performed by the core include computation processes of arithmetic operations on data stored in memory objects, and possibly address calculations. In embodiments of the present invention, core inputs in computer memory and tasks in compute memory are requested, referenced, and accessed through their corresponding memory objects.

[0051] In the context of this invention, a memory object refers to a storage format for information in non-aggregated computer memory, such as a distributed store. Here, information relates to data in the broadest sense and includes, among other things, core inputs (instructions / code, data / operands), metadata associated with the information stored in the memory object, higher-level instructions such as tasks, subroutines and functions, or combinations thereof. Thus, a memory object is a container for organizing and storing content-related information. The information is typically stored self-contained in the memory object. Each memory object has its own logical address space associated with it, thereby making the content stored in the memory object accessible. Memory objects further have object identifiers (OIDs) assigned to them, thereby allowing different coexisting memory objects to be distinguished. Furthermore, memory objects may be subdivided into fragments, and fragments may be further structured into chunks, which may include nested chunks. Each fragment of each memory object is stored in a unique location in physical computer memory, preferably in a non-aggregated memory system, indicated by a physical memory address. When information is exchanged between the compute node core and compute memory during memory access operations (e.g., load / store operations), addresses must be translated between the logical address space of the memory object and the corresponding physical address in compute memory, such as a distributed store.

[0052] In the context of this invention, a view of a memory object refers to a subset of fragments contained within this memory object that are prepared for more efficient use by the core. Preparation of data fragments includes sorting, vectorization, or linearization operations, data / numeric type conversions, etc. A view can be created for a single fragment. Thus, the computing core recognizes the memory object through the view derived from it, and computation tasks, as well as other tasks such as data formatting and prefetching, are typically defined with respect to the view of the memory object. A view can be the result of data address computation, i.e., a data fragment extracted from the memory object according to a view definition that itself is stored in the memory object as metadata within a metadata chunk, for example, in the case of a chunk-based memory object. A view can be trivial linear access, stride access, block sub-selection, or any access pattern that can be represented by the view definition.

[0053] Memory management in most modern computer processors is based on virtual memory, in which the operating system allocates a virtual address space to programs, i.e., processes, loaded for execution. With virtualization implementations, programs perceive memory as a contiguous, directly addressable space (flat or linear memory). The virtual address space can be allocated independently to each running process. These computer processors exclusively use virtual memory addresses to reference and access information located within virtual memory byte by byte, and therefore rely on a memory management unit (MMU) that translates virtual memory addresses to corresponding physical memory addresses (references to locations in main memory, such as RAM, or in secondary storage devices paged out to a hard drive, for example). Virtual memory address calculation is performed routinely by the computer processor, either in a dedicated address calculation unit or within the processor's execution unit itself. Furthermore, relatively slow access to main memory or primary memory is typically mitigated by the use of hierarchical multilevel cache memory, which is an intermediate memory device located between the processor and main memory to reduce read latency and CPU stalls.

[0054] In contrast to memory management techniques and associated memory management units used by conventional computer processors, the processing unit in the embodiments of the present invention is configured to reference and access information, i.e., memory content, as fragments of memory objects. Similar to address spaces in virtual memory implementations, memory objects represent contiguous blocks of memory that are directly accessed by processes running on the processing unit. Each contiguous block of memory associated with a memory object is subdivided into multiple linearly addressable fragments, thus forming a logical address space for information retrieval by the process. The logical memory addresses of fragments contained within a memory object have physical counterparts; logical memory addresses are mapped to corresponding physical addresses that refer to locations in main memory. In contrast to virtual memory, memory objects are not exclusively linked to a process. In fact, a process can create multiple independent memory objects. Memory objects can be created as an efficient means for organizing and accessing memory content that is related to each other to form units of information storage. This includes, for example, low-level features such as constants, data variables, address variables, and opcodes / microcode, and more importantly, high-level programming language features such as arrays, callable units (e.g., functions, methods, subroutines), loops, matrices, vectors, trees, lists, classes and objects (as known from object-oriented programming), and threads. These and other features may be mixed and combined into predefined or user-defined units of information storage, i.e., memory objects. Finally, the stored information accessible through memory objects may include metadata, e.g., metadata about the memory object itself (creation date, creating entity, size, permission) and / or metadata about the information stored in fragments of the memory object, e.g., information about the data type and size of the fragment chunks (code chunk, data chunk, ...).

[0055] Different memory objects are distinguishable and can be referred to via a unique object identifier (abbreviated as OID). A memory object may be empty when created and may not yet contain a valid reference to information stored in main memory, for example. The fragments contained within each memory object typically have a physical memory cell of a predetermined size, e.g., equivalent to one byte, and can be associated with any kind of information, including, but not limited to, instructions (code), numerical data or operands consumed by the processing unit when an instruction is executed, and metadata associated with the memory object. Each fragment has a specific location within the memory object used for addressing, indicated by an offset variable. Contiguous fragments within the logical address space of a memory object can be structured into chunks, such as a data chunk associated with storing a vector or array element, or a code chunk associated with storing a set of program instructions, and can be referred to collectively as chunks.

[0056] A specific fragment of memory can be referenced by a combination of an object identifier and a fragment offset relating to the memory object containing the fragment. This combination is unique across all coexisting memory objects that access information stored in computer memory, such as shared primary memory in a supercomputer with numerous computing cores, each capable of generating and holding multiple memory objects. As a result, memory fragments generally become inaccessible when the corresponding memory object ceases to exist, for example, upon the destruction of a memory object. On the other hand, the logical address space of each memory object is isolated from the logical address spaces of all other coexisting memory objects, thereby improving computer security. Memory objects are an efficient implementation against unauthorized memory access because a memory object requesting access to a fragment belonging to a different memory object does not know the identifier of the different memory object. Memory access can still be permitted by the operating system, for example, by communicating the relevant OID to the requesting memory object, provided that the requesting memory object has been created with the necessary privileges or has been granted the necessary privileges. Special functional units (security units) can be provided on computing nodes to handle security-related aspects, such as the assignment of privileges or authorized access rights, either alone or in conjunction with the operating system.

[0057] An additional layer of security is added by the innovative fact that different fragments of a single, identical memory object are stored in vastly different, unrelated locations within physical memory; that is, each memory object, when stored in physical memory, is scattered across the entire available physical address space. For example, the calculated physical memory addresses of the stored memory object fragments are distributed uniformly and randomly. At the same time, different memory objects are stored in physical memory without overlapping. This makes it impossible for one memory object to overwrite fragments belonging to a different memory object.

[0058] Another difference between memory objects and traditional virtual memory implementations is that the logical memory addresses of memory object fragments do not undergo address translation based on page table entries; instead, the physical address of the location in computer memory corresponding to each fragment of the memory object is calculated. This is a significant advantage compared to traditional computer architectures centered on the concept of paging, as the calculation of the corresponding physical memory address does not require the use of a large page table. This frees up memory resources that would otherwise be allocated to storing the page table. It also eliminates the need to synchronize page tables maintained for different nodes in multi-node computers where information sharing between individual nodes is desired. The computation-based mapping of the logical address of each fragment to its corresponding physical address does not face the communication overhead and power consumption resulting from page table synchronization. Furthermore, the time required to perform the calculation of the physical memory address of each fragment from the aforementioned unique combination of OID and fragment offset is fixed. This is in contrast to the often unpredictable delays of information retrieval in the event of a cache miss, which most traditional computing architectures experience.

[0059] A further advantage of embodiments of the present invention over most conventional computing architectures and memory management techniques is that hierarchical multilevel cache memory is not required. This saves hardware resources and energy associated with the implementation and management of these traditional cache hierarchies. In fact, because contiguous fragments of memory objects are generally not mapped to nearby locations in a computer's main memory, data locality (in physical memory) cannot be utilized as efficiently as in conventional computing architectures with multilevel cache hierarchies. Furthermore, without one or more levels of shared cache memory in a multicore processor architecture, memory management does not face cache coherency issues.

[0060] Nevertheless, embodiments of the present invention can provide a special type of object cache that temporarily stores the contents or parts thereof of one or more memory objects. For example, one or more object caches of a compute node according to embodiments of the present invention can temporarily store copies of recently accessed fragments and / or fragments whose offsets are near the offsets of previously accessed fragments in one or more memory objects.

[0061] Figure 1 schematically represents a memory object according to an embodiment of the present invention. The memory object 10 is referenced via a unique OID and comprises a linear space 11 of logical memory addresses, such as 32-bit logical memory addresses 0x00000000 to 0xFFFFFFFF. However, the logical address space of the memory object may be smaller or larger depending on the total size of physical memory attributable to the supercomputer in which the computing node is deployed. Each logical memory address is stored in the computer's primary memory and points to a fragment 12 of information that forms a larger semantic unit of memory content, i.e., part of the memory object 10. The offset variable "OFFSET" is an index for a particular fragment to its position in the address space of the memory object to which it belongs. The fragments 12 are of the same size, for example, each equivalent to one byte in physical storage, but different fragment sizes may be defined initially. Wider fragments have the advantage of allowing access to more memory content with a single load / store command, but also require a wider data bus. Contiguous regions within the logical address space 11 of memory objects are clustered and allocated to chunks 13 containing multiple fragments. As described below, memory objects are central objects handled by a memory object controller (MOC), which uses them to organize how content-related information is stored and accessed in physical computer memory.

[0062] Referring now to Figure 2, a first embodiment of the compute node is described. The compute node 20 comprises a processor core 21, a memory object management unit 22, and a communication interface 23, such as a network interface or system bus, that links the compute node to an external memory object store 24. The memory object store may be assembled from a plurality of separate memory modules 24a to 24e that cooperate to form a distributed store. Each memory model 24a to 24e typically includes a complementary communication interface and its own local memory controller (not shown). The contents of each of the aforementioned memory objects are physically stored in the memory object store 24, which functions as the computer main memory for the core 21 of the compute node 20. The compute node of the first embodiment has only one compute core, but a compute node according to a different embodiment of the present invention may include multiple cores, such as two, four, eight, or more, that share a common memory object management unit among all compute cores of the compute node.

[0063] Core 21 may be any register-based processor core having a RISC, CISC, or VLIW architecture as known in the art. A typical core 21 comprises, as components, several functional hardware units (e.g., instruction units, execution units, control units) and register sets. The functional units may be constructed from several different or repeated sub-components, and the execution units may comprise, for example, one or more arithmetic logic units (ALUs), one or more floating-point units (FPUs), load-store units, and so on. Core 21 is operably connected to store 24 via memory object management unit 22, which determines the physical memory address of the fragment in the memory object currently referenced by core 21. In embodiments of the present invention, a system bus may be provided to directly connect core 21 to memory object management unit 22. Alternatively, the connection between the core and memory object management unit may be established by a communication channel that forms part of a larger network-on-chip communications subsystem (NoC).

[0064] While the core of a compute node can, in principle, reference individual data fragments of memory objects for computational tasks, it is more efficient for the core to access and consume informational content from a linear data space, such as vectors of data in the strict sense or sequential instructions (e.g., as generated by a compiler). Therefore, a compute node according to an embodiment of the present invention can optionally include at least one data formatting unit that assembles the referenced memory object fragments into a "view," thereby improving information utilization, i.e., data utilization, by one or more cores of the compute node. Data fragments can be extracted from a memory object according to a view definition, which is itself stored in the memory object and executed by a dedicated data address compute unit. A particular view of a memory object can be generated by sorting and / or vectorizing a subset of fragments contained in the original memory object, so that the core can linearly consume the prepared view.

[0065] A typical situation where sorting and / or vectorizing data fragments is advantageous is when accessing elements of an array, where the access follows a specific pattern (e.g., using a specific stride parameter). The preparation of a suitable view for each core by at least one data formatting unit may be optimized by the compiler at compile time. Individual memory object fragments, or views generated from accessed memory objects, may be temporarily placed in the core's scratchpad memory or register file until consumed by the core. More generally, the cores of a compute node can perform compute tasks on any memory object, and different memory objects are distinguished by different core-level addresses (e.g., the OID is part of the core-level address).

[0066] The memory object management unit 22 is similar to the memory management unit (MMU) of conventional computer architectures in that it includes hardware for address translation. Traditionally, the MMU can perform paging, i.e., the division of the virtual address space into pages, and the translation of virtual addresses to physical memory addresses using a page table. Page table entries may be held in the MMU's private memory, or partially in a translation lookaside buffer (TLB) operably connected to the MMU and assisting the MMU in address translation tasks. A TLB miss causes the CPU or operating system to initiate a page walking process to search for the page table entry in main memory. The MMU also plays a role in raising a page fault exception if a page table entry is not found in main memory for the virtual memory address currently being translated. Thus, conventional MMUs spend considerable time and energy performing address translation by lookup operations in the page table. In contrast, the memory object management unit (MOMU) according to embodiments of the present invention derives, by computation, the physical memory address corresponding to the memory object fragment referenced by the core. More specifically, the MOMU calculates the physical memory address as a function of the fragment offset and OID of the memory object containing the referenced fragment. In a preferred embodiment of the present invention, the MOMU calculates the physical memory address according to a hash function. Therefore, no lookup operation against page table entries is required, there is no need to maintain the page table in computer memory, and no page fault exceptions occur.

[0067] The internal operation of an exemplary memory object management unit (MOMU) will be briefly described with reference to the block diagram in Figure 7. The MOMU 700 is operably connected to a communication network interface 701, such as a network-on-chip (NoC) interface, through which messages related to the MOMU are exchanged. A message classification unit 702 is located in the first stage to detect the type of message received by the MOMU 700 from the network interface 701 (e.g., read or write message, OID request, etc.) and directs it to the next stage, either a key extraction unit 703 or an OID generation unit 704. The key extraction unit 703 selects the OID, fragment offset, and, if indicated, the chunk number in the requested memory object from the incoming request and assembles them into a hash key. Other message content may be removed by the key extraction unit 703. The hash key thus obtained is passed as input to a hash calculation unit 705, which is programmed to calculate a hash value from the received hash key. Hash calculation is described in more detail, for example, Donald Knuth, “The Art of Computer Programming, Volume 3: Sorting and Searching”, Addison-Wesley, 1973. The hash value, or a portion of the hash value, corresponds to the network address of the requested information and is sent to the subsequent message builder unit 707. From the initially calculated hash value, it is possible to derive modified hash values, for example, a first modified hash value destined for an information request to a distributed store, and a second modified hash value destined for information access in local cache memory. For example, the number of bits in the initial hash value may be reduced to cover the memory space of the local cache memory and / or the distributed store. Alternatively, the entire number of bits in the initial hash value may be used for the distributed store.Hash mixdown from N bits to fewer bits is described, for example, in Luykx, Atul, et al. “Two-permutation-based hashing with binary mixing” Journal of Mathematical Cryptology, vol.9, no.3, 2015, pp.139-150. The message builder unit 707 reassembles the request to the distributed memory object store and passes it to the message router 708. The message router 708 then directs the reassembled request to either the global network interface 709, which provides connectivity to the distributed store 711, or to the local cache memory 710 of the compute node, if it resides on the same node as the MOMU. In an exemplary embodiment of the present invention, the message router 708 may direct the reassembled “write / storage” message to both the distributed store and the local cache memory, while the reassembled “read / load” message is first directed to the local cache memory, and only if a local cache miss exists, the reassembled “read / load” message is then sent to the distributed store. Therefore, in the case of a "read miss," the flow of information with the appearance of a message packet is message builder unit 707 - message router 708 - local cache memory 710 - message router 708 - global network interface 709 - distributed store 711 - global network interface 709 - message router 708 - local (e.g., on-chip) network interface 701. In the case of a "read hit," the corresponding flow is message builder unit 707 - message router 708 - local cache memory 710 - message router 708 - local (e.g., on-chip) network interface 701. Thus, the message router determines the destination or sequence of destinations for the reassembled message packet. The message types described above are non-limiting examples of the various message types that the message router can process.For example, the MOMU's message classification unit 702 can verify access rights, generate an "access denied" message, and send it to the network interface 701 via the message router 708.

[0068] The OID generation unit 704 issues a new OID to a requester (OID request) when, for example, the task management unit of a computer core assigns the result of a mathematical calculation or transformation (such as addition, multiplication, scaling operations on numbers or vectors, or more complex arithmetic functions such as logarithms, exponential functions, or trigonometric functions) to a new memory object that does not currently exist. In an alternative embodiment, the MOMU's OID generation unit can be implemented in the task management unit of a compute node, for example, by embedding it. The issued new OID may be generated as a snowflake ID, for example, by combining timestamp information and a node identifier (unique) to obtain a unique memory object identifier (OID). Certain types of messages relate to memory object operations that target metadata associated with a memory object, such as metadata stored in a dedicated metadata chunk for the object, or metadata inserted at the beginning or appended to the end of a data / code chunk in a chunk-based storage format for the memory object. The operation of these memory object operations is handled by the object operation unit 706 of the MOMU 700. These may include, but are not limited to, the splitting or merging of two different memory objects. The object manipulation unit may be implemented as a conventional computing core, such as in RISC-V, which runs software programmed to modify the metadata of memory objects.

[0069] Referring again to Figure 2, the MOMU further packages and / or encodes the information stored in the accessed memory object fragment and combines it with the computed physical memory address of the memory object fragment. Naturally, when accessing a memory object fragment under a load command from a compute core, no information is included. The resulting data packet or data frame is submitted to the communication interface 23, which schedules the transmission of the submitted data packet or frame over the communication link or network interconnecting the compute node 20 with the memory modules 24a-24e of the distributed memory object store 24. Furthermore, the communication interface 23 receives data packets or frames from the memory modules 24a-24e of the distributed memory object store 24, which contain information obtained from the stored memory object fragment referenced by the compute core in the context of the load command. The response, e.g., data packet or frame concerning the requested object fragment, is routed by the MOMU and passed to the compute core, which includes steps to make on-chip communication more efficient by optionally removing details unnecessary for on-chip (e.g., intra-node) communication from the received data packet.

[0070] Memory modules 24a-24e are storage units having their respective communication interfaces, which are endpoints of a communication network, such as a cluster network, and are receivers of data packets or frames transmitted by interface unit 23. In a preferred embodiment of the present invention, a portion of the calculated physical address, for example, the k most significant bits, directly form the corresponding network address of one memory module, while the least significant bits may be used by the local memory controller of the receiving memory module to determine the location in local memory attributable to the memory module. Alternatively, the network address of the physical address may be obtained by calculation or a lookup table. Conversely, in the context of a load command, memory modules 24a-24e function as transmitting nodes of the network, while the interface 23 of the compute node is the receiver of data packets or frames containing information about the requested memory object fragment. The memory modules may be self-contained memory devices or may be contained within other devices, such as other compute nodes.

[0071] Figure 3 illustrates the steps MOMU takes when converting memory object fragments into data packets for submission over a cluster network and storage in a distributed physical memory system. Memory object 30 represents a contiguous block of memory perceived by processor cores, containing a first chunk 31 allocated to metadata and a second chunk 32 containing data. Computation cores refer to the information of fragment 33 within chunk 32 in the context of store commands. An OID is associated with memory object 30. MOMU generates an OID when creating the memory object, typically when a task using this memory object is first triggered.

[0072] In step 34, the MOMU calculates the absolute offset of fragment 33 in memory object 30 using the chunk index and the relative fragment offset in the second chunk 32. The absolute offset and OID of fragment 33 are inputs to a hash function that the MOMU calculates in step 35. Next, the MOMU prepares a data packet 36 for submission via the network interface 23, the first part 37 of the data packet 36 containing the output of the hash function from step 35, and the second part 38 containing information to be stored in the physical memory location of fragment 33. The most significant bit of the first part 37 is identical to the network address of the cluster network and thus identifies the recipient of the data packet 36, in this case corresponding to one of the memory modules 24a to 24e. The least significant bit is used by the memory controller of the receiving memory module to determine the location in its own local memory where the fragment information is stored.

[0073] Therefore, the evaluation of the hash function output (i.e., the hash value) in step 35 corresponds to the calculation of the physical memory address of the memory content to be loaded or stored. The calculated hash value is used as an index to a large distributed hash table for information retrieval. The distributed hash table is formed by the combined memory of all memory modules of the distributed object store 24. Hash table entries correspond to the memory content stored in each memory object fragment and referenced by a physical memory address (a local memory address which is the hash key). By using a hash function, it is ensured that the physical memory addresses are distributed in a uniformly random manner and that information can be accessed with uniform latency. Thus, fragments belonging to a single memory object are scattered throughout the distributed memory. The information corresponding to each fragment can be stored, for example, by duplication, in multiple locations in the distributed memory object store to save backup copies in case one of the memory modules fails. Preferably, the hash function used by MOMU is a least collision hash function. Collisions are detectable at the level of the individual memory modules that make up the distributed memory object store, and write requests are checked locally against the allocation table. If a collision is detected locally, the memory controller associated with the memory module relocates the stored information to an alternative storage location in memory. The allocation table can always provide at least one alternative storage location. Furthermore, the MOMU can calculate a set of hash values ​​(orthogonal hashes) for each fragment stored in distributed memory, for example, up to four, or even more different physical memory locations. If a collision is detected for the first hash value in the set and an alternative storage location is also occupied, the fragment of information to be stored can be transferred to the memory module corresponding to the next hash value in the set. In the rare case where collisions are detected for all hash values ​​in the set and all alternative storage locations are also occupied, the distributed object store considers main memory to be full and issues a write failure.This allows the operating system to move some of the memory content to secondary memory storage.

[0074] A second embodiment of the compute node is described here with reference to Figure 4. The compute node 400 of the second embodiment includes the same components as the compute node of the first embodiment, namely, compute core arrays 410a to 410d, a memory object management unit (MOMU) 420, and a network interface 430 as a node interface to a communication system or subsystem 437. Furthermore, the compute node 400 includes at least one task management unit 411, a data formatting unit 412, an address calculation unit 413, a node-local cache memory 415 with associated controllers 416, and node-local store units 424a to 424d, each with its respective controllers 425a to 425d. The data formatting unit 412 and the address calculation unit 413 may each be associated with their respective compute core arrays 410a to 410d and located in a composite data formatting / address calculation unit 414 serving their respective compute core arrays 410a to 410d. Preferably, a high-speed register file or scratchpad memory 426 is provided between the data formatting / address calculation units 414a to 414d and the computing core arrays 410a to 410d, and each core array includes multiple computing cores, such as two, four, or more than four computing cores per array. A communication network 427, such as a network-on-chip, interconnects various components of the computing node 400, each having an appropriate network interface or communication port 431.

[0075] In a typical application, node-local store units 424a-424d, such as DRAM modules or non-volatile memory modules, effectively contribute to the primary memory of a computing device composed of multiple similar or identical interconnected compute nodes. Therefore, node-local store units 424a-424d are generally treated as distributed store units of a larger computer memory system structured as a distributed store 440 for memory objects. In other words, store units 424a-424d are elements of a non-aggregated memory model. In embodiments of the present invention, store units 424a-424d may be assembled on a circuit board for a compute node 400, or integrated on a chip for a compute node. Memory access to each store unit 424a-424d is controlled locally by the corresponding memory controllers 425a-425d.

[0076] The conventional compute core operates periodically in a mode that enables the calculation of virtual memory addresses for fetching future data from main memory and the correct formatting of the fetched data for efficient use by the core's execution units. The CPU cycles allocated to these tasks are lost for the actual compute tasks performed on the data fetched from main memory. In this embodiment, the data formatting unit 412 and the address compute unit 413 are functional units that perform the data formatting task and the address compute task, respectively, but are located outside the compute core. This separation from the compute core is also called "isolated memory" and has the advantage that the data formatting unit, the address compute unit, and the compute core can be operated to perform different tasks simultaneously. Importantly, the compute core can spend more cycles on compute tasks, while the address compute unit 413 determines the logical memory addresses of memory object fragments (e.g., data / operands for compute) that will be consumed by compute cores 410a-410d in the near future. The address compute unit 413 can execute kernels to perform view definitions associated with memory objects, for example, as specified in the metadata chunk of the memory object. Memory object fragments loaded or prefetched from the distributed memory object store 440 are referenced by logical addresses calculated by the address calculation unit 413 and then translated by the memory object management unit 420. While the majority of address calculations are handled by the address calculation unit 413, the calculation cores 410a-410d may still be used for address calculation tasks. The data formatting unit 412 then processes the information from the memory object fragments loaded or prefetched from the distributed store 440. Formatting the data (including instructions) allows the calculation cores to perform calculations more efficiently. Information is not always stored in computer memory in a way that is appropriate or optimal for the calculation task.For example, instructions may need to be merged into more complex instructions for the computing core, array elements may need to be aligned and rearranged into a vectorized format that can be used in vector operations by the computing core, or numerical format conversion may be required. Therefore, during operation, the data formatting unit prepares the loaded or prefetched information for future use by the computing core. The information preparation tasks performed by the data formatting unit include data ordering, data vectorization, and data type casting. The correctly formatted information is then transferred to the scratchpad memory 426 and buffered in the scratchpad memory 426 until it begins to be consumed by one of the computing cores. Conveniently, the data formatting unit also buffers the difference in information consumption rates between one, i.e., the computing core, and the other, i.e., the information fetch process. The former is stable and synchronous, while the latter is asynchronous and out of order.

[0077] As described above, the MOMU 420 is a functional unit of the compute node that processes memory objects, for example, it is hardwired or executes kernels that cause the creation and destruction of memory objects at the request of the task management unit 411, generates an OID associated with the new memory object and communicates it to the task management unit 411, and controls access to fragments within the memory object when reading information from or writing information to the distributed (memory object) store. In embodiments of the present invention, the MOMU can generate the OID as a snowflake ID, for example, by combining timestamp information and a node identifier (unique) to obtain a unique memory object identifier (OID). The MOMU can also initialize, execute, and control prefetch operations on memory object fragments. Load / store commands are sent to and processed by the MOMU 420 in the same way as described with reference to Figure 3, except that prefetched and / or repeatedly used memory object fragments, or local copies of entire memory objects, may be held in the node local cache memory 415. Therefore, in this embodiment, the MOMU 420 first checks whether the node local cache memory 415 holds information about a memory object fragment that is referenced by any of the compute cores or prefetched by the MOMU. For this purpose, the MOMU can calculate a (shorter) local hash code from the fragment offset and the OID of the memory object containing the fragment, and submit the local hash code to the memory controller 416, which checks the local hash code against its private local cache hash table. If the requested fragment exists as an entry in the local cache hash table, the information stored in the fragment is transferred to the MOMU; otherwise, the MOMU continues calculating the physical memory address for accessing the requested fragment in the distributed object store 440 and prepares the corresponding network packet.In embodiments of the present invention, MOMU is not limited to the aforementioned tasks, and can also manipulate metadata stored in fragments of memory objects for purposes such as merging or splitting memory objects, thereby further supporting memory object operations required for immutable data structures.

[0078] In contrast to conventional computing architectures, information stored in a memory object framework does not need to be subject to lengthy retrieval. In a traditional multi-node computing cluster implementation, when information is fetched from a remote node, the requesting node first checks all its cache levels before the operating system queries it, sends a message to the remote node, retrieves the requested data, and finally passes it to the requesting node. In embodiments of the present invention, such a complex information retrieval process is unnecessary because the MOMU of the requesting computing node can accurately locate the requested information in memory by calculating the physical memory address. The network package can then be directly issued to a distributed memory controller having a storage unit that holds the information. Therefore, since neither the operating system nor the central database is involved in information retrieval, the resources and latency associated with information access are significantly reduced.

[0079] Preferably, the node's communication interface, rather than the node's MOMU, handles the encryption / decryption of information, such as the encoding and decryption of information transmitted over the communication network. This further reduces information retrieval / memory access latency because encryption is not required for information already present in the node's local cache memory.

[0080] In embodiments of the present invention, security functions may be integrated into the network interface 430, such as a node-local firewall. A subunit of interface 430, or a separate network management unit of the computing node 400, can pre-coordinate and monitor the flow of data packets within the node 400 and in the extended network or communication system interconnecting various computing nodes of a multi-node computer.

[0081] The security management unit 450 may verify and grant access rights to memory objects actively managed by the MOMU 420 and manage privileges associated with memory objects processed by the MOMU 420. The security management unit 450 may perform these tasks independently or in conjunction with the operating system.

[0082] The task management unit 411 distributes and assigns tasks to other components of the compute node 400 at runtime. Each task can be divided into a series of subtasks, including, but not necessarily in this order, the assignment of compute subtasks to various compute cores of arrays 410a-410d, the assignment of data formatting subtasks to the data formatting unit 412, the assignment of address compute subtasks to the address compute unit 413, fragment prefetching by the MOMU 420, and memory object creation / destruction requests to the MOMU 420. Not all subtasks are necessarily always present in each sequence, and gaps may exist. Furthermore, the task management unit 411 controls the timing and order of execution of assigned tasks by other compute node components. For this purpose, a corresponding kernel may be executed by the task management unit. In a preferred embodiment of the present invention, the task management unit 411 controls the runtime behavior of assigned tasks so that different compute node components operate in parallel, i.e., execute different tasks simultaneously. Furthermore, the task management unit 411 may implement task pipelines for tasks where subtasks must be completed sequentially. Therefore, the data formatting unit 412, the address calculation unit 413, and the MOMU 420 may be assigned different tasks at each stage of the pipeline, but they collaborate to complete a single task by progressively passing the output of the subtask of the current stage to the relevant unit in the next stage of the pipeline.

[0083] In embodiments of the present invention, one or more task management units of a compute node may be implemented as traditional cores that create task objects (a special subtype of general-purpose memory objects) and pass the OIDs (task OIDs) associated with the task objects to data formatting units and compute cores for execution. Thus, task management units according to embodiments of the present invention can also generate new memory objects, such as task objects, which, like any other type of memory object, can be stored in the distributed memory object store (computer main memory) and local cache memory. This greatly simplifies the problem of task migration between different compute nodes. Furthermore, the task management unit may be programmed to run a task scheduler (which is itself a task object) that coordinates the execution of other task objects, such as task objects, related to data prefetching, data formatting, computation, memory object manipulation, etc. Task objects typically define behavior for “views” derived from memory objects.

[0084] In embodiments of the present invention, in which multiple task management units are provided, for example, as part of the same compute node or distributed across several compute nodes of a multi-node supercomputer, the initial task management unit is selected to start in boot mode, thereby allowing the compute node or supercomputer to accept user code (program code) as input, divide the user code into subtasks, and pass these subtasks to other task management units that can accept subtasks for execution. In the context of the present invention, task scheduling can be considered a specific type of task in itself and may be part of the machine's firmware (i.e., compute node firmware).

[0085] Figure 5 shows an example of how a task management unit of a compute node according to an embodiment of the present invention organizes the assignment of tasks to the pipeline. In stage I of the pipeline, task "N" assigned by the task management unit is busy in the MOMU, which performs a prefetch operation on memory object fragments as a subtask. The fragments are prefetched from the distributed object store "DSTORE". Also in stage I of the pipeline, a previously assigned task "N-1" is busy in the data formatting unit, which performs a data formatting operation on the fragment information prefetched during the process of task "N-1". Also in stage I, an even earlier task "N-2" is busy in the compute core, which performs a compute subtask on the already formatted fragment information. The data formatting unit, compute core, and MOMU execute their respective subtasks simultaneously in stage I.

[0086] In stage II of the pipeline, the prefetched fragment information for task "N" is transferred to the data formatting unit, which formats the prefetched information for task "N". Simultaneously, the compute core handles the formatted information for task "N-1", and the MOMU begins prefetching information corresponding to new fragments for further tasks "N+1" assigned by the task management unit. Task "N-2" may be completed at this point, or it may be busy with another subtask, such as writing back results from the compute core.

[0087] Similarly, in stage III of the pipeline, the prefetched fragment information for task "N+1" is transferred to the data formatting unit. Simultaneously, the compute core begins processing the formatted information for task "N," and the MOMU prefetches new fragment information from the distributed object store "DSTORE" according to yet another task "N+2" assigned by the task management unit. By this point, task "N-1" may also be completed or busy with another subtask, such as writing back results from the compute core. The task management unit can efficiently utilize the hardware partitions of the compute nodes by continuously assigning new tasks to keep the pipeline described above constantly filled.

[0088] Furthermore, the present invention relates to a multi-node multiprocessing computer for high-performance computing applications. The multiprocessing computer, or supercomputer, comprises a plurality of compute nodes and non-aggregated memory according to an embodiment of a first aspect of the present invention. The compute nodes have the advantage of being, in fact, highly scalable base units for assembling a supercomputer. Here, multiprocessing relates to the parallel execution of multiple computer programs, such as application processes or user programs, and the resources required for them. Each process can create one or more threads, which are smaller executable entities related to the process, typically sharing the same memory resources. Furthermore, while a task typically refers to a unit of work being performed, a process specifies what needs to be done, regardless of what is currently "being done". In embodiments, a small operating system that performs threading (e.g., FreeRTOS) can be provided.

[0089] In a computer, multiple nodes are operationally connected via a communication network, i.e., they form a set of interconnected network nodes. The communication network typically comprises an internal node communication subsystem, e.g., a network-on-chip, and an inter-node communication subsystem, e.g., an Ethernet network and / or a fiber optic communication network. All computing nodes have a network interface module and are addressable by a portion of the network address. All internal node components are reachable via the full network address.

[0090] Decentralized computer memory is provided by multiple store units distributed across multiple locations, i.e., by multiple store units that form distributed computer main memory. Access to the distributed store units is controlled independently for each store unit by a local memory controller. In certain embodiments of the present invention, all store units constituting decentralized computer memory are distributed and assembled across compute nodes. Thus, each compute node can include at least one store unit, for example, multiple store units, as part of the computer memory or distributed store as a whole. In other embodiments of the present invention, the store units constituting decentralized computer memory are provided as self-contained memory modules and are independent of compute nodes. In yet another embodiment, the store units constituting decentralized computer memory are provided as a combination of self-contained memory modules and node-internal store units. Thus, in a typical embodiment of the present invention, there are more store units than compute nodes.

[0091] Figure 6 is a block diagram of supercomputer 600, which includes a number of computing nodes organized into separate clusters 620-1 to 620-4. Each cluster of computer 600 comprises multiple computing nodes accessible via a communication network 637, for example, via a cluster-level network interface 630. For example, cluster 620-1 comprises eight identical computing nodes 610-1 to 610-8, each having a node-local store unit that contributes to a larger distributed computer memory. The cluster's network interface 630 may include a firewall. Furthermore, separate node-external memory modules 641-1 to 641-3 may be connected to the network 637. These memory modules 641-1 to 641-3, for example, DRAM modules, may be equipped with their respective network communication interfaces and memory controllers, and can be used, for example, as store units for unaggregated computer memory when the physical memory provided by the node-local store units is insufficient. Each store unit of the distributed computer memory (distributed object store) has its own associated network address.

[0092] In embodiments of the present invention, the local memory controller of each store unit may hold and maintain the corresponding local portion of a global distributed hash table. The local portion of the distributed hash table may include the least significant bit (LSB) of a physical memory address computed by the MOMU of a compute node requesting memory access, for example, as the hash code output of a hash function, while the most significant bit (MSB) of the same computed physical memory address is used as the network address of the store unit. Naturally, the roles of the MSB and LSB can be reversed, and the number of bits used as the network address may vary depending on the overall size of the distributed computer memory. This allows for easy future scaling of the computer memory to a larger overall memory size, for example, doubling or power-of-two scaling of the amount of computer memory. Ideally, the store units constituting the non-aggregated computer memory would have similar or identical characteristics, for example, with respect to data transfer rate, read / write latency, and module memory size.

[0093] In embodiments of the present invention, the stored information fragments of each individual memory object are scattered throughout computer memory, for example, randomly and uniformly distributed in the physical address space. A MOMU processing a particular memory object can quickly calculate the physical memory address corresponding to the memory fragment referenced by the computing cores of the same computing node, including the MOMU, and thus can collect information fragments scattered across numerous remote store units (for example, if the relevant fragment is not found in the node's local cache memory). Stored information fragments may be shared among multiple processes running on computing cores belonging to different computing nodes. Memory objects generated by a process on one computing node are usually inaccessible to a MOMU processing memory objects generated by computing cores on different computing nodes because they have different OIDs. However, computing nodes can still exchange information about shared memory resources over a communication network. To access memory content on behalf of different computing nodes, the MOMU or a dedicated security unit on the computing node responsible for managing this content must first verify that the correct access rights have been granted. If access rights have not been granted, the MOMU or security unit may issue an "access denied" message. OIDs may be shared like any other data; that is, the value of an OID is an output from one task and an input to another. Typically, only the task that created / owned the OID has write access, while other tasks have read access. Software locks may be used to handle write permissions set by the programmer.

[0094] As mentioned earlier, each computing node that makes up a supercomputer can have object cache memory that stores memory objects or related parts of them at the node level. Even if it is feasible to maintain two or more local copies of a memory object or fragment thereof on a corresponding number of computing nodes, not all computing nodes are permitted to modify or write to memory objects that are shared resources. More specifically, only the node-specific creator task that caused the creation of the memory object as a shared resource has the right to modify the informational content of the memory object, while tasks that are not native to the node do not have these rights. Tasks running on different nodes can still read the informational content of shared memory objects stored in local object cache memory. Typically, when a producer task is running on the same object, a consumer task is not created (data dependency check in the task graph), and therefore, the programming model of this computer architecture can predict write mismatches in shared memory objects.

[0095] Referring briefly to Figure 9, the flow of information requests consumed by computing cores is shown through a computer system 90 including two computing nodes 400a and 400b (each containing four cores). Each computing core 410 can send requests or await processing of memory object views, which are sent via an intra-node network, such as an on-chip communication network. The address calculation unit 414 of each node translates the memory object view request into a corresponding request for a specific set of memory object fragments by performing address calculations within a contiguous logical address space associated with the relevant memory object. The MOMU 420 then calculates the physical address of the corresponding memory location in the distributed store for each of the requested fragments. A portion of the calculated physical addresses is used as network addresses in the global communication network. Requests for fragments at a given physical memory address are sent over the network in the form of data packets and received by the relevant store unit 424 of the distributed object store. Each store unit of the distributed object store may comprise several memory modules.

[0096] Furthermore, the present invention relates to a memory access method for a computer, such as a supercomputer. The computer comprises a plurality of computing nodes, such as a computing node according to an embodiment of the first aspect of the present invention, and a distributed store as the computer's main memory. Typically, the distributed store comprises or is composed of a plurality of independent store units, and access to each store unit is controlled by its respective memory controller (i.e., a non-aggregated computer memory model). Some or all of the store units may be assembled or integrated into a computing node, that is, each computing node may have one or more of a plurality of store units (e.g., memory modules such as DRAM modules). Nevertheless, some store units may be provided separately from the computing nodes. The memory access method can also be used for centralized computer memory, i.e., non-distributed computer main memory. Memory access may be requested when a computing core located in one of the computing nodes seeks to load or store information necessary for performing a computing task. The information input to the computing core is accessible on the distributed store, where it is organized and stored as fragments of memory objects. The computing core does not know the storage locations of various memory object fragments in the computer's distributed memory, but references a desired fragment through the location of the fragment in the respective logical address space associated with the memory object containing the referenced fragment, for example, as part of a view. Computation tasks processed by the computing core are typically assigned by task management units on different computing nodes, and therefore the tasks themselves are treated as memory objects that can move between nodes.

[0097] Now, with reference to Figure 8, the steps of an exemplary memory access method 800 will be described.

[0098] In step 801, information accessible within the distributed store is organized into memory objects as the underlying storage format. This step is implicitly performed as soon as the user's program defines all its data structures, functions, subroutines, etc., which are then converted into different types of memory objects at compile time. Thus, information is always presented within the context of memory objects. Computation results or intermediate computation results, as well as inputs from I / O devices, are also identified by memory objects, which can be dynamically allocated at runtime by the MOMU of the compute node, for example, upon request by the task management unit.

[0099] In step 802, a unique identifier (OID) is assigned to each memory object, and each memory object is subdivided into multiple fragments such that different fragments of the memory object have different offsets in the logical address space associated with the memory object. As a result, the accessible information in the distributed store corresponds to the fragments of the memory object. The MOMU and task management unit of each compute node can assign an OID to a memory object at creation time using a snowflake ID generator that takes, for example, a timestamp and compute node ID as input.

[0100] Next, in step 803, a unique identifier (OID) for each memory object containing a fragment referenced by one or more computing cores of the computer is determined, for example, by isolating the OID-containing portion of the core address used by the requesting core, and in the subsequent determination step 804, the offset for each fragment referenced by one or more computing cores is obtained. The fragment offsets can be derived from the memory object view by the data address calculation unit of the computing node executing the view definition.

[0101] In step 805, the physical address of the memory location in the distributed store is computed for each referenced fragment as a function of the fragment offset and the unique identifier of the memory object corresponding to the fragment. This step may include the computation of a hash value by evaluating a hash function, which can be performed by a hash computer in the MOMU of the compute node.

[0102] Ultimately, the fragments referenced by one or more computing cores are accessed from their respective memory locations in the distributed store via their corresponding physical addresses (806).

[0103] In embodiments of the present invention, the memory access method may further include the step of distributing each memory object fragment accessible in the distributed store across multiple store units by calculating the physical address of each memory location in the distributed store according to a random uniform distribution function, such as a hash function, preferably a least collision hash function. Hash collisions can be resolved in the manner described above. Furthermore, the method may include formatting of the accessed memory object fragments to improve information utilization by the computing core. The formatting task may include sorting, linearizing, vectorizing, pattern accessing, and subselecting of the accessed memory object fragments before consumption by the computing core. The formatting task object may specify the memory address of a scratchpad memory or a register file of the computing core where the formatted memory object, i.e., the view, is temporarily located. The kernel running on the computing core then consumes the view from the scratchpad. In addition, a portion of the scratchpad may be allocated to "kernel-in-scope" temporary data in a heap-based manner.

[0104] In embodiments of the present invention, the memory access method is also applicable to information referenced by the MOMU of a compute node in the context of an assigned prefetch task defined with respect to a memory object, a part of a memory object (e.g., a fragment, a chunk), or a view derived from a memory object. Furthermore, the memory access method is also applicable to information accessed on node-local cache memory for faster retrieval before being accessed in a distributed store in the event of a cache miss.

[0105] Furthermore, a computation task may itself be part of a larger program, such as user-specified program code, which is represented and stored in the form of memory objects. During compilation or runtime of the program code, the code is transformed or broken down into a series of tasks according to a data flow / computation graph, which has task objects as graph nodes, for example, so that each task contains information about its predecessors and successors.

[0106] In embodiments of the present invention, the execution of program code typically includes the following three parts: (i) A kernel that runs on a compute core and can take a view as input and generate a view as output (executable code for the compute core); (ii) A kernel (executable code for the data unit) that runs on a data formatting and address calculation unit (abbreviated as a data unit) that processes memory objects or parts thereof into a view according to the view definition; (iii) Execution of a task splitting and / or concatenation algorithm that divides a task into smaller tasks and / or reassembles the smaller tasks into more complex tasks (executable code for the task management unit).

[0107] In embodiments of the present invention, tasks and associated task objects can be further classified into different classes and conditionally converted in the following manner. (a) A normal task that can be moved across machines (computers) but is too large to run on a single computing core and therefore needs to be divided into subtasks. The resulting subtasks can be either normal tasks or atomic tasks. (b) Atomic tasks that can be moved across the machine, are small enough to fit into the computing cores, and therefore cannot be further subdivided. (c) A kernel task is an atomic task compiled / linked for local execution, each consisting of a task kernel and a view definition. Kernel tasks are linked to groups of compute cores, e.g., clusters, managed by a task management unit.

[0108] The following describes an exemplary execution flow with reference to Figure 10. At startup, the user or programmer provides a top-level main task, which includes the kernel, view definitions, and code for task partitioning / combining algorithms, preferably as three-part code (shown as a square box in the figure) provided in a mathematical library file. Typically, the programmer then combines elements provided by the software library with elements provided in the mathematical library file, for example, by calling sparse matrix-vector multiplication on vector A and matrix B. Compiling the user program can generate a process graph based on the main task, whose graph vertices are task objects.

[0109] Next, the scheduler / task management unit takes the main task object and performs a split / combine to create smaller tasks. These smaller tasks can be moved to other task management units, and the split / combine process can be repeated until the subtasks fit into the compute cores. The resulting "atomic" task cannot be further subdivided but is still traversable / schedulable across all task management units in the machine (i.e., inter-node task scheduling). Load balancing can be improved by moving tasks to different task management units in the machine (located on the same or different compute nodes). Once a task management unit in a machine accepts an atomic task for execution, it is compiled and linked locally to form a "kernel task" (i.e., kernel plus view definition) and placed in a local execution queue. The task management unit then assigns that kernel task to compute cores (which run the "kernel") and data formatting units (which run the "view definition"). Kernel tasks typically can no longer be moved to other nodes.

[0110] The data formatting unit executes the view definition, requests the MOMU to read the necessary fragments of the input object, and places the obtained data as a view in scratchpad memory. The compute core then accesses the view in scratchpad memory, performs mathematical operations on it, and returns the output data to scratchpad memory. Now the data formatting unit can access the computed output view from scratchpad memory and translate it into addresses in memory objects that may be output memory objects. The data formatting unit then sends a write request to the MOMU for those objects. After the MOMU and distributed store have confirmed that they have stored the (output) memory objects, the data formatting unit notifies its respective task management unit that the kernel task has finished. If other tasks are waiting in the queue, the task management unit schedules the next kernel task.

[0111] Although the present invention has been illustrated and described in detail in the drawings and the foregoing description, such illustrations and descriptions should be understood as illustrative or typical, not limiting. Other variations of the disclosed embodiments can be understood and achieved by those skilled in the art by examining the drawings, this disclosure and the appended claims in the practice of the claimed invention. In the claims, the phrase “comprising” does not exclude other elements or steps, and the indefinite article “a” or “an” does not exclude the plural. No reference numeral in the claims should be construed as limiting.

Claims

1. A compute node (400) for a multi-node multiprocessing computer, which has distributed memory (440) as physical main computer memory, At least one computing core (410) that performs a computation process on information organized and stored in a memory object (10), wherein the information is referred to as one or more fragments (12) contained within the memory object, and each different fragment of the memory object has a different offset in the logical address space (11) associated with the memory object, and each memory object is assigned a unique identifier (OID), A memory object management unit (420) operably connects the at least one computing core (410a to 410d) to the distributed memory (440) in order to access the fragments referenced by the at least one computing core (410a to 410d) from their respective memory locations in the distributed memory, wherein each memory location in the distributed memory is accessible via a corresponding physical address, the memory object management unit (420) comprises The memory object management unit (420) computes the physical address corresponding to the memory location of each fragment to be accessed in the distributed memory as a function of the fragment offset in the logical address space associated with the memory object containing the fragment, and the unique identifier assigned to the memory object containing the fragment, by the compute node (400).

2. The computing node (400) according to claim 1, further comprising one or more memory modules (424; 425) as storage units for each of the distributed memory (440).

3. The computing node (400) according to claim 1 or 2, wherein the physical addresses calculated by the memory object management unit (420) for each different fragment of the memory object are randomly and uniformly distributed in the physical address space of the distributed memory.

4. The computing node (400) according to any one of claims 1 to 3, wherein the at least one computing core (410) and the memory object management unit (420) are connectable to a communication network (427; 437) via their respective network interfaces (431) and are accessible on the communication network via their respective network addresses.

5. The compute node (400) according to any one of claims 1 to 4, further comprising a data formatting unit (412) for preparing memory object fragments for future use by at least one compute core (410).

6. A compute node (400) according to any one of claims 1 to 5, further comprising node-local cache memory (415; 416) for temporarily storing prefetched memory object fragments and / or outputs from the at least one compute core stored as memory object fragments.

7. The compute node (400) according to claims 5 and 6, further comprising the at least one compute core (410) as each stage of the pipeline, the data formatting unit (412), and a task management unit (411) that allocates tasks for parallel pipeline execution by the memory object management unit (420).

8. The compute node (400) according to any one of claims 1 to 7, wherein the memory object management unit (420) computes at least one physical address of each fragment referenced by the at least one compute core (410) by applying at least one hash function to the combination of the unique identifier and the fragment offset.

9. A multiprocessing computer (600) comprising a plurality of computing nodes (610) according to any one of claims 1 to 8, a distributed memory for memory objects including a plurality of storage units (641), and a communication network (637) operably connecting each of the plurality of computing nodes to each of the plurality of storage units, wherein at least a portion of the plurality of storage units are included in the computing nodes and a unique network address is assigned to each storage unit.

10. The multiprocessing computer (600) according to claim 9, wherein each of the plurality of storage units is included in one of the computing nodes, and each computing node comprises at least one storage unit of the distributed memory.

11. A multiprocessing computer (600) according to claim 9 or 10, wherein a unique node identifier is assigned to each of the plurality of compute nodes, and the memory object management unit generates the unique identifier of the memory object on request based on a timestamp and the node identifier of each compute node.

12. A multiprocessing computer (600) according to any one of claims 9 to 11, wherein a portion of each physical address matches the network address of one of the storage units.

13. To use the multiprocessing computer (600) according to any one of claims 9 to 12 for high-performance computing.

14. A memory access method (800) for a multi-node multi-processing computer (600) that has distributed memory as physical main computer memory, The information accessible in the distributed memory is organized into a memory object (10) as a storage format (801), Each memory object (10) is assigned a unique identifier (OID), each memory object is subdivided into multiple fragments (12), and different fragments of the memory object have different offsets in the logical address space (11) associated with the memory object, so that the information accessible in the distributed memory corresponds to the fragments of the memory object (802), (803) Determining the unique identifier (OID) of the memory object which contains each of the fragments referenced by one or more computing cores of the computer, Determining the respective offsets of the fragments referenced by the one or more computing cores (804), For each fragment referenced by one or more computing cores, the physical address of the memory location in the distributed memory is calculated as a function of the fragment offset and the unique identifier of the memory object corresponding to the fragment (805), Accessing the fragment referenced by one or more computing cores from the respective memory locations in the distributed memory via the corresponding physical address (806), A memory access method that includes this.

15. The distributed memory comprises a plurality of storage units, and the method is The fragments of each memory object accessible in the distributed memory are distributed to the plurality of storage units by calculating the physical addresses of each memory location in the distributed memory according to a random uniform distribution function. The memory access method (800) according to claim 14, further comprising:

Images